Method and apparatus for sparse matrix-vector multiplication based on RISC-V

Abstract

This disclosure relates to a method and apparatus for processing sparse matrix-vector multiplication based on RISC-V. The method includes: dividing an N:M sparse weight matrix into P×Q N:M weight matrix blocks, where each weight matrix block has a dimension of V×V; dividing an input feature vector into Q feature vector segments of length V; extracting all non-zero weight elements in each weight matrix block in row-major order to generate a dense weight vector for that weight matrix block, and generating an index vector corresponding to each weight matrix block, the index vector indicating the column offset position of each non-zero weight element in the corresponding weight matrix block within its respective row; processing at least one weight matrix block and its corresponding feature vector segment in parallel within a vector execution unit of a RISC-V processor; and obtaining the output vector by accumulating all parts and vectors associated with the same output row of the output vector.

Description

Technical Field

[0001] This disclosure relates to the field of computers, and more specifically, to a method and apparatus for processing sparse matrix-vector multiplication based on RISC-V. Background Technology

[0002] The sparsity pattern in model weights is a core technical element of deep neural network compression models, as it fundamentally determines the final trade-off between accuracy and inference efficiency. Element-wise sparsity is achieved through individual weight pruning, with minimal impact on model accuracy. However, this format leads to irregular computations and scattered memory accesses, making it extremely difficult to achieve efficient utilization on GPUs. In fact, the trade-off between accuracy and efficiency has always been a long-standing challenge for sparse deep neural networks.

[0003] Currently, N:M sparseness is widely used in deep learning algorithms. This mode requires that non-zero weights be locally balanced, for example, by mandating that there must be N non-zero elements in every M consecutive elements. Thanks to its more regular computation and memory access patterns, N:M sparseness allows for efficient computation through customized hardware. Summary of the Invention

[0004] According to embodiments of this disclosure, a method for processing sparse matrix-vector multiplication based on RISC-V is provided. The method includes: dividing an N:M sparse weight matrix of dimension D×K into P×Q weight matrix blocks, wherein each weight matrix block has a dimension of V×V, P, Q, and V are all positive integers, and satisfy P = D / V, Q = K / V, wherein the weight matrix blocks adopt an N:M structured sparse pattern, N and M are both positive integers, and satisfy 1 ≤ N < 0. M; Divide the input feature vector of length K into Q feature vector segments, where each feature vector segment has a length of V; Extract all non-zero weight elements of each weight matrix block in row-major order to generate a dense weight vector of that weight matrix block, and generate an index vector corresponding to each weight matrix block, the index vector indicating the column offset position of each non-zero weight element in the corresponding weight matrix block within its row; Perform matrix-vector multiplication of at least one weight matrix block with the corresponding feature vector segment in parallel in the vector execution unit of the RISC-V processor, including: Dynamically selecting and rearranging feature data elements from the corresponding feature vector segment according to the index vector of at least one weight matrix block to generate rearranged feature vector segments aligned with the dense weight vector of the at least one weight matrix block, and performing vectorized multiplication and accumulation of the dense weight vector of the at least one weight matrix block with the rearranged feature vector segments to obtain a partial sum vector; and Obtain the output vector of length D by accumulating all partial sum vectors associated with the same output row of the output vector.

[0005] According to embodiments of this disclosure, an apparatus for processing sparse matrix-vector multiplication based on RISC-V is provided, wherein the apparatus includes a processor circuit configured to perform the above-described method.

[0006] According to embodiments of the present disclosure, a computer-readable storage medium is provided having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to perform the methods described above.

[0007] According to embodiments of this disclosure, a computer program product is provided, including instructions, wherein the instructions, when executed by a processor, cause the processor to perform the methods described above. Attached Figure Description

[0008] Embodiments of this disclosure will be described by way of example, not limitation, in conjunction with the accompanying drawings, wherein similar reference numerals denote similar elements, and wherein:

[0009] Figure 1 The diagram illustrates a block diagram of an example processor and / or SoC 100, which may have one or more cores and an integrated memory controller.

[0010] Figure 2 The illustration shows a flowchart of a method for processing sparse matrix-vector multiplication based on RISC-V according to an embodiment of the present disclosure.

[0011] Figure 3 The illustration shows a schematic diagram of a hardware microarchitecture implementation for executing custom instructions according to an embodiment of the present disclosure.

[0012] Figure 4 The illustration shows a flowchart of a sparse matrix-vector multiplication acceleration method with a sparsity of 1:8 according to an embodiment of the present disclosure.

[0013] Figure 5A The illustration shows an exemplary schematic diagram of sparse weight matrix block partitioning according to an embodiment of the present disclosure.

[0014] Figure 5B The illustration shows an exemplary schematic diagram of the weight distribution of a sparse weight matrix block in a 1:8 sparse mode.

[0015] Figure 5C The illustration shows an exemplary diagram of the dense weight vector generated for a sparse weight matrix block.

[0016] Figure 5D An exemplary schematic diagram of an index structure for a 1:8 sparse pattern is illustrated according to an embodiment of the present disclosure.

[0017] Figure 6The illustration shows a flowchart of a sparse matrix-vector multiplication acceleration method with a sparsity of 2:8 according to an embodiment of the present disclosure.

[0018] Figure 7A The illustration shows an exemplary schematic diagram of sparse weight matrix block partitioning according to an embodiment of the present disclosure.

[0019] Figure 7B The illustration shows an exemplary schematic diagram of the weight distribution of a sparse weight matrix block in a 2:8 sparse mode.

[0020] Figure 7C The illustration shows an exemplary diagram of the dense weight vector generated for a 2:8 sparse weight matrix block.

[0021] Figure 7D An exemplary schematic diagram of an index structure for a 2:8 sparse pattern is illustrated according to an embodiment of the present disclosure.

[0022] Figure 8A The illustration shows an exemplary schematic diagram of sparse weight matrix block partitioning according to an embodiment of the present disclosure.

[0023] Figure 8B The illustration shows an exemplary diagram of the weight distribution of a sparse weight matrix block in a 4:8 sparse mode.

[0024] Figure 8C The illustration shows an exemplary diagram of the dense weight vector generated for a 4:8 sparse weight matrix block.

[0025] Figure 8D An exemplary schematic diagram of an index structure for a 4:8 sparse pattern is illustrated according to an embodiment of the present disclosure.

[0026] Figure 9 An exemplary schematic diagram of the weight distribution of a vector-level structured sparse matrix according to an embodiment of the present disclosure is shown.

[0027] Figure 10 An exemplary schematic diagram of a block-level structured sparse matrix according to an embodiment of the present disclosure is shown.

[0028] Figure 11 This is a block diagram illustrating components capable of reading instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) and performing any one or more of the methods discussed herein, according to some example embodiments. Detailed Implementation

[0029] The features and exemplary embodiments of various aspects of this application will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a comprehensive understanding of this application. However, it will be apparent to those skilled in the art that this application can be implemented without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of this application by illustrating examples. This application is by no means limited to any specific configuration presented below, but covers any modifications, substitutions, and improvements to elements, components, and algorithms without departing from the spirit of this application. Well-known structures and techniques are not shown in the accompanying drawings and the following description in order to avoid unnecessary obfuscation of this application.

[0030] Furthermore, the various operations will be described as multiple discrete operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations must depend on the order. In particular, these operations do not need to be performed in the order presented.

[0031] The phrases “in an embodiment,” “in one embodiment,” and “in some embodiments” are used repeatedly throughout this document. These phrases do not typically refer to the same embodiment; however, they may refer to the same embodiment. Unless the context otherwise specifies, the terms “comprising,” “having,” and “including” are synonyms. The phrases “A or B” and “A / B” mean “(A), (B) or (A and B).”

[0032] Figure 1 A block diagram of an example processor and / or SoC 100 is illustrated, which may have one or more cores and an integrated memory controller. The processor 100 illustrated by solid-line boxes has a single core 102(A), system proxy unit circuitry 110, and a set of one or more interface controller unit circuitry 116, while alternative processors 100 can be illustrated by dashed-line boxes having multiple cores 102(A)-(N), a set of one or more integrated memory control unit circuitry 114 in the system proxy unit circuitry 110, dedicated logic 108, and a set of one or more interface controller unit circuitry 116.

[0033] Different implementations of processor 100 may include: 1) a CPU, where dedicated logic 108 is integrated graphics and / or scientific (throughput) logic (which may include one or more cores, not shown), and cores 102(A)-(N) are one or more general-purpose cores (e.g., general-purpose ordered cores, general-purpose out-of-order cores, or a combination of both); 2) a coprocessor, where cores 102(A)-(N) are a large number of dedicated cores primarily for graphics and / or scientific (throughput) purposes; and 3) a coprocessor, where cores 102(A)-(N) are a large number of general-purpose ordered cores. Thus, processor 100 may be a general-purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (General-Purpose Graphics Processing Unit), a high-throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. The processor 100 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of process technologies, such as complementary metal oxide semiconductor (CMOS), bipolar CMOS (BiCMOS), P-type metal oxide semiconductor (PMOS), or N-type metal oxide semiconductor (NMOS).

[0034] The memory hierarchy includes one or more levels of cache cell circuitry 104(A)-(N) within cores 102(A)-(N), a group of one or more shared cache cell circuitry 106, and external memory (not shown) coupled to the group of integrated memory controller cell circuitry 114. The group of one or more shared cache cell circuitry 106 may include one or more intermediate level caches, such as level 2 (L2), level 3 (L3), level 4 (4), or other levels of cache, such as the last-level cache (LLC), and / or combinations thereof. While in some examples interface network circuitry 112 (e.g., a ring interconnect) provides an interface to dedicated logic 108 (e.g., integrated graphics logic), the group of shared cache cell circuitry 106, and system agent cell circuitry 110, alternative examples use any number of known techniques to interface to these units. In some examples, one or more circuits in the shared cache cell circuitry 106 maintain consistency with cores 102(A)-(N). In some examples, the interface controller unit circuit 116 couples these cores to one or more other devices 118, such as one or more I / O devices, storage devices, one or more communication devices (e.g., wireless networks, wired networks, etc.).

[0035] In some examples, one or more of cores 102(A)-(N) have multi-threading capabilities. System agent unit circuitry 110 includes those components that coordinate and operate cores 102(A)-(N). System agent unit circuitry 110 may include, for example, power control unit (PCU) circuitry and / or display unit circuitry (not shown). The PCU may be (or may include) the logic and components required to regulate the power state of cores 102(A)-(N) and / or dedicated logic 108 (e.g., integrated graphics logic). Display unit circuitry is used to drive one or more externally connected displays.

[0036] Core 102(A)-(N) can be homogeneous in terms of instruction set architecture (ISA). Alternatively, core 102(A)-(N) can also be heterogeneous in terms of ISA; that is, a subset of core 102(A)-(N) may be able to execute one ISA, while other cores may be able to execute only a subset of that ISA or be able to execute another ISA. In one embodiment, processor core 102(A)-(N) may employ all or part of the Reduced Instruction Set Computer (RISC-V) instruction set architecture. In another embodiment, processor core 102(A)-(N) may employ various other instruction set architectures, such as x86, ARM, MIPS, etc.

[0037] The N:M sparse mode benefits from its more regular computation and memory access patterns, making efficient implementation based on customized hardware possible. However, in general-purpose processors for open-source, scalable instruction set architectures like RISC-V, there is still a lack of hardware solutions that can systematically utilize N:M sparse characteristics to accelerate core operations (especially sparse matrix-vector multiplication (SpMV)).

[0038] Compared to intensive general-purpose matrix multiplication (GEMM), existing sparse GEMM implementations are significantly less computationally efficient and struggle to achieve effective energy savings. This is rooted in the irregularity and dispersion of sparse data storage, which prevents parallel architectures like GPUs from fully utilizing their high-bandwidth storage and massively parallel computing capabilities. This not only results in low utilization of computing resources (e.g., multiplication units) but also leads to additional power waste due to the inefficient use of memory bandwidth caused by the dispersed access of sparse weight matrices. Similarly, performing N:M sparse matrix-vector multiplication on a RISC-V general-purpose CPU also suffers from high software decoding overhead, low computational parallelism, and irregular memory access due to the lack of dedicated hardware support.

[0039] To address this, this disclosure proposes a method and apparatus for accelerating sparse matrix-vector multiplication on RISC-V CPUs. By co-designing the extended instruction set and microarchitecture, direct hardware mapping of sparse computation at the instruction level is achieved, enabling general-purpose CPU pipelines to achieve near-dense computation regularity and high throughput. This significantly improves the inference speed and energy efficiency of sparse neural networks on the RISC-V platform while maintaining the original accuracy of the model.

[0040] Specifically, this disclosure proposes a sparse index structure designed for efficient hardware decoding, and defines a RIS structure based on this structure that can fully utilize RIS. The vector rearrangement multiplication instruction for V computing resources significantly improves the execution efficiency of N:M sparse matrix vector multiplication by regularizing and reorganizing non-zero data in a sparse matrix into a dense matrix and achieving vectorized dense computation. This technology is for RIS. The V ecosystem provides key IP support for the widespread application of AI Transformers and other fields: it can not only significantly accelerate sparse computing in the current mainstream sparse rate range (such as N:M sparse mode with a sparse rate of 50% to 90%), but also directly complete sparse data recombination and computation mapping through hardware, simplifying the software processing flow and achieving significant energy efficiency optimization while improving computing throughput.

[0041] Figure 2The figure illustrates a flowchart of a method for processing sparse matrix-vector multiplication based on RISC-V according to an embodiment of the present disclosure. Method 200 may include steps S202, S204, S206, S208, and S210. However, in some embodiments, method 200 may include more or fewer different steps, and the present disclosure places no limitation thereon.

[0042] At step S202, an N:M sparse weight matrix with dimensions D×K is divided into P×Q weight matrix blocks, where each weight matrix block has dimensions V×V, and P, Q, and V are all positive integers, and satisfy P = D / V, Q = K / V. Here, each weight matrix block adopts an N:M structured sparse pattern, and N and M are both positive integers, and satisfy 1 ≤ N < M.

[0043] In an embodiment of the present disclosure, the parameter V is generally co-designed with the vector register length (VLEN) of the processor and the operation data bit width. For example, when VLEN = 256 bits and 8-bit integer data is used, a single vector register can accommodate 32 elements. At this time, V can be configured as, for example, 8, 16, or 32 to ensure that the data of a single weight matrix block or eigenvector segment can be efficiently loaded into the vector register for processing. For example, for a 1024×1024 weight matrix (D = K = 1024), if V = 8, then P = 128 and Q = 128 can be obtained, thereby regularly dividing the original matrix into 128×128 8×8 matrix blocks.

[0044] In an embodiment of the present disclosure, the calculation acceleration optimization is mainly performed for a deep neural network model adopting an N:M structured sparse pattern, especially for the widespread medium and high sparsity scenarios, such as typical scenarios where the sparsity rate is in the range of 50% to 90%. The structured sparsity N:M is a mandatory constraint on the distribution of non-zero elements in the sparse weight matrix and the corresponding weight matrix block. In an embodiment of the present disclosure, the value of M can be 8, and correspondingly, N can take values of 1, 2, or 4, corresponding to three sparsity rates of 87.5% (1:8), 75% (2:8), and 50% (4:8), respectively. It should be understood that the specific values of N and M above are only examples, and the technical solution of the present disclosure is equally applicable to other sparse configurations where M is other values (such as 4, 16, 32, etc.) and satisfies N < M, and its hardware execution process and data organization method can be realized through adaptive adjustment.

[0045] At step S204, an input eigenvector with length K is divided into Q eigenvector segments, where the length of each eigenvector segment is V.

[0046] In the embodiments of this disclosure, the partitioning of the feature vector and the column partitioning of the weight matrix are strictly aligned, and both share the same segment length V. Specifically, the feature vector X of length K is uniformly divided into Q continuous segments (X0, X1, ..., X...) of length V. Q-1 ), where Q = K / V. This partition ensures that the j-th feature vector segment X j Mathematically, this corresponds precisely to all rows in the j-th column of the weight matrix W. During hardware execution, each feature vector segment X of length V... j This can be loaded into the vector register as a complete, well-organized block of data. This alignment partitioning is crucial, as it allows subsequent operations on any weight matrix block W to be performed. ij The computation of the matrix block (in the i-th row and j-th column) depends only on a single eigenvector segment X. j .

[0047] In step S206, all non-zero weight elements in each weight matrix block are extracted in row-major order to generate a dense weight vector for that weight matrix block, and an index vector corresponding to each weight matrix block is generated, which indicates the column offset position of each non-zero weight element in the corresponding weight matrix block within its respective row in the weight matrix block.

[0048] Specifically, for a V×V weight block, it is scanned in row-major order, and all non-zero weights are extracted and packaged consecutively to form a dense weight vector. Simultaneously, an index value is generated for each non-zero weight, which records only the column offset position (offset 0 to V-1) of the non-zero weight within the original matrix block's row. For example, for an 8×8 and 2:8 sparse matrix block, a dense weight vector containing 16 weights and an index vector containing 16 index values ​​(each ranging from 0 to 7) are generated. In embodiments of this disclosure, each element in the index vector can be encoded using the minimum number of bits required to uniquely represent V different column offset positions; for example, when V=8, each offset is represented by 3 bits.

[0049] At step S208, the matrix-vector multiplication of at least one weight matrix block and the corresponding feature vector segment is processed in parallel in the vector execution unit of the RISC-V processor, including: dynamically selecting and rearranging feature data elements from the corresponding feature vector segment according to the index vector of at least one weight matrix block to generate rearranged feature vector segments aligned with the dense weight vector of at least one weight matrix block, and performing vectorized multiplication and accumulation on the dense weight vector of at least one weight matrix block and the rearranged feature vector segment to obtain a partial sum vector.

[0050] In embodiments of this disclosure, parallel processing of matrix-vector multiplication of at least one weight matrix block and corresponding feature vector segment can be triggered in response to the execution of a single RISC-V custom vector instruction, the semantics of which are configured to: perform dynamic selection and rearrangement of feature data elements and vectorized multiplication and accumulation operations within one instruction cycle.

[0051] The computational iteration process defined in step S208 corresponds in hardware to the atomic operation performed by executing a single custom vector instruction. In the embodiments of this disclosure, the custom vector instruction is a vector rearrangement multiplication (vreordermul) instruction, whose operands include: source and target vector registers storing dense weight vectors of at least one weight matrix block, a first source vector register storing index vectors corresponding to at least one weight matrix block, and a second source vector register storing feature vector segments corresponding to at least one weight matrix block. This instruction explicitly receives dense weight vectors, index vectors, and original feature vector segments through its operands. Its microarchitecture is designed as follows: first, based on the column offset information encoded by the index vector, the corresponding data elements are selected from the feature vector segments and sequentially reorganized into a new rearranged feature vector segment whose data order is strictly aligned with the dense weight vector; then, the rearranged vector and the dense weight vector are fed into a fully parallel multiply-accumulate unit for operation to generate a partial sum vector corresponding to one or more output row positions. This "rearrangement-computation" fusion operation is completed within a single instruction, realizing zero-overhead hardware conversion from sparse data format to regularized dense computation.

[0052] The vector rearrangement multiplication (vreordermul) instruction is a custom instruction based on the RISC-V vector extension instruction set, with the instruction format vreordermul.vv vd, vs2, vs1. In the embodiments of this disclosure, a single computation iteration corresponds in hardware to the atomic operation performed by executing one of these custom vector instructions.

[0053] The vreordermul instruction strictly follows the RISC-V Vector Extension (RVV) encoding standard, and its 32-bit binary instruction encoding format is shown in Table 1:

[0054] Table 1

[0055]

[0056] As shown in Table 1, the vreordermul directive can include the following fields:

[0057] Opcode (bits [6:0]): RVV instruction opcode, with a fixed opcode 1010111, which is reserved in the RISC-V instruction set for all vector operations (OP-V encoding space).

[0058] vd (bits [11:7]): Source and destination registers. Before the vreordermul instruction is executed, it stores a dense set of data from the eigenvector; after the vreordermul instruction is executed, it stores the result of the element-wise multiplication operation.

[0059] Function code funct3 (bits [14:12]): Encoded as 000, it indicates that this instruction is a vector-to-vector operation type, meaning that all source operands come from vector registers.

[0060] vs2 (bits [19:15]): Specifies the source register for storing the vector of data to be strobed. This vector stores non-zero weight values ​​loaded from memory consecutively.

[0061] vs1 (bits [24:20]): Specifies the source register for storing the index vector. Each element in the index vector is a relative position index within the data block with a non-zero weight.

[0062] Function code funct6 (bits [31:26]): encoded as 101100. This 6-bit field uniquely identifies the specific operation vreordermul within the OP-V encoding space and is the core field for the decoder to identify the "vector rearrangement multiplication" function.

[0063] Execution control field vm (bit

[25] ): Vector mask control bit. When vm=0, instruction execution is controlled by the corresponding bit in the vector mask register v0, and only the channel with the mask "1" is operated on; when vm=1, the mask function is disabled, and all vector channels participate in the calculation.

[0064] The core operation of this vector rearrangement multiplication instruction is index selection multiplication. First, the hardware dynamically selects and reorders data elements from vs2 according to the pattern specified by the index vector in vs1, forming a dense temporary vector. Then, it performs element-wise multiplication of this temporary vector with the original data vector in vd, and writes the result back to the vd register. This process is completed within a single instruction, achieving a hardware-level transformation from irregular sparse data to regularized dense computation.

[0065] Figure 3The illustration shows a schematic diagram of a hardware microarchitecture implementation for executing the custom instruction vreordermul.vv according to an embodiment of the present disclosure. This architecture maps the semantics of the instruction to a dedicated hardware data path, achieving end-to-end hardware acceleration from sparse data decoding and dynamic rearrangement to parallel multiplication and accumulation.

[0066] like Figure 3 As shown, the sparse weight matrix is ​​input in a predefined N:M structured format. For example, a hardware decoder can parse this data stream in real time according to the set N:M pattern, thereby performing the following operations: extracting all non-zero weights of each block of the weight matrix and reorganizing the extracted non-zero weights into a continuous dense data stream to store in a ping-pong weight buffer; generating local indices of the non-zero weights within their respective blocks to store in an index buffer; and generating shift signals to control the sliding of the feature vector window according to the block processing order (e.g., shifting left by eight bits per calculation cycle). During instruction execution, the source vector register vs1 provides the index vector, the contents of which are mapped to the index buffer. Another source vector register vs2 provides the currently processed feature vector segment. The destination register vd is used to provide the currently processed dense weight vector. Based on the index values ​​in vs1, feature elements that precisely correspond to the non-zero weights stored in vd can be selected in parallel from the data in vs2, thereby realizing the "data rearrangement" operation described in the embodiment. The rearrangement multiplication unit receives the regularized dense weight stream from the ping-pong weight buffer and the rearranged feature vector data stream. This unit performs multi-path parallel multiplication and accumulation operations within one clock cycle, directly corresponding to the "reordermul.vv" semantics of the instruction. Subsequently, the vector reduction and summation unit further accumulates and integrates the intermediate results, finally writing the output back to the instruction's target register vd. This hardware architecture integrates sparse computation tasks that traditionally require multiple instructions and complex software logic into a single, deeply pipelining hardware operation triggered by a single instruction.

[0067] In embodiments of this disclosure, the number of non-zero weights processed in parallel within a vector execution unit can be less than or equal to the maximum number of parallel multiplication operations supported by the vector execution unit in one instruction cycle. It should be understood that the above configuration is intended to achieve peak hardware utilization, but in practical applications, based on specific performance and power consumption tradeoffs, a single processing run can also be configured to occupy fewer computing units than the maximum available number, for example, when processing boundary blocks or performing fine-grained power management.

[0068] In the embodiments of this disclosure, the number of non-zero weights processed in parallel within the vector execution unit is configured based on the processor's vector register length (VLEN), operational data width, and N:M sparse mode. The execution granularity of step S208 achieves optimal matching with hardware resources. In one embodiment, when VLEN is 256 bits and the operational data width is 8 bits, a single instruction can perform 32 multiplications in parallel. For the 1:8 sparse mode (8 non-zero weights per block), the hardware processes 4 weight matrix blocks in parallel per cycle to fill all 32 computation units; for the 2:8 mode (16 non-zero weights per block), 2 matrix blocks are processed in parallel; for the 4:8 mode (32 non-zero weights per block), processing 1 matrix block is sufficient to fully utilize computational resources. This design ensures flexible adaptation to different sparsity rates and always transforms sparse computation into hardware-friendly regular vector operations, thereby significantly reducing power consumption caused by irregular memory accesses while improving instruction throughput.

[0069] At step S210, the output vector is obtained by accumulating all parts and vectors associated with the same output row of the output vector of length D.

[0070] Since a single weight matrix block contributes only a portion of the result for a row in the final output vector, it is necessary to traverse all column matrix blocks (Q in total) corresponding to that row and accumulate the partial sums and vectors generated in each calculation iteration. This accumulation process is typically performed cyclically in an accumulator register (or accumulator file) of the processor. At the software level, this manifests as a loop, the loop body of which executes the single calculation iteration described in step S208 (e.g., by executing the vreordermul.vv instruction). Through this loop, the weights and feature data corresponding to all column blocks of that row are processed sequentially, and the intermediate results are continuously accumulated until the complete result of that output row is obtained. Then, the system moves to the next row and repeats this process until all P output rows are calculated, forming the final output vector Y.

[0071] To clearly illustrate the implementation of this disclosure, a typical scenario will be described below. Assuming a RISC-V vector extension architecture is used, the vector register length VLEN = 256 bits. When using 8-bit integer or floating-point data types, a single vector instruction can process 32 independent data elements simultaneously. Those skilled in the art should understand that the specific value of VLEN and the data type bit width can be adjusted according to the actual hardware configuration and application requirements without affecting the universality of this disclosure.

[0072] In the subsequent description of the embodiments, specific settings will be made with the weight matrix dimensions D = K = 1024, the vector processing granularity V = 8, and the sparse mode parameter M = 8. For three sparse configurations of N = 1, 2, and 4 respectively, the data storage format, index organization method, and instruction execution process will be elaborated in detail. It should be noted that the above values of N are only examples, and the solutions of the present disclosure are equally applicable to other sparse configurations where M is other values (such as 4, 16, etc.) and N < M. Its hardware execution process and data organization method can be achieved by adaptively adjusting the index structure and instruction parameters, and such variations should all fall within the protection scope of the present disclosure.

[0073] Embodiment 1: Sparse matrix-vector multiplication with a sparsity of 1:8

[0074] Figure 4 The figure shows a flowchart of a method for accelerating sparse matrix-vector multiplication with a sparsity of 1:8 (N:M = 1:8) according to an embodiment of the present disclosure. Taking the sparse matrix-vector multiplication (Y = W*X) of a weight matrix W with a sparsity of 1:8 and a dense feature vector X as an example, as described above, where the dimension of the sparse weight matrix W is D×K (for example, 1024×1024); the length of the dense feature vector X is K (for example, 1024); and the length of the output vector Y is D (for example, 1024). The process of this sparse matrix-vector multiplication may include the following steps:

[0075] At step S410, the sparse weight matrix W can be divided into P×Q (for example, P = 128, Q = 128) matrix blocks of V*V (for example, 8*8), and each matrix block can be represented as W ij (where 0 ≤ i < P, 0 ≤ j < Q), and the feature vector X with a length of K is correspondingly divided into Q feature vector segments X j . Figure 5A The figure shows an exemplary schematic diagram of the P×Q matrix blocks into which the sparse weight matrix W is divided. As Figure 5A shown, the sparse weight matrix W is divided into matrix blocks W 00 , W 01 , W 02 , … W (p-1)(q-1) . Figure 5B The figure shows an exemplary schematic diagram of the weight distribution of a sparse weight matrix block (for example, W 00 ). Among them, the white square area represents zero-valued elements, and the gray squares represent non-zero-valued elements. As Figure 5B shown, this matrix block has a sparsity of 1:8, that is, each row includes 1 non-zero value and 7 zero values. Those skilled in the art can understand that the above division method and sparsity are only examples, and the solutions of the present disclosure are equally applicable to other block sizes and sparse configurations.

[0076] At step S420, for each weight matrix block W ij Generate dense weight vector W j And the corresponding index vector, the dense weight vector W j Only the weight matrix block W is included. ij The non-zero elements, and the index vector is used to indicate the weight vector matrix block W. ij Each non-zero element in the array is located in the column position of its respective row.

[0077] Specifically, the dense weight vector W j It is generated through the following operation: each weight matrix block W ij Non-zero elements in the matrix are extracted in row-major order and stored contiguously in memory (e.g., SRAM) to form a dense weight vector W. j . Figure 5C The illustration shows an exemplary diagram of generating dense weight vectors from sparse weight matrix blocks. Figure 5C As shown, for the weight matrix block W ij The generated dense weight vector W j All elements in W are non-zero, and W j The elements in the matrix correspond to the weight matrix block W in sequence. ij The non-zero weight of each row in the text, for example, W j The first element in is W ij The non-zero values ​​in the first row, W j The second element in is W ij The non-zero values ​​in the second row, and so on. At the level of the entire sparse weight matrix W, its partitioned weight matrix blocks are also sequentially extracted for non-zero elements in row-major order and stored contiguously in memory. For example, W 00 W 01 W 02 and W 03 Each dense weight vector will be stored contiguously in, for example, SRAM.

[0078] Figure 5D The illustration shows an exemplary schematic diagram of an index structure for a 1:8 sparse pattern according to an embodiment of the present disclosure. Figure 5D As shown, this index structure can be represented in hardware as multiple parallel index vectors, such as the four index vectors shown in this embodiment: Idx[0], Idx[1], Idx[2], and Idx[3]. Idx[0], Idx[1], Idx[2], and Idx[3] are used to record four matrix blocks (e.g., W) stored consecutively in memory. 00 W 01 W 02 and W 03Each index vector has its own non-zero weight position information. Each index vector is associated with a specific dense weight vector W. j Strict alignment, and used to indicate each non-zero weight value in the data vector within its original matrix block W. ij The column offset position within the row where it is located, for example, Idx[0] is for matrix block W. 00 The index vector, Idx[1] is for matrix block W 01 The index vector, Idx[2], is for matrix block W 02 The index vector, and Idx[3] is for the matrix block W 03 The index vector. Specifically, each element in the index vector Idx is encoded with a fixed bit width (e.g., 3 bits) to represent the column offset (0 to 7) of its corresponding non-zero weight in the row of its original matrix block. For the 1:8 sparse mode, each element in each index vector (e.g., Idx[0], Idx[1], Idx[2], or Idx[3]) is encoded with 3 bits to uniquely represent the column position (0 to 7) of its corresponding non-zero weight in its row. Therefore, for a complete 8×8 weight matrix block, the total length of its corresponding index vector is 8 elements × 3 bits / element = 24 bits. In other words, 24 bits are sufficient to record the precise position of all 8 non-zero weights in the row of an 8×8 weight matrix block.

[0079] For example, targeting Figure 5B The example matrix block of the 1:8 sparse pattern shown has an index vector Idx[0] containing 8 elements, each indicating the specific column position of the unique non-zero element in the corresponding row of the matrix block. Specifically, the value of the k-th element (k=0,1,…,7) of Idx[0] represents the column offset of the unique non-zero weight in the k-th row of the matrix block from the 8 possible positions (numbered 0 to 7) in that row. For example:

[0080] - Idx[0][0] = 0 indicates that the non-zero weight of the 0th row is located in the 0th column of that row;

[0081] - Idx[0][1] = 1 indicates that the non-zero weight of the first row is located in the first column of that row;

[0082] - Idx[0][2] = 7 indicates that the non-zero weight of the second row is located in the 7th column of that row;

[0083] -Idx[0][3] = 5 indicates that the non-zero weight of the 3rd row is located in the 5th column of that row;

[0084] - Idx[0][4] = 7 indicates that the non-zero weight of the 4th row is located in the 7th column of that row;

[0085] - Idx[0][5] = 2 indicates that the non-zero weight of the 5th row is located in the 2nd column of that row;

[0086] -Idx[0][6] = 2 indicates that the non-zero weight of the 6th row is located in the 2nd column of that row;

[0087] - Idx[0][7] = 3 indicates that the non-zero weight of the 7th row is located in the 3rd column of that row.

[0088] This index structure strictly corresponds to the non-zero weighted data vectors extracted from the same matrix block and stored consecutively, together forming the basis for subsequent efficient data rearrangement and calculation.

[0089] Under a specific hardware configuration, this embodiment employs a design of four index vectors (e.g., Idx[0] to Idx[3]) to achieve peak utilization of computing resources. Specifically, when the vector register length (VLEN) is 256 bits and the data width is 8 bits, a single vector instruction can perform up to 32 multiplications in parallel. For a scenario with a sparsity of 1:8 (i.e., each 8×8 matrix block contains 8 non-zero weights), the optimal strategy to fill all 32 computing units in one instruction cycle is to process 4 matrix blocks (a total of 32 non-zero weights) simultaneously. Therefore, four independent index vectors are needed to record the column positions of the non-zero weights in these 4 matrix blocks, thereby supporting the parallel rearrangement and multiplication of all data in a single hardware cycle. Those skilled in the art should understand that the number of index vectors is a configurable parameter designed in conjunction with the VLEN, data width, and sparsity mode (N:M) of the target platform. The "four" in this embodiment is merely an optimized example under the specific configuration.

[0090] In step S430, for the four consecutive weight matrix blocks currently being processed, based on the index vector generated for each matrix block, the corresponding matrix block W is used to determine the weights. ij The corresponding feature vector segment X j The selected data elements are rearranged to generate four weighted blocks W. ij Aligned four rearranged feature vector segments X j Specifically, based on the column offset information provided in the index vector, it is possible to determine the column offset from the feature vector segment X. j The corresponding data elements are selected in parallel and reordered according to the index-specified order to generate rearranged feature vector segments X. j Rearrange the feature vector segment X j The data order is the same as the densely stored weight vector W generated in step S420. j The non-zero weights in the algorithm are strictly aligned in order, thus ensuring that each weight is paired with the correct feature value in subsequent steps.

[0091] In one example, the specific gating logic of the rearrangement operation can be illustrated by the following pseudocode example:

[0092] / / For matrix block W 00 Using its corresponding index vector Idx[0], eight elements are selected from the feature vector segment X0 of the feature vector X to form a matrix block W. 00 The rearranged feature vector segment X0' is obtained by multiplying the dense weight vector W0;

[0093] X0'= {X[Idx[0][0]], X[Idx[0][1]], X[Idx[0][2]], X[Idx[0][3]], X[Idx[0][4]],

[0094] / / The feature vector window is logically shifted 8 elements to the left to move to the next feature vector segment X1;

[0095] X = X << 8;

[0096] / / For matrix block W 01 Using its corresponding index vector Idx[1], 8 elements are selected from the feature vector segment X1 of the feature vector X to form a rearranged feature vector segment X1r for multiplication with the dense weight vector W1 of the matrix block W01;

[0097] X1'= {X[Idx[1][0]], X[Idx[1][1]], X[Idx[1][2]], X[Idx[1][3]], X[Idx[1][4]],

[0098] X = X << 8;

[0099] / / Repeat the above process, using Idx[2] and Idx[3] to generate the data for W. 02 and W 03 The rearranged feature vector segments X2' and X3' are used for multiplication;

[0100] X2'={X[idx[2][0]], X[idx[2][1]], X[idx[2][2]], X[idx[2][3]], X[idx[2][4]],

[0101] X = X << 8;

[0102] X3'={ X[idx[3][0]], X[idx[3][1]], X[idx[3][2]],

[0103] X = X << 8;

[0104] …

[0105] Based on the above rearrangement operation, the complete row calculation process can be unfolded. Taking the calculation of one sub-vector segment Y[0:7] of the output vector Y as an example:

[0106] For the first weight matrix block W in the first row 00 The calculation: by executing the vreordermul.vv instruction, according to W 00 The index is used to rearrange the corresponding feature vector segment X0 into X0', thus completing W. 00 The dense weight vector is multiplied and accumulated with the vectorized form of X0' to obtain the partial sum vector T[0]. 00 ;

[0107] Iterative calculation of subsequent weight matrix blocks in the first row: the eigenvector window moves, processing W sequentially. 01 W 02 And so on, obtaining partial sums and T[0] respectively. 01 T[0] 02 wait;

[0108] Traverse all Q weight matrix blocks in the first row, repeating the above "rearrangement-calculation-shifting" process to obtain the partial sum vector T[0]. 00 , T[0] 01 ,,…,T[0] 0(Q-1) .

[0109] In step S440, the final output is obtained by summing the partial sum vectors from step S430. Specifically, the partial sum vectors generated by all column blocks in the first row are summed: Y[0:7] = T[0] 00 + T[0] 01 + … + T[0] 0Q-1 This allows us to obtain the first sub-vector segment Y0 of the output vector Y.

[0110] The calculation of the entire sparse weight matrix W is achieved by repeating the process from S430 to S440 row by row (i=0 to P-1). The calculation of each row independently produces an output sub-vector segment Y. i Once all rows have been calculated, the complete output vector Y is obtained, as shown in step S450. In the 1:8 sparse mode of this example, since D=K=1024 and V=8, P=Q=128. Specifically, calculating a single row Y... i It requires traversing its 128 column blocks, while the hardware can process 4 column blocks in parallel per iteration. Therefore, calculating a single row only requires 32 inner iterations. In summary, completing the entire matrix-vector multiplication Y=W*X requires executing P*32 = 128 * 32 = 4096 core instructions.

[0111] The above operations can be efficiently implemented in hardware using the vreordermul.vv instruction defined in this disclosure. This allows for the simultaneous processing of four consecutive 8×8 matrix blocks (e.g., W) within a single instruction cycle. 00 W 01 W 02 W 03 The hardware will perform the following register configuration and instruction operations:

[0112] Register loading:

[0113] Load the dense weight data corresponding to these four matrix blocks, which have been continuously packed and stored, into the target vector register vd, i.e., vd = {W 00 W 01 W 02 W 03};

[0114] Load the current four consecutive feature vector segments (X(0:7),X(8:15),X(16:23),X(24:31)) into the source vector register vs2, i.e. vs2 = {X(0:7), X(8:15), X(16:23), X(24:31)}.

[0115] Load the index vectors (idx0, idx1, idx2, idx3) corresponding to these four matrix blocks into the source vector register vs1, i.e., vs1 = {idx0, idx1, idx2, idx3}.

[0116] Command execution:

[0117] Execute a single vector instruction: vreordermul.vv vd, vs2, vs1.

[0118] This instruction atomically performs the following operations within the hardware: First, based on the four index vectors in vs1, it dynamically selects and rearranges the corresponding data from the four feature vector segments in vs2 in parallel; then, it performs parallel multiplication and summation of the four rearranged data sets with the corresponding four weights in vd, and writes the results back to vd. This process completes all effective calculations for four matrix blocks (a total of 32 non-zero weights) within one instruction cycle, achieving peak utilization of computing resources.

[0119] Example 2: Sparse matrix-vector multiplication with a sparsity ratio of 2:8

[0120] Figure 6 The diagram illustrates a flowchart of an accelerated sparse matrix-vector multiplication method with a sparsity of 2:8 (N:M=2:8) according to an embodiment of the present disclosure. The method takes the multiplication of a weight matrix W with a sparsity of 2:8 and a dense feature vector X as an example, where the matrix dimension, block parameters (D=K=1024, V=8, P=Q=128), and overall objective (Y=W*X) are the same as in Embodiment 1. The sparse matrix-vector multiplication process may include the following steps:

[0121] In step S610, the sparse weight matrix W is divided into P×Q V×V (8×8) matrix blocks Wij, and the feature vector X is correspondingly divided into Q feature vector segments Xj of length V, in the same way as in Example 1. Figure 7A The diagram illustrates an exemplary schematic of P×Q matrix blocks partitioned from a sparse weight matrix W. Figure 7A As shown, the sparse weight matrix W is divided into matrix blocks W 00 W 01 W 02 ...W (P-1)(Q-1) . Figure 7B The diagram illustrates an exemplary distribution of weights for each sparse weight matrix block in a 2:8 sparsity mode, where white squares represent zero-value elements and gray squares represent non-zero-value elements. Figure 7B As shown, the matrix block has a sparsity of 2:8, meaning that each row includes 2 non-zero values ​​and 6 zero values.

[0122] In step S620, for each weight matrix block W ij Generate dense weight vector W j And the corresponding index structure. Unlike Example 1, in the 2:8 mode, each 8×8 matrix block contains 16 non-zero weights, therefore W j It is a vector containing all 16 non-zero elements in the block. This index structure is used to indicate the column offset of each non-zero element in the block within its respective row.

[0123] Specifically, the dense weight vector Wj Generate by the following operation: Transform matrix block W ij The 16 non-zero elements are extracted in row-major order (i.e., the two non-zero weights of row 0, row 1, ..., row 7 are stored sequentially) and stored contiguously in memory (e.g., SRAM), thus forming a dense weight vector W of length 16. j . Figure 7C The illustration shows an exemplary diagram of the dense weight vector generated from a 2:8 sparse weight matrix block. At the level of the entire sparse weight matrix W, the weight matrix blocks are also sequentially extracted with non-zero elements in row-major order and stored contiguously in memory. For example, W... 00 W 01 W 02 and W 03 Each dense weight vector will be stored contiguously in, for example, SRAM.

[0124] Figure 7D The illustration shows an exemplary schematic diagram of an index structure for a 2:8 sparse mode according to an embodiment of the present disclosure. The index structure is represented in hardware as multiple parallel index vectors, such as the two index vectors shown in this embodiment: Idx[0] and Idx[1]. Each index vector includes 16 elements, with the 8 values ​​in the first row corresponding to the column offsets of the two non-zero elements in each row of the matrix block from row 0 to row 3, and the 8 numbers in the second row corresponding to the column offsets of the two non-zero elements in each row of the matrix block from row 4 to row 7.

[0125] Idx[0] and Idx[1] are used to record the non-zero weight position information of two matrix blocks (e.g., W00 and W01) that are stored consecutively in memory, so as to realize the peak calculation of the two blocks (32 non-zero elements in total) in one cycle.

[0126] For example, targeting Figure 7B The example matrix block W shown is a 2:8 sparse pattern. 00 , Figure 7D The corresponding index vector Idx[0] is shown, containing 16 elements indicating the specific column positions of the non-zero elements in each row of the matrix block. Specifically, for Figure 7B Example matrix block W 00 , Figure 7D The elements in Idx[0] shown represent:

[0127] - Idx[0][0] = 0 indicates that the first non-zero weight in row 0 is located in column 0 of that row;

[0128] - Idx[0][1] = 1 indicates that the second non-zero weight of row 0 is located in column 1 of that row;

[0129] - Idx[0][2] = 3 indicates that the first non-zero weight of the first row is located in the third column of that row;

[0130] - Idx[0][3] = 7 indicates that the second non-zero weight of the first row is located in the 7th column of that row;

[0131] - Idx[0][4] = 4 indicates that the first non-zero weight in the second row is located in the fourth column of that row;

[0132] - Idx[0][5] =6 indicates that the second non-zero weight in the second row is located in the 6th column of that row;

[0133] - Idx[0][6] = 2 indicates that the first non-zero weight in the 3rd row is located in the 2nd column of that row;

[0134] - Idx[0][7] = 7 indicates that the second non-zero weight of the 3rd row is located in the 7th column of that row;

[0135] -Idx[0][8] = 1 indicates that the first non-zero weight in the 4th row is located in the 1st column of that row;

[0136] - Idx[0][9] = 5 indicates that the second non-zero weight in the 4th row is located in the 5th column of that row;

[0137] - Idx[0]

[10] = 3 indicates that the first non-zero weight in the 5th row is located in the 3rd column of that row;

[0138] - Idx[0]

[11] = 4 indicates that the second non-zero weight in the 5th row is located in the 4th column of that row;

[0139] - Idx[0]

[12] = 2 indicates that the first non-zero weight in the 6th row is located in the 3rd column of that row;

[0140] - Idx[0]

[13] =6 indicates that the second non-zero weight in the 6th row is located in the 6th column of that row;

[0141] - Idx[0]

[14] = 0 indicates that the first non-zero weight in the 7th row is located in the 0th column of that row;

[0142] - Idx[0]

[15] = 5 indicates that the second non-zero weight of the 7th row is located in the 5th column of that row.

[0143] Under a specific hardware configuration, this embodiment employs a design with two index vectors (e.g., Idx[0] and Idx[1]) to achieve peak utilization of computing resources. Specifically, when the vector register length (VLEN) is 256 bits and the data width is 8 bits, a single vector instruction can perform up to 32 multiplications in parallel. For a scenario with a sparsity of 2:8 (i.e., each 8×8 matrix block contains 16 non-zero weights), the optimal strategy to fill all 32 computing units within one instruction cycle is to process two matrix blocks simultaneously (2 blocks × 16 non-zero weights / block = 32 calculations). Those skilled in the art should understand that the number of index vectors is a configurable parameter designed in conjunction with the VLEN, data width, and sparsity mode (N:M) of the target platform. The "two" in this embodiment is merely an optimized example under the specific configuration.

[0144] At step S630, for the two weight matrix blocks currently being processed (e.g., W) ij W i(j+1) Based on the index vector generated for each matrix block, from the corresponding feature vector segment (X) j ,X j+1 Data elements are selected from the two weight blocks and rearranged to generate four rearranged feature vector segments (X) aligned with them. j ', X j+1 ').

[0145] Specifically, the hardware uses the first 8 elements of the index vector Idx[0] (indicating the matrix block W) 00 X0'[0] is generated by gating elements from the feature vector segment X0 based on the non-zero weight column positions of rows 0-3 of the same index vector Idx[0] (indicating the non-zero weight column positions of rows 4-7 of matrix block W00). Similarly, X0'[1] is generated by gating elements from X0 based on the last 8 elements of the same index vector Idx[1] (indicating the non-zero weight column positions of rows 4-7 of matrix block W00). 01 X1'[0] is generated by selecting elements from the feature vector segment X1 based on the non-zero weight column positions of rows 0-3; at the same time, X1' is generated based on the last 8 elements of the same index vector Idx[1] (indicating matrix block W). 01 The non-zero weight column positions of rows 4-7 are used to select elements from X1 to generate X1'[1]. This operation can be illustrated by the following pseudocode logic example:

[0146] / / For matrix block W 00 Based on Idx[0], the feature vector segment X0 is rearranged to generate X0'[0] and X0'[1];

[0147] X0'[0] = {X[Idx[0][0]], X[Idx[0][1]], X[Idx[0][2]], X[Idx[0][3]], X[Idx[0][4]],

[0148] X0'[1] = {X[Idx[0][8]], X[Idx[0][9]], X[Idx[0]

[10] ], X[Idx[0]

[11] ], X[Idx[0]

[15] ]};

[0149] / / The feature vector window is logically shifted 8 elements to the left to move to the next feature vector segment X1;

[0150] X = X << 8;

[0151] / / For matrix block W 01 Based on Idx[1], the feature vector segment X1 is rearranged to generate X1'[0] and X1'[1];

[0152] X1'[0]= {X[Idx[1][0]], X[Idx[1][1]], X[Idx[1][2]], X[Idx[1][3]],

[0153] X1'[1] = {X[Idx[1][8]], X[Idx[1][9]], X[Idx[1]

[10] ], X[Idx[1]

[11] ], X[Idx[1]

[15] ]};

[0154] X = X << 8;

[0155] Matrix block W 00 The first set of non-zero weights (the first 8 elements W of its dense vector) 00 [0]) is vectorized and multiplied and accumulated with X0'[0] to obtain a partial sum vector T[0]00, and matrix block W 01 The first set of non-zero weights (the first 8 elements W of its dense vector) 01[0]) is vectorized and multiplied and summed with X1'[0] to obtain a partial sum vector T[0]01. The partial sum vectors T[0]00 and T[0]01 will contribute to the first sub-vector segment (such as Y[0:3]) in the final output vector. Matrix block W 00 The second set of non-zero weights (the last 8 elements W of the dense vector) 00 [1]) is vectorized and multiplied with X0'[1] to obtain a partial sum vector T[1]00, and matrix block W 01 The second set of non-zero weights (the last 8 elements W of its dense vector) 01 [1]) is vectorized and multiplied with X1'[1] to obtain a partial sum vector T[1]01. The partial sum vectors T[1]00 and T[1]01 will contribute to the second sub-vector segment (such as Y[4:7]) in the final output vector.

[0156] At step S640, the final output is obtained by accumulating the partial sums of the vector. For each 4-element subarray of the output vector, the final output is obtained by accumulating all relevant partial sums. For example:

[0157] Y[0:3] = T[0]=T[0]00 + T[0]01 + … + T[0]0(Q-1) (such as Figure 5A (The sum of the products of the first eight non-zero values ​​of each matrix block in the first row of W and their corresponding eigenvalues)

[0158] Y[4:7] = T[1]= T[1]00 + T[1]01 + … + T[1]0(Q-1) (The sum of the last eight non-zero values ​​of each matrix block in the first row multiplied by their corresponding eigenvalues)

[0159] Y[8:11]=T[2]= T[2]00 + T[2]01 + … + T[2]0(Q-1) (The sum of the products of the first eight non-zero values ​​of each matrix block in the second row and their corresponding eigenvalues)

[0160] Y[12:15]=T[3]= T[3]00 + T[3]01 + … + T[3]0(Q-1) (The sum of the last eight non-zero values ​​of each matrix block in the second row multiplied by their corresponding eigenvalues)

[0161] And so on.

[0162] The calculation of the entire sparse weight matrix W is to repeat the process from S630 to S640 row by row (i = 0 to P-1). At S650, the complete output vector Y is obtained after all rows have been calculated.

[0163] With a VLEN=256-bit configuration and a data width of 8 bits, a single vector instruction can perform 32 multiplications in parallel. A 2:8 sparse block provides 16 valid computations, so the optimal strategy per cycle is to process two such matrix blocks simultaneously (32 computations in total), corresponding to the use of two index vectors (such as Idx[0] and Idx[1]). Those skilled in the art should understand that the organization of index vectors and the parallel processing strategy are configurable parameters designed in conjunction with the sparse mode (N:M) and hardware resources. In this example, since D=K=1024 and V=8, P=Q=128. Calculating a pair of output rows (Y[i], Y[i+1]) requires traversing all 128 column blocks, processing two column blocks in each iteration, so calculating a pair of output rows requires 64 inner iterations. Since the total number of rows P=128, completing the entire matrix-vector multiplication Y=W*X requires executing 64 * 128 = 8192 core instructions.

[0164] The above operations can be efficiently implemented in hardware using the vreordermul.vv instruction defined in this disclosure. This allows for the simultaneous processing of two consecutive 8×8 matrix blocks (e.g., W) within a single instruction cycle. 00 and W 01 The hardware can perform the following register configurations and instruction operations:

[0165] Register loading:

[0166] Load the dense weight data corresponding to these two matrix blocks, which have been continuously packed and stored, into the target vector register vd, i.e., vd = {W 00 W 01};

[0167] Load the current two consecutive feature vector segments (X(0:7), X(0:7), X(8:15), X(8:15)) into the source vector register vs2, i.e. vs2 = {X(0:7), X(0:7), X(8:15), X(8:15)}.

[0168] Load the index vectors (idx0, idx1) corresponding to these two matrix blocks into the source vector register vs1, i.e., vs1 = {idx0, idx1}.

[0169] Command execution:

[0170] Execute a single vector instruction: vreordermul.vv vd, vs2, vs1.

[0171] This instruction atomically performs the following operations within the hardware: First, based on the two index vectors in vs1, it dynamically selects and rearranges the corresponding data from the two feature vector segments in vs2 in parallel; then, it performs parallel multiplication and summation of the rearranged feature vector data with the corresponding weight vector in vd, and writes the result back to vd. This process completes all effective calculations for two matrix blocks (a total of 32 non-zero weights) within one instruction cycle, achieving peak utilization of computing resources.

[0172] Example 3: Sparse matrix-vector multiplication with a sparsity ratio of 4:8

[0173] This embodiment takes a sparsity of 4:8 (i.e., N:M=4:8) as an example. The matrix dimension, block parameters (D=K=1024, V=8, P=Q=128) and overall objective (Y=W*X) are the same as those in the previous embodiment. The main difference is that the sparsity density is higher, so there are some adjustments in data organization, index structure and instruction parallelism strategy.

[0174] First, similar to Examples 1 and 2, the sparse weight matrix W is divided into P×Q V×V (8×8) matrix blocks W. ij Furthermore, the feature vector X is correspondingly divided into Q feature vector segments X of length V. j . Figure 8A The illustration shows an exemplary diagram of P×Q matrix blocks divided by a sparse weight matrix W.

[0175] Figure 8B The diagram illustrates an exemplary weight distribution for each sparse weight matrix block in a 4:8 sparse pattern, where blank areas represent zero-value elements and shaded areas represent non-zero-value elements. Figure 8B As shown, this matrix block has a 4:8 sparsity, meaning each row includes 4 non-zero values ​​and 4 zero values. Therefore, each 8×8 matrix block contains a total of 32 non-zero weights. Correspondingly, for each weight matrix block W... ij Generate dense weight vector W j And the corresponding index structure. In 4:8 mode, each 8×8 matrix block contains 32 non-zero weights, therefore W j It is a vector containing all 32 non-zero elements in the matrix block. This index structure is used to indicate the column offset of each non-zero element in the block within its respective row.

[0176] Specifically, the dense weight vector W j Generate by the following operation: Transform matrix block W ijThe 32 non-zero elements in the data are extracted and stored consecutively in memory (e.g., SRAM) in row-major order (i.e., storing the four non-zero weights of row 0, row 1, ..., row 7 in sequence), thus forming a dense weight vector W of length 32. j . Figure 8C The illustration shows an exemplary diagram of the dense weight vector generated for a 4:8 sparse weight matrix block.

[0177] Figure 8D An exemplary schematic diagram of an index structure for a 4:8 sparse pattern according to an embodiment of the present disclosure is shown. The index structure is represented in hardware as an index vector Idx[0] of length 32. This vector stores the column offsets of all non-zero elements consecutively in row-major order; that is, the first four elements indicate the column positions of the four non-zero weights in row 0, the next four elements indicate the column positions of the four non-zero weights in row 1, and so on. Each column offset is encoded using 3 bits (0-7). Therefore, the index data for a matrix block requires a total of 32 × 3 bits = 96 bits.

[0178] With a typical configuration of a 256-bit vector register length (VLEN) and an 8-bit data width, 32 multiplications can be performed in a single cycle. An 8×8 block of 4:8 contains exactly 32 non-zero weights. Therefore, the optimal hardware strategy is to process a complete 4:8 sparse matrix block in one instruction cycle, thereby achieving 100% utilization of the computing unit.

[0179] When the hardware performs calculations, it targets the weight matrix block W being processed. ij Based on its corresponding index vector, from the corresponding feature vector segment X j The selected feature elements are rearranged to generate a weighted block W. ij Aligned rearranged feature vector segments X j '.

[0180] Specifically, the hardware uses all 32 values ​​of the index vector Idx[0] to generate the feature vector segment X in parallel. j The dynamic selection process identifies 32 corresponding eigenvalues, forming a rearranged eigenvector segment X. j The rearrangement operation is integrated with the multiply-accumulate calculation at the hardware level and completed in one instruction. The 32 products calculated in each block are grouped by row (every 4 products correspond to one row of the original matrix), and the sums within each group are accumulated to obtain 8 partial sums, which are then added to the corresponding positions in the output vector Y.

[0181] The calculation of the entire sparse weight matrix W is to repeat the above block processing row by row (i = 0 to P-1) until all P = 128 rows are completed, and then the final output Y is obtained.

[0182] The above operations can be efficiently implemented in hardware using the vreordermul.vv instruction defined in this disclosure. This allows processing a 4:8 matrix block (e.g., W) within one instruction cycle. 00 The hardware can perform the following register configurations and instruction operations:

[0183] Register loading:

[0184] matrix block W 00 The dense weight vector (32 weights) is loaded into the target vector register vd, i.e., vd = {W 00}

[0185] Load the current feature vector segment X(0:7) into the source vector register vs2, i.e., vs2={X(0:7), X(0:7), X(0:7), X(0:7)}.

[0186] matrix block W 00 The index vector IDX is loaded into the source vector register vs1, i.e., vs1 = {Idx0}.

[0187] Command execution:

[0188] Execute a single vector instruction: vreordermul.vv vd, vs2, vs1.

[0189] This instruction atomically performs the following operations within the hardware: First, based on the index vector in vs1, 32 eigenvalues ​​are selected in parallel from the feature segments of vs2; then, these eigenvalues ​​are multiplied in parallel with the 32 weights in vd, and the products of every four belonging to the same row are summed to generate eight partial sums, which are accumulated at the corresponding output positions in vd. This process completes all 32 effective calculations for a 4:8 matrix block within one instruction cycle, achieving full utilization of computational resources.

[0190] In this example, since D=K=1024 and V=8, P=Q=128. Calculate an output line Y. i It requires traversing all 128 column blocks. Since one column block needs to be processed in each iteration (in the case of 4:8 sparsity), computing a single row requires 128 inner iterations. Because the total number of rows P = 128, completing the entire matrix-vector multiplication Y = W * X requires P * Q = 128 * 128 = 16384 core instructions. Although the total number of instructions may be higher than in sparser models, the computational density (100% effective computation) and regularity of each iteration reach their peak, and the overall energy efficiency and throughput are still significantly better than traditional unstructured sparse implementations.

[0191] Examples 1 to 3 described in detail the acceleration method for element-level structured sparsity. Its core feature is that the sparsity constraint acts on the most basic scalar element granularity, that is, in each consecutive M scalar weights, only N non-zero values ​​are forced. This fine-grained normalization provides the basis for hardware implementation of zero-overhead dynamic data rearrangement.

[0192] It is worth emphasizing that the "dense storage-compact index-dynamic rearrangement" acceleration paradigm proposed in this disclosure has universality. It not only supports the most flexible element-level sparsity, but its hardware architecture and instruction set can also naturally support coarser-grained structured sparse patterns such as vector-wise and block-wise with minimal adaptation. These coarse-grained patterns can be regarded as strongly constrained subsets or special cases of specific N:M element-level sparse patterns. While maintaining full utilization of computing units, they can further simplify the index structure, thereby meeting the trade-off requirements of different models between compression ratio and hardware efficiency. The vector-wise and block-wise patterns, two typical coarse-grained sparse patterns, will be described in detail in Examples 4 and 5, respectively. Both patterns can be regarded as specific implementations of the general 4:8 sparse pattern described in Example 3 after applying specific structured constraints: the vector-wise sparse pattern constrains that non-zero elements in each row must be continuously clustered in the first or second half; the block-wise sparse pattern further relaxes the constraint granularity to the row group level. Although the granularity of the sparsity constraints differs, both maintain consistency with 4:8 element-level sparsity in terms of computational density (32 non-zero weights per block) and hardware execution pipeline. The core difference lies in the extreme simplification of the index structure.

[0193] Example 4: Matrix-vector multiplication based on vector-level structured sparse patterns

[0194] The vector-level structured sparse pattern described in this embodiment is a special case of 4:8 sparse with the property of "half-row clustering". In each 8×8 weight matrix block, the four non-zero elements of each row are forced to be continuously distributed in the first four or last four columns of the row.

[0195] Figure 9 An exemplary schematic diagram of a vector-level structured sparse matrix according to an embodiment of the present disclosure is shown. Specifically, for four consecutive 8×8 weight matrix blocks (W... 00 W 01 W 10 W 11 Its sparse pattern is directly encoded as four compact index vectors:

[0196] Idx00:[0, 1, 0, 0, 0, 1, 1, 0]

[0197] Idx01:[1, 0, 0, 1, 1, 1, 0, 0]

[0198] Idx10:[0, 1, 1, 0, 1, 0, 1, 0]

[0199] Idx11:[0, 1, 1, 0, 0, 1, 0, 1]

[0200] In this mode, each weight matrix block still contains 32 non-zero weights, and its dense weight vector is organized in exactly the same way as in Example 3. The key difference lies in the simplification of the index: each element in the index vector only requires 1 bit (0 or 1) to indicate the "half" where the non-zero element of the corresponding row is located. Specifically, each index vector contains 8 elements, corresponding to the sparse mode identifiers of the 8 rows in the corresponding matrix block: an index value of 0 indicates that the non-zero element of the row is located in columns 0-3 (pre-mode), and an index value of 1 indicates that it is located in columns 4-7 (post-mode). For example, [0,1,0,...] in Idx00 represents W 00 The 0th row uses the front mode, the 1st row uses the back mode, the 2nd row uses the front mode, and so on.

[0201] The hardware execution flow follows a unified `vreordermul.vv` instruction. After the hardware decodes the 8 bits in the index vector in parallel, it continuously selects all 4 feature values ​​for each row from the corresponding half-region (the first 4 or the last 4 elements) of the feature vector segment. Since non-zero weights are also stored contiguously within the row, the rearranged feature data and weight data are precisely aligned, thus enabling all 32 effective calculations to be completed in one cycle. Compared to the general 4:8 sparsity of Example 3, this mode, while maintaining the same computational density and hardware efficiency, sharply reduces the index overhead of each matrix block from 96 bits (8 rows × 4 non-zero / row × 3 bits / position) to 8 bits (8 rows × 1 bit / row), significantly reducing storage and transmission overhead.

[0202] Example 5: Matrix-vector multiplication based on block-level structured sparse pattern

[0203] This embodiment describes a more extreme coarse-grained sparsity pattern—block-level sparsity, which is also a strongly constrained special case of the aforementioned 4:8 sparsity. This pattern divides an 8×8 block into two row groups (RG0: rows 0-3; RG1: rows 4-7) and forces all non-zero weights within each row group to be consecutively located within a predetermined interval of 4 columns (such as the first 4 columns or the last 4 columns).

[0204] Figure 10An exemplary schematic diagram of a block-level structured sparse matrix according to an embodiment of the present disclosure is shown. Specifically, for four consecutive 8×8 weight matrix blocks (W... 00 W 01 W 10 W 11 Its sparse pattern is directly encoded as four extremely simple block-level indices:

[0205] - W 00 Index: 01

[0206] - W 01 Index: 10

[0207] - W 10 Index: 10

[0208] - W 11 Index: 10

[0209] Each index is 2 bits, and its meaning is defined as follows: An 8×8 block is divided into two row groups: row group RG0 (rows 0-3) and row group RG1 (rows 4-7). For any row group, a bit value of 0 indicates that all effective (non-zero) weights of that row group are located in the first 4 columns (columns 0-3); a bit value of 1 indicates that all weights are located in the last 4 columns (columns 4-7). For example, index 01 represents W. 00 The effective weights of row group RG0 are in the first 4 columns, and the effective weights of row group RG1 are in the last 4 columns.

[0210] In this mode, the organization of the dense weight vector remains consistent with embodiments 3 and 4. Its index is further simplified: each 8×8 block only requires a 2-bit block-level index (e.g., 01), where bit 0 controls the half-zone selection of row group RG0 and bit 1 controls the half-zone selection of row group RG1.

[0211] The hardware execution flow is also based on the vreordermul.vv instruction. After decoding this 2-bit index, the hardware generates the same gating logic for all rows in the entire row group. For example, for index 10, the hardware will continuously select data from the four rows of the latter half of the feature vector segment (RG0) and from the four rows of the first half (RG1). This design, while maintaining the same peak computational efficiency (32 times / cycle) and computational density (50% sparsity) as in Example 4, further reduces the index overhead of each matrix block from 8 bits at the vector level to 2 bits, achieving extreme storage efficiency.

[0212] The instruction set and microarchitecture design disclosed herein can be flexibly extended to support the aforementioned various structured sparse patterns. By configuring the instruction format or the processor's control status register (CSR), the type of sparse pattern being processed (element-level / vector-level / block-level) can be indicated, and the corresponding index decoding and data paths can be activated accordingly, thereby providing unified and efficient acceleration support for various sparse deep learning models.

[0213] The N:M sparse matrix-vector multiplication acceleration scheme based on custom RISC-V instructions provided in this invention has a significant advantage in computational efficiency compared to traditional software implementations based on the standard RISC-V vector instruction set. To clearly illustrate this advantage, a quantitative analysis is performed below through a comparison of instruction overhead.

[0214] On conventional RISC-V vector processors that do not use the custom instructions of this invention, performing N:M sparse matrix-vector multiplication typically requires a cyclic combination of multiple standard vector instructions (e.g., vector multiplication vmul.vv and vector sliding vslideup.vx) to simulate the selection and alignment operations of sparse data. This approach is inefficient, primarily because it requires performing multiple conditional multiplications on the few valid non-zero elements in each sparse block, interspersed with numerous data shift operations to adjust the positions of eigenvectors.

[0215] For illustration, Table 2 shows the instruction overhead required by the traditional implementation and the proposed solution to process a basic computation block (e.g., 8 elements) under three typical sparse modes:

[0216] Table 2

[0217]

[0218]

[0219] As shown in Table 2, traditional methods require approximately 32 instructions (including loop control and data shifting) to complete the effective computation of a 1:8 extreme sparse pattern, with the majority of instruction cycles consumed in invalid zero-value operations and data preparation. In contrast, this disclosure introduces a dedicated `vreordermul.vv` instruction, embedding the aforementioned complex "index decoding-data rearrangement-multiplication-accumulation" process into the hardware microarchitecture. For any N:M pattern, the operations required to compute a sparse block can be atomically completed by this single instruction.

[0220] The embodiments disclosed herein aim to systematically address several key performance bottlenecks in N:M sparse matrix-vector multiplication (SpMV). First, addressing the problem of low operand loading efficiency caused by the scattered storage of non-zero data, this scheme designs a regularized non-zero data storage sequence structure, allowing non-zero elements in the weight matrix to be stored contiguously in memory (such as SRAM or DDR), thereby ensuring that the processor can quickly load operands in a highly efficient burst transfer mode, reducing memory access latency. Second, to reduce the decoding overhead of sparse indexes, this scheme proposes a compact index structure that uses very few bits to record the relative position of each non-zero element within its data block. This simplified design not only significantly reduces storage overhead but also makes the hardware decoding circuit extremely simple and efficient, laying the hardware foundation for implementing dedicated vector instructions (such as `vreordermul`) completed in a single cycle. Finally, to address the constraint of non-zero data dispersion on the parallelism of computational units, this scheme achieves regular alignment of all effective computational elements by densely reorganizing non-zero data and dynamically hardware rearranging the input feature vector. In this way, the processor's vector computation unit can be fully utilized. For example, on a RISC-V processor with a vector register length (VLEN) of 256 bits and support for 8-bit integer operations, a single instruction can perform 32 multiplication operations in parallel within one cycle, achieving the theoretical peak computational throughput. Furthermore, the method proposed in this disclosure exhibits good scalability. For sparse matrix multiplication (SpMM) operations, it can be decomposed into multiple sparse matrix-vector multiplication subproblems, and the aforementioned storage, indexing, and computation schemes can be applied, thereby achieving significant performance and energy efficiency improvements in a wider range of sparse linear algebra computations.

[0221] Figure 11 This is a block diagram illustrating components capable of reading instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) and executing any one or more of the methods discussed herein, according to some example embodiments. Specifically, Figure 7 shows a schematic diagram of hardware resource 1100, which includes one or more processors (or processor cores) 1110, one or more memory / storage devices 1120, and one or more communication resources 1130, wherein each of these processors, memory / storage devices, and communication resources may be communicatively coupled via bus 1140 or other interface circuitry. For embodiments utilizing node virtualization (e.g., Network Functions Virtualization (NFV)), a hypervisor 1102 may be executed to provide an execution environment for one or more network slices / subslices, thereby utilizing hardware resource 1100.

[0222] Processor 1110 may include, for example, processor 1112 and processor 1114. Processor 1110 may be, for example, a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP) such as a baseband processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a radio frequency integrated circuit (RFIC), another processor (including those discussed herein), or any suitable combination thereof.

[0223] Memory / storage device 1120 may include main memory, disk storage devices, or any suitable combination thereof. Memory / storage device 720 may include, but is not limited to, any type of volatile, non-volatile, or semi-volatile memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory, etc.

[0224] Communication resource 1130 may include interconnect or network interface controllers, components, or other suitable devices for communicating with one or more peripheral devices 1104 or one or more databases 1106 or other network elements via network 1108. For example, communication resource 1130 may include wired communication components (e.g., for coupling via USB, Ethernet, etc.), cellular communication components, near field communication (NFC) components, Bluetooth® (or Bluetooth® Low Energy) components, Wi-Fi® components, and other communication components.

[0225] Instructions 1150 may include software, programs, application programs, applets, or other executable code for causing at least any one of processors 1110 to perform any one or more of the methods discussed herein. Instructions 1150 may reside wholly or partially within processor 1110 (e.g., in the processor's cache), memory / storage device 1120, or any suitable combination thereof. Furthermore, any portion of instructions 1150 may be transferred from any combination of peripheral device 1104 or database 1106 to hardware resource 1100. Therefore, the memory of processor 1110, memory / storage device 1120, peripheral device 1104, and database 1106 are examples of computer-readable and machine-readable media.

[0226] Some examples may be implemented or be implemented as an article of art or at least a computer-readable medium. The computer-readable medium may include a non-transitory storage medium for storing logic. In some examples, the non-transitory storage medium may include one or more types of computer-readable storage media capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and so on. In some examples, the logic may include various software elements, such as software components, programs, applications, computer programs, application programs, system programs, machine programs, operating system software, middleware, firmware, software modules, routines, subroutines, functions, methods, procedures, software interfaces, APIs, instruction sets, computational code, computer code, code segments, computer code segments, words, values, symbols, or any combination thereof.

[0227] According to some examples, computer-readable media may include non-transitory storage media to store or maintain instructions that, when executed by a machine, computing device, or system, cause that machine, computing device, or system to perform methods and / or operations according to the described examples. Instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, etc. Instructions may be implemented according to a predetermined computer language, manner, or syntax to instruct a machine, computing device, or system to perform specific functions. Instructions may be implemented using any suitable high-level, low-level, object-oriented, visual, compiled, and / or interpreted programming language.

[0228] One or more aspects of at least one example can be implemented by representative instructions representing various logic within a processor, stored on at least one machine-readable medium, which, when read by a machine, computing device, or system, cause the machine, computing device, or system to manufacture logic to perform the techniques described herein. This representation, referred to as an "IP core," can be stored on a tangible machine-readable medium and provided to various customer or manufacturing facilities for loading into the manufacturing machine that actually manufactures the logic or processor.

[0229] The phrase "an example" or "an example" does not necessarily refer to the same example or embodiment. Any aspect described herein may be combined with any other aspect or similar aspect described herein, whether or not these aspects are described with reference to the same drawings or elements. The division, omission, or inclusion of block functions depicted in the drawings does not imply that hardware components, circuits, software, and / or elements used to implement these functions will necessarily be divided, omitted, or included in the embodiments.

[0230] Examples can be described using the terms “coupling” and “connection” and their derivatives. These terms are not necessarily intended to be synonyms. For example, a description using the terms “connection” and / or “coupling” may indicate that two or more elements are in direct physical or electrical contact with each other. However, the term “coupling” may also refer to two or more elements that are not in direct contact with each other but still cooperate or interact with each other.

[0231] The terms “first,” “second,” and the like are not used herein to indicate any order, quantity, or importance, but rather to distinguish one element from another. The term “a” herein does not imply a limitation on quantity, but rather indicates the presence of at least one mentioned item. The term “assertion” as used herein when referring to a signal refers to a state in which the signal is valid and can be achieved by applying any logic level (whether logic 0 or logic 1) to the signal. The terms “subsequently” or “afterward” can mean immediately following or following one or more other events. According to alternative embodiments, other sequences of steps may also be performed. Furthermore, depending on the specific application, additional steps may be added or removed. Any combination of variations can be used, and many variations, modifications, and alternative embodiments will be understood by those skilled in the art who benefit from this disclosure.

[0232] Unless otherwise specifically stated, disjunctive language such as the phrase "at least one of X, Y, or Z" is understood in context to generally state that an item, term, etc., can be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended, nor should it imply, that certain embodiments require the presence of each of at least one X, at least one Y, or at least one Z. Furthermore, unless otherwise specifically stated, connective language such as the phrase "at least one of X, Y, and Z" should also be understood to refer to X, Y, Z, or any combination thereof, including "X, Y, and / or Z".

Claims

1. A method for processing sparse matrix-vector multiplication based on RISC-V, comprising: The N:M sparse weight matrix of dimension D×K is divided into P×Q weight matrix blocks, where each weight matrix block has dimension V×V, P, Q, and V are all positive integers and satisfy P = D / V and Q = K / V. The weight matrix blocks adopt the N:M structured sparse pattern, where N and M are both positive integers and satisfy 1 ≤ N < M. The input feature vector of length K is divided into Q feature vector segments, where the length of each feature vector segment is V; Extract all non-zero weight elements in each weight matrix block in row-major order to generate a dense weight vector for that weight matrix block, and generate an index vector corresponding to each weight matrix block. The dense weight vector contains only non-zero elements, and the index vector is used to indicate the column offset position of each non-zero weight element in the corresponding weight matrix block within its respective row in the weight matrix block. Parallel processing of matrix-vector multiplication of at least one weight matrix block and a corresponding feature vector segment in the vector execution unit of a RISC-V processor includes: dynamically selecting and rearranging feature data elements from the corresponding feature vector segment based on the index vector of the at least one weight matrix block to generate a rearranged feature vector segment aligned with the dense weight vector of the at least one weight matrix block; and performing vectorized multiplication of the dense weight vector of the at least one weight matrix block and the rearranged feature vector segment to obtain a partial sum vector; and The output vector is obtained by summing all the partial sums associated with the same output row of the output vector of length D. The parallel processing of matrix-vector multiplication of at least one weight matrix block and its corresponding feature vector segment is triggered in response to the execution of a single RISC-V custom vector instruction. The semantics of this custom vector instruction are configured to perform dynamic selection and rearrangement of feature data elements, as well as vectorized multiplication and accumulation, within a single instruction cycle. The custom vector instruction is a vector rearrangement multiplication instruction, whose operands include: a target vector register storing the dense weight vectors of the at least one weight matrix block, a first source vector register storing the index vectors corresponding to the at least one weight matrix block, and a second source vector register storing the feature vector segments corresponding to the at least one weight matrix block.

2. The method according to claim 1, wherein, The number of non-zero weights processed in parallel in the vector execution unit is less than or equal to the maximum number of parallel multiplication operations supported by the vector execution unit in one instruction cycle.

3. The method according to claim 1, wherein, The number of non-zero weights processed in parallel within the vector execution unit is configured based on the processor's vector register length VLEN, the operation data bit width, and the N:M sparse mode.

4. The method according to claim 3, wherein, The RISC-V processor has a vector register length (VLEN) of 256 bits and an operation data width of 8 bits.

5. The method according to claim 4, wherein, When V=M=8 and N=1, the number of the at least one weight matrix block is 4.

6. The method according to claim 4, wherein, When V=M=8 and N=2, the number of the at least one weight matrix block is 2.

7. The method according to claim 4, wherein, When V=M=8 and N=4, the number of the at least one weight matrix block is 1.

8. The method according to claim 1, wherein, Each element in the index vector has a value ranging from 0 to V-1.

9. The method according to claim 8, wherein, Each element in the index vector is encoded using the minimum number of binary bits required to uniquely represent the offset positions of V different columns.

10. The method according to claim 9, wherein, When V=8, each element in the index vector is encoded using 3 bits.

11. The method according to claim 1, wherein, The sparsity of the sparse weight matrix is ​​between 50% and 90%.

12. The method according to claim 1, wherein, V is an integer multiple of M.

13. The method according to claim 1, wherein, M=8, and N=1, 2 or 4.

14. A device for processing sparse matrix-vector multiplication based on RISC-V, wherein, The apparatus includes a processor circuit configured to perform the method of any one of claims 1-13.

15. A computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to perform the method according to any one of claims 1-13.

16. A computer program product comprising instructions, wherein the instructions, when executed by a processor, cause the processor to perform the method according to any one of claims 1-13.

Images