Zero skipping sparsification technique for reducing data movement

Abstract

The present application provides a zero-out sparsity technique to reduce data movement between memory and an accelerator and reduce the computational workload of the accelerator. The technique includes detecting zero and near-zero values on the memory, non-zero values are transferred to the accelerator for computation, and zero and near-zero values are written back to the memory as zero values.

Description

Background Technology

[0001] Scalar, vector, and matrix multiplication are common computations performed by computing devices. These operations are frequently used in machine learning, artificial intelligence, neural networks, data analysis, and more. For example, many applications compute vector and / or matrix multiplications of input or activation matrices and weight vectors or matrices. This computation can be performed hundreds, thousands, or even more times.

[0002] Please refer to Figure 1 The matrix-vector multiplication of a c×r matrix A with r-row column vectors X involves the dot product of X with each row of A. The dot product is achieved by multiplying each element in the first row of matrix A with the corresponding element in each column of vector X, summing the partial products, and repeating this process for each row of matrix A. Matrix-matrix multiplication can be viewed as multiple matrix-vector multiplications. Figure 2 The diagram illustrates an exemplary matrix-matrix multiplication of a c×r matrix A and an r×c matrix X. It should also be noted that matrix-matrix multiplication and matrix-vector multiplication are identical when one of the matrices is a single-row matrix (e.g., a row vector) or a single-column matrix (e.g., a column vector).

[0003] Now for reference Figure 3 The diagram illustrates an example sparse matrix. A sparse matrix is ​​a matrix in which most of its elements have a value of zero. Conversely, a matrix in which most of its elements have a value of non-zero is called a dense matrix. Matrices are typically stored as two-dimensional arrays. Each entry in the array represents an element a of the matrix. i,j Access is via two indices, i and j. For an m×n matrix, the storage required to store the matrix is ​​proportional to m×n. For sparse matrices, the storage required can be reduced by storing only the non-zero element values. Various data structures exist for storing sparse matrices in compressed formats, including but not limited to key dictionaries, lists of lists, lists of coordinates, compressed sparse rows, and compressed sparse columns.

[0004] Now for reference Figure 4This illustrates an exemplary Compressed Sparse Row (CSR) data structure based on conventional techniques. A CSR data structure represents a sparse matrix with three arrays: an array of row pointers, an array of column indices, and an array of values. The value array contains non-zero values. The column index array indicates the column containing the non-zero value in a given row. The row pointer array indicates the starting position of the non-zero value in the value array for the corresponding row. Similarly, a Compressed Sparse Column (CSC) data structure can represent a sparse matrix with an array of column pointers, an array of row indices, and an array of values. Generally, compressed matrix data structures, such as CSR and CSC, reduce the storage consumption of the matrix. Compressed matrix data structures, such as CSR or CSC, can also reduce computational effort by eliminating calculations involving zero-value matrix elements. CSR and CSC techniques offer improved performance when the sparsity rate is high due to compression and decompression overhead. Other conventional sparse techniques include Eyerisis and Compressed Direct Memory Access (DMA) engines. In Eyerisis, element processing is designed to skip zero inputs, primarily for power reduction. Eyerisis can improve performance under computationally constrained conditions, but it does not benefit data movement from memory to the processor / accelerator. In a compressed DMA engine, zero data is masked and data is decoded before computation. However, a compressed DMA engine only benefits data transfer and requires decompression before computation.

[0005] Most sparse techniques are implemented on processors or accelerators, not on the memory subsystem. Most sparse techniques utilize compression to improve computational efficiency. Off-chip storage devices, such as volatile memory (e.g., DRAM) and non-volatile memory (e.g., FLASH), typically have insufficient bandwidth to support high-speed processors and accelerators, thus becoming a performance bottleneck. Therefore, there is a continuous need for improved data transfer and computation techniques for sparse scalar, vector, and matrix computations. Summary of the Invention

[0006] This application can be best understood by referring to the following description and figures, which illustrate embodiments of the application for a zero-hopping sparsity technique for reducing data movement.

[0007] In one embodiment, the zero-jumping sparse computing system may include a processor and a memory. The processor may include a processor sparse control unit and a command interface; the processor sparse control unit may be configured to encapsulate a memory access request in a zero-jumping sparse command; the processor's command interface may be configured to send the zero-jumping sparse command and receive a given element from a first group of elements and a second group of elements. The processor may be configured to perform computation operations on the corresponding given elements of the first and second groups of elements to generate a result element. The processor's command interface may also be configured to send the result element. The memory may include a memory cell array, a command interface, and a memory sparse control unit; the memory's command interface may be configured to receive zero-jumping sparse commands; the memory sparse control unit may be configured to access a first group of elements in the memory cell array, determine the zero and near-zero element values ​​in the first group, and generate a first zero element mask based on the determined zero and near-zero elements of the first group; the memory sparse control unit may also be configured to access a second group of elements in the memory cell array, determine the zero and near-zero element values ​​in the second group, and generate a second zero element mask based on the determined zero and near-zero elements of the second group; the memory... The sparse control unit can also be configured to generate a computational zero mask based on the computational operation of the zero-jumping sparse command, the first zero element mask, and the second zero element mask; the memory sparse control unit can also be configured to write zero values ​​into the memory cell array as part of the computational result based on the computational zero mask; the command interface of the memory can also be configured to send given elements from the first and second groups of elements corresponding to the non-zero indicators in the computational zero mask and receive the result elements; the memory sparse control unit can also be configured to write the result elements into the memory cell array as an additional part of the computational result of the first and second groups of elements based on the computational zero mask.

[0008] In another embodiment, the processor may include a processor sparse control unit configured to generate a first hop-zero sparse command encapsulating a memory access request; the processor may also include a command interface configured to send the first hop-zero sparse command, receive a first zero-element mask and given elements from a first set of elements, and receive a second zero-element mask and given elements from a second set of elements. The processor's processor sparse control unit may also be configured to generate a first computational zero mask based on computational operations of the first hop-zero sparse command, the first zero-element mask, and the second zero-element mask; the processor may be configured to perform computational operations on each given element of the first and second sets of elements to generate a first result element. The processor's command interface can also be configured to send the first result element; the memory may include a memory cell array, a command interface, and a memory sparsity control unit; the memory's command interface can be configured to receive the first zero-jump sparse command; the memory sparsity control unit can be configured to access a first group of elements in the memory cell array, determine the zero and near-zero element values ​​in the first group, and generate a first zero element mask based on the determined zero and near-zero element values ​​of the first group; the memory sparsity control unit can also be configured to access a second group of elements in the memory cell array, determine the zero and near-zero element values ​​in the second group, and generate a second zero element mask based on the determined zero and near-zero elements of the second group; the memory's command interface can also be configured to send a given element from the first group of elements corresponding to a non-zero indicator in the first zero element mask, transmit the first zero element mask, send a given element from the second group of elements corresponding to a non-zero indicator in the second zero element mask, transmit the second zero element mask, and receive the first result element. The memory sparsity control unit can also be configured to write the first result element as the calculation result of the first and second groups of elements into the memory cell array.

[0009] This invention provides a simplified description of the selection of concepts further described in the following detailed description. This content is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Attached Figure Description

[0010] Embodiments of this application are illustrated in the accompanying drawings by way of example rather than limitation, and wherein the same reference numerals refer to similar elements, and wherein: Figure 1 The matrix-vector multiplication method based on conventional techniques is shown. Figure 2 An exemplary matrix-matrix multiplication method based on conventional techniques is shown. Figure 3 An exemplary sparse matrix based on conventional techniques is shown. Figure 4 An exemplary compressed sparse row (CSR) data structure based on conventional techniques is shown. Figure 5 A computing system according to an aspect of this application is shown. Figures 6A-6B A method for zero-jumping sparse computation based on some aspects of this application is shown. Figures 7A-7D A method for zero-jumping sparse computation based on some aspects of this application is shown. Detailed Implementation

[0011] Reference will now be made in detail to embodiments of this application, examples of which are illustrated in the accompanying drawings. While this application will be described in conjunction with these embodiments, it should be understood that they are not intended to limit the application to these embodiments. Rather, this application is intended to cover substitutions, modifications, and equivalents that may be included within the scope of this application as defined by the appended claims. Furthermore, numerous specific details are set forth in the following detailed description of this application to provide a thorough understanding of the application. However, it should be understood that this application may be practiced without these specific details. In other instances, well-known methods, processes, components, and circuits have not been described in detail to avoid unnecessarily obscuring aspects of this application.

[0012] The following embodiments of this application are presented in the form of routines, modules, logic blocks, and other symbolic representations of operations on data within one or more electronic devices. Description and representation are means by which those skilled in the art most effectively communicate the essence of their work to those skilled in the art. Routines, modules, logic blocks, and / or the like are, herein and generally, considered as self-consistent sequences of processes or instructions that lead to desired results. These processes include physical operations on physical quantities. Typically, though not always, these physical operations take the form of electrical or magnetic signals capable of being stored, transmitted, compared, and otherwise operated in an electronic device. For convenience and in reference to common usage, with reference to embodiments of this application, these signals are referred to as data, bits, values, elements, symbols, characters, terms, numbers, strings, etc.

[0013] However, it should be remembered that these terms will be interpreted as referring to physical operations and quantities and are merely convenient labels and will be further interpreted in light of terms commonly used in the art. Unless otherwise clearly stated in the following discussion, it should be understood that, through the discussion of this application, the use of terms such as "receiving" refers to the actions and processes of electronic devices such as processing and converting data. Data is represented as physical (e.g., electronic) quantities within the logic circuits, registers, memories, and / or similar elements of electronic devices and is converted into other data similarly represented as physical quantities within the electronic device.

[0014] In this application, the use of antonymous conjunctions is intended to encompass conjunctions. The use of definite or indefinite articles is not intended to indicate quantity. In particular, references to “the” or “a” object are also intended to indicate one of a possible plurality of such objects. The use of the terms “comprising,” “including,” “including,” “comprise,” etc., specifies the presence of the element, but does not preclude the presence or inclusion of one or more other elements and / or groups thereof. It should also be understood that although the terms first, second, etc., may be used herein to describe various elements, such elements should not be limited by these terms. These terms are used herein to distinguish one element from another. For example, without departing from the scope of the embodiments, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. It should also be understood that when an element is referred to as “coupled” to another element, it may be directly or indirectly connected to the other element, or there may be intermediate elements. Conversely, when an element is referred to as “directly connected” to another element, there are no intermediate elements. It should also be understood that the term “and or” includes any and all combinations of one or more related elements. It should also be understood that the wording and terminology used in this article are for descriptive purposes and should not be considered as limiting.

[0015] Now for reference Figure 5 This illustration depicts a computing system according to some aspects of the present invention. The computing system 500 may include one or more accelerators 505 and one or more memories 510. The computing system 500 may include many other subsystems that are not essential for understanding the aspects of this application and are therefore not described herein. The one or more accelerators 505 may be, but are not limited to, a central processing unit (CPU) or a graphics processing unit (GPU). The one or more memories 510 may be, but are not limited to, volatile memories such as dynamic random access memory (DRAM), or non-volatile memories such as static random access memory (SRAM), flash memory, or the like. For ease of description, aspects of the present invention will be described below with regard to the case of one accelerator 505 and one memory 510. However, the computing system 500 may include any combination of one or more accelerators 505 and one or more memories 510.

[0016] Accelerator 505 may include an accelerator sparse control unit 515 and an accelerator command interface 520. Accelerator 505 may include many other subsystems not essential for understanding aspects of this application and therefore not described herein. Accelerator sparse control unit 515 may include a zero-jump command unit 525 and an optional mask information processor 530. Memory 510 may include a memory cell array 535, a memory sparse control unit 540, and a memory command interface 545. In one implementation, for example, the memory sparse control unit may be part of a memory controller (MC) integrated with memory 510. In another embodiment, the memory sparse control unit may be part of a separate memory controller (MC) coupled to one or more memory subsystems. Memory 510 may include many other subsystems not essential for understanding aspects of this application and therefore not described herein. Memory sparse control unit 540 may include a zero-counting unit 550, a mask information processor 555, and a write-back control unit 560. The sparse control units 515 and 540 of the accelerator 505 and memory 510, and the command interfaces 520 and 545 can be configured to achieve zero-jumping sparse computation, as shown in the reference. Figure 6A and Figure 6B Further described.

[0017] Zero-jump sparse computation may include, at 605, generating a zero-jump sparse command encapsulating a memory access request. The zero-jump sparse command may be generated by accelerator 505. In one implementation, accelerator sparse control unit 515 may be configured to encapsulate the memory access request within the zero-jump sparse command. In one embodiment, zero-jump command unit 525 of accelerator sparse control unit 515 may be configured to generate a zero-jump sparse command including a zero-jump command indicator (TAGID), a computation opcode (OP), first and second set element data types (DataType), the address of the first set of elements (Addr1), the address of the second set of elements (Addr2), the target address of the computation result (Addrdst), and the data length (length). The first set of elements may be the value of a first scalar, vector, matrix, etc. The second set of elements may be the value of a second scalar, vector, matrix, etc. The computation opcode (OP) may be an opcode specifying an addition, subtraction, multiplication, division, etc., operation using the first and second set of elements. In one embodiment, the zero-jump sparse command may further include a near-zero threshold (NZ_Th) value. In another implementation, a near-zero threshold can be provided to the memory during a separate initialization process. At 610, a zero-jump sparse command can be sent from accelerator 505. In one implementation, accelerator command interface 520 can send a zero-jump sparse command generated by accelerator sparse control unit 515 to memory 510.

[0018] At 615, a zero-jump sparse command can be received by memory 510. In one implementation, memory command interface 545 can receive the zero-jump sparse command from accelerator 505. At 620, a first set of elements can be accessed to determine zero and near-zero element values ​​in the first set of elements, and a first zero element mask can be generated based on the determined zero and near-zero element values. In one embodiment, memory sparsity control unit 540 can be configured to access a first set of elements stored in memory cell array 535, specified by the address (Addr1) and data length (length) of the first set of elements in the zero-jump sparse command. Zero counting unit 550 can be configured to determine zero and near-zero element values ​​in the first set of elements. For example, zero counting unit 550 can determine that a value in the first set of elements is a near-zero value when it is less than a specified near-zero threshold (NZ_Th). Similarly, the near-zero threshold (NZ_Th) can be included in the zero-jump sparse command or provided in a separate initialization process. Zero counting unit 550 can also utilize the data type (DataType) in the zero-jump sparse command to determine zero and near-zero values. For example, if the element value is encoded in integer or fixed-point format, leading zeros in the integer part can be counted to determine if a given element value is near zero. If the element value is encoded in floating-point format, leading zeros in the exponent field can be counted to determine if a given element value is near zero. The mask information processor 555 can generate a first zero-element mask, where zero and near-zero element values ​​are indicated by a first state (e.g., a '0' bit value) and non-zero values ​​are indicated by a second state (e.g., a '1' bit value).

[0019] At 625, a second set of elements can be accessed to determine zero and near-zero element values ​​in the second set, and a second zero element mask is generated based on the determined zero and near-zero element values. In one embodiment, the memory sparsity control unit 540 can be configured to access a second set of elements stored in the memory cell array 535, the second set of elements being specified by the address (Addr2) and data length (length) of the second set of elements in a jump-zero sparse command. The zero-counting unit 550 can be configured to determine zero and near-zero values ​​in the second set of elements. For example, the zero-counting unit 550 can determine that a value in the second set of elements is near-zero when it is less than a specified near-zero threshold (NZ_Th). Similarly, the near-zero threshold (NZ_Th) can be included in the jump-zero sparse command or provided in a separate initialization process. The zero-counting unit 550 can also utilize the data type (DataType) in the jump-zero sparse command to determine zero and near-zero values. The mask information processor 555 can generate a second zero element mask, wherein zero and near-zero element values ​​are indicated by a first state (e.g., '0' bit value) and non-zero values ​​are indicated by a second state (e.g., '1' bit value).

[0020] At 630, a computed zero-element mask can be generated based on the computational operation of the zero-skip sparse command, the first zero-element mask, and the second zero-element mask. In one implementation, the mask information processor 555 can generate a computed zero-element mask comprising a set of indicators, a computational opcode (OP) based on the zero-skip sparse command, and various states of the first and second zero-element masks, wherein zero and near-zero element values ​​are indicated by a first state (e.g., '0' bit value) and non-zero values ​​are indicated by a second state (e.g., '1' bit value). For example, for a multiplication computation, if one or both of the respective indicators of the first and second zero-element masks are in the first state (e.g., '0'), the corresponding indicator of the computed zero-element mask can be set to the first state (e.g., '0'). However, if both indicators in the first and second zero-element masks are in the second state of the multiplication computation (e.g., '1'), the corresponding indicator of the computed zero-element mask can be set to the second state (e.g., '1'). In another example, for addition, if each indicator of the first and second zero-element masks is in a first state (e.g., '0'), the corresponding indicator for calculating the zero mask can be set to the first state (e.g., '0'). However, if one or the other, or both, of the first and second zero-element masks is in a second state for addition (e.g., '1'), the corresponding indicator for calculating the zero mask can be set to the second state (e.g., '1').

[0021] At 635, the zero value based on the zero-mask can be written back as the corresponding part of the computation result of the first and second sets of elements. In one implementation, for any indicator in the zero-mask that is set to a first state (e.g., '0'), the write-back control unit 560 can be configured to write back the zero value to the corresponding element position at the target address (Addrdst) specified in the zero-jump sparse command.

[0022] At 640, memory 510 can send a given element from the first and second groups corresponding to a non-zero indicator in the computed zero mask. In one implementation, memory command interface 545 can transmit a given element from the first and second groups (corresponding to an indicator in the computed zero mask set to the second state) to accelerator 505. Memory command interface 545 can optionally transmit a zero-jump command indicator (TAGID) from a zero-jump sparse command, along with the given element. The zero-jump command indicator (TAGID) identifies the given element and the given zero-jump sparse command associated with it.

[0023] At 645, accelerator 505 can receive given elements from memory 510 in response to a zero-skip sparse command. In one implementation, the given elements from the first and second groups can be received from memory command interface 545 by accelerator command interface 520. A zero-skip command indicator (TAGID) can also be received by accelerator command interface 520 along with the given elements. Similarly, the zero-skip command indicator (TAGID) can be used to identify the given zero-skip sparse command associated with the given element. At 650, computational operations can be performed on the respective given elements from the first and second groups to generate result elements. In one implementation, accelerator 505 can perform computational operations specified by the computation opcode (OP) of the zero-skip sparse command on the received given elements from the first and second groups. At 655, the result elements can be sent by accelerator 505. In one implementation, accelerator command interface 520 can transfer the result elements for the zero-skip sparse command to memory 510. The accelerator command interface 520 can send a zero-jump command indicator (TAGID) and a result element from a zero-jump sparse command as needed. Furthermore, the zero-jump command indicator (TAGID) can be used to identify a given zero-jump sparse command associated with a given result element.

[0024] At 660, memory 510 can receive the result element of a zero-jump sparse command. In one implementation, memory command interface 545 can receive the result element from accelerator command interface 520. Memory command interface 545 can accept a zero-jump command indicator (TAGID) and the result element from the zero-jump sparse command as needed. Furthermore, the zero-jump command indicator (TAGID) can be used to identify a given zero-jump sparse command associated with a given result element. At 665, the computation results of the first and second sets of elements can be written to memory 510 based on the received result element and the computed zero mask. In one implementation, write-back control unit 560 can be configured to write back the received result element as an additional part of the computation results of the first and second sets of elements to memory cell array 535 at the target address (Addrdst) based on the computed zero mask. It should be understood that the process at 605-666 can be iteratively repeated for each of the plurality of zero-jump sparse commands. Furthermore, the accelerator sparse control unit and the memory sparse control unit can be configured to be enabled and disabled to selectively perform zero-jump sparse computation when appropriate.

[0025] Referring now to Table 1, exemplary zero-jumping sparse computation according to some aspects of this application is illustrated. Exemplary zero-jumping sparse computation may be a vector-vector multiplication of N-element first and second vectors (vector 1 and vector 2). Although the element values ​​in Table 1 are listed as decimal numbers for ease of understanding, it should be understood that these values ​​are typically encoded as binary numbers. Table 1 As described above, first and second masks (mask 1 and mask 2) can be generated based on the zero and near-zero element values ​​in the first and second vectors (vector 1 and vector 2), respectively. For example, if the near-zero threshold (NZ_Th) is set to 0.1, the floating-point value of the first element of the first vector can be determined to be close to zero, and the corresponding indicator in the first mask (mask 1) can be set to 0. Similarly, elements 0.001, 1E-08, and 2E-05 can be determined to be close to 0, and the corresponding indicator in the first mask (mask 1) can also be set to 0. Additionally, the corresponding indicator for non-zero elements can be set to 1. The zero and near-zero element values ​​of the second vector (vector 2) can be determined, and the corresponding indicators in the second mask (mask 2) can be set similarly. If one or both of the corresponding indicators in the first and second masks (mask 1 and mask 2) are zero, a computational zero mask for multiplication can be generated by setting the corresponding indicator in the computational zero mask to 0. If the corresponding indicators in the first and second masks (mask 1 and mask 2) are both set to 1, then the corresponding indicators in the zero mask can be set to 1. As shown in Table 1, the indicators for the third and (N-1)th elements in the zero mask are set to 1. Accordingly, the given elements 3.65 and 56.3 of the first vector (vector 1) and the given elements 62.3 and 8.52 of the second vector (vector 2) can be sent from memory to the accelerator for multiplying the given elements to generate the resulting elements 227.4 and 479.7. The resulting elements can be sent from the accelerator to memory for storage as an additional part of the computation result.

[0026] Referring now to Table 2, another exemplary zero-jumping sparse computation according to some aspects of this application is shown. The exemplary zero-jumping sparse computation may be vector-vector addition. Table 2 Additionally, if the corresponding indicators in the first and second masks (mask 1 and mask 2) are both zero, a computed zero mask can be generated by setting the corresponding indicator in the computed zero mask to 0. If one or both corresponding indicators in the first and second masks (mask 1 and mask 2) are set to 1, the corresponding indicator in the computed zero mask can be set to 1.

[0027] Several aspects of this application can advantageously reduce bandwidth utilization at the interface between accelerator 205 and memory 210. Depending on the sparsity of one of the element groups, bandwidth utilization can be reduced proportionally to the degree of sparsity. For example, if a group of elements is characterized by a sparsity of 50% (e.g., half of the elements are zero or near-zero values), bandwidth utilization at the interface between accelerator 205 and memory 210 can be reduced by approximately 50%. Similarly, the computational workload on the processor can also be reduced.

[0028] Now for reference Figures 7A-7D This paper illustrates a zero-jump sparse computation method according to some aspects of this application. At 702, the zero-jump sparse computation may include generating a first zero-jump sparse command encapsulating a memory access request. The zero-jump sparse command may be generated by accelerator 505. In one implementation, accelerator sparse control unit 515 may be configured to encapsulate the memory access request in the first zero-jump sparse command. In one embodiment, zero-jump command unit 525 of accelerator sparse control unit 515 may be configured to generate the first zero-jump sparse command, which includes a zero-jump command indicator (TAGID), a computation opcode (OP), data types (DataType) of the first and second sets of elements, the address (Addr1) of the first set of elements, the address (Addr2) of the second set of elements, the target address (Addrdst) of the computation result, and the data length. The first set of elements may be the values ​​of a first scalar, vector, matrix, etc. For example, the first set of elements may be a weight vector, matrix, etc. The second set of elements may be the values ​​of a second scalar, vector, matrix, etc. For example, the second set of elements may be a feature map vector, matrix, etc. The computation opcode (OP) can be an opcode that specifies addition, subtraction, multiplication, division, etc., using the first and second sets of elements. In one embodiment, the zero-jump sparse command may also include a near-zero threshold (NZ_Th). In another embodiment, the near-zero threshold can be provided to the memory during a separate initialization process. At 704, the first zero-jump sparse command can be sent from accelerator 505. In one implementation, accelerator command interface 520 can send the first zero-jump sparse command generated by accelerator sparse control unit 515 to memory 510.

[0029] At 706, memory 510 can receive a first hop-zero sparse command. In one implementation, memory command interface 545 can receive the first hop-zero sparse command from accelerator 505. At 708, a first set of elements can be accessed to determine zero and near-zero element values ​​in the first set and to generate a first zero element mask based on the determined zero and near-zero element values. In one embodiment, memory sparsity control unit 540 can be configured to access a first set of elements stored in memory cell array 535, specified by the address (Addr1) and data length of the first set of elements in the first hop-zero sparse command. Zero counting unit 550 can be configured to determine zero and near-zero values ​​in the first set of elements. For example, zero counting unit 550 can determine that a value in the first set of elements is near-zero when it is less than a specified near-zero threshold (NZ_Th). Similarly, the near-zero threshold (NZ_Th) can be included in the hop-zero sparse command or provided during a separate initialization process. Zero counting unit 550 can also utilize the data type (DataType) in the hop-zero sparse command to determine zero and near-zero values. For example, if the element value is encoded as an integer or fixed-point format, leading zeros in the integer part can be counted to determine if a given element value is near zero. If the element value is encoded in floating-point format, leading zeros in the exponent field can be counted to determine if a given element value is near zero. The mask information processor 555 can generate a first zero-element mask, where zero and near-zero element values ​​are indicated by a first state (e.g., a '0' bit value) and non-zero values ​​are indicated by a second state (e.g., a '1' bit value).

[0030] At 710, a first zero-element mask and a given element from a first group corresponding to a non-zero indicator in the first zero-element mask can be sent from memory 510. In one implementation, memory command interface 545 can send the first zero-element mask and the given element to accelerator 505. At 712, accelerator 505 can receive the first zero-element mask and the given element. In one implementation, accelerator command interface 520 can receive the first zero-element mask from memory 510 and the given element from a first group corresponding to a non-zero indicator in the first zero-element mask.

[0031] At 714, a second set of elements can be accessed to determine the zero and near-zero element values ​​in the second set and to generate a second zero element mask based on the determined zero and near-zero element values. In one embodiment, the memory sparsity control unit 540 can be configured to access a second set of elements stored in the memory cell array 535, specified by the address (Addr2) and data length of the second set of elements in the first jump-zero sparse command. The zero-counting unit 550 can be configured to determine the zero and near-zero values ​​in the second set of elements. For example, the zero-counting unit 550 can determine that a value in the second set of elements is a near-zero value when it is less than a specified near-zero threshold (NZ_Th). Similarly, the near-zero threshold (NZ_Th) can be included in the jump-zero sparse command or provided in a separate initialization process. The zero-counting unit 550 can also utilize the data type (DataType) in the jump-zero sparse command to determine the zero and near-zero values. Mask information processor 555 can generate a second zero-element mask, where zero and near-zero element values ​​are indicated by a first state (e.g., '0' bit value) and non-zero values ​​are indicated by a second state (e.g., '1' bit value). At 716, the second zero-element mask and a given element from a second set corresponding to a non-zero indicator in the second zero-element mask can be sent from memory 510. In one implementation, memory command interface 545 can send the second zero-element mask and the given element to accelerator 505.

[0032] At 718, the accelerator 505 can receive a second zero-element mask and a given element. In one implementation, the accelerator command interface 520 can receive the second zero-element mask sent by the memory 510 and a given element from a second set corresponding to a non-zero indicator in the second zero-element mask. At 720, a first computational zero mask can be generated based on a first hop-zero sparse command, a computational operation of the first zero-element mask, and the second zero-element mask. In one implementation, based on the computational operation code (OP) of the first hop-zero sparse command and the respective states of the first and second zero-element masks, the mask information processor 530 of the accelerator 505 can generate a first computational zero-element mask including a set of indicators, where zero and near-zero element values ​​are indicated by a first state (e.g., '0' bit values) and non-zero values ​​are indicated by a second state (e.g., '1' bit values). For example, for multiplication, if any one or both of the indicators of the first and second zero-element masks are in a first state (e.g., '0'), the corresponding indicator of the first computed zero mask can be set to the first state (e.g., '0'). However, for multiplication, if both of the indicators of the first and second zero-element masks are in a second state (e.g., '1'), the corresponding indicator of the first computed zero mask can be set to the second state (e.g., '1'). In another example, for addition, if both of the indicators of the first and second zero-element masks are in a first state (e.g., '0'), the corresponding indicator of the first computed zero mask can be set to the first state (e.g., '0'). However, for addition, if one or both of the indicators of the first and second zero-element masks are in a second state (e.g., '1'), the corresponding indicator of the first computed zero mask can be set to the second state (e.g., '1').

[0033] At 722, computational operations can be performed on each given element from the first and second groups to generate a first result element. In one implementation, accelerator 505 can perform a computational operation specified by the computation opcode (OP) of a first zero-skip sparse command on the received given elements from the first and second groups. At 724, accelerator 505 can send the first result element and a first computational zero mask. In one implementation, accelerator command interface 520 can transfer the first result element and the first computational zero mask for the zero-skip sparse command to memory 510, and accelerator command interface 520 can send a zero-skip command indicator (TAGID) from the first zero-skip sparse command along with the result element as needed. Again, the zero-skip command indicator (TAGID) can be used to identify a given zero-skip sparse command associated with a given result element.

[0034] At 726, memory 510 may receive a first result element and a first computational zero mask of a first zero-skip sparse command. In one implementation, memory command interface 545 may receive the first result element and the first computational zero mask from accelerator command interface 520. Memory command interface 545 may optionally receive a zero-skip command indicator (TAGID) and the first result element from the first zero-skip sparse command. Again, the zero-skip command indicator (TAGID) can be used to identify a given zero-skip sparse command associated with a given result element. At 728, the first computation result of the first and second sets of elements may be written to memory 510 based on the received first result element and the first computational zero mask. In one implementation, write-back control unit 560 may be configured to write back the received first result element along with applicable zero values ​​as the computation result of the first and second sets of elements to the memory cell array 535 at the target address (Addrdst) based on the first computational zero mask.

[0035] At 730, a second-hop zero-sparse command encapsulating the memory access request can be generated. In one implementation, the accelerator sparse control unit 515 can be configured to encapsulate the memory access request in a second-hop zero-sparse command. In one embodiment, the hop-zero command unit 525 of the accelerator sparse control unit 515 can be configured to generate a second-hop zero-sparse command including a hop-zero command indicator (TAGID), a computation opcode (OP), a third set of element data type (DataType), an address of the third set of elements (Addr3), a second target address of the computation result (Addrdst), and a data length (length). The third set of elements can be the value of a first scalar, vector, matrix, etc. For example, the third set of elements can be another feature map vector, matrix, etc. The computation opcode (OP) can be an opcode specifying the use of the third set of elements for addition, subtraction, multiplication, division, etc. In one implementation, the hop-zero sparse command can also include a near-zero threshold (NZ_Th) value. In another embodiment, the near-zero threshold can be provided to the memory during a separate initialization process. At 732, the second-hop zero-sparse command can be sent from the accelerator 505. In one implementation, the accelerator command interface 520 can send a second-hop zero-sparse command generated by the accelerator sparse control unit 515 to the memory 510.

[0036] At 734, memory 510 can receive a second-hop zero-sparse command. In one implementation, memory command interface 545 can receive the second-hop zero-sparse command from accelerator 505. At 736, a third set of elements can be accessed to determine the zero and near-zero element values ​​in the third set, and a third zero element mask can be generated based on the determined zero and near-zero element values. In one implementation, memory sparsity control unit 540 can be configured to access a third set of elements stored in memory cell array 535, specified by the address (Addr3) and data length of the first set of elements in the second-hop zero-sparse command. Zero counting unit 550 can be configured to determine the zero and near-zero values ​​in the third set of elements. For example, zero counting unit 550 can determine that a value in the third set of elements is near-zero when it is less than a specified near-zero threshold (NZ_Th). Alternatively, the near-zero threshold (NZ_Th) can be included in the first and / or second-hop zero-sparse command or provided during a separate initialization process. Zero-counting unit 550 can also utilize the data type (DataType) in the first and / or second hop zero sparse command to determine zero and near-zero values. For example, if the element value is encoded as an integer or fixed-point format, leading zeros in the integer part can be counted to determine if a given element value is near zero. If the element value is encoded in floating-point format, leading zeros in the exponent field can be counted to determine if a given element value is near zero. Mask information processor 555 can generate a third zero-element mask, where zero and near-zero element values ​​are indicated by a first state (e.g., a '0' bit value), and non-zero values ​​are indicated by a second state (e.g., a '1' bit value). At 738, the third zero-element mask and a given element from the third set of elements corresponding to a non-zero indicator in the third zero-element mask can be sent from memory 510. In one implementation, memory command interface 545 can transmit the third zero-element mask and the given element to accelerator 505.

[0037] At 740, accelerator 505 can receive a third zero mask and a given element. In one implementation, accelerator command interface 520 can receive a third zero element mask from memory 510 and a given element from a third group corresponding to a non-zero indicator in the third zero element mask. At 742, a second computational zero mask can be generated based on computational operations of the second hop zero sparsity command, the first zero element mask, and the third zero element mask. At 744, computational operations can be performed on each given element from the first and third groups to generate a second result element. In one implementation, accelerator 505 can perform computational operations on the received given elements from the first and third groups, specified by the computational operation code (OP) of the first or second hop zero sparsity command. For example, vector multiplication can be performed between the weights of the first group of elements and the feature map of the third group of elements. At 746, accelerator 505 can transmit the second result element and the second computational zero mask. In one implementation, accelerator command interface 520 can transmit the second result element and the second computational zero mask for the hop zero sparsity command to memory 510. The accelerator command interface 520 can send the zero-jump command indicator (TAGID) from the first zero-jump sparse command along with the resulting element as needed. Again, the zero-jump command indicator (TAGID) can be used to identify the given zero-jump sparse command associated with a given resulting element.

[0038] At 748, memory 510 may receive a second result element and a second computational zero mask for a second zero-skip sparse command. In one implementation, memory command interface 545 may receive the second result element and the second computational zero mask from accelerator command interface 520. Memory command interface 545 may receive a zero-skip command indicator (TAGID) along with the second result element from the second zero-skip sparse command as needed. Again, the zero-skip command indicator (TAGID) may be used to identify a given zero-skip sparse command associated with a given result element. At 750, the second computation result of the first and third sets of elements may be written to memory 510 based on the received second result element and the second computational zero mask. In one implementation, write-back control unit 560 may be configured to write back the second received result element along with applicable zero values ​​as the computation result of the first and third sets of elements to memory cell array 535 located at a second destination address (Addrdst) based on the second computational zero mask.

[0039] Referring now to Table 3, another exemplary zero-jumping sparse computation according to some aspects of this application is shown. The exemplary zero-jumping sparse computation can be a vector-vector multiplication between a first vector and multiple additional vectors. For example, the vector-vector multiplication can be between a weight vector (vector 1) and multiple feature maps (vector 2, vector 3, etc.). Table 3

[0040] As described above, the first and second masks (mask 1 and mask 2) can be generated from memory based on the zero and near-zero element values ​​in the first and second vectors (vector 1 and vector 2), respectively. The first and second masks (mask 1 and mask 2), along with the given elements corresponding to the non-zero indicators in the first and second masks (mask 1 and mask 2), can be sent from memory to the accelerator. For example, given elements 3.65 and 56.3 of the first vector (vector 1), corresponding to the non-zero indicators in the first mask (mask 1), and given elements 62.3 and 8.52, corresponding to the non-zero indicators in the second mask (mask 2), can be sent to the accelerator along with the first and second masks (mask 1 and mask 2). The accelerator can generate a first computationally zero mask for the multiplication of the first and second vectors (vector 1 and vector 2). The accelerator can then perform multiplication on the given elements corresponding to the non-zero indicators in the first computationally zero mask to generate first result elements 2654.1, 54721, and 6854.7. Next, the memory can generate a third mask (mask 3) based on the zero and near-zero element values ​​in the third vector (vector 3). The third mask (mask 3) along with the given elements corresponding to the non-zero indicators in the third mask (mask 3) can be sent from the memory to the accelerator. For example, the given elements 2654.1, 154, 54721, and 6854.7 of the third vector (vector 3), corresponding to the non-zero indicators in the third mask (mask 3), can be sent to the accelerator along with the third mask (mask 3). The accelerator can generate a second computational zero mask for the multiplication of the first and third vectors (vector 1 and vector 3). Then, the accelerator can perform the multiplication of the given elements corresponding to the non-zero indicators in the second computational zero mask to generate the second result elements 9687.5, 574571, and 385920.

[0041] Some aspects of this application can advantageously improve performance when performing scalar, vector, and matrix operations using sparse vectors and matrices. Some aspects of this application advantageously reduce data transfer between the accelerator and memory, and reduce accelerator computation. Therefore, some aspects of this application can advantageously improve the performance of computing systems in applications such as artificial intelligence (AI) and machine learning (ML) that utilize sparse vectors and matrices.

[0042] The following examples relate to specific technical embodiments and indicate specific features, elements, or steps that may be used or otherwise combined in implementing such embodiments.

[0043] Example 1 includes a processor comprising: a processor sparse control unit configured to encapsulate a memory access request in a zero-jump sparse command; a command interface configured to transmit the zero-jump sparse command and receive given elements from a first set of elements and a second set of elements; the processor being configured to perform computation operations on each given element of the first set of elements and the second set of elements to generate a result element; and the command interface being further configured to send the result element.

[0044] Example 2 includes a processor according to Example 1, wherein: the first set of elements includes one of a scalar, vector, or matrix; and the second set of elements includes one of a scalar, vector, or matrix.

[0045] Example 3 includes a processor according to Example 1, wherein the computational operation includes one of addition, subtraction, multiplication, or division.

[0046] Example 4 includes a processor according to Example 1, wherein the processor sparse control unit includes: a zero-jump command unit configured to generate the zero-jump sparse command, the zero-jump sparse command including a zero-jump command indicator (TAGID), a computation opcode (OP), data types of first and second group elements (DataType), an address of the first group elements (Addr1), an address of the second group elements (Addr2), a target address of the computation result (Addrdst), and a data length (length).

[0047] Example 5 includes a memory comprising: a memory cell array; a command interface configured to receive a zero-jump sparse command; and a memory sparsity control unit configured to: access a first set of elements in the memory cell array, determine zero and near-zero element values ​​in the first set, and generate a first zero element mask based on the determined zero and near-zero elements of the first set; access a second set of elements in the memory cell array, determine zero and near-zero element values ​​in the second set, and generate a second zero element mask based on the determined zero and near-zero elements of the second set; generate a computed zero mask according to a computational operation of the zero-jump sparse command, the first zero element mask, and the second zero element mask; and write the zero values ​​based on the computed zero mask as part of the computation result to the memory cell array; the command interface is further configured to send a given element from the first and second sets of elements corresponding to a non-zero indicator in the computed zero mask and receive a result element; and the memory sparsity control unit is further configured to write the result element based on the computed zero mask as an additional part of the computation result of the first and second sets of elements to the memory cell array.

[0048] Example 6 includes the memory according to Example 5, and further includes: a zero-counting unit of the memory sparse control unit, the zero-counting unit being configured to determine zero and near-zero element values ​​based on a near-zero threshold and the data types of the first and second sets of elements.

[0049] Example 7 includes the memory according to Example 6, and further includes: a mask information processor of the memory sparse control unit, the mask information processor being configured to generate a first zero-element mask, wherein zero and near-zero elements in a first set of elements are indicated by a first state and non-zero elements in the first set of elements are indicated by a second state; and being configured to generate a second zero-element mask, wherein zero and near-zero elements in a second set of elements are indicated by the first state and non-zero elements in the second set of elements are indicated by the second state.

[0050] Example 8 includes a memory according to Example 5, wherein the zero-jump sparse command includes a zero-jump command indicator (TAGID), a computation opcode (OP), data types of the first and second groups of elements (DataType), the address of the first group of elements (Addr1), the address of the second group of elements (Addr2), the target address of the computation result (Addrdst), and the data length (length).

[0051] Example 9 includes the memory according to Example 5, and further includes: a write-back control unit of the memory sparse control unit, the write-back control unit being configured to: write a zero value based on the computed zero mask as part of the computation result to the memory cell array; and write the result element based on the computed zero mask as an additional part of the computation result of the first and second sets of elements to the memory cell array.

[0052] Example 10 includes a method for accelerating zero-jump sparse computation, comprising: generating a zero-jump sparse command encapsulating a memory access request; sending the zero-jump sparse command; receiving a given element from a first set of elements and a given element from a second set of elements in response to the zero-jump sparse command; performing a computation operation on each given element from the first and second sets of elements to generate a result element; and sending the result element.

[0053] Example 11 includes the zero-jump sparse computation acceleration method of Example 10, wherein the zero-jump sparse command includes a zero-jump command indicator (TAGID), a computation opcode (OP), the number of data type elements (DataType) of the first and second sets, the address of the first set of elements (Addr1), the address of the second set of elements (Addr2), the target address of the computation result (Addrdst), and the data length (length).

[0054] Example 12 includes the zero-jumping sparse computation acceleration method of Example 11, wherein the zero-jumping sparsity command further includes a zero-connection threshold (NZ_Th).

[0055] Example 13 includes the zero-jumping sparse computation acceleration method of Example 10, wherein the computation operation includes one of addition, subtraction, multiplication or division.

[0056] Example 14 includes the zero-jumping sparse computation acceleration method of Example 13, wherein: the first set of elements includes one of scalars, vectors, or matrices; and the second set of elements includes one of scalars, vectors, or matrices.

[0057] Example 15 includes a zero-skipping sparse computation storage method, comprising: receiving a zero-skipping sparse command for encapsulated memory access request; accessing a first group of elements in response to the zero-skipping sparse command, determining zero and near-zero elements in the first group of elements, and generating a first zero element mask based on the determined zero and near-zero element values ​​of the first group of elements; accessing a second group of elements in response to the zero-skipping sparse command, determining zero and near-zero elements in the second group of elements, and generating a second zero element mask based on the determined zero and near-zero element values ​​of the second group of elements; generating a computed zero mask according to a computation operation of the zero-skipping sparse command, the first zero element mask, and the second zero element mask; writing the zero values ​​based on the computed zero mask as corresponding portions of the computation results of the first and second groups of elements; transmitting a given element from the first and second groups of elements corresponding to a non-zero indicator in the computed zero mask; receiving a result element; and writing the result element as an additional portion of the computation results of the first and second groups of elements according to the computed zero mask.

[0058] Example 16 includes the zero-jumping sparse computation storage method of Example 15, wherein the computation operation includes one of addition, subtraction, multiplication or division.

[0059] Example 17 includes the zero-jumping sparse computation storage method of Example 15, wherein: the first set of elements includes one of scalars, vectors, or matrices; and the second set of elements includes one of scalars, vectors, or matrices.

[0060] Example 18 includes the zero-jump sparsity calculation storage method of Example 15, wherein the zero-jump sparsity command includes a zero-jump command indicator (TAGID), a calculation opcode (OP), the data types of the first and second groups of elements (DataType), the address of the first group of elements (Addr1), the address of the second group of elements (Addr2), the target address of the calculation result (Addrdst), and the data length (length).

[0061] Example 19 includes the zero-jumping sparse computation storage method of Example 15, wherein zero element values ​​and near-zero element values ​​are determined based on a near-zero threshold and the data types of the first and second sets of elements.

[0062] Example 20 includes the zero-jumping sparse computation memory method of Example 15, wherein zero and near-zero elements in the first group of elements are indicated by a first state in a first zero element mask, and non-zero elements in the first group of elements are indicated by a second state in a first zero element mask, zero and near-zero elements in the second group of elements are indicated by a first state in a second zero element mask, and non-zero elements in the second group of elements are indicated by a second state in a second zero element mask.

[0063] Example 21 includes a processor comprising: a processor sparse control unit configured to generate a first hop-zero sparse command encapsulating a memory access request; a command interface configured to send the first hop-zero sparse command, receive a first zero-element mask and a given element from a first set of elements, and receive a second zero-element mask and a given element from a second set of elements; the processor sparse control unit is further configured to generate a first computed zero mask based on a computational operation of the first hop-zero sparse command, the first zero-element mask, and the second zero-element mask; the processor is configured to perform a computational operation on each given element of the first set of elements and the second set of elements to generate a first result element; the command interface is further configured to send the first result element.

[0064] Example 22 includes the processor according to Example 21, further comprising: the processor sparse control unit further configured to generate a second hop-zero sparse command encapsulating a memory access request; the command interface further configured to send the second hop-zero sparse command and receive a third zero-element mask and a given element of a third set of elements; the processor sparse control unit further configured to generate a second computational zero mask based on computational operations of the second hop-zero sparse command, the first zero-element mask, and the third zero-element mask. The processor is configured to perform computational operations on each given element of the first set of elements and the third set of elements to generate a second result element; the command interface is further configured to send the second result element.

[0065] Example 23 includes a processor according to Example 22, wherein the computational operation includes one of addition, subtraction, multiplication or division.

[0066] Example 24 includes a memory comprising: a memory cell array; a command interface configured to receive a first zero-hopping sparse command; and a memory sparsity control unit configured to: access a first set of elements in the memory cell array, determine zero and near-zero element values ​​in the first set, and generate a first zero element mask based on the determined zero and near-zero elements of the first set; and access a second set of elements in the memory cell array, determine zero and near-zero element values ​​in the second set, and generate a second zero element mask based on the determined zero and near-zero elements of the second set; the command interface is further configured to send a given element from the first set of elements corresponding to a non-zero indicator in the first zero element mask, send the first zero element mask, send a given element from the second set of elements corresponding to a non-zero indicator in the second zero element mask, send the second zero element mask, and receive a first result element; the memory sparsity control unit is further configured to write the first result element as a calculation result of the first set of elements and the second set of elements into the memory cell array.

[0067] Example 25 includes the memory according to Example 24, further comprising: the command interface being further configured to receive a second-hop zero-sparse command; the memory sparsity control unit being further configured to: access a third set of elements in the memory cell array, determine zero and near-zero element values ​​in the third set of elements, and generate a third zero element mask based on the determined zero and near-zero elements of the third set of elements; the command interface being further configured to send a given element from the third set of elements corresponding to a non-zero indicator in the third zero element mask, send the third zero element mask, and receive a second result element; the memory sparsity control unit being further configured to write the second result element as the calculation result of the first and third sets of elements into the memory cell array.

[0068] Example 26 includes the memory according to Example 24, and further includes: a zero-counting unit of the memory sparsity control unit, the zero-counting unit being configured to determine zero element values ​​and near-zero element values ​​based on a near-zero threshold and the data types of the first, second, and third groups of elements.

[0069] Example 27 includes a method for accelerating zero-skip sparse computation, comprising: generating a first zero-skip sparse command encapsulating a memory access request; sending the first zero-skip sparse command; in response to the first zero-skip sparse command, receiving a given element from a first set of elements, a first zero element mask, a given element from a second set of elements, and a second zero element mask; generating a first computational zero mask based on computational operations from the first zero-skip sparse command, the first zero element mask, and the second zero element mask; performing computational operations on each given element from the first and second sets of elements based on the first computational mask to generate a first result element; and sending the first result element.

[0070] Example 28 includes the zero-skip sparse computation acceleration method of Example 27, further comprising: generating a second zero-skip sparse command encapsulating a memory access request; sending the second zero-skip sparse command; receiving a given element from a third set of elements and a third zero element mask in response to the second zero-skip sparse command; generating a second computational zero mask based on computational operations of the second zero-skip sparse command, the first zero element mask, and the third zero element mask; performing computational operations on each given element from the first set of elements and the third set of elements based on the second computational mask to generate a second result element; and sending the second result element.

[0071] Example 29 includes the zero-jumping sparse computation acceleration method of Example 27, wherein the computation operation includes one of addition, subtraction, multiplication or division.

[0072] Example 30 includes a zero-skipping sparse computation storage method, comprising: receiving a first zero-skipping sparse command for encapsulated memory access requests; accessing a first set of elements in response to the first zero-skipping command, determining zero and near-zero elements in the first set of elements, and generating a first zero element mask based on the determined zero and near-zero element values ​​of the first set of elements; accessing a second set of elements in response to the first zero-skipping command, determining zero and near-zero elements in the second set of elements, and generating a second zero element mask based on the determined zero and near-zero element values ​​of the second set of elements; sending a given element from the first set of elements corresponding to a non-zero indicator in the first zero element mask, the first zero element mask, and a given element from the second set of elements corresponding to a non-zero indicator in the second zero element mask; receiving a first result element and a first computed zero mask; and writing the first result element according to the first computed zero mask.

[0073] Example 31 includes the zero-jumping sparsity computation and storage method of Example 30, further comprising: receiving a second zero-jumping sparsity command encapsulating a storage access request; accessing a third set of elements in response to the second zero-jumping command, determining zero and near-zero elements in the third set of elements, and generating a third zero element mask based on the determined zero and near-zero element values ​​of the third set of elements; sending a given element corresponding to a non-zero indicator of the third zero element mask from the third set of elements and the third zero element mask; receiving a second result element and the second computed zero mask; and writing the second result element according to the second computed zero mask.

[0074] Example 32 includes the zero-jumping sparse computation storage method of Example 30, wherein the computation operation includes one of addition, subtraction, multiplication or division.

[0075] Example 33 includes a zero-jump sparse computing system comprising: a processor including: a processor sparse control unit configured to encapsulate a memory access request in a zero-jump sparse command; a command interface configured to send the zero-jump sparse command and receive given elements from a first set of elements and a second set of elements; the processor being configured to perform computation operations on each given element of the first set of elements and the second set of elements to generate a result element; and the command interface being further configured to send the result element. The computing system further includes a memory comprising: a memory cell array; a command interface configured to receive a zero-jumping sparse command; and a memory sparse control unit configured to: access a first set of elements in the memory cell array, determine zero and near-zero element values ​​in the first set, and generate a first zero element mask based on the determined zero and near-zero elements of the first set; access a second set of elements in the memory cell array, determine zero and near-zero element values ​​in the second set, and generate a second zero element mask based on the determined zero and near-zero elements of the second set; generate a computed zero mask according to the zero-jumping sparse command, the first zero element mask, and the second zero element mask; and write the zero values ​​based on the computed zero mask as part of the computation result to the memory cell array; the command interface is further configured to send a given element corresponding to a non-zero indicator in the computed zero mask from the first and second sets of elements and receive a result element; and the memory sparse control unit is further configured to write the result element as an additional part of the computation result of the first and second sets of elements based on the computed zero mask to the memory cell array.

[0076] Example 34 includes a zero-jumping sparse computing system, which includes: A processor, comprising: a processor sparse control unit configured to generate a first hop-zero sparse command for encapsulating a memory access request; a command interface configured to send the first hop-zero sparse command, receive a first zero-element mask and a given element from a first set of elements, and receive a second zero-element mask and a given element from a second set of elements; the processor sparse control unit is further configured to generate a first computational zero mask based on computational operations of the first hop-zero sparse command, the first zero-element mask, and the second zero-element mask; the processor is configured to perform computational operations on each given element of the first set of elements and the second set of elements to generate a first result element; the command interface is further configured to send the first result element. The processor includes a memory, which includes: a memory cell array; a command interface configured to receive a first zero-hopping sparse command; and a memory sparse control unit configured to: access a first group of elements in the memory cell array, determine the zero and near-zero element values ​​in the first group, and generate a first zero element mask based on the determined zero and near-zero elements of the first group; and access a second group of elements in the memory cell array, determine the zero and near-zero element values ​​in the second group, and generate a second zero element mask based on the determined zero and near-zero elements of the second group; the command interface is further configured to send a given element from the first group of elements corresponding to a non-zero indicator in the first zero element mask, send the first zero element mask, send a given element from the second group of elements corresponding to a non-zero indicator in the second zero element mask, send the second zero element mask, and receive a first result element; the memory sparse control unit is further configured to write the first result element as the calculation result of the first group and the second group of elements into the memory cell array.

[0077] Example 35 includes a zero-jump sparse computing system according to Example 34, wherein the processor further includes: a processor sparse control unit further configured to generate a second zero-jump sparse command encapsulating a memory access request; the command interface further configured to send the second zero-jump sparse command and receive a third zero-element mask and a given element of a third set of elements; the processor sparse control unit further configured to generate a second computational zero mask based on computational operations of the second zero-jump sparse command, the first zero-element mask, and the third zero-element mask; the processor is configured to perform computational operations on each given element of the first set of elements and the third set of elements to generate a second result element; the command interface is further configured to send the second result element. The memory further includes: the command interface is further configured to receive the second zero-jump sparse command; the memory sparse control unit is further configured to: access a third group of elements in the memory cell array, determine the zero and near-zero element values ​​in the third group of elements, and generate a third zero element mask based on the determined zero and near-zero elements of the third group of elements; the command interface is further configured to send a given element from the third group of elements corresponding to a non-zero indicator in the third zero element mask, send the third zero element mask, and receive the second result element; the memory sparse control unit is further configured to write the second result element as the calculation result of the first and third groups of elements into the memory cell array.

[0078] The foregoing description of specific embodiments of the application has been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the application to the precise forms disclosed, and it will be apparent that many modifications and variations can be made in accordance with the foregoing teachings. The embodiments were chosen and described in order to best explain the principles of the application and its practical application, thereby enabling others skilled in the art to best utilize the application and its various embodiments with various modifications suited to the intended particular use. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A processor, comprising: A processor sparse control unit, configured to encapsulate memory access requests in zero-jump sparse commands; A command interface configured to transmit a zero-jump sparse command to the memory according to the memory access request, wherein the zero-jump sparse command includes a zero-jump command identifier and a computation opcode; and to receive, in response to the zero-jump sparse command, a given element corresponding to a non-zero indicator in the computed zero mask from a first set of elements and a second set of elements, and the zero-jump command identifier, which identifies the zero-jump sparse command; the computed zero mask is generated by the memory based on the computation operation of the zero-jump sparse command, a first zero-element mask generated based on the zero element values ​​and near-zero element values ​​of the first set of elements, and a second zero-element mask generated based on the zero element values ​​and near-zero element values ​​of the second set of elements; The processor is configured to perform computation operations on a given element from the first and second groups of elements, based on the computation opcode of the zero-jumping sparse command, to generate a result element; and The command interface is also configured to send the result element and the zero-jump command identifier to the memory.

2. The processor according to claim 1, wherein: The first set of elements includes one of the following: scalar, vector, or matrix; and The second set of elements includes one of the following: scalar, vector, or matrix.

3. The processor according to claim 1, wherein the computational operation includes one of addition, subtraction, multiplication or division.

4. The processor as described in claim 1, characterized in that, The processor sparse control unit includes: A zero-jump command unit is configured to generate the zero-jump sparse command, which includes the data types of a first group of elements and a second group of elements, the addresses of the first group of elements and the second group of elements, the target address of the calculation result, and the data length; wherein, after sending the zero-jump sparse command to the memory, the memory is configured as follows: Access the first set of elements stored in the memory cell array based on the address of the first set of elements and the data length specified in the zero-jump sparse command; The zero and near-zero element values ​​in the first group of elements are determined based on the data type of the first group of elements specified in the zero-jumping sparse command. Generate the first zero-element mask, where zero-element values ​​and near-zero-element values ​​are represented by the first state, and non-zero-element values ​​are represented by the second state; Access the second set of elements stored in the memory cell array based on the address of the second set of elements and the data length specified in the zero-jump sparse command; The zero and near-zero element values ​​in the second group of elements are determined based on the data type of the second group of elements specified in the zero-jumping sparse command. Generate a second zero-element mask, where zero-element values ​​and near-zero-element values ​​are represented by the first state, and non-zero-element values ​​are represented by the second state; Generate a computational zero mask based on the computation opcode, the first zero element mask, and the second zero element mask specified in the jump-zero sparse command; For any indicator set to the first state in the zero mask computation, write the zero element value back to the corresponding element position in the first and second groups of elements, with the target address being the target address specified in the zero-jump sparse command, and Transmit the zero-jump command identifier and the given element from the first group of elements and the second group of elements to the processor.

5. A memory, comprising: Storage cell array; A command interface configured to receive zero-jumping sparse commands; A memory sparse control unit, wherein the memory sparse control unit is configured as follows: Access the first group of elements in the storage cell array, determine the zero element value and near-zero element value in the first group, and generate a first zero element mask based on the zero element value and near-zero element value of the first group of elements; Access the second group of elements in the storage cell array, determine the zero element value and near-zero element value in the second group of elements, and generate a second zero element mask based on the zero element value and near-zero element value of the second group of elements; A zero-compute mask is generated based on the computation operation of the zero-jump sparse command, the first zero-element mask, and the second zero-element mask; and The zero element value based on the calculated zero mask is written into the storage cell array as part of the calculation result; The command interface is also configured to send a given element from the first group of elements and the second group of elements corresponding to a non-zero indicator in the computed zero mask to the processor, and to receive a result element sent by the processor. and The memory sparse control unit is further configured to write the resulting element as an additional part of the calculation result of the first set of elements and the second set of elements based on the computation zero mask into the memory cell array.

6. The memory of claim 5, further comprising: The zero-counting unit of the memory sparse control unit is configured to determine zero and near-zero element values ​​based on a near-zero threshold and the data types of the first group of elements and the second group of elements.

7. The memory of claim 6, further comprising: The mask information processor of the memory sparse control unit is configured to generate the first zero element mask, wherein the zero element values ​​and near-zero element values ​​in the first group of elements are indicated by a first state, and the non-zero element values ​​in the first group of elements are indicated by a second state. And generate the second zero element mask, wherein the zero element values ​​and near-zero element values ​​in the second group of elements are indicated by a first state, and the non-zero element values ​​in the second group of elements are indicated by a second state.

8. The memory according to claim 5, wherein the zero-jump sparse command includes a zero-jump command identifier, a calculation opcode, the data types of the first group of elements and the second group of elements, the address of the first group of elements, the address of the second group of elements, the target address of the calculation result, and the data length.

9. The memory according to claim 5, further comprising: The write-back control unit of the memory sparse control unit is configured to: The zero element value based on the calculated zero mask is written into the storage cell array as part of the calculation result; and The resulting element is written into the storage cell array as an additional part of the calculation results of the first group of elements and the second group of elements based on the calculated zero mask.

10. A zero-jumping sparse computing accelerator method, comprising: Generate zero-jump sparse commands for packaged memory access requests; Send the zero-jump sparse command to the memory, wherein the zero-jump sparse command includes a zero-jump command identifier and a computation opcode; The receiving memory, in response to the zero-jump sparse command, sends a given element from a first set of elements and a given element from a second set of elements corresponding to a non-zero indicator in the zero-mask computation, as well as a zero-jump command identifier, wherein the zero-jump command identifier identifies the zero-jump sparse command; the zero-mask computation is generated by the memory based on the zero-jump sparse command computation operation, a first zero-element mask generated based on the zero-element values ​​and near-zero-element values ​​of the first set of elements, and a second zero-element mask generated based on the zero-element values ​​and near-zero-element values ​​of the second set of elements; Based on the computation opcode of the zero-jumping sparse command, a computation operation is performed on the given element corresponding to the non-zero indicator in the computation zero mask from the first and second groups of elements received, to generate the result element; and Send the result element and the zero-jump command identifier to memory.

11. The zero-jumping sparse computation accelerator method according to claim 10, wherein the zero-jumping sparse command includes the data type of the first group of elements and the second group of elements, the address of the first group of elements, the address of the second group of elements, the target address of the computation result, and the data length; wherein, The memory performs the following operation in response to the zero-jump sparse command: Access the first set of elements stored in the memory cell array based on the address of the first set of elements and the data length specified in the jump-zero sparse command; The zero and near-zero element values ​​in the first group of elements are determined based on the data type of the first group of elements specified in the zero-jumping sparse command. Generate the first zero-element mask, where zero-element values ​​and near-zero-element values ​​are represented by the first state; non-zero-element values ​​are represented by the second state; Access the second set of elements stored in the memory cell array based on the address of the second set of elements and the data length specified in the zero-jump sparse command; The zero and near-zero element values ​​in the second group of elements are determined based on the data type of the second group of elements specified in the zero-jumping sparse command. Generate a second zero-element mask, where zero-element values ​​and near-zero-element values ​​are represented by the first state, and non-zero-element values ​​are represented by the second state; Generate a computational zero mask based on the computation opcode, the first zero element mask, and the second zero element mask specified in the jump-zero sparse command; For any indicator set to the first state in the zero mask computation, write the zero element value back to the corresponding element position in the first and second groups of elements, with the target address being the target address specified in the zero-jump sparse command, and Transmit the zero-jump command identifier and the given element from the first group of elements and the second group of elements to the processor.

12. The zero-jumping sparse computing accelerator method of claim 11, wherein the zero-jumping sparse command further includes a near-zero threshold.

13. The zero-jumping sparse computing accelerator method of claim 10, wherein the computing operation includes one of addition, subtraction, multiplication or division operations.

14. The zero-jumping sparse computing accelerator method as described in claim 13, wherein: The first set of elements includes one of the following: scalar, vector, or matrix; and The second set of elements includes one of the following: scalar, vector, or matrix.

15. A zero-jumping sparse computing storage method, comprising: Receive zero-jump sparse commands for packaged memory access requests; In response to the zero-jumping sparse command, access the first group of elements, determine the zero and near-zero elements in the first group of elements, and generate a first zero element mask based on the determined zero and near-zero element values ​​of the first group of elements. In response to the zero-jumping sparse command, access the second group of elements, determine the zero element values ​​and near-zero element values ​​in the second group of elements, and generate a second zero element mask based on the zero element values ​​and near-zero element values ​​of the second group of elements; A zero-compute mask is generated based on the computation operation of the zero-jump sparse command, the first zero-element mask, and the second zero-element mask; Write the zero element value based on the calculated zero mask into the memory as the corresponding part of the calculation results of the first group of elements and the second group of elements. Send the given element corresponding to the non-zero indicator in the calculated zero mask from the first group of elements and the second group of elements to the processor, and receive the result element sent by the processor; and Based on the calculated zero mask, the resulting element is written into memory as an additional part of the calculation results of the first group of elements and the second group of elements.

16. The zero-jumping sparse computation storage method as described in claim 15, wherein the computation operation includes one of addition, subtraction, multiplication, or division operations.

17. The zero-jumping sparse computation and storage method as described in claim 15, wherein: The first set of elements includes one of the following: scalar, vector, or matrix; and The second set of elements includes one of the following: scalar, vector, or matrix.

18. The zero-jump sparse computation and storage method as described in claim 15, wherein the zero-jump sparse command includes a zero-jump command identifier, a computation opcode, the data types of the first group of elements and the second group of elements, the address of the first group of elements, the address of the second group of elements, the target address of the computation result, and the data length.

19. The zero-jumping sparse computation and storage method as described in claim 15, wherein determining the zero element value and the near-zero element value based on the near-zero threshold, the data type of the first group of elements, and the second group of elements includes: When an element value is less than a specified near-zero threshold, the values ​​in the first group of elements and the second group of elements are determined to be near-zero element values.

20. The zero-jumping sparse computation and storage method of claim 15, wherein the zero element values ​​and near-zero element values ​​in the first group of elements are indicated by a first state in the first zero element mask, and the non-zero element values ​​in the first group of elements are indicated by a second state in the first zero element mask, and the zero element values ​​and near-zero element values ​​in the second group of elements are indicated by a first state in the second zero element mask, and the non-zero element values ​​in the second group of elements are indicated by a second state in the second zero element mask.

21. A zero-jumping sparse computing system, comprising: The processor includes; A processor sparse control unit, configured to wrap memory access requests in zero-jump sparse commands; A processor command interface configured to send the zero-jump sparse command, the zero-jump sparse command including a zero-jump command identifier and a computation opcode, and to receive from memory a given element corresponding to a non-zero indicator in a computational zero mask from a first set of elements and a second set of elements, and the zero-jump command identifier, the zero-jump command identifier identifying the zero-jump sparse command. The processor is configured to perform a computation operation on a given element from the first set of elements and the second set of elements received from the memory to generate a result element; and The processor command interface is also configured to send the result element and the zero-jump command identifier to the memory; and The memory includes; Storage cell array; A memory command interface configured to receive the zero-jump sparse command; The memory sparsity control unit is configured as follows: Access the first group of elements in the storage cell array, determine the zero element value and near-zero element value in the first group, and generate a first zero element mask based on the zero element value and near-zero element value of the first group of elements; Access the second group of elements in the storage cell array, determine the zero element value and near-zero element value in the second group of elements, and generate a second zero element mask based on the zero element value and near-zero element value of the second group of elements; A zero-compute mask is generated based on the computation operation of the zero-jump sparse command, the first zero-element mask, and the second zero-element mask; and The zero element value based on the calculated zero mask is written into the storage cell array as part of the calculation result; The memory command interface is also configured to send a given element from the first set of elements and the second set of elements, corresponding to a non-zero indicator in the computed zero mask, to the processor and to receive a result element sent by the processor. and The memory sparse control unit is further configured to write the resulting element as an additional part of the calculation result of the first set of elements and the second set of elements based on the computation zero mask into the memory cell array.

Images