Storage particle, storage controller, storage chip, storage device and equipment

Abstract

The invention discloses a storage particle, a storage controller, a storage chip, a storage device and equipment, relates to the technical field of computers, and is used for reducing the requirement of a model parameter matrix for memory capacity in a large-scale calculation process and solving the problem that the large-scale calculation performance is influenced by the storage medium interface bus rate. The storage device comprises a storage controller used for sending a plurality of model parameter sub-matrixes included in a model parameter matrix to a plurality of storage particles so as to store the plurality of model parameter sub-matrixes in the plurality of storage particles; the plurality of storage particles are used for concurrently calculating intermediate calculation results of the input data matrix and at least one model parameter sub-matrix stored in the storage particles, and sending the intermediate calculation results obtained by the storage particles to the storage controller; and the storage controller is also used for calculating the calculation results of the input data matrix and the model parameter matrix according to the plurality of intermediate calculation results correspondingly obtained by the plurality of storage particles.

Description

Storage particles, storage controllers, storage chips, storage devices and equipment Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a storage particle, a storage controller, a storage chip, a storage device and equipment. Background Art

[0002] During deep learning inference calculations, it is often necessary to multiply the input data matrix with the model parameter matrix to obtain the result before performing subsequent inference operations. This multiplication of the input data matrix and the model parameter matrix is ​​typically implemented using a computing system.

[0003] Currently, computing systems typically include a host computer and memory. The host computer includes a processor, memory, and a memory interface, and the memory and the host computer are connected via the memory interface. When the computing system performs model calculations, the host computer reads the model parameter file required for the calculation from the memory and writes it to the host computer's memory. The processor then reads the model parameter file from the memory and performs calculations to obtain the final calculation results.

[0004] However, as computing scale continues to grow, the model parameter files required for calculations are becoming larger and larger, and the demand for memory capacity in computing systems is also increasing. However, memory is expensive, and expanding memory in existing computing systems is relatively difficult. Therefore, the memory of computing systems has become a bottleneck for supporting large-scale computing.

[0005] Summary of the Invention

[0006] This application provides a storage particle, storage controller, storage chip, storage device, and equipment for reducing the memory capacity requirements of model parameter matrices during large-scale computing, while also resolving the problem of storage medium interface bus rates being unable to meet the bandwidth requirements of large-scale computing, thereby improving computing performance. To achieve the above objectives, the embodiments of this application employ the following technical solutions:

[0007] In a first aspect, a storage particle is provided, which can be an unpackaged storage die or a storage chip. The storage particle includes: one or more storage arrays for receiving and storing at least one model parameter submatrix output by a storage controller, the at least one model parameter submatrix being a model parameter submatrix among a plurality of model parameter submatrices included in a model parameter matrix, and the other model parameter submatrices in the plurality of model parameter submatrices except the at least one model parameter submatrix being stored in other storage particles arranged with the storage particle array; a calculation circuit for calculating an intermediate calculation result of an input data matrix and the at least one model parameter submatrix, the intermediate calculation result being used to calculate a calculation result of the input data matrix and the model parameter matrix. The multiple model parameter submatrices corresponding to the model parameter matrix are dispersedly stored in a plurality of different storage particles (for example, dispersedly stored in a plurality of different storage particles according to the concurrency of the storage particles), each storage particle can calculate an intermediate calculation result of the input data matrix and the at least one model parameter submatrix stored therein by the calculation circuit, and the multiple intermediate calculation results corresponding to the multiple storage particles can be used to calculate the calculation result of the input data matrix and the model parameter matrix.

[0008] In the above technical solution, the storage device stores at least one model parameter submatrix included in the model parameter matrix via a storage array, and uses a computation circuit to calculate the intermediate calculation results between the input data matrix and the at least one model parameter submatrix. In other words, by providing a computation circuit within the storage device, the storage device supports local memory computing. In this case, the multiple storage devices required for computing the model parameter matrix are distributed across multiple storage devices. The calculation of the input data matrix and the model parameter matrix is ​​performed through computations performed by multiple storage devices, thereby improving the performance of the storage device in performing matrix calculations. This solution can reduce the memory capacity requirements for the model parameter matrix during large-scale computations, while also resolving the problem of the storage medium interface bus rate failing to meet the bandwidth requirements of large-scale computations, thereby improving computational performance.

[0009] In one possible implementation of the first aspect, the storage particle further includes a control circuit configured to output the intermediate calculation result, such as controlling the calculation circuit to output the intermediate calculation result. In this possible implementation, the control circuit of the storage particle can output the intermediate calculation result to a storage controller, so that the storage controller calculates the calculation result of the input data matrix and the model parameter matrix based on the multiple intermediate calculation results corresponding to the multiple storage particles, thereby improving computing performance.

[0010] In a possible implementation of the first aspect, the control circuit is further configured to: receive a storage request, the storage request including the at least one model parameter submatrix; and store the at least one model parameter submatrix in the storage array according to the storage request. In this possible implementation, the control circuit of each storage cell may store the at least one model parameter submatrix included in the storage request in the storage array, so that multiple model parameter submatrices of the model parameter matrix can be stored in different storage cells according to the concurrency of the storage cells.

[0011] In a possible implementation of the first aspect, the control circuit is further configured to receive address indication information and, based on the address indication information, read the at least one model parameter submatrix from the storage array to the calculation circuit. For example, the address indication information includes the physical address and size of each model parameter submatrix. In this possible implementation, the control circuit reads the at least one model parameter submatrix from the storage array to the calculation circuit based on the address indication information, so that the calculation circuit concurrently calculates intermediate calculation results of the input data matrix and the at least one model parameter submatrix, thereby improving matrix calculation performance.

[0012] In one possible implementation of the first aspect, the control circuit is further configured to receive a computation request, the computation request including the input data matrix and the address indication information. In this possible implementation, by including the input data matrix and the address indication information in the computation request, the number of interactions between the storage device and the storage controller can be reduced, thereby improving interaction efficiency.

[0013] In one possible implementation of the first aspect, the storage cell further includes: an input cache for caching the input data matrix; and / or an output cache for caching the intermediate calculation results; and / or a read cache for caching the at least one model parameter submatrix. In these possible implementations, the efficiency of the computing circuit in the storage cell in reading the input data matrix and the at least one model parameter submatrix, as well as the efficiency of outputting the intermediate calculation results, can be improved, thereby improving the computational efficiency of the computing circuit.

[0014] In one possible implementation of the first aspect, the at least one model parameter submatrix is ​​obtained by dividing the model parameter matrix according to the concurrency of the storage granule. Optionally, the storage granule includes multiple storage planes, each of which stores a model parameter submatrix. In this possible implementation, by dividing the model parameter matrix required for calculation according to the granularity of storage granule read concurrency and storing it on multiple storage granules, the multiple storage granules can concurrently calculate the calculation results of the input data matrix and the model parameter matrix, thereby improving the performance of matrix calculation.

[0015] In a possible implementation of the first aspect, the control circuit is further configured to send storage status information to the storage controller, where the storage status information indicates the concurrency of the storage granule. In this possible implementation, sending the storage status information to the storage controller enables the storage controller to determine the concurrency of the storage granule based on the storage status information.

[0016] In a second aspect, a memory chip is provided, comprising a plurality of memory cells, the plurality of memory cells being configured to perform calculations on an input data matrix and a model parameter matrix, wherein each memory cell is a memory cell provided by the first aspect or any possible implementation of the first aspect; wherein the calculation results of the input data matrix and the model parameter matrix are obtained based on a plurality of intermediate calculation results corresponding to the plurality of memory cells. The plurality of memory cells may be arranged in an array.

[0017] According to a third aspect, a storage controller is provided, which is used to couple with a plurality of storage particles; wherein the plurality of storage particles store a plurality of model parameter sub-matrices included in a model parameter matrix, and each storage particle stores at least one model parameter sub-matrix; the storage controller is used to send the input data matrix to the plurality of storage particles respectively; the storage controller is also used to obtain a plurality of intermediate calculation results, and calculate the calculation results of the input data matrix and the model parameter matrix based on the plurality of intermediate calculation results; wherein the plurality of intermediate calculation results are obtained by the plurality of storage particles respectively calculating the input data matrix and the at least one model parameter sub-matrix stored therein.

[0018] In a possible implementation of the third aspect, the storage controller is further configured to: send the multiple model parameter sub-matrices to the multiple storage particles according to the concurrency of the multiple storage particles, so as to store the multiple model parameter sub-matrices in the multiple storage particles.

[0019] In a possible implementation manner of the third aspect, the storage controller is further configured to: divide the model parameter matrix into the multiple model parameter sub-matrices.

[0020] In a possible implementation manner of the third aspect, the number of the multiple model parameter sub-matrices is related to the number of the multiple storage particles and the number of the multiple storage planes in each storage particle.

[0021] In a possible implementation manner of the third aspect, the storage controller is further configured to: obtain storage status information of each storage particle in the multiple storage particles, and determine the concurrency of the multiple storage particles according to the storage status information of the multiple storage particles.

[0022] In a possible implementation manner of the third aspect, the storage controller is further configured to: send storage requests to the multiple storage particles respectively, where the storage request corresponding to each storage particle includes at least one model parameter sub-matrix from the multiple model parameter sub-matrices.

[0023] In a possible implementation manner of the third aspect, the storage controller is further used to: send address indication information to the multiple storage particles respectively, and the address indication information corresponding to each storage particle is used to read the at least one model parameter submatrix stored in the storage particle.

[0024] In a possible implementation of the third aspect, the storage controller is further configured to: send computing requests to the multiple storage particles respectively, where the computing request corresponding to each storage particle includes the input data matrix and address indication information corresponding to the storage particle.

[0025] In a fourth aspect, a storage device is provided, which includes: a plurality of storage particles provided by the first aspect or any possible implementation of the first aspect or a storage chip provided by the second aspect, and a storage controller provided by the third aspect or any possible implementation of the third aspect.

[0026] In a fifth aspect, an electronic device is provided, which includes a processor and a storage device as provided in the fourth aspect.

[0027] It can be understood that the beneficial effects that can be achieved by any of the memory chips, memory controllers, memory devices and electronic devices provided above can correspond to the beneficial effects of the memory particles provided above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] FIG1 is a schematic diagram of the structure of a computing system;

[0029] FIG2 is a schematic diagram of the structure of another computing system;

[0030] FIG3 is a schematic structural diagram of a storage device provided in an embodiment of the present application;

[0031] FIG4 is a schematic diagram of storing a model parameter matrix according to an embodiment of the present application;

[0032] FIG5 is a schematic diagram of a process for storing a model parameter matrix according to an embodiment of the present application;

[0033] FIG6 is a schematic diagram of a process for calculating an input data matrix and a model parameter matrix according to an embodiment of the present application;

[0034] FIG7 is a schematic structural diagram of a storage particle provided in an embodiment of the present application;

[0035] FIG8 is a schematic structural diagram of a memory chip provided in an embodiment of the present application;

[0036] FIG9 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following sections discuss the making and use of various embodiments in detail. However, it should be understood that many applicable inventive concepts provided herein can be implemented in a variety of specific contexts. The specific embodiments discussed herein are merely illustrative of specific ways to implement and use the present application and technology and do not limit the scope of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art.

[0038] Various circuits or other components may be described or referred to as being "configured to" perform one or more tasks. In this case, "configured to" is used to imply structure by indicating that the circuit / component includes structure (e.g., circuitry) that performs the one or more tasks during operation. Thus, even when a specified circuit / component is not currently operational (e.g., not turned on), the circuit / component may be referred to as being configured to perform the task. Circuits / components used with the phrase "configured to" include hardware, such as circuitry that performs an operation, etc.

[0039] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, a, b and c; where a, b and c can be single or multiple.

[0040] The embodiments of this application use terms such as "first" and "second" to distinguish objects with similar names, functions, or effects. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or order of execution. The term "coupled" is used to indicate an electrical connection, including direct connection via wires or connectors or indirect connection via other devices. Therefore, "coupling" should be considered a broadly defined electronic communication connection.

[0041] In this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0042] Before introducing the embodiments of the present application, the application scenarios involved in the present application are first introduced and explained.

[0043] During deep learning inference calculations, it is often necessary to multiply the input data matrix with the model parameter matrix. The resulting multiplication is then used to perform subsequent inference operations. With the rise of various neural network models (such as large language models, image processing models, and image rendering models), the size of the model parameter matrix has gradually increased, and the corresponding computational effort has also exploded. This multiplication of the input data matrix and the model parameter matrix is ​​typically performed using a computing system, resulting in an increasing amount of memory required to store the model parameter matrix within the computing system.

[0044] As shown in Figure 1, a computing system generally includes a host 10 and a memory 20. The host 10 includes a processor 11, a memory 12, and a memory interface 13. The memory 20 and the host 10 are connected via the memory interface 13. When the computing system performs a model calculation, the host 10 reads the model parameter matrix required for the calculation from the memory 20 and writes it to the memory 12 of the host 10. The processor 11 reads the model parameter matrix from the memory 12 and calculates to obtain the final calculation result.

[0045] However, as computing scale continues to grow, the model parameter matrices required for calculations are becoming larger and larger, and the demand for memory capacity in computing systems is also increasing. However, memory is expensive, and expanding memory in existing computing systems is relatively difficult. Therefore, the memory of computing systems has become a bottleneck for supporting large-scale computing.

[0046] In the related art, the computing function of the host is embedded into the solid state drive (SSD) storage controller, thereby reducing the demand for memory capacity for large-scale computing. In conjunction with Figure 1, as shown in Figure 2, the memory 20 includes an SSD storage controller 21, a NAND (or nand) storage medium 22, a storage medium interface 23 and a memory interface 24. Among them, the memory interface 13 of the host 10 is connected to the memory interface 24 of the memory 20; the SSD storage controller 21 is connected to the NAND storage medium 22 through the storage medium interface 23, and the SSD storage controller 21 is also connected to the memory interface 24. In this computing system, the SSD storage controller 21 has matrix computing capabilities, and can read the model parameter matrix stored in the NAND storage medium 22 into the SSD storage controller 21 for calculation according to the computing operation issued by the host 10.

[0047] In the above scheme, the SSD storage controller 21 and the NAND storage medium 22 are connected via a storage medium interface 23. The storage medium interface 23 generally uses a standard interface protocol, such as the Open NAND Flash Interface (ONFI) protocol. These interface protocols have an upper limit on the data transmission rate. As the computational scale increases, the transmission rate of the storage medium interface 23 becomes a bottleneck for reading the model parameter matrix, thereby affecting the performance of the overall computation. In addition, the higher the rate required by the storage medium interface 23, the greater the transient current generated during the data transmission process.

[0048] Based on this, the embodiments of the present application provide a storage particle, a storage controller, a storage chip, a storage device, and an apparatus. In the solution provided by the embodiments of the present application, the computing circuit is embedded in the storage particle, and the model parameter matrix required for the computing process is divided and stored according to the concurrency of multiple storage particles. The parallel computing method of multiple storage particles is used to provide high-performance computing capabilities for the computing system. Therefore, the embodiments of the present application can reduce the memory capacity requirements of the model parameter matrix during large-scale computing, and also solve the problem that the storage medium interface bus rate cannot meet the bandwidth required for large-scale computing.

[0049] FIG3 is a schematic diagram of the structure of a storage device provided in an embodiment of the present application. The storage device can be used to implement the calculation of an input data matrix and a model parameter matrix based on a computing operation issued by a host. The storage device includes a storage controller 30 and a plurality of storage particles 40 coupled to the storage controller 30. The plurality of storage particles 40 may include storage particles 41 to 4n, where n is an integer greater than 1. Optionally, the plurality of storage particles 40 may be arranged in an array.

[0050] The storage device may be a memory or a storage device having a memory, and the memory may be a non-volatile memory or a volatile memory. Exemplarily, the memory may be any one of the following: an embedded multimedia card (eMMC), a universal flash storage (UFS), a NAND flash memory, or a non-flash (norflash), a solid state drive (SSD), a SAS SSD, a SATA SSD, a non-volatile memory express (NVMe) SSD, a memory, a static random access memory (SRAM), a dynamic random access memory (DRAM), or a disk, etc., and the embodiments of the present application do not impose specific restrictions on this.

[0051] In addition, the plurality of storage particles 40 may be a plurality of unpackaged storage dies or a plurality of storage chips, each of which may be integrated with one or more storage dies. For example, the plurality of storage particles 40 may be a plurality of NAND storage dies.

[0052] Furthermore, the model parameter matrix (or model parameter file) can be a model parameter matrix corresponding to any neural network model. For example, the neural network model can be a large language model, or an image processing (e.g., image rendering, image recognition) model, or other neural network models. The calculation between the input data matrix and the model parameter matrix can be called a matrix calculation. When the method is applied to the field of image processing, the matrix calculation can also be called an image calculation.

[0053] In the storage device, the storage controller 30 may be configured to send the multiple model parameter sub-matrices included in the model parameter matrix to the multiple storage granules 40, so as to store the multiple model parameter sub-matrices in the multiple storage granules 40. The multiple model parameter sub-matrices may be pre-divided or divided by the storage controller 30. For example, the storage controller 30 may divide the model parameter matrix into multiple model parameter sub-matrices and store the multiple model parameter sub-matrices in the multiple storage granules 40. Each storage granule in the multiple storage granules 40 may store at least one model parameter sub-matrix from the multiple model parameter sub-matrices.

[0054] In one possible embodiment, the storage controller 30 may be configured to send the multiple model parameter sub-matrices to the multiple storage granules 40 according to the concurrency of the multiple storage granules 40. In one implementation, the storage controller 30 may be configured to divide the model parameter matrix into multiple model parameter sub-matrices according to the concurrency of the multiple storage granules 40, and send a storage request to each of the multiple storage granules 40 according to the concurrency of the multiple storage granules 40. The storage request sent to each storage granule (or the storage request corresponding to each storage granule) includes at least one model parameter sub-matrix from the multiple model parameter sub-matrices. Accordingly, each storage granule in the multiple storage granules 40 receives the storage request and stores the at least one model parameter sub-matrix included in the storage request. The model parameter sub-matrix may also be referred to as a slice or a shard, and thus the model parameter matrix may include multiple slices or multiple shards. The concurrency mentioned above may refer to the total number of storage cells in the multiple storage granules 40 connected to the storage controller 30 that can be accessed in parallel. The storage cell may refer to a storage cell of a certain storage granularity, and the storage cell may also be referred to as a concurrency unit.

[0055] Optionally, the storage controller 30 is further configured to obtain storage status information of each storage particle in the plurality of storage particles, and determine the concurrency of the plurality of storage particles based on the storage status information of the plurality of storage particles. The storage status information of each storage particle can be used to indicate the status of the plurality of concurrent units of the storage particle. When the status of a concurrent unit is busy, the concurrent unit cannot be used for concurrent operations. When the status of a concurrent unit is idle, the concurrent unit can be used for concurrent operations. The storage controller 30 can actively obtain the storage status information of each storage particle, or each storage particle can send the corresponding storage status information to the storage controller 30.

[0056] Optionally, the number of the multiple model parameter sub-matrices is related to the number of the multiple storage particles 40 and the number of multiple storage planes in each storage particle. In one possible implementation, the multiple storage particles 40 include n storage particles, each storage particle includes m storage planes, and if the size of the model parameter matrix is ​​A rows and B columns and the size of each model parameter is s bytes, then the storage controller 30 can divide the model parameter matrix into y model parameter sub-matrices, each model parameter sub-matrix has a size of x, and then y and x satisfy the following formulas (1) and (2). y = m × n (1) x = A × B × s / y (2)

[0057] For example, the size of the model parameter matrix is ​​4096×4096, the size of each model parameter is 4 bytes, the multiple storage particles 40 include 8 storage particles, and each storage particle includes 4 storage surfaces, that is, A=4096, B=4096, s=4, n=8, m=4. According to the above description, the number of the multiple model parameter sub-rectangles y=32, and the size of each model parameter sub-rectangle x=2097152 bytes. Further, as shown in Figure 4, if the 8 storage particles are represented as particles 41 to 48, the 32 model parameter sub-rectangles are represented as slices 1 to slice 32, and the 4 storage surfaces are represented as surfaces 1 to 4, then the storage controller 30 can be used to: store slices 1 to slice 4 in the 4 storage surfaces of particle 1 respectively; store slices 5 to slice 8 in the 4 storage surfaces of particle 2 respectively; store slices 9 to slice 12 in the 4 storage surfaces of particle 3 respectively; store slices 13 to slice 16 in the 4 storage surfaces of particle 4 respectively; store slices 17 to slice 20 in the 4 storage surfaces of particle 5 respectively; store slices 21 to slice 24 in the 4 storage surfaces of particle 6 respectively; store slices 25 to slice 28 in the 4 storage surfaces of particle 7 respectively; and store slices 29 to slice 32 in the 4 storage surfaces of particle 8 respectively.

[0058] It will be appreciated that, while the above description uses the example of each storage granule including multiple concurrent storage planes, in actual applications, each storage granule may also include multiple concurrent storage areas of other granularities. In this case, the number of the multiple model parameter submatrices may also be related to the number of the multiple storage granules 40 and the number of storage areas of the other granularities in each storage granule. Exemplarily, the other granularities may include channels, banks, super blocks, blocks, sub-blocks, or pages, etc., and the present embodiment does not impose any specific limitations on this.

[0059] Exemplarily, as shown in FIG5 , the process of the storage controller 30 storing the model parameter matrix in the multiple storage particles may include: S11. Receiving a model parameter matrix save request from a host (or processor), the save request including the size (A and B) of the model parameter matrix and the size s of each model parameter; S12. Calculating the number y of multiple slices based on the number n of the multiple storage particles 40 and the number m of storage surfaces included in each storage particle; S13. Calculating the size x of each slice based on the number y; S14. Determining the physical address at which each slice is saved to the multiple storage particles 40; S15. Saving the multiple slices in different storage surfaces of different storage particles based on the above-mentioned physical addresses; S16. After the multiple storage particles 40 are saved, determining the mapping relationship between the logical address and the physical address saved for each slice, as well as the size of the slice; S17. Sending a response message to the host indicating that the model parameter matrix save is complete to notify the host that the model parameter matrix save is complete.

[0060] In the storage device, the storage controller 30 is also used to: send input data matrices and address indication information to the multiple storage particles 40 respectively, and the address indication information sent to each storage particle (or called the address indication information corresponding to each storage particle) is used to read at least one model parameter submatrix stored in the storage particle.

[0061] In which, when the storage controller 30 sends the input data matrix and address indication information to each storage particle, it can send the input data matrix and the address indication information to the storage particle through the same message, or it can send the input data matrix and the address indication information to the storage particle through different messages. The embodiment of the present application does not specifically limit this.

[0062] In a possible embodiment, the storage controller 30 may send a computing request to each storage particle in the multiple storage particles 40 respectively. The computing request sent to each storage particle (or the computing request corresponding to each storage particle) may include an input data matrix and address indication information corresponding to the storage particle.

[0063] The input data matrix may be sent by the host to the storage controller 30. In one possible example, the host may send a calculation request Req0 to the storage controller 30, where the calculation request Req0 includes the logical address information of the input data matrix and the model parameter matrix, and the calculation request Req0 is used to request the execution of the calculation of the input data matrix and the model parameter matrix. Accordingly, as shown in FIG6 , the process of the storage controller 30 sending the input data matrix and address indication information to each storage particle may include: S21. receiving the calculation request Req0; S22. reading the input data matrix in the calculation request Req0; S23. reading the logical address information of the model parameter matrix; S24. determining the physical address and slice size of each slice based on the logical address information; and S25. sending calculation requests Req1 to Reqn to the multiple storage particles 40, each calculation request including the input data matrix and corresponding address indication information, where the address indication information may include the corresponding physical address and slice size.

[0064] In the storage device, each of the plurality of storage cells 40 may include a storage array 1 and a computing circuit 2. Optionally, the computing circuit 2 may be a matrix computing circuit. In practical applications, the computing circuit 2 may also be other mathematical computing circuits, such as a vector computing circuit, an array computing circuit, or a trigonometric function computing circuit, and the present embodiment does not impose any specific limitations thereto.

[0065] The storage array 1 can be used to store at least one model parameter submatrix; the calculation circuit 2 can be used to calculate an intermediate calculation result between the input data matrix and the at least one model parameter submatrix, and the intermediate calculation result is used to calculate the calculation result of the input data matrix and the model parameter matrix. The number of storage arrays 1 included in each storage cell can be one or more, and this embodiment of the application does not impose specific limitations on this.

[0066] Accordingly, when each storage particle calculates an intermediate calculation result, the multiple storage particles 40 can correspondingly calculate multiple intermediate calculation results. The multiple storage particles 40 can also be used to send their respective intermediate calculation results to the storage controller 30. In this way, the storage controller 30 can obtain the multiple intermediate calculation results and calculate the calculation results of the input data matrix and the model parameter matrix based on the multiple calculation results. Afterwards, the storage controller 30 can also send the calculation results of the input data matrix and the model parameter matrix to the host.

[0067] Optionally, as shown in FIG7 , each storage cell may further include a control circuit 3. This control circuit 3 may be configured to output intermediate calculation results, such as controlling calculation circuit 2 to output calculation results. Furthermore, as shown in FIG7 , each storage cell may further include an input buffer 4 for caching the input data matrix; and / or an output buffer 5 for caching the intermediate calculation results; and / or a read buffer 6 for caching the at least one model parameter submatrix. FIG7 illustrates this using storage cell 41 as an example.

[0068] In one possible embodiment, any one of the plurality of storage cells 40 may be further configured to receive and store at least one model parameter submatrix. Exemplarily, the control circuit 3 is further configured to: receive a storage request from the storage controller 30, the storage request including at least one model parameter submatrix; and store the at least one model parameter submatrix in the storage array 1 according to the storage request.

[0069] In another possible embodiment, for any storage granule among the plurality of storage granules 40, the storage granule may further be configured to receive an input data matrix and address indication information, the address indication information being used to read the at least one model parameter submatrix. Exemplarily, the control circuit 3 may further be configured to: receive a calculation request from the storage controller 30, the calculation request including the input data matrix and the address indication information. For example, the address indication information may include the physical address and corresponding size information of each sub-rectangle in the at least one model parameter submatrix; and read the at least one model parameter submatrix from the storage array 1 to the calculation circuit 2 according to the address indication information. The calculation circuit 2 may be configured to: calculate an intermediate calculation result of the input data matrix and the at least one model parameter submatrix. Optionally, the control circuit 3 may read the at least one model parameter submatrix according to the maximum read concurrency granularity of the storage granule.

[0070] Optionally, when the control circuit 3 receives the calculation request, the control circuit 3 may also cache the input data matrix in the input cache 4; when the control circuit 3 reads the at least one model parameter submatrix from the storage array 1, the control circuit 3 may also cache the at least one model parameter submatrix in the read cache 6; when performing the calculation, the calculation circuit 2 may obtain the input data matrix from the input cache 4 and obtain the at least one model parameter submatrix from the read cache; the calculation circuit 2 may also cache the intermediate calculation result in the output cache 5 when obtaining the intermediate calculation result.

[0071] It can be understood that the above-mentioned input cache 4, output cache 5 and read cache 6 can also be integrated into the computing circuit 2, so that the control circuit 3 can transmit the received input data matrix and the read at least one model parameter sub-matrix to the computing circuit 2, and the computing circuit 2 can also locally cache the intermediate calculation results obtained. The embodiment of the present application does not impose specific restrictions on this.

[0072] In a possible example, as shown in Figure 6, for any storage particle among the multiple storage particles 40, the storage particle can perform the following steps: S26. The control circuit 3 receives the calculation request Reqi of the storage controller 30, where the value range of i is 1 to n; S27. The control circuit 3 reads the input data matrix in the calculation request and caches it in the input cache 4; S28. The control circuit 3 reads the address indication information in the calculation request, and reads the at least one model parameter submatrix from the storage array 1 to the calculation circuit 2 according to the address indication information. The address indication information may include the physical address and slice size corresponding to the at least one model parameter submatrix; S29. The calculation circuit 2 calculates the input data matrix and the at least one model parameter submatrix; S30. When the calculation is completed, the calculation circuit 2 caches the intermediate calculation results in the output cache 5. The storage controller 30 may then perform the following steps: S31. Read the intermediate calculation results in the output buffers 5 of the plurality of storage granules 40 to obtain a plurality of intermediate calculation results; S32. Calculate the input data matrix and the model parameter matrix based on the plurality of intermediate calculation results; S33. The storage controller 30 notifies the host of the completion of the calculation and sends the calculation results to the host. In this way, the host can obtain the calculation results.

[0073] Optionally, for any storage particle, when the storage particle sends corresponding storage status information to the storage controller 30 , the control unit 3 may specifically send the corresponding storage status information to the storage controller 30 .

[0074] In an embodiment of the present application, the storage controller 30 can store multiple model parameter sub-matrices included in the model parameter matrix in the multiple storage particles 40 according to the concurrency of the multiple storage particles 40. The multiple storage particles 40 can concurrently calculate the intermediate calculation results of the input data matrix and at least one model parameter sub-matrix stored in each of them, and send the intermediate calculation results obtained by each of them to the storage controller 30. In this way, the storage controller 30 can calculate the calculation results of the input data matrix and the model parameter matrix based on the multiple intermediate calculation results obtained by the multiple storage particles 40. Compared with the solution shown in Figure 2 above, the embodiment of the present application can support the function of near-memory calculation by setting a calculation circuit in the storage particle without changing the structure of the storage device, and store the model parameter matrix required for calculation on the storage particle according to the granularity of the storage particle read concurrency, thereby improving the performance of the storage device for matrix calculation. This solution can reduce the demand for memory capacity of the model parameter matrix in large-scale calculations, and also solve the problem that the storage medium interface bus rate cannot meet the bandwidth required for large-scale calculations.

[0075] Based on this, an embodiment of the present application further provides a storage particle, the structure of which may be as shown in the storage particle 41 in FIG3 , or the storage particle 41 as shown in FIG7 . The storage particle may include: a storage array 1, for receiving and storing at least one model parameter submatrix output by a storage controller, wherein the at least one model parameter submatrix is ​​one of a plurality of model parameter submatrices included in a model parameter matrix, and other model parameter submatrices in the plurality of model parameter submatrices except the at least one model parameter submatrix are stored in other storage particles arranged with the storage particle array; a calculation circuit 2, for calculating an intermediate calculation result of an input data matrix and the at least one model parameter submatrix; wherein the intermediate calculation result is used to calculate a calculation result of the input data matrix and the model parameter matrix.

[0076] Optionally, the at least one model parameter submatrix is ​​obtained by dividing the model parameter matrix according to the concurrency of the storage particle. Exemplarily, the storage particle includes multiple storage planes, and each storage plane of the multiple storage planes stores a model parameter submatrix.

[0077] Furthermore, the storage particle further includes: a control circuit 3, configured to output the intermediate calculation result, such as controlling the calculation circuit 2 to output the intermediate calculation result.

[0078] In a possible implementation, the control circuit 3 is further configured to: receive a storage request, the storage request including the at least one model parameter sub-matrix; and store the at least one model parameter sub-matrix in the storage array 1 according to the storage request.

[0079] In another possible implementation, the control circuit 3 is further configured to: receive address indication information, and read the at least one model parameter sub-matrix from the storage array 1 to the calculation circuit 2 according to the address indication information.

[0080] Optionally, the control circuit 3 is further configured to receive a calculation request, where the calculation request includes an input data matrix and address indication information.

[0081] Furthermore, the storage particle also includes: an input cache 4 for caching the input data matrix; and / or, an output cache 5 for caching the intermediate calculation results; and / or, a read cache 6 for caching the at least one model parameter submatrix.

[0082] In another embodiment of the present application, a memory chip is provided, as shown in FIG8 . The memory chip includes a plurality of memory particles, any one of which may be the memory particle provided above. Optionally, the plurality of memory particles are packaged in the same packaging structure, and the plurality of memory particles may be a plurality of memory particles arranged in an array.

[0083] In another aspect of the present application, a storage controller is provided, which is used to couple with multiple storage particles; wherein the multiple storage particles store multiple model parameter sub-matrices included in the model parameter matrix, and each storage particle stores at least one model parameter sub-matrix.

[0084] The storage controller is configured to: send the input data matrix to each of the plurality of storage cells; obtain a plurality of intermediate calculation results; and calculate calculation results of the input data matrix and the model parameter matrix based on the plurality of intermediate calculation results. The plurality of intermediate calculation results are obtained by the plurality of storage cells respectively calculating the input data matrix and at least one model parameter submatrix stored in each of the plurality of storage cells.

[0085] Optionally, the storage controller is further configured to store the multiple model parameter sub-matrices in the multiple storage granules according to the concurrency of the multiple storage granules. Furthermore, the storage controller is further configured to divide the model parameter matrix into multiple model parameter sub-matrices. Exemplarily, the number of the multiple model parameter sub-matrices is related to the number of the multiple storage granules and the number of storage planes in each storage granule.

[0086] In a possible implementation, the storage controller is further configured to: send storage requests to the multiple storage particles respectively, where the storage request corresponding to each storage particle includes at least one model parameter sub-matrix from the multiple model parameter sub-matrices.

[0087] Optionally, the storage controller is further configured to: send address indication information to the multiple storage particles respectively, where the address indication information corresponding to each storage particle is used to read at least one model parameter submatrix stored in the storage particle.

[0088] In another possible implementation, the storage controller is further configured to: send computing requests to the multiple storage particles respectively, where the computing request corresponding to each storage particle includes the input data matrix and address indication information corresponding to the storage particle.

[0089] In another embodiment of the present application, an electronic device is further provided, as shown in FIG9 . The electronic device may include a processor and the storage device provided above, and the storage device may be any one of the storage devices provided above.

[0090] Optionally, the storage controller and the plurality of storage particles may be coupled via a storage medium interface. Exemplarily, the storage controller may include a storage control unit and a storage medium interface coupled to the plurality of storage particles. Optionally, the processor and the storage controller in the storage device may be coupled via a memory interface. Exemplarily, the processor is coupled to a memory interface, and the storage controller further includes a memory interface. The processor and the memory interface may also be collectively referred to as a host.

[0091] In practical applications, the electronic device may include but is not limited to: mobile phones, tablet computers, laptops, PDAs, ultra-mobile personal computers (umPCs), mobile internet devices (MIDs), netbooks, camcorders, cameras, wearable devices (such as smart watches and smart bracelets, etc.), vehicle-mounted equipment (such as cars, bicycles, electric vehicles, airplanes, ships, trains, high-speed trains, etc.), virtual reality (VR) equipment, augmented reality (AR) equipment, wireless terminals in industrial control, smart home devices (such as refrigerators, televisions, air conditioners, electricity meters, etc.), intelligent robots, workshop equipment, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, or wireless terminals in smart homes, flying equipment (such as intelligent robots, hot air balloons, drones, airplanes), etc.

[0092] It can be understood that all the contents in the embodiments corresponding to the above-mentioned storage device can be referred to the embodiments of the storage particles, the embodiments of the storage chip, the embodiments of the storage controller, and the embodiments of the electronic device, and the embodiments of the present application will not be repeated here.

[0093] In the several embodiments provided in this application, it should be understood that the disclosed storage devices, storage particles, storage controllers, and electronic devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another device, or ignoring or not implementing some features.

[0094] The units described as separate components may or may not be physically separate, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. The readable storage medium may include: a USB flash drive, a mobile hard drive, a read-only memory, a random access memory, a magnetic disk, or an optical disk, etc., which can store program code. Based on this understanding, the technical solution of the embodiment of the present application, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product.

[0096] In another embodiment of the present application, a readable storage medium is also provided, which stores computer execution instructions. When a device (which may be a single-chip microcomputer, chip, etc.) or a processor executes the function of the storage particles in the above embodiment.

[0097] In another embodiment of the present application, a readable storage medium is also provided, which stores computer execution instructions. When a device (which may be a single-chip microcomputer, chip, etc.) or a processor executes the functions of the above-mentioned storage controller.

[0098] In another embodiment of the present application, a computer program product is provided, which includes computer instructions stored in a readable storage medium; at least one processor of the device can read the computer instructions from the readable storage medium, and at least one processor executes the computer instructions to enable the device to store particles in the above-mentioned embodiment.

[0099] In another embodiment of the present application, a computer program product is also provided, which includes computer instructions, which are stored in a readable storage medium; at least one processor of the device can read the computer instructions from the readable storage medium, and at least one processor executes the computer instructions to enable the device to perform the functions of the storage controller in the above method embodiment.

[0100] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A storage particle, characterized in that, the storage particle includes: a storage array for receiving and storing at least one model parameter sub - matrix output by a storage controller, where the at least one model parameter sub - matrix is a model parameter sub - matrix among multiple model parameter sub - matrices included in a model parameter matrix, and other model parameter sub - rectangles among the multiple model parameter sub - matrices are stored in other storage particles provided with the storage particle array; a calculation circuit for calculating an intermediate calculation result of an input data matrix and the at least one model parameter sub - matrix; wherein, the intermediate calculation result is used to calculate a calculation result of the input data matrix and the model parameter matrix.

2. The storage particle according to claim 1, characterized in that, the storage particle further includes: a control circuit for controlling the calculation circuit to output the intermediate calculation result.

3. The storage particle according to claim 2, characterized in that, the control circuit is further configured to: receive a storage request, where the storage request includes the at least one model parameter sub - matrix; store the at least one model parameter sub - matrix in the storage array according to the storage request.

4. The storage particle according to claim 2 or 3, characterized in that, the control circuit is further configured to: receive address indication information, and read the at least one model parameter sub - matrix from the storage array to the calculation circuit according to the address indication information.

5. The storage particle according to claim 4, characterized in that, the control circuit is further configured to: receive a calculation request, where the calculation request includes the input data matrix and the address indication information.

6. The storage particle according to any one of claims 1 - 5, characterized in that, the storage particle further includes: an input cache for caching the input data matrix; and / or, an output cache for caching the intermediate calculation result; and / or, a read cache for caching the at least one model parameter sub - matrix.

7. The storage particle according to any one of claims 1 - 6, characterized in that, the at least one model parameter sub - matrix is obtained by slicing the model parameter matrix according to the concurrency degree of the storage particle.

8. The storage particle according to claim 7, characterized in that, the storage particle includes multiple storage planes, and each storage plane among the multiple storage planes stores one model parameter sub - matrix.

9. The storage particle according to any one of claims 2 - 8, characterized in that, the control circuit is further configured to: send storage status information to the storage controller, where the storage status information is used to indicate the concurrency degree of the storage particle.

10. A storage chip, characterized in that, the storage chip includes multiple storage particles for performing calculations of an input data matrix and a model parameter matrix, and each storage particle is the storage particle according to any one of claims 1 - 9; wherein, the calculation result of the input data matrix and the model parameter matrix is calculated based on multiple intermediate calculation results corresponding to the multiple storage particles.

11. A storage controller, characterized in that, the storage controller is used to be coupled with a plurality of memory grains; wherein, a plurality of model parameter sub-matrices included in a model parameter matrix are stored in the plurality of memory grains, and each memory grain correspondingly stores at least one model parameter sub-matrix; the storage controller is used to respectively send an input data matrix to the plurality of memory grains; the storage controller is further used to obtain a plurality of intermediate calculation results, and calculate a calculation result of the input data matrix and the model parameter matrix according to the plurality of intermediate calculation results; wherein, the plurality of intermediate calculation results are obtained by the plurality of memory grains respectively calculating the input data matrix and the at least one model parameter sub-matrix stored therein.

12. The storage controller according to claim 11, characterized in that, the storage controller is further used to send the plurality of model parameter sub-matrices to the plurality of memory grains according to the concurrency of the plurality of memory grains, so as to store the plurality of model parameter sub-matrices in the plurality of memory grains.

13. The storage controller according to claim 12, characterized in that, the storage controller is further used to divide the model parameter matrix into the plurality of model parameter sub-matrices.

14. The storage controller according to claim 12 or 13, characterized in that, the number of the plurality of model parameter sub-matrices is related to the number of the plurality of memory grains and the number of a plurality of memory planes in each memory grain.

15. The storage controller according to any one of claims 12-14, characterized in that, the storage controller is further used to obtain storage status information of each of the plurality of memory grains, and determine the concurrency of the plurality of memory grains according to the storage status information of the plurality of memory grains.

16. The storage controller according to any one of claims 11-15, characterized in that, the storage controller is further used to respectively send storage requests to the plurality of memory grains, and a storage request corresponding to each memory grain includes at least one model parameter sub-matrix of the plurality of model parameter sub-matrices.

17. The storage controller according to any one of claims 11-16, characterized in that, the storage controller is further used to respectively send address indication information to the plurality of memory grains, and the address indication information corresponding to each memory grain is used to read the at least one model parameter sub-matrix stored in the memory grain.

18. The storage controller according to claim 17, characterized in that, the storage controller is further used to respectively send calculation requests to the plurality of memory grains, and a calculation request corresponding to each memory grain includes the input data matrix and the address indication information corresponding to the memory grain.

19. A storage device, characterized in that, the storage device includes: a plurality of memory grains according to any one of claims 1-9 or a memory chip according to claim 10, and a storage controller according to any one of claims 11-18.

20. An electronic device, characterized in that, The electronic device includes a processor and a storage device as described in claim 19.

Images