Operation method, computing-in-memory apparatus, storage controller, and device

Abstract

An operation method, a computing-in-memory apparatus, a storage controller, and a device, relating to the technical field of electronics, and for use in improving the performance of large model computing. A storage computing circuit in the computing-in-memory apparatus is used for implementing a multiplication operation of a matrix and a vector. The storage computing circuit comprises a first storage row and a second storage row which are located in different storage arrays. A plurality of pieces of first data of the matrix are stored in the first storage row, and a plurality of pieces of second data of the matrix are stored in the second storage row. The method comprises: a computing-in-memory apparatus receiving a first operation signal, wherein the first operation signal is used for instructing to execute a first computing operation on a plurality of pieces of first data and a vector; receiving a first activation signal, wherein the first activation signal is used for instructing to execute an activation operation on a second storage row during the first computing operation; and receiving a second operation signal, wherein the second operation signal is used for instructing to, when the first computing operation ends, start executing a second computing operation on a plurality of pieces of second data and the vector.

Description

An operation method, memory-compute integrated device, memory controller and device

[0001] The present application claims priority from the Chinese patent application No. 202410612824.8 filed on May 16, 2024 with the State Intellectual Property Office, and titled "An operation method, memory-compute integrated device, memory controller and device", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of electronic technology, in particular to an operation method, memory-compute integrated device, memory controller and device. BACKGROUND

[0003] Current large models have important applications in multiple fields, such as chat generative pre-trained transformer (chatGPT) for AI question and answer chat generation, and sora model for text-to-video. Current large models are mainly based on transformer architecture, with parameter quantity in the order of 10 billion to 100 billion, and future parameter quantity evolving towards more than 100 billion, thereby causing large models to require massive computing and storage bandwidth. In particular, large model inference computation contains a large number of memory-bound rectangular vector operation (GEMV) operators, further exacerbating the storage bandwidth bottleneck.

[0004] Processing in memory (PIM) is a technology that implements computation in memory. By utilizing the bandwidth inside the memory, the PIM technology can greatly improve the performance of large model computation. Therefore, how to improve the bandwidth utilization inside the memory in the PIM architecture to improve the performance of large model computation is a technical problem to be solved. SUMMARY

[0005] The present application provides an operation method, memory-compute integrated device, memory controller and device for improving the bandwidth utilization inside the memory-compute integrated device to improve the performance of large model computation.

[0006] To achieve the above-mentioned purpose, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, an operation method is provided and applied to a storage-computing integrated device. The storage-computing integrated device includes at least one storage computing circuit, each of which is configured to implement multiplication of a matrix and a vector, i.e., multiplication of each row of data in the matrix and the same vector, which can be stored in the storage computing circuit. The matrix includes a plurality of first data and a plurality of second data, which can include different data in the same row of the matrix or data in different rows of the matrix. The storage computing circuit includes a first storage row and a second storage row located in different storage arrays, the plurality of first data being stored in the first storage row, and the plurality of second data being stored in the second storage row. For example, the storage computing circuit includes a first storage array and a second storage array, the plurality of first data being stored in the first storage row of the first storage array, and the plurality of second data being stored in the second storage row of the second storage array. The method includes: the storage-computing integrated device receiving a first operation signal or a first operation command, the first operation signal being used to instruct to perform a first computing operation on the plurality of first data and the vector; the storage-computing integrated device receiving a first activation signal or a first activation command, the first activation signal being used to instruct to perform an activation operation on the second storage row during the first computing operation; and the storage-computing integrated device receiving a second operation signal or a first operation command, the second operation signal being used to instruct to start a second computing operation on the plurality of second data and the vector at the end of the first computing operation.

[0008] In the above technical solution, the storage-computing integrated device can receive the first activation signal and the second operation signal, perform the activation operation on the second storage row during the first computing operation according to the first activation signal, and start the second computing operation at the end of the first computing operation according to the second operation signal, thereby avoiding the activation operation occupying extra time in the two computing operations, improving the bandwidth utilization of the storage-computing integrated device, and further improving the computing performance of the storage-computing integrated device.

[0009] In any possible implementation manner of the first aspect, the method further includes: the storage-computing integrated device receiving a first pre-charge signal or a first pre-charge command, the first pre-charge signal being used to instruct to perform a pre-charge operation on the first storage row during the second computing operation. In the above possible implementation manner, the storage computing circuit in the storage-computing integrated device can perform the pre-charge operation on the first storage row during the second computing operation according to the first pre-charge signal, thereby avoiding the pre-charge operation occupying extra time in the two computing operations, improving the bandwidth utilization of the storage array in the storage-computing integrated device, and further improving the computing performance of the storage-computing integrated device.

[0010] In any possible implementation form of the first aspect, each storage computing circuit further comprises a computing unit, a first row buffer corresponding to a storage array where the first storage row is located, and a second row buffer corresponding to a storage array where the second storage row is located; and the method further comprises: when receiving the first operation signal, the first row buffer acquires the plurality of first data from the first storage row and buffers; the computing unit acquires the plurality of first data from the first row buffer and performs the first computing operation on the plurality of first data and the vector; when receiving the second operation signal, the second row buffer acquires the plurality of second data from the second storage row and buffers; and the computing unit acquires the plurality of second data from the second row buffer at the end of the first computing operation and performs the second computing operation on the plurality of second data and the vector. The above possible implementation form can ensure that the storage computing circuit in the storage-computing integrated device activates the storage row required for the next computing operation in the last computing operation, and pre-charges the storage row required for the next computing operation in the next computing process, thereby improving the internal bandwidth utilization of the storage-computing integrated device, and further improving the computing performance of the storage-computing integrated device.

[0011] In any possible implementation form of the first aspect, the storage-computing integrated device comprises a plurality of storage computing circuits, and the plurality of storage computing circuits are configured to implement different matrix-vector multiplication operations. In the above possible implementation form, the storage-computing integrated device can implement different matrix-vector multiplication operations through the plurality of storage computing circuits, so that the storage-computing integrated device can perform different operations in parallel, thereby improving the operation efficiency and the computing performance.

[0012] In any possible implementation form of the first aspect, the plurality of storage computing circuits perform the activation operation asynchronously; or the method further comprises: the storage-computing integrated device receives a synchronous activation signal, and the synchronous activation signal is configured to instruct the plurality of storage computing circuits to perform the activation operation synchronously. In the above possible implementation form, the plurality of storage computing circuits perform the activation operation asynchronously, which can reduce the peak power consumption caused by the activation operation; and the plurality of storage computing circuits perform the activation operation synchronously through the synchronous activation signal, which can reduce the signaling interaction between the storage-computing integrated device and the storage controller.

[0013] In any possible implementation form of the first aspect, the plurality of storage computing circuits perform the first computing operation or the second computing operation asynchronously; or the method further comprises: receiving, by the storage computing device, a synchronous computing signal, the synchronous computing signal being used to indicate that the plurality of storage computing circuits perform the first computing operation or the second computing operation synchronously. In the possible implementation form, the plurality of storage computing circuits perform the computing operation asynchronously, which can reduce the peak power consumption caused by the computing operation; the plurality of storage computing circuits perform the computing operation synchronously through the synchronous computing signal, which can reduce the signaling interaction between the storage computing device and the storage controller.

[0014] In any possible implementation form of the first aspect, the plurality of storage computing circuits perform the pre-charging operation asynchronously; or the method further comprises: receiving, by the storage computing device, a synchronous pre-charging signal, the synchronous pre-charging signal being used to indicate that the plurality of storage computing circuits perform the pre-charging operation synchronously. In the possible implementation form, the plurality of storage computing circuits perform the pre-charging operation asynchronously, which can reduce the peak power consumption caused by the pre-charging operation; the plurality of storage computing circuits perform the pre-charging operation synchronously through the synchronous pre-charging signal, which can reduce the signaling interaction between the storage computing device and the storage controller.

[0015] In a second aspect, a computing method is provided, applied to a storage controller, the storage controller being used to be coupled with a storage computing device, the storage computing device comprising at least one storage computing circuit, each storage computing circuit being used to implement a multiplication operation of a matrix and a vector, the vector being stored in the storage computing circuit, the matrix comprising a plurality of first data and a plurality of second data, the storage computing circuit comprising a first storage row and a second storage row located in different storage arrays, the plurality of first data being stored in the first storage row, the plurality of second data being stored in the second storage row, the method comprising: sending, by the storage controller, a first computing signal, the first computing signal being used to indicate that a first computing operation is performed on the plurality of first data and the vector; sending, by the storage controller, a first activation signal, the first activation signal being used to perform an activation operation on the second storage row in a process of the first computing operation; sending, by the storage controller, a second computing signal, the second computing signal being used to start a second computing operation on the plurality of second data and the vector at the end of the first computing operation.

[0016] In any possible implementation form of the second aspect, the method further comprises: sending, by the storage controller, a first pre-charging signal, the first pre-charging signal being used to perform a pre-charging operation on the first storage row in a process of the second computing operation.

[0017] In any possible implementation form of the second aspect, the storage computing device comprises a plurality of storage computing circuits, the plurality of storage computing circuits being used to implement different multiplication operations of matrices and vectors.

[0018] In any possible implementation form of the second aspect, the plurality of storage computing circuits perform the activation operation asynchronously; or the method further comprises: the storage controller sending a synchronous activation signal, the synchronous activation signal being used to instruct the plurality of storage computing circuits to perform the activation operation synchronously.

[0019] In any possible implementation form of the second aspect, the plurality of storage computing circuits perform the first computing operation or the second computing operation asynchronously; or the method further comprises: the storage controller sending a synchronous operation signal, the synchronous operation signal being used to instruct the plurality of storage computing circuits to perform the first computing operation or the second computing operation synchronously.

[0020] In any possible implementation form of the second aspect, the plurality of storage computing circuits perform the pre-charge operation asynchronously; or the method further comprises: sending a synchronous pre-charge signal, the synchronous pre-charge signal being used to instruct the plurality of storage computing circuits to perform the pre-charge operation synchronously.

[0021] In a third aspect, there is provided a storage-computing integrated device, which can implement the functions performed by the storage-computing integrated device in the above method. The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software comprises one or more modules corresponding to the above functions.

[0022] In a possible embodiment of the third aspect, the storage-computing integrated device comprises an interface circuit and at least one storage computing circuit, which are configured to support the device to perform the corresponding functions in the above method. In an example, the interface circuit is configured to receive operation signals, activation signals, pre-charge signals and the like in the above method, and the at least one storage computing circuit is configured to perform corresponding operations according to the operation signals, activation signals, pre-charge signals and the like.

[0023] In a fourth aspect, there is provided a storage controller, which can implement the functions performed by the storage controller in the above method. The functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software comprises one or more modules corresponding to the above functions.

[0024] In a possible embodiment of the fourth aspect, the storage controller comprises a processing circuit and an interface circuit, which are configured to support the storage controller to perform the corresponding functions in the above method. In an example, the processing circuit is configured to generate operation signals, activation signals, pre-charge signals and the like in the above method, and the interface circuit is configured to send the operation signals, activation signals, pre-charge signals and the like.

[0025] In a fifth aspect, a storage device is provided, comprising the storage and computing integrated device and the storage controller; wherein the storage controller is the storage controller provided in the fourth aspect or any possible implementation manner of the fourth aspect; or the storage and computing integrated device is the storage and computing integrated device provided in the third aspect or any possible implementation manner of the third aspect.

[0026] In a sixth aspect, an electronic device is provided, comprising the processor and the storage device provided in the fifth aspect.

[0027] In a seventh aspect, a computer readable storage medium is provided, which stores a computer program or instructions, when the computer program or instructions are run, the method provided in the first aspect or any possible implementation manner of the first aspect is implemented.

[0028] In an eighth aspect, a computer readable storage medium is provided, which stores a computer program or instructions, when the computer program or instructions are run, the method provided in the second aspect or any possible implementation manner of the second aspect is implemented.

[0029] In a ninth aspect, a computer program product is provided, which comprises: a computer program (also can be called code, or instruction), when the computer program is run, the computer program makes the computer execute the method provided in the first aspect or any possible implementation manner of the first aspect.

[0030] In a tenth aspect, a computer readable storage medium is provided, which stores a computer program or instructions, when the computer program or instructions are run, the method provided in the second aspect or any possible implementation manner of the second aspect is implemented.

[0031] It can be understood that the beneficial effects that can be achieved by the above-mentioned fifth aspect to tenth aspect can correspond to the beneficial effects in the first aspect or any possible implementation manner of the first aspect provided above, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0032] FIG. 1 is a structural schematic diagram of a storage and computing integrated device provided in an embodiment of the present application;

[0033] FIG. 2 is a schematic diagram of an ACT operation, a MUL operation and a PCH operation provided in an embodiment of the present application;

[0034] FIG. 3 is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0035] FIG. 4 is a structural schematic diagram of a storage device provided in an embodiment of the present application;

[0036] FIG. 5 is a schematic diagram of a matrix and vector multiplication operation of an all-memory computing device according to an embodiment of the present application;

[0037] FIG. 6 is a flow diagram of an operation method according to an embodiment of the present application;

[0038] FIG. 7 is a flow diagram of another operation method according to an embodiment of the present application;

[0039] FIG. 8 is a schematic diagram of an ACT operation, a MUL operation, and a PCH operation according to an embodiment of the present application;

[0040] FIG. 9 is a schematic diagram of another ACT operation, a MUL operation, and a PCH operation according to an embodiment of the present application;

[0041] FIG. 10 is a schematic diagram of yet another ACT operation, a MUL operation, and a PCH operation according to an embodiment of the present application;

[0042] FIG. 11 is a schematic diagram of a plurality of signals and a plurality of operations involved in an operation of a DRAM according to an embodiment of the present application;

[0043] FIG. 12 is a schematic diagram of a plurality of signals and a plurality of operations involved in an operation of an all-memory computing device according to an embodiment of the present application;

[0044] FIG. 13 is a schematic diagram of a plurality of signals and a plurality of operations involved in an operation of another all-memory computing device according to an embodiment of the present application;

[0045] FIG. 14 is a schematic diagram of another all-memory computing device according to an embodiment of the present application;

[0046] FIG. 15 is a schematic diagram of another memory device according to an embodiment of the present application. DETAILED DESCRIPTION

[0047] The making and using of various embodiments are discussed in detail below. It should be appreciated that the specific embodiments discussed are merely illustrative of specific ways to make and use the application and this technology and do not limit the scope of the present application.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0049] Circuits or other components can be described as, or said to be, “configured to” perform a task or tasks. In such contexts, “configured to” is used to connote structure by indicating that a circuit / component includes structure (e.g., circuitry) that performs the task or tasks during operation. As such, the circuit / component can be said to be configured to perform the task or tasks even when the specified circuit / component is not currently operational (e.g., is not on). The circuits / components used with the “configured to” language can include, for example, a structural configuration, hardware, or a combination of both, designed to perform the task or tasks during operation.

[0050] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In the present application, “at least one” means one or more, and “multiple” means two or more. “And / or” describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character “ / ” generally represents an “or” relationship between the associated objects. “At least one of the following” or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b or c can represent a, b, c, a and b, a and c, b and c, a, b and c; where a, b and c can be single or multiple.

[0051] The embodiments of the present application use “first” and “second” and the like to distinguish objects with similar names or functions or roles. Those skilled in the art can understand that “first” and “second” and the like do not limit the quantity and execution order. The word “coupled” is used to represent electrical connection, including direct connection through wires or connection terminals or indirect connection through other devices. Therefore, “coupled” should be regarded as a broad sense of electronic communication connection.

[0052] It should be noted that in the present application, the words “exemplary” or “for example” are used to mean serving as an example, instance, or illustration. Any embodiment or design scheme described as “exemplary” or “for example” in the present application should not be interpreted as being more preferred or advantageous than other embodiments or design schemes. Rather, the use of “exemplary” or “for example” is intended to present relevant concepts in a concrete manner.

[0053] Before introducing the embodiments of the present application, first, the application scenarios involved in the present application will be introduced and described.

[0054] Current large models have important applications in many fields, such as chat generative pre-trained transformer (chatGPT) for AI question and answer, and sora model for text-to-video. Current large models are mainly based on transformer architecture, with parameter quantity in the order of 10 billion to 100 billion, and future parameter quantity is evolving towards more than 100 billion, thereby causing large models to require massive computing and storage bandwidth. Especially, large model inference computing contains a large number of memory-bound rectangular vector operation (GEMV) operators, which further aggravates the storage bandwidth bottleneck. The matrix vector operation can refer to the multiplication operation of matrix and vector.

[0055] Processing in memory (PIM) is a technology of implementing computation in memory, which utilizes the bank level bandwidth of memory in the process of computation. The bank level bandwidth can be improved by one order of magnitude compared with the bandwidth of each channel in the memory, so that the PIM technology can greatly improve the performance of large model computation. The device supporting PIM can also be called an integrated memory and computing device, which is collectively referred to as an integrated memory and computing device in this paper.

[0056] FIG. 1 is a structural schematic diagram of an integrated memory and computing device provided by an embodiment of the present application, which includes an input output (IO) interface circuit (or interface circuit), a global buffer, a plurality of memory banks, a plurality of processing units (PUs) and a plurality of accumulators (Accs). Each memory bank in the plurality of memory banks can be provided with a processing unit PU and an accumulator Acc coupled next to it, and each memory bank, the processing unit PU and the accumulator Acc corresponding to the memory bank can be collectively referred to as a memory computing circuit in this paper. In addition, the plurality of processing units PU and the plurality of accumulators Acc can also be coupled with the global buffer through a wire, and the IO interface circuit can be used to support the integrated memory and computing device to communicate with the outside, such as transmitting data and commands, etc. The above memory bank can also be referred to as a storage bank, and the memory bank is denoted as BA.

[0057] Optionally, the memory bank includes one or more memory arrays, and each memory array corresponds to a row buffer.

[0058] In a possible embodiment, each memory bank and the corresponding processing unit PU in the storage-computing integrated device can be used to implement a matrix-vector multiplication operation, the matrix can be stored in a storage array in the corresponding memory bank, each row of data in the matrix can be stored in a storage row in the storage array, and one row of data in the matrix is obtained from the storage array each time the operation is performed. In an example, as shown in FIG. 2, taking a memory bank in a dynamic random access memory (DRAM) as an example, an activation (ACT) operation, also referred to as a row activation operation, needs to be performed each time one row of data in the matrix is read, and then the data is read and written into a row buffer of the corresponding storage array, and then the data in the row buffer is read and multiplication (MUL) operation is performed in time, and when the data in the row buffer is calculated, the storage row where the row of data is located needs to be precharged (PCH) before the next row of data in the matrix is read, and then the ACT operation and the MUL operation are performed on the storage row where the next row of data is located. In FIG. 2, taking the DRAM including 16 memory banks and denoted as BA0 to BA15 as an example, the corresponding operations of each memory bank in implementing one row of data operation include the ACT operation, the MUL operation and the PCH operation, and different memory banks can be used to implement different matrix-vector multiplication operations.

[0059] The ACT operation, the MUL operation and the PCH operation are sequentially performed, and there is an additional time for the ACT operation and the PCH operation between two MUL operations of the same memory bank. In the above-mentioned FIG. 2, the time of the ACT operation is denoted as tRCD, the time of the PCH operation is denoted as tRP, and the time of the MUL operation is denoted as tGEMV. If the ACT operation and the PCH operation each need 18 nanoseconds (ns) and the MUL operation needs 256 ns, the bandwidth utilization rate of the overall memory bank is 87.7%, that is, the bandwidth utilization rate of the overall memory bank is low, thereby reducing the performance of the storage-computing integrated device for implementing large model calculation. Therefore, while improving the performance of the storage-computing integrated device for large model calculation, how to improve the bandwidth utilization rate of the memory bank in the storage-computing integrated device is a technical problem to be solved.

[0060] Based on this, an operation method is provided in the embodiments of the present application, which can be used to implement the continuity of two calculation operations in a matrix-vector operation, avoid the additional time occupied by the ACT operation and / or the PCH operation in the two calculation operations, and thereby improve the bandwidth utilization rate of the memory bank in the storage-computing integrated device. The method can be applied to various electronic devices with a storage-computing integrated device.

[0061] Optionally, the electronic device can include, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a mobile internet device (MID), a camera, a wearable device (for example, a smart watch, a smart bracelet, a pedometer, etc.), a sound equipment, an audio and video player, a set-top box, a game console, a printer, a mouse, a keyboard, a vehicle-mounted device (for example, a device on a vehicle such as a car, an airplane, a ship, a train, and a high-speed rail), a wireless terminal in virtual reality (VR), a wireless terminal in augmented reality (AR), a wireless terminal in industrial control, a smart home device (for example, a refrigerator, a television, an air conditioner, an electricity meter, etc.), a smart robot, a workshop device, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, or a wireless terminal in a smart home, a flight device (for example, a smart robot, a hot air balloon, a drone, an airplane), and the like.

[0062] The structure of the electronic device will be described below by taking the electronic device as a mobile phone as an example. As shown in FIG. 3, the electronic device can include radio frequency (RF) circuit 110, memory 120, input unit 130, display unit 140, sensor 150, audio circuit 160, processor 170, and power supply 180, and the like. Optionally, the display unit 140 can be the display device in the above.

[0063] The RF circuit 110 can be used to transceive information, or receive or send a signal in a call process. In particular, after receiving the downlink information of a base station, the processor 170 is processed; in addition, the data of the uplink is sent to the base station. Usually, the RF circuit 110 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and the like. In addition, the RF circuit 110 can also communicate with the network and other devices through the mode of wireless communication.

[0064] The memory 120 can be used to store data, software programs, and modules; can include a program storage area and a data storage area, wherein the program storage area can store operating systems, application programs required by at least one function, such as a sound playing function, an image playing function, etc.; and the data storage area can store data created according to the use of the electronic device, such as audio data, image data, a phone book, etc. In addition, the electronic device can include a high-speed random access memory, and can also include a nonvolatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. In the embodiments of the present application, the memory can include, but is not limited to, a synchronous dynamic random access memory (SDRAM), a double data rate dynamic random access memory (DDR SRAM), and a memory bus type dynamic random access memory (RDRAM), etc.

[0065] The input unit 130 can be used to receive inputted digital or character information, and to generate key signal inputs related to the user settings and function control of the electronic device. The input unit 130 can include a touch screen 131 and other input devices 132. The touch screen 131 can collect the touch operation of a user thereon or therearound (such as the operation of a user using a finger, a stylus, or any suitable object or accessory on or near the touch screen), and drive the corresponding connection device according to the pre-set program. Optionally, the other input devices 132 can include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control button, a power on / off button, etc.), a trackball, a mouse, a joystick, etc.

[0066] The display unit 140 can be used to display information input by a user or provided to the user, as well as various menus of the electronic device, etc. In one example, the display unit 140 can include a display screen 141, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, a touch screen 131 can cover the display screen 141, and when the touch screen 131 detects a touch operation thereon or adjacent thereto, it transmits to the processor 170 to determine the type of touch event, and then the processor 170 provides a corresponding visual output on the display screen 141 according to the type of touch event. Although in FIG. 3, the touch screen 131 and the display screen 141 are implemented as two independent components to realize the input and output functions of the electronic device, in some embodiments, the touch screen 131 and the display screen 141 can be integrated to realize the input and output functions of the electronic device.

[0067] The sensor 150 can include one or more sensors for providing various aspects of state evaluation for the electronic device. Among them, the sensor 150 can include a light sensor, which can be used in imaging applications, i.e., as a component of a camera or a camera. In addition, the sensor 150 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor, through which the acceleration / deceleration of the electronic device, the orientation, the open / close state, the relative positioning of components, or the temperature change of the electronic device, etc. can be detected.

[0068] The audio circuit 160, the speaker, and the microphone can provide an audio interface between the user and the electronic device. The audio circuit 160 can convert the received audio data into an electrical signal and transmit it to the speaker, which converts it into a sound signal output. On the other hand, the microphone collects sound signals and converts them into electrical signals, which are received by the audio circuit 160 and converted into audio data, which are then output to the RF circuit 110 for transmission to, for example, another mobile phone, or to the storage 120 for further processing.

[0069] The processor 170 is the control center of the electronic device, connects all parts of the electronic device through various interfaces and lines, performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 120 and calling data stored in the memory 120, thereby monitoring the entire electronic device. Optionally, the processor 170 can include one or more processing units, which can include but are not limited to: a central processing unit (CPU), a network processing unit (NPU), a graphic processing unit (GPU), an image signal processor (ISP), a tensor processing unit (TPU), a data processing unit (DPU), a digital signal processor (DSP), a microcontroller or a microprocessor, etc. Further, the processor 170 can also include other hardware circuits or accelerators, such as an application specific integrated circuit (ASIC), a complex programmable logic device (CPLD) or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Optionally, the processor 170 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc.

[0070] The electronic device can also include a power supply 180 (such as a battery) to power various components. The power supply 180 can be logically connected to the processor 170 through a power management system, so that the power management system can realize functions such as management of charging, discharging, and power consumption management. Optionally, the power management system can support both fast charging technology and non-fast charging technology. In actual application, the power management system can charge the battery in the power supply 180 through fast charging technology, or charge the battery in the power supply 180 through non-fast charging technology.

[0071] The electronic device can also include a wireless fidelity (WiFi) module, a Bluetooth module, etc., which will not be described herein. Those skilled in the art can understand that the structure of the electronic device shown in FIG. 3 does not constitute a limitation on the electronic device, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0072] FIG. 4 is a structural schematic diagram of a storage device provided by an embodiment of the present application, the storage device comprising a storage controller and a storage-computing integrated device, the storage controller being configured to control the storage-computing integrated device, for example, the storage controller being configured to control the storage-computing integrated device to implement multiplication operation of a matrix and a vector, and control the storage-computing integrated device to implement writing and reading of data, etc.

[0073] In the storage device, the storage-computing integrated device comprises at least one storage computing circuit, each of the at least one storage computing circuit comprising a computing unit and a storage bank. The computing unit can comprise the processing unit PU as described above, or comprise the processing unit PU and the accumulation unit Acc, and embodiments of the present application do not make specific limitations in this regard. Each storage bank can comprise a plurality of storage arrays and a plurality of row buffers, the plurality of storage arrays corresponding to the plurality of row buffers one by one, each storage array comprising a plurality of storage rows, and the plurality of storage arrays being configured to store different data in the matrix.

[0074] Different storage arrays correspond to different row buffers, or different storage arrays can correspond to different address ranges, or be distinguished by different word lines and / or bit lines corresponding to different storage arrays, and embodiments of the present application do not make specific limitations in this regard.

[0075] Optionally, each storage bank can further comprise a row decoder, a column decoder, a sense amplifier (SA), and a middle in-out routing (MIO), etc. The row decoder, the column decoder, the sense amplifier (SA), and the middle in-out routing (MIO) can be configured to realize functions such as decoding of a row address, decoding of a column address, data amplification, and data transmission, etc. in the process of storing and reading data in the storage bank.

[0076] Optionally, the plurality of storage arrays comprises a first storage array and a second storage array, the first storage array corresponding to a first row buffer, and the second storage array corresponding to a second row buffer, the first storage array and the second storage array each comprising a plurality of storage rows, a first storage row in the first storage array being configured to cache a plurality of first data in the matrix, and a second storage row in the second storage array being configured to cache a plurality of second data in the matrix, the first storage row being any one of the plurality of storage rows in the first storage array, and the second storage row being any one of the plurality of storage rows in the second storage array.

[0077] For example, as shown in FIG. 5, if the storage-computing integrated device is used to implement multiplication operation of a matrix and a vector, and the matrix is an N*M matrix, the vector is an M-dimensional column vector, and the operation result of the multiplication operation of the matrix and the vector is an N-dimensional column vector, the storage-computing circuit can include a plurality of storage arrays MAT, the plurality of storage arrays including a first storage array MAT1 and a second storage array MAT2, the plurality of first data D11 to D1M can be stored in a first storage row in the first storage array MAT1, and the plurality of second data D21 to D2M can be stored in a second storage row in the second storage array MAT2, D11 to D1M representing data of a first row in the matrix, and D21 to D2M representing data of a second row in the matrix. FIG. 5 also shows other circuits in the storage-computing circuit, such as row buffers in the storage bank, row decoders, column decoders, and sense amplifiers, etc.

[0078] Further, the storage-computing integrated device also includes a global buffer and a peripheral circuit, which can include an interface circuit, a power supply circuit, a driving circuit, a global sense amplifier (GSA), a global in-out routing (GIO), etc. The interface circuit can be used to support information interaction between the storage-computing integrated device and other devices, for example, the interface circuit can transmit various signals or commands (CMD) from the storage controller to the storage-computing circuit. The global buffer can be used to buffer data required by the storage-computing circuit for calculation, for example, used to store the vector in the multiplication operation of the matrix and the vector, and the storage-computing circuit can obtain and buffer the data from the global buffer during specific operation. The power supply circuit can be used to provide power supply for the storage-computing circuit. The driving circuit can be used to drive the storage-computing circuit to perform one or more operations under the control of the storage controller, for example, the one or more operations can include but are not limited to ACT operation, MUL operation, and PCH operation, etc.

[0079] FIG. 6 is a flowchart of an operation method provided by an embodiment of the present application. The method can be applied to the storage device provided above, which includes a storage controller and a storage-computing integrated device, each storage-computing circuit in the storage-computing integrated device being used to implement multiplication operation of a matrix and a vector, i.e., implement multiplication operation of data of each row in the matrix and the same vector, and the method includes the following steps.

[0080] S201: The storage-computing integrated device receives a first operation signal from the storage controller, the first operation signal being used to instruct to perform a first computing operation on a plurality of first data and a vector.

[0081] The matrix includes a plurality of first data and a plurality of second data, which can include different data in the same row of the matrix or data in different rows of the matrix. For example, the plurality of first data can include data in an (i-1)-th row R[i-1] of the matrix, and the plurality of second data can include data in an i-th row R[i] of the matrix, i.e., the plurality of second data can be data in a next row of the plurality of first data. i can be in a range of 2 to M, where M represents a total number of rows of the matrix.

[0082] In addition, the plurality of first data and the plurality of second data can be stored in different storage arrays of the storage computing circuit. For example, the storage computing circuit can include a plurality of storage arrays, including a first storage array and a second storage array, the plurality of first data is stored in a first storage row of the first storage array, and the plurality of second data is stored in a second storage row of the second storage array. The first storage row can be any storage row of a plurality of storage rows in the first storage array, and the second storage row can be any storage row of a plurality of storage rows in the second storage array. The vector can be stored in the storage computing circuit, specifically in a computing unit of the storage computing circuit.

[0083] Optionally, the matrix can be sent to the storage computing integrated device by the storage controller in advance and stored in the storage computing circuit of the storage computing integrated device. For example, the storage controller can send the matrix to the storage computing integrated device and instruct the storage computing integrated device to store different data of the matrix in different matrices of the storage computing circuit, such as storing the plurality of first data in the first storage row of the first storage array and storing the plurality of second data in the second storage row of the second storage array.

[0084] In a possible embodiment, in the process of the storage controller controlling the storage computing integrated device to implement the multiplication operation of the matrix and the vector, the storage controller can send a first operation signal to the storage computing circuit in the storage computing integrated device; the storage computing circuit can receive the first operation signal and perform a first computing operation on the plurality of first data and the vector according to the first operation signal. Optionally, the first operation signal can also be referred to as a first operation command, and the first computing operation can be referred to as a first multiplication operation or a first multiply-accumulate operation.

[0085] Further, the plurality of first data is stored in a first storage row in a first storage array of the storage computing circuit, and the storage computing circuit further comprises a first row buffer corresponding to the first storage array. The storage computing circuit performs a first computing operation on the plurality of first data and the vector according to a first operation signal, which can comprise: when the storage computing circuit receives the first operation signal, the first row buffer obtains the plurality of first data from the first storage row and buffers; the computing unit obtains the plurality of first data from the first row buffer and performs a first computing operation on the plurality of first data and the vector.

[0086] Optionally, the computing unit can obtain the plurality of first data from the first row buffer and the corresponding data in the vector in time, and perform a computing operation on each obtained first data and the corresponding data in the vector. For example, the plurality of first data comprises R11, R12 and R13, and the data in the vector comprises V1, V2 and V3, then: the computing unit can first obtain R11 and V1 and perform R11 x V1 to obtain S1; then, the computing unit obtains R12 and V2 and performs R12 x V2 to obtain S2; finally, the computing unit obtains R13 and V3 and performs R13 x V3 to obtain S3. When the computing unit further comprises an accumulation unit Acc, the accumulation unit Acc can perform accumulation operation on S1 and S2 to obtain S21 when S1 and S2 are obtained, and perform accumulation on S21 and S3 to obtain the result of the first computing operation when S3 is obtained.

[0087] S202: The storage-computing integrated device receives a first activation signal from the storage controller, and the first activation signal is used to indicate that an activation operation is performed on the second storage row in the process of the first computing operation.

[0088] In the second storage row, a plurality of second data is stored, and the plurality of second data is the data required by the storage-computing integrated device for computing operation after the first computing operation is completed. The first activation signal is used to indicate that an activation operation is performed on the second storage row in the process of the first computing operation, that is, an activation operation is performed on the storage row where the data required for the next computing operation is located before the last computing operation is completed.

[0089] In a possible embodiment, during the process in which the storage computing circuit performs the first computing operation on the plurality of first data and the vector according to the first operation signal, the storage controller can send a first activation signal for a second storage row to the storage-computing integrated device; the peripheral circuit in the storage-computing integrated device can receive the first activation signal, and perform an activation operation on the second storage row according to the first activation signal during the first computing operation, or perform an activation operation on the second storage row according to the first activation signal before the first computing operation ends, which can be specifically performed by the driving circuit in the peripheral circuit. In this way, the storage-computing integrated device can perform an activation operation on the storage row in which the next row of data is located during the computing process of the previous row of data, thereby avoiding that the activation operation occupies extra time. Optionally, the first activation signal can also be referred to as a first activation command.

[0090] Optionally, the storage-computing integrated device can also perform an activation operation on the first storage row before the first computing operation, which can be specifically performed before the previous computing operation ends before the first computing operation. That is, for any computing operation in the process of the matrix-vector multiplication, the storage computing circuit can perform an activation operation on the storage row in which the data required for the next computing operation is located during the process of the previous computing operation. For example, 1ACT shown in FIG. 5 can represent an activation operation performed on the storage row in which the data D21 to D2M in the second storage array MAT2 is stored, and 2ACT can represent an activation operation performed on the storage row in which the data required for the next computing operation is stored.

[0091] S203: The storage-computing integrated device receives a second operation signal from the storage controller, where the second operation signal is used to indicate that the second computing operation on the plurality of second data and the vector is started when the first computing operation ends.

[0092] In a possible embodiment, during the process in which the storage-computing integrated device implements the matrix-vector multiplication, the storage controller can send a second operation signal to the storage computing circuit in the storage-computing integrated device, where the second operation signal is used to indicate that the second computing operation on the plurality of second data and the vector is started when the first computing operation ends; and the storage computing circuit can receive the second operation signal, and perform the second computing operation on the plurality of second data and the vector according to the second operation signal when the first computing operation ends. Optionally, the second operation signal can also be referred to as a second operation command, and the second computing operation can be referred to as a second multiplication operation or a second multiply-accumulate operation.

[0093] Further, the plurality of second data is stored in a second storage row in a second storage array of the storage computing circuit, and the storage computing circuit further comprises a second row buffer corresponding to the second storage array. The storage computing circuit performs a second computing operation on the plurality of second data and the vector according to a second operation signal, which can comprise: when the storage computing circuit receives the second operation signal, the second row buffer obtains the plurality of second data from the second storage row and buffers; and the computing unit obtains the plurality of second data from the second row buffer and performs a second computing operation on the plurality of second data and the vector.

[0094] Optionally, the computing unit can obtain the plurality of second data from the second row buffer and obtain corresponding data in the vector in time, and perform a computing operation on each obtained second data and corresponding data in the vector. For example, the plurality of second data comprises R21, R22 and R23, and the data in the vector comprises V1, V2 and V3, then: the computing unit can first obtain R21 and V1 and perform R21 x V1 to obtain W1; then, the computing unit obtains R22 and V2 and performs R22 x V2 to obtain W2; finally, the computing unit obtains R23 and V3 and performs R23 x V3 to obtain W3. When the computing unit further comprises an accumulation unit Acc, the accumulation unit Acc can perform accumulation operation on W1 and W2 to obtain W21 when W1 and W2 are obtained, and perform accumulation operation on W21 and W3 to obtain the result of the second computing operation when W3 is obtained.

[0095] In the embodiments of the present application, the storage computing device can receive a first activation signal and a second operation signal, and perform an activation operation on the second storage row in the process of the first computing operation according to the first activation signal, and perform a second computing operation at the end of the first computing operation according to the second operation signal, thereby avoiding the activation operation occupying additional time in the two computing operations, and improving the bandwidth utilization of the storage body in the storage computing device, and further improving the computing performance of the storage computing device.

[0096] Further, the storage computing device can further perform a pre-charge operation on the first storage row in the process of the second computing operation, that is, the storage computing device can perform a pre-charge operation on the storage row where the data read in the previous computing operation in the process of the next computing operation, so as to avoid the pre-charge operation occupying additional time. Correspondingly, as shown in FIG. 7, the method further comprises the following steps in combination with FIG. 6.

[0097] S204: The storage computing device receives a first pre-charge signal from the storage controller, and the first pre-charge signal is used to indicate that a pre-charge operation is performed on the first storage row in the process of the second computing operation.

[0098] In a possible implementation, during the process in which the storage computing circuit performs the second computing operation on the plurality of second data and the vector according to the second operation signal, the storage controller can send a first pre-charge signal for the first storage row to the storage-computing integrated device; the peripheral circuit in the storage-computing integrated device receives the first pre-charge signal, and performs a pre-charge operation on the first storage row according to the first pre-charge signal during the second computing operation. In this way, the storage computing circuit can perform the pre-charge operation on the storage row in which the previous row of data is located during the computation of the next row of data, thereby avoiding the pre-charge operation occupying additional time. Optionally, the first pre-charge signal can also be referred to as a first pre-charge command.

[0099] Optionally, the peripheral circuit in the storage-computing integrated device can also perform a pre-charge operation on the storage row in which the data read in the computing operation before the first computing operation is located during the first computing operation. That is, for any computing operation in the process of the matrix-vector multiplication, the storage-computing integrated device can perform a pre-charge operation on the storage row in which the data required by the previous computing operation is located during the next computing operation.

[0100] Further, the storage-computing integrated device can include a plurality of storage computing circuits, which can be used to implement different matrix-vector multiplications, and each storage computing circuit can be implemented according to the operation method provided above. For example, the plurality of storage computing circuits include a first storage computing circuit, a second storage computing circuit, and a third storage computing circuit, the first storage computing circuit is used to implement the multiplication of a first matrix and a first vector, the second storage computing circuit is used to implement the multiplication of a second matrix and a second vector, and the third storage computing circuit is used to implement the multiplication of a third matrix and a third vector. The first matrix, the second matrix, and the third matrix are different, and / or the first vector, the second vector, and the third vector are different.

[0101] The plurality of different matrices corresponding to the plurality of storage computing circuits in the above operation can also be different sub-rectangles in the same matrix, that is, the plurality of storage computing circuits can also be used to implement the operation of different parts in the multiplication of a matrix and a vector, and the embodiments of the present application do not make specific limitations in this regard.

[0102] In one possible embodiment, the plurality of storage-computing circuits can be configured to perform the activation operation synchronously; and / or, the plurality of storage-computing circuits can be configured to perform the computing operation synchronously; and / or, the plurality of storage-computing circuits can be configured to perform the pre-charge operation synchronously. For example, as shown in FIG. 8, when the plurality of storage-computing circuits includes 16 storage-computing circuits, the storage-computing device can be configured to perform the activation operation, the computing operation and the pre-charge operation for the storage row in which the data in the i-1th row or the i-th row of a matrix is located simultaneously when the 16 storage-computing circuits are configured to perform the multiplication operation of different matrices and vectors. In FIG. 8, the storage banks in the 16 storage-computing circuits are denoted as BA0 to BA15, and the activation operation, the computing operation and the pre-charge operation for the storage row in which the data in the i-1th row is located are denoted as ACT R[i-1], MUL R[i-1] and PCH R[i-1] respectively, and the activation operation, the computing operation and the pre-charge operation for the storage row in which the data in the i-th row is located are denoted as ACT R[i], MUL R[i] and PCH R[i] respectively.

[0103] Optionally, when the plurality of storage-computing circuits are configured to perform the activation operation synchronously, the storage controller can send the corresponding activation signals to the storage-computing device simultaneously, or send a synchronous activation signal to the storage-computing device, where the synchronous activation signal is configured to instruct the plurality of storage-computing circuits to perform the activation operation synchronously. When the plurality of storage-computing circuits are configured to perform the computing operation synchronously, the storage controller can send the corresponding computing signals to the storage-computing device simultaneously, or send a synchronous computing signal to the storage-computing device, where the synchronous computing signal is configured to instruct the plurality of storage-computing circuits to perform the computing operation synchronously. When the plurality of storage-computing circuits are configured to perform the pre-charge operation synchronously, the storage controller can send the corresponding pre-charge signals to the storage-computing device simultaneously, or send a synchronous pre-charge signal to the storage-computing device, where the synchronous pre-charge signal is configured to instruct the plurality of storage-computing circuits to perform the pre-charge operation synchronously.

[0104] For example, as shown in Table 1 below, the signals (or commands CMD) supported by the storage controller of the storage-computing device can be different from the signals (or commands CMD) supported by the storage controller of the current DRAM. Specifically, the storage controller of the storage-computing device can support not only the various signals corresponding to the storage controller of the current DRAM, but also a synchronous activation signal BMACT, a synchronous computing signal BMMUL and a synchronous pre-charge signal BMPCH.

[0105] Table 1

[0106] In one possible embodiment, the plurality of storage-computing circuits can be configured to perform the activation operation asynchronously; and / or, the plurality of storage-computing circuits can be configured to perform the computing operation asynchronously; and / or, the plurality of storage-computing circuits can be configured to perform the pre-charge operation asynchronously.

[0107] In one example, as shown in FIG. 9, when the plurality of storage-computing circuits comprises 16 storage-computing circuits, for the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the i-1th row or the ith row in the matrix is located, the storage-computing integrated device can be configured to perform the activation operation asynchronously, and perform the computing operation and the pre-charge operation synchronously. In FIG. 9, the storage banks in the 16 storage-computing circuits are denoted as BA0 to BA15, and the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the i-1th row is located are denoted as ACT R[i-1], MUL R[i-1] and PCH R[i-1] respectively, the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the ith row is located are denoted as ACT R[i], MUL R[i] and PCH R[i] respectively, and the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the i+1th row is located are denoted as ACT R[i+1], MUL R[i+1] and PCH R[i+1] respectively.

[0108] In one example, as shown in FIG. 10, when the plurality of storage-computing circuits comprises 16 storage-computing circuits, for the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the i-1th row or the ith row in the matrix is located, the storage-computing integrated device can be configured to perform the activation operation and the pre-charge operation asynchronously, and perform the computing operation asynchronously. In FIG. 10, the storage banks in the 16 storage-computing circuits are denoted as BA0 to BA15, and the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the i-1th row is located are denoted as ACT R[i-1], MUL R[i-1] and PCH R[i-1] respectively, the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the ith row is located are denoted as ACT R[i], MUL R[i] and PCH R[i] respectively, and the activation operation, the computing operation and the pre-charge operation of the storage row where the data of the i+1th row is located are denoted as ACT R[i+1], MUL R[i+1] and PCH R[i+1] respectively.

[0109] In the above FIGS. 8-10, the time of the activation operation is denoted as tRCD, the time of the pre-charge operation is denoted as tRP, and the time of the computing operation is denoted as tGEMV, and the time of the activation operation and the pre-charge operation is 18 ns, and the time of the computing operation is 256 ns.

[0110] For the sake of understanding, the following will take the current DRAM for performing the multiplication operation of matrix and vector, and the plurality of signals and the plurality of operations involved in the operation of the memory-computing integrated device for implementing the multiplication operation of matrix and vector as examples to illustrate the scheme provided by the embodiments of the present application. The plurality of signals can include a clock signal CLK and a signal output by a storage controller, the signal output by the storage controller including an activation signal sACT, an operation signal sMUL and a pre-charge signal sPCH. The plurality of operations can include an activation operation ACT on the storage rows in the first storage bank BA1 and the second storage bank BA2, an operation operation MUL and a pre-charge operation PCH, the operation operation MUL of the first computing unit PU1 corresponding to the first storage bank BA1, and the operation operation MUL of the second computing unit PU2 corresponding to the second storage bank BA2.

[0111] FIG. 11 shows a schematic diagram of the plurality of signals and the plurality of operations involved in the operation of the current DRAM. Among them, the storage controller outputs the signals sACT, sMUL and sPCH in sequence and discontinuously, the first storage bank BA1 and the second storage bank BA2 perform the activation operation ACT based on the signal sACT, then the first computing unit PU1 and the second computing unit PU2 perform the operation operation MUL based on the signal sMUL, and finally the first storage bank BA1 and the second storage bank BA2 perform the activation operation ACT based on the signal sPCH. In the above operation process, there is an extra time between the two adjacent calculation operations MUL of each computing unit in the first computing unit PU1 and the second computing unit PU2, and the extra time is the time occupied by the activation operation ACT and the pre-charge operation PCH.

[0112] FIG. 12 and FIG. 13 show a schematic diagram of the plurality of signals and the plurality of operations involved in the operation of the memory-computing integrated device. Among them, the storage controller outputs sACT, sMUL and sPCH in sequence and continuously, the first storage bank BA1 and the second storage bank BA2 perform the activation operation ACT based on the signal sACT before the end of the last calculation operation MUL, the first computing unit PU1 and the second computing unit PU2 perform the next calculation operation MUL based on the signal sMUL after the end of the last calculation operation MUL, and the first storage bank BA1 and the second storage bank BA2 perform the pre-charge operation PCH corresponding to the last calculation operation MUL in the process of the next calculation operation MUL based on the signal sPCH. In the above operation process, the two adjacent operation operations MUL of each computing unit in the first computing unit PU1 and the second computing unit PU2 are continuous, i.e. there is no extra time. In FIG. 12, the activation operation, the calculation operation and the pre-charge operation of different storage-computing circuits in the memory-computing integrated device are taken as examples for illustration; in FIG. 13, the activation operation, the calculation operation and the pre-charge operation of different storage-computing circuits in the memory-computing integrated device are taken as examples for illustration.

[0113] In the embodiment of the present application, the storage-computing integrated device can receive a first activation signal, a second operation signal and a first pre-charge signal, perform an activation operation on the second storage row in the process of the first computing operation according to the first activation signal, perform a second computing operation at the end of the first computing operation according to the second operation signal, and perform a pre-charge operation on the first storage row in the process of the second computing operation according to the first pre-charge signal, thereby realizing the continuity of the two computing operations in the matrix and vector operation, avoiding the additional time occupied by the activation operation and the pre-charge operation in the two computing operations, thereby improving the bandwidth utilization rate of the storage body in the storage-computing integrated device to 100%, and further improving the computing performance of the storage-computing integrated device.

[0114] Based on this, the embodiment of the present application also provides a storage-computing integrated device. As shown in FIG. 14, the storage-computing integrated device includes an interface circuit and at least one storage-computing circuit. Optionally, each storage-computing circuit in the at least one storage-computing circuit can include a computing unit and a storage body, and the storage body can include a plurality of storage arrays and a plurality of row buffers, etc. Further, each storage body can also include a row decoder, a column decoder, a sense amplifier and an intermediate input-output trace, etc. Optionally, the storage-computing integrated device can also include a global buffer and a peripheral circuit. The peripheral circuit can include an interface circuit, a power supply circuit, a driving circuit, a global flexible amplifier and a global input-output trace, etc.

[0115] In the embodiment of the present application, the interface circuit can be used to support the device to perform one or more steps in S201, S202, S203 or S204 in the above-mentioned method embodiment; and the at least one storage-computing circuit can be used to support the device to perform the steps of performing a first computing operation on a plurality of first data and a vector according to a first operation signal, performing an activation operation on a second storage row, performing a second computing operation on a plurality of second data and a vector according to a second operation signal, and performing a pre-charge on a first storage row, etc. in the above-mentioned method embodiment.

[0116] The embodiment of the present application also provides a storage controller used for coupling with a storage-computing integrated device. In the embodiment of the present application, the storage controller can be used to perform one or more steps in transmitting a first operation signal, a first activation signal, a second operation signal, a first pre-charge signal, a synchronous activation signal, a synchronous computing signal or a synchronous pre-charge signal in the above-mentioned method embodiment.

[0117] In another embodiment of the present application, a storage device is also provided, as shown in FIG. 15, which includes the in-memory computing device and the storage controller, the in-memory computing device is any one of the in-memory computing devices provided above, and is configured to perform the steps of the in-memory computing device in the method embodiments provided above; the storage controller is configured to perform the steps of the storage controller in the method embodiments provided above.

[0118] In another embodiment of the present application, an electronic device is also provided, which includes the processor and the storage device provided above.

[0119] Optionally, in the electronic device, the storage controller can be integrated with the in-memory computing device, or can be integrated with the processor, and the embodiments of the present application do not make specific limitations in this regard.

[0120] It can be understood that all the related contents of the steps involved in the above method embodiments can be cited into the embodiments of the in-memory computing device, the embodiments of the storage controller, and the embodiments of the electronic device, and the embodiments of the present application will not be repeated here.

[0121] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0122] The units described as separate components can or can not be physically separate, and the components shown as units can be one physical unit or multiple physical units, that is, can be located in one place, or can be distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0123] The integrated units, if implemented in the form of software function units and sold or used as independent products, can be stored in a readable storage medium, which can include: U disk, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk, and various storage program codes. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence or say the part of the technical solutions of the present application that make contributions to the prior art or the whole or part of the technical solutions.

[0124] In another embodiment of the present application, a readable storage medium is also provided, which stores computer-executable instructions, and when a device (which can be a single-chip microcomputer, a chip, etc.) executes the steps of the storage controller in the above method embodiments.

[0125] In another embodiment of the present application, a readable storage medium is also provided, which stores computer-executable instructions, and when a device (which can be a single-chip microcomputer, a chip, etc.) executes the steps of the storage controller in the above method embodiments.

[0126] In another embodiment of the present application, a readable storage medium is also provided, which stores computer-executable instructions, and when a device (which can be a single-chip microcomputer, a chip, etc.) executes the steps of the storage controller in the above method embodiments.

[0127] In another embodiment of the present application, a readable storage medium is also provided, which stores computer-executable instructions, and when a device (which can be a single-chip microcomputer, a chip, etc.) executes the steps of the storage controller in the above method embodiments.

[0128] Finally, it should be noted that the above description is merely a specific implementation of the present application, and the protection scope of the present application is not limited thereto, and any change or replacement within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An arithmetic method characterized by comprising: The method is applied to a storage-computing integrated device, the storage-computing integrated device comprises at least one storage-computing circuit, each storage-computing circuit is used for implementing multiplication operation of a matrix and a vector, the matrix comprises a plurality of first data and a plurality of second data, the storage-computing circuit comprises a first storage row and a second storage row located in different storage arrays, the plurality of first data is stored in the first storage row, and the plurality of second data is stored in the second storage row, and the method comprises: The storage-computing integrated device receives a first operation signal, and the first operation signal is used for indicating that a first calculation operation is performed on the plurality of first data and the vector; The storage-computing integrated device receives a first activation signal, and the first activation signal is used for indicating that an activation operation is performed on the second storage row in the process of the first calculation operation; The storage-computing integrated device receives a second operation signal, and the second operation signal is used for indicating that a second calculation operation is performed on the plurality of second data and the vector when the first calculation operation ends.

2. The method of claim 1, wherein, The method further comprises: The storage-computing integrated device receives a first pre-charge signal, and the first pre-charge signal is used for indicating that a pre-charge operation is performed on the first storage row in the process of the second calculation operation.

3. The method according to claim 1 or 2, characterized in that, Each storage-computing circuit further comprises a calculation unit, a first row buffer corresponding to a storage array where the first storage row is located, and a second row buffer corresponding to a storage array where the second storage row is located; and the method further comprises: When the first operation signal is received, the first row buffer acquires the plurality of first data from the first storage row and caches; and the calculation unit acquires the plurality of first data from the first row buffer and performs the first calculation operation on the plurality of first data and the vector; When the second operation signal is received, the second row buffer acquires the plurality of second data from the second storage row and caches; and the calculation unit acquires the plurality of second data from the second row buffer when the first calculation operation ends, and performs the second calculation operation on the plurality of second data and the vector.

4. The method according to any one of claims 1 to 3, characterized in that, The storage-computing integrated device comprises a plurality of storage-computing circuits, and the plurality of storage-computing circuits are used for implementing multiplication operation of different matrices and vectors.

5. The method of claim 4, wherein, The plurality of storage-computing circuits asynchronously perform the activation operation; Alternatively, the method further comprises: The storage-computing integrated device receives a synchronous activation signal, and the synchronous activation signal is used for indicating that the plurality of storage-computing circuits synchronously perform the activation operation.

6. The method according to claim 4 or 5, characterized in that, The plurality of storage-computing circuits asynchronously perform the first calculation operation or the second calculation operation; or, the method further comprises: The storage-computing integrated device receives a synchronous operation signal, and the synchronous operation signal is used for indicating that the plurality of storage-computing circuits synchronously perform the first calculation operation or the second calculation operation.

7. The method according to any one of claims 4-6, characterized in that, The plurality of storage-computing circuits asynchronously perform the pre-charge operation; or, the method further comprises: The storage-computing integrated device receives a synchronous pre-charge signal, and the synchronous pre-charge signal is used for indicating that the plurality of storage-computing circuits synchronously perform the pre-charge operation.

8. An arithmetic method characterized by comprising: The method is applied to a storage controller, the storage controller is used for being coupled with a storage-computing integrated device, the storage-computing integrated device includes at least one storage computing circuit, each storage computing circuit is used for implementing multiplication operation of a matrix and a vector, the matrix includes a plurality of first data and a plurality of second data, the storage computing circuit includes a first storage row and a second storage row located in different storage arrays, the plurality of first data is stored in the first storage row, and the plurality of second data is stored in the second storage row, and the method includes: The storage controller sends a first operation signal, and the first operation signal is used to indicate that a first calculation operation is performed on the plurality of first data and the vector; The storage controller sends a first activation signal, and the first activation signal is used to perform an activation operation on the second storage row in the process of the first calculation operation; The storage controller sends a second operation signal, and the second operation signal is used to start a second calculation operation on the plurality of second data and the vector at the end of the first calculation operation.

9. The method of claim 8, wherein, The method further includes: The storage controller sends a first pre-charge signal, and the first pre-charge signal is used to perform a pre-charge operation on the first storage row in the process of the second calculation operation.

10. The method according to claim 8 or 9, characterized in that, The storage-computing integrated device includes a plurality of storage computing circuits, and the plurality of storage computing circuits are used for implementing multiplication operation of different matrices and vectors.

11. The method of claim 10, wherein, The plurality of storage computing circuits asynchronously perform the activation operation; Alternatively, the method further includes: The storage controller sends a synchronous activation signal, and the synchronous activation signal is used to indicate that the plurality of storage computing circuits synchronously perform the activation operation.

12. The method according to claim 10 or 11, characterized in that, The plurality of storage computing circuits asynchronously perform the first calculation operation or the second calculation operation; or, the method further includes: The storage controller sends a synchronous operation signal, and the synchronous operation signal is used to indicate that the plurality of storage computing circuits synchronously perform the first calculation operation or the second calculation operation.

13. The method according to any one of claims 10-12, characterized in that, The plurality of storage computing circuits asynchronously perform the pre-charge operation; or, the method further includes: The storage controller sends a synchronous pre-charge signal, and the synchronous pre-charge signal is used to indicate that the plurality of storage computing circuits synchronously perform the pre-charge operation.

14. A storage-computing integrated device, comprising: The storage-computing integrated device includes at least one storage computing circuit, each storage computing circuit in the at least one storage computing circuit is used for implementing multiplication operation of a matrix and a vector, the matrix includes a plurality of first data and a plurality of second data, each storage computing circuit includes a first storage row and a second storage row located in different storage arrays, the plurality of first data is stored in the first storage row, and the plurality of second data is stored in the second storage row; and the device further includes: An interface circuit is used for receiving a first operation signal, and the first operation signal is used to indicate that a first calculation operation is performed on the plurality of first data and the vector; The interface circuit is further used for receiving a first activation signal, and the first activation signal is used to indicate that an activation operation is performed on the second storage row in the process of the first calculation operation; The interface circuit is further configured to receive a second operation signal, the second operation signal being used to indicate that a second computation operation on the plurality of second data and the vector is started at the end of the first computation operation.

15. The apparatus of claim 14, wherein, The interface circuit is further configured to receive a first precharge signal, the first precharge signal being used to indicate that a precharge operation on the first storage row is performed during the second computation operation.

16. The apparatus of claim 14 or 15, wherein, Each storage computation circuit further includes a computation unit, a first row buffer corresponding to a storage array where the first storage row is located, and a second row buffer corresponding to a storage array where the second storage row is located: The first row buffer is configured to obtain and buffer the plurality of first data from the first storage row when the first operation signal is received. The computation unit is configured to obtain the plurality of first data from the first row buffer and perform a first computation operation on the plurality of first data and the vector. The second row buffer is configured to obtain and buffer the plurality of second data from the second storage row when the second operation signal is received. The computation unit is configured to obtain the plurality of second data from the second row buffer at the end of the first computation operation and perform a second computation operation on the plurality of second data and the vector.

17. The apparatus of any of claims 14-16, wherein, The storage computation circuit is configured to perform the first computation operation on the plurality of first data and the vector.

18. The apparatus of claim 17, wherein, The plurality of storage computation circuits perform the activation operation asynchronously; or the interface circuit is further configured to receive a synchronous activation signal, the synchronous activation signal being used to indicate that the plurality of storage computation circuits perform the activation operation synchronously.

19. The apparatus of claim 17 or 18, wherein, The plurality of storage computation circuits perform the first computation operation or the second computation operation asynchronously; or the interface circuit is further configured to receive a synchronous operation signal, the synchronous operation signal being used to indicate that the plurality of storage computation circuits perform the first computation operation or the second computation operation synchronously.

20. The apparatus of any of claims 17-19, wherein, The plurality of storage computation circuits perform the precharge operation asynchronously; or the interface circuit is further configured to receive a synchronous precharge signal, the synchronous precharge signal being used to indicate that the plurality of storage computation circuits perform the precharge operation synchronously.

21. A storage controller, comprising: The storage controller is configured to be coupled with a storage computation device, the storage computation device including at least one storage computation circuit, each storage computation circuit being configured to implement a multiplication operation of a matrix and a vector, the matrix including a plurality of first data and a plurality of second data, the storage computation circuit including a first storage row and a second storage row located in different storage arrays, the plurality of first data being stored in the first storage row, and the plurality of second data being stored in the second storage row. The storage controller is configured to send a first operation signal, the first operation signal being used to indicate that a first computation operation on the plurality of first data and the vector is performed. The storage controller is further configured to send a first activation signal, the first activation signal being used to perform an activation operation on the second storage row during the first computation operation. The storage controller is further configured to send a second operation signal, the second operation signal being used to start a second computation operation on the plurality of second data and the vector when the first computation operation ends.

22. The storage controller of claim 21, wherein, The storage controller is further configured to send a first precharge signal, the first precharge signal being used to perform a precharge operation on the first storage row during the second computation operation.

23. The storage controller of claim 21 or 22, wherein, The storage and computation integrated device includes a plurality of storage computation circuits, the plurality of storage computation circuits being used to implement different matrix and vector multiplication operations.

24. The storage controller of claim 23, wherein, The plurality of storage computation circuits perform the activation operation asynchronously; or the storage controller is further configured to send a synchronous activation signal, the synchronous activation signal being used to instruct the plurality of storage computation circuits to perform the activation operation synchronously.

25. The storage controller of claim 23 or 24, wherein, The plurality of storage computation circuits perform the first computation operation or the second computation operation asynchronously; or the storage controller is further configured to send a synchronous operation signal, the synchronous operation signal being used to instruct the plurality of storage computation circuits to perform the first computation operation or the second computation operation synchronously.

26. The storage controller of any of claims 23-25, wherein, The plurality of storage computation circuits perform the precharge operation asynchronously; or the storage controller is further configured to send a synchronous precharge signal, the synchronous precharge signal being used to instruct the plurality of storage computation circuits to perform the precharge operation synchronously.

27. A memory device, comprising: The storage device includes a storage and computation integrated device and a storage controller, the storage and computation integrated device being the device of any one of claims 14-20, and the storage controller being the storage controller of any one of claims 21-26.

28. An electronic device, comprising: The electronic device includes a processor and the storage device of claim 27.

29. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions, when the instructions are run on a device, causing the device to perform the method of any one of claims 1-13.

30. A computer program product, characterised in that, The computer program product includes a computer program, when the computer program is run on a device, causing the device to perform the method of any one of claims 1-13.

Images