Compute-in-memory chip, instruction scheduling method, and related apparatus

Abstract

The present application discloses a compute-in-memory chip, an instruction scheduling method, and a related apparatus. The compute-in-memory chip comprises an instruction memory, an instruction scheduler, and at least one compute-in-memory memory; each compute-in-memory memory comprises at least one storage array; the instruction memory is used for acquiring a first tensor instruction to be executed; and the instruction scheduler is used for scheduling, on the basis of the association relationship between the first tensor instruction and a second tensor instruction and the state of a target storage array needing to be operated for executing the first tensor instruction, the first tensor instruction to the compute-in-memory memory to which the target storage array belongs so that the compute-in-memory memory executes the first tensor instruction. According to embodiments of the present application, diversified compute-in-memory computing can be supported, efficient out-of-order execution scheduling of a tensor instruction set is achieved on the basis of the compute-in-memory chip, and the requirements of compute-in-memory technology for high concurrency and high throughput rate are met.

Description

Storage and computing integrated chip, instruction scheduling method and related devices

[0001] This application claims priority to the Chinese patent application with application number 202410283167.7 filed with the State Intellectual Property Office of China on March 12, 2024, and priority to the Chinese patent application with the invention name “Storage and computing integrated chip, instruction scheduling method and related devices”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of chip technology, and in particular to a storage and computing integrated chip, an instruction scheduling method and related devices. Background Art

[0003] In recent years, with the rapid development of emerging applications and the explosive growth of information, computing scale has rapidly expanded and computing forms have become increasingly diversified, placing higher demands on the computing performance, storage performance, and scalability of hardware computing systems. To address the computing bottlenecks caused by the von Neumann architecture, integrated memory and compute technology has emerged as one of the most promising chip architecture solutions for high computing power, high energy efficiency, and high scalability. By implementing embedded computing functions within the memory chip, the scale and frequency of data transfer between the storage module and the compute module are significantly reduced, significantly improving the computing power and energy efficiency of the computing system. Existing mainstream instruction sets are generally scalar instruction sets, such as the fifth-generation Reduced Instruction Set Computer (RISC-V), the Microprocessor without Interlocked Pipelined Stages (MIPS) instruction set, and the X86 instruction set. Using scalar instruction sets for integrated memory and compute tasks will significantly increase the number of instructions, resulting in excessive compilation delays and difficulty meeting the high concurrency and high throughput requirements of integrated memory and compute technology. Summary of the Invention

[0004] The embodiments of the present application provide a storage-computing integrated chip, an instruction scheduling method and related devices, which can support diversified storage-computing integrated calculations, and based on the storage-computing integrated chip, implement efficient out-of-order execution scheduling of tensor instruction sets, meeting the requirements of storage-computing integrated technology for high concurrency and high throughput.

[0005] In a first aspect, an embodiment of the present application provides a memory-computing integrated chip, comprising an instruction memory, an instruction scheduler, and at least one memory-computing integrated memory; each memory-computing integrated memory comprises at least one storage array;

[0006] An instruction memory for obtaining a first tensor instruction to be executed;

[0007] An instruction scheduler is configured to schedule the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated for executing the first tensor instruction, so that the storage-computing integrated memory executes the first tensor instruction;

[0008] The second tensor instruction is a tensor instruction cached in the instruction scheduler except the first tensor instruction.

[0009] It can be seen that the embodiment of the present application provides a storage-computing integrated chip architecture that supports out-of-order execution scheduling of tensor instructions. The tensor instructions to be executed that enter the storage-computing integrated chip first enter the instruction memory cache. After being processed by the instruction memory, the tensor instructions to be executed are determined by the instruction scheduler based on the association relationship between them and other tensor instructions in the cache and the state of the target storage array required to execute the tensor instruction. Whether the tensor instruction meets the scheduling conditions is determined. If it is satisfied, the instruction scheduler schedules the tensor instruction to the corresponding storage-computing integrated memory so that the corresponding storage-computing integrated memory executes the tensor instruction, thereby realizing out-of-order execution scheduling of tensor instructions, and then meeting the requirements of storage-computing integrated technology for high concurrency and high throughput, and realizing high parallelism of multiple complex operators.

[0010] In one possible implementation, in terms of scheduling the first tensor instruction to the storage-computation integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated when executing the first tensor instruction, the instruction scheduler is specifically configured to:

[0011] When it is determined based on the association relationship that the first tensor instruction does not have a second tensor instruction with a preceding dependency and the target storage array is in an idle state, the first tensor instruction is scheduled to the storage-computing integrated memory to which the target storage array belongs.

[0012] In this implementation, for the tensor instructions to be executed cached in the instruction scheduler, if it is determined based on the association relationship between the tensor instruction and other tensor instructions that the tensor instruction has no pre-dependency relationship, and the storage array corresponding to the tensor instruction is in an idle state, then it can be determined that the tensor instruction meets the scheduling conditions, so that it can be scheduled to the corresponding storage and computing integrated memory for execution, thereby realizing the out-of-order execution scheduling of tensor instructions.

[0013] In one possible implementation, each storage-computing integrated memory further includes a control unit;

[0014] In terms of scheduling the first tensor instruction to the storage-computation integrated memory to which the target storage array belongs, the instruction scheduler is specifically used to: schedule the first tensor instruction to a control unit of the storage-computation integrated memory to which the target storage array belongs;

[0015] The control unit is configured to, if the target storage array is being operated to execute a third tensor instruction, wait for the third tensor instruction to be completed and then operate the target storage array to execute the first tensor instruction; if the target storage array is in an idle state, operate the target storage array to execute the first tensor instruction.

[0016] In this implementation, the tensor instructions entering the storage-computing integrated memory are cached by the control unit. Based on the pipeline operation mode, the control unit will again determine whether the target storage array is being operated to execute other tensor instructions, so as to immediately execute the first tensor instruction or wait according to the situation to avoid instruction execution conflicts.

[0017] In one possible implementation, in operating the target storage array to execute the first tensor instruction, the control unit is specifically configured to:

[0018] The first tensor instruction is executed by operating the target storage array based on the instruction type of the first tensor instruction.

[0019] In this implementation, the control unit can perform specific operations on the target storage array based on the instruction type of the first tensor instruction to execute the first tensor instruction.

[0020] In one possible implementation, in terms of operating a target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction, the control unit is specifically configured to:

[0021] If the instruction type is an array control instruction, the storage and calculation function of the row or column of the target storage array is disabled, or the storage and calculation function of the row or column of the target storage array is enabled.

[0022] In this implementation, when the tensor instruction to be processed is an array control instruction, the control unit can control the state of the target storage array to enable or disable the storage and computing function of the storage array.

[0023] In one possible implementation, each integrated storage and computing memory further includes a near-memory computing circuit; and in terms of operating a target storage array based on an instruction type of the first tensor instruction to execute the first tensor instruction, the control unit is specifically configured to:

[0024] If the instruction type is an in-memory calculation instruction, the input tensor is read, and the near-memory calculation circuit is called to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

[0025] In this implementation, although the storage-computing integrated memory integrates multiple computing modes of in-memory computing and near-memory computing, if the tensor instruction to be processed is an in-memory computing instruction, the control unit can also perform a single in-memory computing. The storage-computing integrated operation is flexible, and the reading of data and the calling of the near-memory computing circuit are carried out simultaneously, which is conducive to improving concurrency and throughput.

[0026] In one possible implementation, the control unit is further configured to cache states of each processing module in the near-memory computing circuit; and in calling the near-memory computing circuit to assist in completing in-memory computation between an input tensor and a first tensor stored in the target storage array, the control unit is specifically configured to:

[0027] When the first processing module in the near-memory computing circuit is in an idle state, the first processing module is called to assist in completing the in-memory computing of the input tensor and the first tensor stored in the target storage array.

[0028] In this implementation, the control unit calls the first processing module to assist in the in-memory calculation based on the state of the first processing module that needs to be called for the in-memory calculation.

[0029] In a possible implementation, the instruction memory is further used to cache an operation code of the first tensor instruction; and the first processing module is determined based on the operation code of the first tensor instruction.

[0030] In this implementation, the control unit can determine which processing module in the near-memory calculation needs to be called for the in-memory calculation based on the operation code of the first tensor instruction.

[0031] In one possible implementation, each integrated storage and computing memory further includes a near-memory computing circuit; and in terms of operating a target storage array based on an instruction type of the first tensor instruction to execute the first tensor instruction, the control unit is specifically configured to:

[0032] If the instruction type is a near memory calculation instruction, then when the near memory calculation circuit has an input tensor, the second tensor stored in the target storage array is read to the near memory calculation circuit, and the near memory calculation circuit is called to complete the near memory calculation of the input tensor and the second tensor; or when the near memory calculation circuit has no input tensor, the second tensor and the third tensor stored in the target storage array are read to the near memory calculation circuit, and the near memory calculation circuit is called to complete the near memory calculation of the second tensor and the third tensor.

[0033] In this implementation, if the tensor instruction to be processed is a near-memory calculation instruction, the control unit can also perform a single near-memory calculation. The integrated storage and calculation operation is flexible, and the reading of data and the calling of the near-memory calculation circuit are carried out simultaneously, which is conducive to improving concurrency and throughput.

[0034] In one possible implementation, the control unit is further configured to cache states of each processing module in the near-memory computing circuit; and in calling the near-memory computing circuit to complete the near-memory computing of the input tensor and the second tensor, the control unit is specifically configured to:

[0035] When the second processing module in the near-memory computing circuit is in an idle state, calling the second processing module to complete the near-memory computing of the input tensor and the second tensor;

[0036] In calling the near-memory computing circuit to complete the near-memory computing of the second tensor and the third tensor, the control unit is specifically configured to:

[0037] When the second processing module is in an idle state, the second processing module is called to complete the near-memory calculation of the second tensor and the third tensor.

[0038] In this implementation, the control unit calls the processing module to perform the near memory calculation based on the state of the second processing module that needs to be called for the near memory calculation.

[0039] In a possible implementation, the instruction memory is further used to cache an operation code of the first tensor instruction; and the second processing module is determined based on the operation code of the first tensor instruction.

[0040] In this implementation, the control unit can determine which processing module in the near-memory computing circuit needs to be called for the near-memory computing based on the operation code of the first tensor instruction.

[0041] In a possible implementation, the storage-computing integrated chip further includes a cache module, and the cache module includes multiple cache areas;

[0042] The control unit is further used to write the calculation result of the first tensor instruction into a target cache area among the multiple cache areas, and store the address of the target cache area in association with the target location; the target location is the storage location of the calculation result in the target storage array.

[0043] In this implementation, after the control unit writes the result of the in-memory calculation or the near-memory calculation into the target cache area, it can associate the address of the target cache area with the storage location of the calculation result in the target storage array to establish a mapping between the cache and the array.

[0044] In one possible implementation, the storage-computing integrated chip further includes a memory output scheduling module;

[0045] The memory output scheduling module is used to send the calculation result of the first tensor instruction to the central processing unit or the dynamic random access memory.

[0046] In this implementation, the integrated storage and computing chip can output the calculation results of tensor instructions to the central processing unit or dynamic random access memory through the memory output scheduling module.

[0047] In a second aspect, an embodiment of the present application provides an instruction scheduling method applied to a memory-integrated chip, the memory-integrated chip including an instruction memory, an instruction scheduler, and at least one memory-integrated memory; each memory-integrated memory includes at least one storage array; the method includes:

[0048] The instruction memory obtains the first tensor instruction to be executed;

[0049] The instruction scheduler schedules the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated for executing the first tensor instruction, so that the storage-computing integrated memory executes the first tensor instruction;

[0050] The second tensor instruction is a tensor instruction cached in the instruction scheduler except the first tensor instruction.

[0051] In one possible implementation, based on the association between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated when executing the first tensor instruction, scheduling the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs includes:

[0052] When the instruction scheduler determines based on the association relationship that the first tensor instruction does not have a second tensor instruction with a preceding dependency and the target storage array is in an idle state, the instruction scheduler schedules the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs.

[0053] In one possible implementation, each integrated storage and computing memory further includes a control unit; and dispatching the first tensor instruction to the integrated storage and computing memory to which the target storage array belongs includes:

[0054] The instruction scheduler schedules the first tensor instruction to a control unit of the storage-computation integrated memory to which the target storage array belongs;

[0055] Execute the first tensor instruction, including:

[0056] If the target storage array is being operated to execute the third tensor instruction, the control unit waits for the third tensor instruction to be executed and then operates the target storage array to execute the first tensor instruction; if the target storage array is in an idle state, the control unit operates the target storage array to execute the first tensor instruction.

[0057] In a possible implementation, operating a target storage array to execute a first tensor instruction includes:

[0058] The control unit operates the target storage array to execute the first tensor instruction based on an instruction type of the first tensor instruction.

[0059] In one possible implementation, operating a target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction includes:

[0060] If the instruction type is an array control instruction, the control unit disables the storage and calculation function of the row or column of the target storage array, or enables the storage and calculation function of the row or column of the target storage array.

[0061] In one possible implementation, each integrated storage and computing memory further includes a near-memory computing circuit; and operating a target storage array based on an instruction type of the first tensor instruction to execute the first tensor instruction includes:

[0062] If the instruction type is an in-memory calculation instruction, the control unit reads the input tensor and calls the near-memory calculation circuit to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

[0063] In one possible implementation, the control unit caches states of each processing module in the near-memory computing circuit; calling the near-memory computing circuit to assist in completing an in-memory computation between an input tensor and a first tensor stored in a target storage array includes:

[0064] When the first processing module in the near memory computing circuit is in an idle state, the control unit calls the first processing module to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

[0065] In a possible implementation, the instruction memory caches an operation code of the first tensor instruction; and the first processing module is determined based on the operation code of the first tensor instruction.

[0066] In one possible implementation, each integrated storage and computing memory further includes a near-memory computing circuit; and operating a target storage array based on an instruction type of the first tensor instruction to execute the first tensor instruction includes:

[0067] If the instruction type is a near memory calculation instruction, the control unit reads the second tensor stored in the target storage array to the near memory calculation circuit when the near memory calculation circuit has an input tensor, and calls the near memory calculation circuit to complete the near memory calculation of the input tensor and the second tensor; or if the near memory calculation circuit has no input tensor, the control unit reads the second tensor and the third tensor stored in the target storage array to the near memory calculation circuit, and calls the near memory calculation circuit to complete the near memory calculation of the second tensor and the third tensor.

[0068] In one possible implementation, the control unit caches states of each processing module in the near-memory computing circuit; calling the near-memory computing circuit to complete the near-memory computing of the input tensor and the second tensor includes:

[0069] The control unit calls the second processing module to complete the near memory calculation of the input tensor and the second tensor when the second processing module in the near memory calculation circuit is in an idle state;

[0070] Call the near-memory computation circuit to complete the near-memory computation of the second and third tensors, including:

[0071] When the second processing module is in an idle state, the control unit calls the second processing module to complete the near-memory calculation of the second tensor and the third tensor.

[0072] In a possible implementation, the instruction memory caches an operation code of the first tensor instruction; and the second processing module is determined based on the operation code of the first tensor instruction.

[0073] In one possible implementation, the integrated storage and computing chip further includes a cache module, and the cache module includes multiple cache areas; and the method further includes:

[0074] The control unit writes the calculation result of the first tensor instruction into a target cache area among the multiple cache areas, and stores the address of the target cache area in association with a target location; the target location is the storage location of the calculation result in the target storage array.

[0075] In one possible implementation, the integrated storage and computing chip further includes a memory output scheduling module; and the method further includes:

[0076] The memory output scheduling module sends the calculation result of the first tensor instruction to the central processing unit or the dynamic random access memory.

[0077] It should be understood that since the method embodiment and the storage-computing integrated chip embodiment are different presentation forms of the same technical concept, the content of the first aspect of the embodiment of this application should be simultaneously adapted to the second aspect of the embodiment of this application, and can achieve the same or similar beneficial effects, and will not be repeated here.

[0078] In a third aspect, an embodiment of the present application provides a memory-computing integrated chip, including an instruction memory, an instruction scheduler, at least one memory-computing integrated memory, and a memory output scheduling module;

[0079] Each integrated storage and computing memory includes a control unit, at least one storage array, a near-storage computing circuit, and a cache module;

[0080] The instruction scheduler is connected to the instruction memory and the control unit respectively, and the control unit is connected to at least one storage array, a near memory computing circuit and a cache module respectively;

[0081] At least one storage array is connected to the near memory computing circuit and the cache module respectively; the near memory computing circuit is connected to the cache module;

[0082] Each storage-computing integrated memory is connected to the memory output scheduling module respectively.

[0083] In one possible implementation, the instruction memory is used to obtain a first tensor instruction to be executed;

[0084] An instruction scheduler is configured to schedule the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated for executing the first tensor instruction, so that the storage-computing integrated memory executes the first tensor instruction;

[0085] The second tensor instruction is a tensor instruction cached in the instruction scheduler except the first tensor instruction.

[0086] In one possible implementation, in terms of scheduling the first tensor instruction to the storage-computation integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated when executing the first tensor instruction, the instruction scheduler is specifically configured to:

[0087] When it is determined based on the association relationship that the first tensor instruction does not have a second tensor instruction with a preceding dependency and the target storage array is in an idle state, the first tensor instruction is scheduled to the storage-computing integrated memory to which the target storage array belongs.

[0088] In one possible implementation, in terms of scheduling the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs, the instruction scheduler is specifically configured to: schedule the first tensor instruction to a control unit of the storage-computing integrated memory to which the target storage array belongs;

[0089] The control unit is configured to, if the target storage array is being operated to execute a third tensor instruction, wait for the third tensor instruction to be completed and then operate the target storage array to execute the first tensor instruction; if the target storage array is in an idle state, operate the target storage array to execute the first tensor instruction.

[0090] In one possible implementation, in operating the target storage array to execute the first tensor instruction, the control unit is specifically configured to:

[0091] The first tensor instruction is executed by operating the target storage array based on the instruction type of the first tensor instruction.

[0092] In one possible implementation, in terms of operating a target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction, the control unit is specifically configured to:

[0093] If the instruction type is an array control instruction, the storage and calculation function of the row or column of the target storage array is disabled, or the storage and calculation function of the row or column of the target storage array is enabled.

[0094] In one possible implementation, in terms of operating a target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction, the control unit is specifically configured to:

[0095] If the instruction type is an in-memory calculation instruction, the input tensor is read, and the near-memory calculation circuit is called to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

[0096] In one possible implementation, the control unit is further configured to cache states of each processing module in the near-memory computing circuit; and in calling the near-memory computing circuit to assist in completing in-memory computation between an input tensor and a first tensor stored in the target storage array, the control unit is specifically configured to:

[0097] When the first processing module in the near-memory computing circuit is in an idle state, the first processing module is called to assist in completing the in-memory computing of the input tensor and the first tensor stored in the target storage array.

[0098] In a possible implementation, the instruction memory is further used to cache an operation code of the first tensor instruction; and the first processing module is determined based on the operation code of the first tensor instruction.

[0099] In one possible implementation, in terms of operating a target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction, the control unit is specifically configured to:

[0100] If the instruction type is a near memory calculation instruction, then when the near memory calculation circuit has an input tensor, the second tensor stored in the target storage array is read to the near memory calculation circuit, and the near memory calculation circuit is called to complete the near memory calculation of the input tensor and the second tensor; or when the near memory calculation circuit has no input tensor, the second tensor and the third tensor stored in the target storage array are read to the near memory calculation circuit, and the near memory calculation circuit is called to complete the near memory calculation of the second tensor and the third tensor.

[0101] In one possible implementation, the control unit is further configured to cache states of each processing module in the near-memory computing circuit; and in calling the near-memory computing circuit to complete the near-memory computing of the input tensor and the second tensor, the control unit is specifically configured to:

[0102] When the second processing module in the near-memory computing circuit is in an idle state, calling the second processing module to complete the near-memory computing of the input tensor and the second tensor;

[0103] In calling the near-memory computing circuit to complete the near-memory computing of the second tensor and the third tensor, the control unit is specifically configured to:

[0104] When the second processing module is in an idle state, the second processing module is called to complete the near-memory calculation of the second tensor and the third tensor.

[0105] In a possible implementation, the instruction memory is further used to cache an operation code of the first tensor instruction; and the second processing module is determined based on the operation code of the first tensor instruction.

[0106] In one possible implementation, the cache module includes multiple cache areas;

[0107] The control unit is further used to write the calculation result of the first tensor instruction into a target cache area among the multiple cache areas, and store the address of the target cache area in association with the target location; the target location is the storage location of the calculation result in the target storage array.

[0108] In a possible implementation, the memory output scheduling module is configured to send a calculation result of the first tensor instruction to a central processing unit or a dynamic random access memory.

[0109] In a fourth aspect, an embodiment of the present application provides a computer device comprising a processor and a storage-computing integrated chip as in any one of the embodiments of the first aspect above, wherein the processor is configured to send a storage-computing integrated task to the storage-computing integrated chip, and the storage-computing integrated chip is configured to execute the storage-computing integrated task and return the execution result of the storage-computing integrated task to the processor.

[0110] In a fifth aspect, an embodiment of the present application provides a computer device comprising a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to implement the method in any one of the embodiments of the second aspect above when executed by the processor.

[0111] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program for execution by a device, and when the computer program is executed, implements the method in any one of the embodiments of the second aspect above.

[0112] In a seventh aspect, an embodiment of the present application provides a computer program product. When the computer program product is run by a device, the device executes a method as in any one of the embodiments of the second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0114] FIG1 is a schematic diagram of a method for filtering a vector using a scalar instruction;

[0115] FIG2 is a schematic diagram showing the requirements of storage-computing-integrated technology for instruction sets;

[0116] FIG3 is a schematic diagram illustrating the requirements of integrated storage and computing technology for chip architecture;

[0117] FIG4 is a schematic diagram illustrating the relationship between a chip architecture, a tensor instruction set, and a scheduling method provided in an embodiment of the present application;

[0118] FIG5 is a schematic diagram of the structure of a storage and computing integrated chip provided in an embodiment of the present application;

[0119] FIG6 is a schematic diagram of the structure of another integrated storage and computing chip provided in an embodiment of the present application;

[0120] FIG7 is a schematic diagram of an instruction association matrix provided in an embodiment of the present application;

[0121] FIG8 is a schematic diagram of an instruction scheduler performing instruction scheduling according to an embodiment of the present application;

[0122] FIG9 is a schematic diagram of the structure of another integrated storage and computing chip provided in an embodiment of the present application;

[0123] FIG10 is a schematic diagram of the structure of another integrated storage and computing chip provided in an embodiment of the present application;

[0124] FIG11 is a schematic diagram of a control unit executing a tensor instruction according to an embodiment of the present application;

[0125] FIG12 is a schematic diagram of the structure of another integrated storage and computing chip provided in an embodiment of the present application;

[0126] FIG13 is a schematic diagram of a near-memory computing circuit provided in an embodiment of the present application;

[0127] FIG14 is a schematic diagram of another control unit executing a tensor instruction according to an embodiment of the present application;

[0128] FIG15 is a schematic diagram of the structure of another integrated storage and computing chip provided in an embodiment of the present application;

[0129] FIG16 is a schematic diagram of a storage and computing integrated system provided in an embodiment of the present application;

[0130] FIG17 is a flow chart of an instruction scheduling method provided in an embodiment of the present application;

[0131] FIG18 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0132] FIG19 is a schematic structural diagram of another computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0133] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0134] References to "embodiments" herein mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0135] As used in this specification, the terms "component", "module", "system", etc. are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both an application running on a terminal device and a terminal device can be a component. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, through local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems via signals).

[0136] First, a brief introduction to the relevant terms and related technical background in this application is given to facilitate understanding by those skilled in the art.

[0137] (1) Read after Write: Read after Write, RAW;

[0138] (2) Write after Read: Write after Read, WAR;

[0139] (3) Write after Write: WAW;

[0140] (4) Operation Code, opcode;

[0141] (5) Brain Floating Point (BF);

[0142] (6) Floating Point: FP;

[0143] (7) Out-of-order Execution (OoOE);

[0144] (8) Central Processing Unit (CPU)

[0145] (9) Graphics Processing Unit (GPU)

[0146] (10) Energy efficiency ratio: the ratio of chip throughput to energy consumption, reflecting the performance per unit energy;

[0147] (11) Register: A small memory used by the CPU to temporarily store instructions, data, or addresses;

[0148] (12) Tensor: A multidimensional array.

[0149] RISC-V is a typical scalar instruction set with a modular design. Its instructions consist of several interchangeable base instruction sets and optional extended instruction sets. The base instruction set specifies instruction encoding, control flow, the number of registers (and their lengths), memory and addressing methods, logical (integer) operations, and other aspects. Using RISC-V (or other existing scalar instruction sets) for integrated storage and computation tasks would result in a significant increase in the number of instructions, excessive compilation latency, and low processing concurrency. As shown in Figure 1, using a conventional scalar instruction set (such as RISC-V) to filter a vector of length 1000 would generate 1000 sets of rd+cmp+st instructions to read, compare, and write back 1000 data points from memory. This would also result in frequent data transfers, resulting in low computational parallelism, poor processing speed, and power consumption. On this basis, the storage-computing integration technology puts forward new requirements for the instruction set, as shown in Figure 2, mainly including: (1) the need for a tensor-based instruction set that supports extremely high data parallelism and processing concurrency; (2) the internal array of the memory requires special operation instructions to support the array-level operations unique to storage-computing integration; (3) the near-memory computing circuit requires additional tensor instructions to complete near-memory computing operations; (4) a new compilation method for complex operators to tensor instruction sets needs to be designed. In summary, the existing scalar instruction set cannot meet the instruction requirements of storage-computing integration technology.

[0150] The integrated storage and computing architecture can be divided into near-memory computing architecture and in-memory computing architecture according to whether it performs calculations within the array. The so-called near-memory computing refers to shortening the physical distance between the memory and the logic processing module, so that the delay and power consumption of data transmission can be reduced; while in-memory computing refers to performing calculations inside the memory, fundamentally eliminating possible data transmission, thereby greatly reducing the system's delay and power consumption. The existing integrated storage and computing architecture is usually a single architecture model, lacking a general-purpose computing and storage-computing architecture that is compatible with near-memory computing and in-memory computing, such as the near-memory computing chip architecture with integrated HBM-PIM and the analog in-memory chip architecture for artificial intelligence (AI). Although these architectures can handle conventional in-memory computing or near-memory computing, they cannot meet the requirements of storage-computing technology for high-concurrency chip architecture. The specific reasons are mainly as follows: (1) The existing chip architecture focuses on processing data correlation between scalars and pays less attention to data correlation between tensors; (2) The existing chip architecture does not consider the state of the storage and computing array, but only focuses on the readiness of the scalar itself; (3) The existing chip architecture does not consider the state of the near-memory circuit, and the state of the near-memory circuit will directly affect whether a certain instruction can call the near-memory computing circuit at a certain moment; (4) The existing chip architecture does not consider the possible non-uniform distribution of tensor data on the array, because scalar storage is usually on the same array. On this basis, storage-computing integrated technology puts forward new requirements for chip architecture, as shown in Figure 3, which mainly include: (1) processing the correlation between tensor data; (2) paying attention to the status of the storage and computing array; (3) paying attention to the status of the modules in the near-memory computing circuit; (4) considering the possible non-uniform distribution of tensor data on the array.

[0151] This shows that current storage-computing integrated technology is still in its early stages. There are many research gaps in instruction sets, compilers, and instruction execution methods. A mature, practical storage-computing integrated hardware computing system has not yet been formed, and there are many problems that need to be solved urgently:

[0152] First, the most suitable data format for the storage-computing architecture is tensors, not scalars. However, the existing instruction set lacks instruction design for tensor processing and operator compilation and execution methods.

[0153] Second, storage-computing integration technology has multiple implementation modes and can use a variety of storage devices, requiring instruction sets and compilers to simultaneously support diverse architecture mappings. However, existing instruction sets and compilation and operation methods only support a single type of architecture or storage device, making it difficult to support diverse computing operators and computing modes.

[0154] Third, the processing efficiency of tensor instructions is greatly affected by the parallelism between instructions. However, the current instruction processing flow basically does not implement out-of-order execution of the tensor instruction set. The concurrency and throughput between instructions are low, which seriously affects the overall processing speed of computing tasks.

[0155] In general, various storage-computing models have their own strengths and weaknesses, requiring appropriate instruction sets and chip architectures for proper configuration. However, existing storage-computing technologies lack a targeted tensor instruction set, a chip architecture that supports it, or a corresponding scheduling method. A tensor instruction set and its efficient scheduling methods are essential for storage-computing technologies to support diverse computing (general-purpose and AI computing), high concurrency, and high throughput.

[0156] In order to address the shortcomings of the existing technology, as shown in Figure 4, this application has developed a tensor instruction set based on a storage-computing integrated chip architecture that supports diversified computing, and designed an efficient scheduling method for the tensor instruction set, so that the new storage-computing integrated architecture can realize diversified storage-computing integrated computing in the tensor domain.

[0157] The technical solution provided in this application is introduced in detail below in conjunction with specific implementation methods.

[0158] Please refer to Figure 5, which is a schematic diagram of the structure of a storage-computing integrated chip provided in an embodiment of the present application. As shown in Figure 5, the storage-computing integrated chip includes an instruction memory, an instruction scheduler, and at least one storage-computing integrated memory (such as N storage-computing integrated memories, N is greater than or equal to 1); each storage-computing integrated memory includes at least one storage array (such as n storage arrays, n is greater than or equal to 1). It should be understood that the storage array usually has corresponding row processors and column processors. Among them:

[0159] An instruction memory is used to obtain a first tensor instruction to be executed. Exemplarily, the first tensor instruction comes from a tensor instruction set, which is obtained by compiling operators such as filtering, sorting, summing, and averaging (including general operators and AI operators). Exemplarily, these operators can be compiled by an instruction compiler executing a storage-computing integration compilation method to obtain a tensor instruction set. The instruction compiler can be a compiler within the storage-computing integration chip, or a compiler independent of the storage-computing integration chip. Among them, the tensor instruction set is an instruction set for processing tensor data (multidimensional arrays). The tensor instruction set is designed to accelerate calculations in fields such as deep learning and artificial intelligence, where large-scale tensor data needs to be calculated in parallel. The tensor instruction set provides specialized instructions and hardware support so that operations on tensor data can be processed in parallel more efficiently. The scalar instruction set is an instruction set for processing single data elements. The operations it performs are usually performed on a single data or a group of independent data elements, such as integers, floating-point numbers, etc. In the scalar instruction set, each instruction processes only one data element and does not involve operations on multiple data elements at the same time.

[0160] Exemplarily, as shown in FIG6 , the instruction memory includes a transparent operator cache and a format conversion module, and the format conversion module includes an instruction fetch unit and an instruction decoding unit. The transparent operator cache is used to cache the tensor instructions to be executed in the tensor instruction set, the instruction fetch unit is used to read the tensor instructions to be executed from the transparent operator cache, and the instruction decoding unit is used to decode and convert the tensor instructions to be executed into the first tensor instruction adapted to the storage and computing integrated chip. Among them, the conversion operation may include, for example, splitting or merging. In this implementation, the instruction memory can convert complex operators into basic tensor instructions adapted to the execution of the storage and computing integrated chip, so that the subsequent storage and computing integrated memory adopts a storage and computing mode suitable for the tensor instruction and calls the adapted computing module to execute the tensor instruction, which is beneficial to reduce the power consumption and delay consumption of operator processing and improve the energy efficiency of processing.

[0161] An instruction scheduler is configured to schedule the first tensor instruction to the integrated storage and computing memory to which the target storage array belongs based on an association between the first tensor instruction and the second tensor instruction and a state of a target storage array required for executing the first tensor instruction, so that the integrated storage and computing memory executes the first tensor instruction. The second tensor instruction is a tensor instruction other than the first tensor instruction cached in the instruction scheduler.

[0162] Exemplarily, for tensor instructions to be executed, the instruction scheduler will cache them. For example, the instruction scheduler includes an instruction storage list, and the first tensor instruction that enters the instruction scheduler from the instruction memory will be cached in the instruction storage list by the instruction scheduler. The instruction storage list can be shown in Table 1, mainly including the instruction number insID, the operator number cmdID corresponding to the instruction, the opcode of the instruction, the length of the data to be operated by the instruction (the unit can be bits), the address read_addr to be read when executing the instruction, the address write_addr to be written when executing the instruction, the storage and computing integrated memory ID to execute the instruction, the storage array read_pos to be operated in the corresponding storage and computing integrated memory for reading the instruction, and the storage array write_pos to be operated in the corresponding storage and computing integrated memory for writing the instruction. Among them, read_addr and write_addr are usually the first address of the data, the storage array read_pos to be operated for reading the instruction can be represented by the coordinates of the position of the storage array, and the storage array write_pos to be operated for writing the instruction can be represented by the coordinates of the position of the storage array, such as (1,4,1,8) and (0,-1,0,-1) in Table 1. After an instruction in Table 1 is scheduled to the corresponding integrated storage and computing memory, subsequent instructions are pushed forward according to the instruction number insID to ensure that the later-entered instructions are stored at the back. Therefore, the strategy of prioritizing the first-entered instructions (but still meeting the scheduling conditions) can be adopted during scheduling.

[0163] Table 1

[0164] Based on Table 1, it can be concluded that the instruction scheduler stores the ID of the integrated storage and computing memory for executing the first tensor instruction and the target storage array to be operated on by executing the first tensor instruction (for example, read data corresponds to read_pos, and write data corresponds to write_pos). The target storage array can be a complete storage array as shown in Figure 5, or it can be a portion of a complete storage array.

[0165] Exemplarily, the instruction scheduler also stores the associations between pending tensor instructions. As shown in Figure 7, the associations between pending tensor instructions can be described using an instruction association matrix. The instruction association matrix is ​​a matrix with the same length and width as the instruction storage list, and the matrix elements represent the instruction associations. For example, the value (i, j) in the matrix element represents the association between the i-th instruction and the j-th instruction, or represents the dependency of the i-th instruction on the j-th instruction. Among them, the value of the matrix element is 0, which means that the two instructions have no association or dependency relationship; the value of the matrix element is 1, which means that there is a WAW association relationship between the i-th instruction and the j-th instruction (for example, the write operation of the i-th instruction depends on the completion of the write operation of the j-th instruction, or the write operation of the j-th instruction depends on the completion of the write operation of the i-th instruction); the value of the matrix element is 2, which means that there is a WAR association relationship between the i-th instruction and the j-th instruction (for example, the read operation of the i-th instruction depends on the completion of the write operation of the j-th instruction, or the read operation of the j-th instruction depends on the completion of the write operation of the i-th instruction); the value of the matrix element is 3, which means that there is a RAW association relationship between the i-th instruction and the j-th instruction (for example, the write operation of the i-th instruction depends on the completion of the read operation of the j-th instruction, or the write operation of the j-th instruction depends on the completion of the read operation of the i-th instruction). Among them, the i-th instruction and the j-th instruction represent any two tensor instructions cached in the instruction storage list.

[0166] Exemplarily, the instruction scheduler also stores the status of the storage arrays in each integrated storage and computing memory. For example, the instruction scheduler includes a storage array status list, as shown in Table 2. The storage array status list stores the current array status of each integrated storage and computing memory, primarily including information such as the integrated storage and computing memory ID, storage array ID, and array status.

[0167] Table 2

[0168] The storage array status list pulls the status of each storage array when scheduling the out-of-order execution of tensor instructions to determine whether the tensor instruction can be sent to the corresponding storage-computing integrated memory, and operates the corresponding storage array (such as read_pos or write_pos in Table 1) for execution, and updates the status of the corresponding storage array during execution. As shown in Table 2, 1 can indicate that the storage array is in working state, and 0 can indicate that the storage array is in idle state.

[0169] As shown in Figure 6, based on the instruction storage list, instruction association matrix and storage array status list in the instruction scheduler, see Figure 8, the first tensor instruction is cached in the instruction storage list after entering the instruction scheduler from the instruction memory. The instruction scheduler determines that there is no pre-dependency relationship for the first tensor instruction based on the association relationship in the instruction association matrix (that is, the execution of the first tensor instruction does not depend on the completion of the execution of any other tensor instruction in the instruction storage list) and determines that the target storage array corresponding to the first tensor instruction is idle based on the storage array status list (that is, it determines whether the first tensor instruction meets the scheduling conditions), and then schedules the first tensor instruction to the storage and computing integrated memory to which the target storage array belongs. It should be understood that if a read operation is required to execute the first tensor instruction, the target storage array is read_pos, and if the first tensor instruction requires a write operation, the target storage array is write_pos.

[0170] For example, as shown in Figure 8, if the first tensor instruction does not meet the scheduling conditions, it waits in the instruction storage list. It should be noted that if the first tensor instruction is scheduled to the corresponding storage-computing integrated memory, the instruction scheduler clears the information of the first tensor instruction in the instruction storage list and the instruction association matrix.

[0171] In this implementation, for the tensor instructions to be executed cached in the instruction scheduler, if it is determined based on the association relationship between the tensor instruction and other tensor instructions that the tensor instruction has no pre-dependency relationship, and the storage array corresponding to the tensor instruction is in an idle state, then it can be determined that the tensor instruction meets the scheduling conditions, so that it can be scheduled to the corresponding storage and computing integrated memory for execution, thereby realizing the out-of-order execution scheduling of tensor instructions.

[0172] It can be seen that the embodiment of the present application provides a storage-computing integrated chip architecture that supports out-of-order execution scheduling of tensor instructions. The tensor instructions to be executed that enter the storage-computing integrated chip first enter the instruction memory cache. After being processed by the instruction memory, the tensor instructions to be executed are determined by the instruction scheduler based on the association relationship between them and other tensor instructions in the cache and the state of the target storage array required to execute the tensor instruction. Whether the tensor instruction meets the scheduling conditions is determined. If it is satisfied, the instruction scheduler schedules the tensor instruction to the corresponding storage-computing integrated memory so that the corresponding storage-computing integrated memory executes the tensor instruction, thereby realizing out-of-order execution scheduling of tensor instructions, and then meeting the requirements of storage-computing integrated technology for high concurrency and high throughput, and realizing high parallelism of multiple complex operators.

[0173] Exemplarily, referring to FIG9 , the storage array, row processor, and column processor shown in FIG5 or FIG6 can be integrated into another array chip, which also integrates an analog-to-digital converter (ADC), a digital-to-analog converter (DAC), a sense amplifier (SA), etc. The storage-computing integrated memory can include n such array chips. Exemplarily, the storage-computing integrated chip also includes a memory output scheduling module, which is used to determine whether the storage-computing integrated memory communicates with the CPU or with the DRAM, for example: determining whether the calculation result of the first tensor instruction is sent to the CPU and / or the dynamic random access memory (DRAM). Among them, the DRAM can be an internal device of the storage-computing integrated chip, or it can be a device independent of the storage-computing integrated chip.

[0174] Exemplarily, referring to FIG10 , each storage-computing-in-one memory further includes a control unit. In terms of scheduling the first tensor instruction to the storage-computing-in-one memory to which the target storage array belongs, the instruction scheduler is specifically used to: schedule the first tensor instruction to the control unit of the storage-computing-in-one memory to which the target storage array belongs. The control unit is used to cache the first tensor instruction that enters. Specifically, the control unit includes an unfinished instruction list, and the first tensor instruction is cached in the unfinished instruction list. As shown in FIG11 , the control unit is used to determine whether the target storage array is executing other tensor instructions, such as the third tensor instruction. If the target storage array is being operated to execute the third tensor instruction, the first tensor instruction must wait until the third tensor instruction is executed before being executed; if the target storage array is in an idle state, the target storage array is operated to execute the first tensor instruction.

[0175] In this implementation, the tensor instructions entering the storage-computing integrated memory are cached by the control unit. Based on the pipeline operation mode, the control unit will again determine whether the target storage array is being operated to execute other tensor instructions, so as to immediately execute the first tensor instruction or wait according to the situation to avoid instruction execution conflicts.

[0176] Exemplarily, as shown in FIG12 , each integrated memory device also includes a near-memory computing circuit. The near-memory computing circuit typically includes multiple processing modules. Processing modules that perform the same processing form an array. For example, all modules that perform NAND processing form a NAND array, all modules that perform AND-OR processing form an AND-OR array, all modules that perform AND processing form an AND array, all modules that perform SHIFT processing form a SHIFT array, all modules that perform floating-point addition processing (FPPADD) form an FPADD array, all modules that perform floating-point multiplication (FPMult) form an FPMult array, and so on. Specifically, referring to FIG13 , the near-memory computing circuit may include an interaction unit, an input routing, an execution unit, and an output unit. The interaction unit receives external input signals and temporarily stores the status of each processing module. The status of each processing module is sent to the interaction unit by the execution unit for update. The interaction unit can synchronize the status of each processing module with the control unit. Signals emitted from the interaction unit are sent to the designated processing module within the execution unit via the input routing. Data processed by each processing module within the execution unit is then sent to the output unit via the output control within the execution unit and output via the output unit. The input to the near-memory computation circuit is the tensor required for computation, the array number being called, and the number of the processing module being called in the array. The input routing can include routing modules such as AND routing and OR routing, and the output unit includes multiple output block banks.

[0177] In this implementation, multiple computing modes, such as in-memory computing and near-memory computing, are integrated into the integrated memory, enabling the integrated memory chip to support diverse computing modes. For example, if the tensor instructions to be processed entering the integrated memory chip contain complex operators that require both in-memory computing and near-memory computing, the integrated memory chip provided in this embodiment of the application can also meet the storage and computing needs of such operators.

[0178] Exemplarily, the control unit is further configured to cache the status of each processing module in the near-memory computing circuit. As shown in Table 3, the control unit includes a near-memory status list, in which the status of each processing module synchronized by the interaction unit of the near-memory computing circuit is presented. This list primarily includes the number of each processing module, the number of the array to which each processing module belongs, and the status of each processing module.

[0179] Table 3

[0180] Among them, 0 indicates that the corresponding processing module is in an idle state, and 1 indicates that the corresponding processing module is in a working state.

[0181] Exemplarily, in operating a target storage array to execute a first tensor instruction, the control unit is specifically configured to: operate the target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction. Instruction types include array control instructions, in-memory computation instructions, and near-memory computation instructions. In this implementation, the control unit can perform specific operations on the target storage array based on the instruction type of the first tensor instruction to execute the first tensor instruction.

[0182] For example, as shown in FIG14 , after the instruction enters the execution instruction phase, the control unit determines the instruction type of the first tensor instruction:

[0183] If the instruction type is an array control instruction, the control unit disables the storage and calculation function of the row or column of the target storage array, or enables the storage and calculation function of the row or column of the target storage array.

[0184] In this implementation, when the tensor instruction to be processed is an array control instruction, the control unit can control the state of the target storage array to enable or disable the storage and computing function of the storage array.

[0185] If the instruction type is an in-memory calculation instruction, the control unit reads the input tensor and calls the near-memory calculation circuit to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array. In-memory calculation usually requires reading external input tensors, such as input data during neural network inference, and the first tensor is the tensor stored in the target storage array. Therefore, the read input tensor can be used to perform in-memory calculations with the first tensor stored in the target storage array. In-memory calculations usually require the assistance of the processing module in the near-memory calculation circuit, so the operations of reading the input tensor and calling the near-memory calculation circuit can be performed simultaneously.

[0186] In this implementation, although the storage-computing integrated memory integrates multiple computing modes of in-memory computing and near-memory computing, if the tensor instruction to be processed is an in-memory computing instruction, the control unit can also perform a single in-memory computing. The storage-computing integrated operation is flexible, and the reading of data and the calling of the near-memory computing circuit are carried out simultaneously, which is conducive to improving concurrency and throughput.

[0187] If the instruction type is a near memory calculation instruction, the control unit reads the second tensor stored in the target storage array to the near memory calculation circuit when there is an input tensor in the near memory calculation circuit, and calls the near memory calculation circuit to complete the near memory calculation of the input tensor and the second tensor; or if there is no input tensor in the near memory calculation circuit, reads the second tensor and the third tensor stored in the target storage array to the near memory calculation circuit, and calls the near memory calculation circuit to complete the near memory calculation of the second tensor and the third tensor. Wherein, if there is no input tensor in the near memory calculation circuit, the control unit needs to determine whether the second tensor and the third tensor required for the near memory calculation can be read at the same time. If they cannot be read at the same time (for example, the second tensor and the third tensor are not stored in the same row), the control unit needs to read the second tensor and the third tensor separately to the near memory calculation circuit, and call the near memory calculation circuit to perform the near memory calculation; if they can be read at the same time, the control unit reads the second tensor and the third tensor to the near memory calculation circuit at the same time, and calls the near memory calculation circuit to perform the near memory calculation. It should be understood that during the near memory calculation, the operations of reading the tensor and calling the near memory calculation circuit can also be performed simultaneously.

[0188] In this implementation, if the tensor instruction to be processed is a near-memory calculation instruction, the control unit can also perform a single near-memory calculation. The integrated storage and calculation operation is flexible, and the reading of data and the calling of the near-memory calculation circuit are carried out simultaneously, which is conducive to improving concurrency and throughput.

[0189] Exemplarily, since the control unit caches the states of each processing module in the near-memory computing circuit, in calling the near-memory computing circuit to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array, the control unit is specifically configured to:

[0190] If the first processing module in the near-memory computation circuit is idle (i.e., the idle processing module corresponding to the calling operation), the first processing module is called to assist in completing the in-memory computation between the input tensor and the first tensor stored in the target memory array. The first processing module is determined by the control unit based on the opcode of the first tensor instruction. If all first processing modules are in working state, then wait.

[0191] In this implementation, the control unit can determine which processing module in the near-memory computing circuit needs to be called for the in-memory calculation based on the opcode of the first tensor instruction. For example, if the opcode indicates that a shift operation is required, the SHIFT module in the near-memory computing circuit needs to be called, and based on the status of the processing module, the processing module is called to assist in the in-memory calculation.

[0192] Exemplarily, in calling the near-memory computing circuit to complete the near-memory computing of the input tensor and the second tensor, the control unit is specifically configured to:

[0193] When the second processing module in the near-memory computing circuit is in an idle state (i.e., the idle processing module corresponding to the calling operation), calling the second processing module to complete the near-memory computing of the input tensor and the second tensor;

[0194] In calling the near-memory computing circuit to complete the near-memory computing of the second tensor and the third tensor, the control unit is specifically configured to:

[0195] When the second processing module is in an idle state, the second processing module is called to complete the near-memory calculation of the second tensor and the third tensor.

[0196] The second processing module is determined based on the operation code of the first tensor instruction. If the first processing modules are all in working state, then wait.

[0197] In this implementation, the control unit can determine which processing module in the near memory computing circuit needs to be called for the near memory computing based on the opcode of the first tensor instruction. For example, if the opcode indicates that a floating-point addition operation is required, the FP ADD module needs to be called, and the processing module is called to perform the near memory computing based on the state of the processing module.

[0198] Exemplarily, as shown in FIG15 , each storage-computing integrated memory further includes a cache module, and the cache module includes multiple cache areas, each cache area having a corresponding physical address. Then in FIG14 , after the calculation is completed, the control unit is further used to write the calculation result of the first tensor instruction into a target cache area (such as an area where no data is stored) in the multiple cache areas, and store the address of the target cache area in association with the target location; the target location is the storage location of the calculation result in the target storage array. Exemplarily, the control center includes a cache control table, which can realize cache scheduling and mapping of cache and storage array. As shown in Table 4, the cache control table mainly includes the address of each cache area in the cache module, the corresponding instruction number insID, the data cached in each cache area, the storage address of the data in the target storage array (usually the first address), and the storage location of the data in the target storage array (usually represented by coordinates, such as (1,4,1,8) in Table 4). This location is composed of the coordinates of the upper left corner and lower right corner taken after all arrays are spliced ​​in rows.

[0199] Table 4

[0200] Among them, the array storage address of 0 means that the data is only in the cache, but not in the array. If the data in a certain cache area is empty, the corresponding address is also empty, and the storage location is displayed as (0,-1,0,-1). The control unit will sort each cache area, and each time the cache area is written or read, the sorting sequence number of the area is set to the last to achieve the least recently used (Least Recently Used, LRU) function. For data that does not have a clear write address, the control unit will write it to the cache area that is currently ranked first, and after writing, the address sorting sequence number of the area will be set to the last. For example, the cache area in Table 4 above can be sorted as shown in Table 5 after being operated:

[0201] Table 5

[0202] Among them, after writing the calculation result of the first tensor instruction into the cache, the control unit will synchronously modify the cache control table.

[0203] In this implementation, after the control unit writes the result of the in-memory calculation or the near-memory calculation into the target cache area, it can associate the address of the target cache area with the storage location of the calculation result in the target storage array to establish a mapping between the cache and the array.

[0204] The researchers of this application analyzed the latency and energy efficiency of analog in-memory computing, digital in-memory computing, near-memory computing, and traditional computing, and came to the conclusions shown in Table 6:

[0205] Table 6

[0206] Through experimental comparison, the present application solution can give full play to the advantages of each computing mode by combining various computing modes, thereby reducing the power consumption and delay consumption of tensor instruction processing, and greatly improving the energy efficiency of processing. It also enables the hardware level to support multiple computing modes such as near-memory computing and in-memory computing at the same time, and for different scenarios, it can be degraded to a single storage computing mode, which improves the flexibility of storage-computing technology. At the operator level, it can support both AI operators and general operators, and convert each operator into an adapted tensor instruction and assign it to a computing mode with advantages, giving full play to the advantages of combining various storage computing modes.

[0207] Please refer to Figure 16, which is a schematic diagram of a storage-computing integrated system provided in an embodiment of the present application. As shown in Figure 16, it includes an instruction compiler and a storage-computing integrated chip. The user's operations and storage-computing integrated tasks in the data are unloaded through the CPU, and the CPU issues the storage-computing integrated tasks to the storage-computing integrated system. In the storage-computing integrated system, the storage-computing integrated tasks issued by the CPU are compiled by the instruction compiler, compiled into tensor instructions executable by the storage-computing integrated chip, and sent to the storage-computing integrated chip for execution. The specific execution process can be referred to the description in the embodiments shown in Figures 5 to 15.

[0208] Please refer to Figure 17, which is a flow chart of an instruction scheduling method provided in an embodiment of the present application. This method can be applied to the integrated storage chip in any of the embodiments shown in Figures 5-15. The integrated storage chip includes an instruction memory, an instruction scheduler, and at least one integrated storage and computing memory; each of the integrated storage and computing memories includes at least one storage array. As shown in Figure 17, the method includes steps 1701-1702:

[0209] 1701: The instruction memory obtains the first tensor instruction to be executed;

[0210] 1702: Based on the association between the first tensor instruction and the second tensor instruction and the state of the target storage array required to execute the first tensor instruction, the instruction scheduler schedules the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs so that the storage-computing integrated memory executes the first tensor instruction.

[0211] The second tensor instruction is a tensor instruction cached in the instruction scheduler except the first tensor instruction.

[0212] In some scenarios, this method can also be implemented based on the storage and computing integrated system shown in Figure 16.

[0213] In one possible implementation, based on the association between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated when executing the first tensor instruction, scheduling the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs includes:

[0214] When the instruction scheduler determines based on the association relationship that the first tensor instruction does not have a second tensor instruction with a preceding dependency and the target storage array is in an idle state, the instruction scheduler schedules the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs.

[0215] In one possible implementation, each integrated storage and computing memory further includes a control unit; and dispatching the first tensor instruction to the integrated storage and computing memory to which the target storage array belongs includes:

[0216] The instruction scheduler schedules the first tensor instruction to a control unit of the storage-computation integrated memory to which the target storage array belongs;

[0217] Execute the first tensor instruction, including:

[0218] If the target storage array is being operated to execute the third tensor instruction, the control unit waits for the third tensor instruction to be executed and then operates the target storage array to execute the first tensor instruction; if the target storage array is in an idle state, the control unit operates the target storage array to execute the first tensor instruction.

[0219] In a possible implementation, operating a target storage array to execute a first tensor instruction includes:

[0220] The control unit operates the target storage array to execute the first tensor instruction based on an instruction type of the first tensor instruction.

[0221] In one possible implementation, operating a target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction includes:

[0222] If the instruction type is an array control instruction, the control unit disables the storage and calculation function of the row or column of the target storage array, or enables the storage and calculation function of the row or column of the target storage array.

[0223] In one possible implementation, each integrated storage and computing memory further includes a near-memory computing circuit; and operating a target storage array based on an instruction type of the first tensor instruction to execute the first tensor instruction includes:

[0224] If the instruction type is an in-memory calculation instruction, the control unit reads the input tensor and calls the near-memory calculation circuit to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

[0225] In one possible implementation, the control unit caches states of each processing module in the near-memory computing circuit; calling the near-memory computing circuit to assist in completing an in-memory computation between an input tensor and a first tensor stored in a target storage array includes:

[0226] When the first processing module in the near memory computing circuit is in an idle state, the control unit calls the first processing module to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

[0227] In a possible implementation, the instruction memory caches an operation code of the first tensor instruction; and the first processing module is determined based on the operation code of the first tensor instruction.

[0228] In one possible implementation, each integrated storage and computing memory further includes a near-memory computing circuit; and operating a target storage array based on an instruction type of the first tensor instruction to execute the first tensor instruction includes:

[0229] If the instruction type is a near memory calculation instruction, the control unit reads the second tensor stored in the target storage array to the near memory calculation circuit when the near memory calculation circuit has an input tensor, and calls the near memory calculation circuit to complete the near memory calculation of the input tensor and the second tensor; or if the near memory calculation circuit has no input tensor, the control unit reads the second tensor and the third tensor stored in the target storage array to the near memory calculation circuit, and calls the near memory calculation circuit to complete the near memory calculation of the second tensor and the third tensor.

[0230] In one possible implementation, the control unit caches states of each processing module in the near-memory computing circuit; calling the near-memory computing circuit to complete the near-memory computing of the input tensor and the second tensor includes:

[0231] The control unit calls the second processing module to complete the near memory calculation of the input tensor and the second tensor when the second processing module in the near memory calculation circuit is in an idle state;

[0232] Call the near-memory computation circuit to complete the near-memory computation of the second and third tensors, including:

[0233] When the second processing module is in an idle state, the control unit calls the second processing module to complete the near-memory calculation of the second tensor and the third tensor.

[0234] In a possible implementation, the instruction memory caches an operation code of the first tensor instruction; and the second processing module is determined based on the operation code of the first tensor instruction.

[0235] In one possible implementation, the integrated storage and computing chip further includes a cache module, and the cache module includes multiple cache areas; and the method further includes:

[0236] The control unit writes the calculation result of the first tensor instruction into a target cache area among the multiple cache areas, and stores the address of the target cache area in association with a target location; the target location is the storage location of the calculation result in the target storage array.

[0237] In one possible implementation, the integrated storage and computing chip further includes a memory output scheduling module; and the method further includes:

[0238] The memory output scheduling module sends the calculation result of the first tensor instruction to the central processing unit or the dynamic random access memory.

[0239] It should be noted that the method embodiment and the storage-computing integrated chip embodiment are different presentation forms of the same technical concept. Therefore, the implementation of each step in Figure 17 can refer to the relevant description in the embodiments shown in Figures 5-15, and can achieve the same or similar beneficial effects, which will not be repeated here.

[0240] Please refer to Figure 18, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 18, the computer device includes a processor 1801 and a storage-computing integrated chip 1802. The processor 1801 is configured to issue a storage-computing integrated task to the storage-computing integrated chip 1802. The storage-computing integrated chip 1802 is configured to execute the storage-computing integrated task and return the execution result of the storage-computing integrated task to the processor 1801. For example: the storage-computing integrated chip 1802 can perform in-memory calculations or near-memory calculations by executing tensor instructions issued by the processor 1801, and return the calculation results. Among them, the storage-computing integrated chip 1802 can be the storage-computing integrated chip in any one of the embodiments shown in Figures 5 to 15.

[0241] Please refer to Figure 19, which is a schematic diagram of the structure of another computer device provided in an embodiment of the present application. The computer device includes at least a processor 1901 and a memory 1902, and the processor 1901 and the memory 1902 are interconnected via a bus 1903. The computer device can be used to execute the relevant steps of the instruction scheduling method. The computer device can be a computer, a computer host, a server, etc. The processor 1901 in the computer device is used to read one or more programs stored in the memory 1902 and execute the method of the embodiment shown in Figure 17.

[0242] The memory 1902 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), and is used to store relevant computer programs and data.

[0243] The processor 1901 may be one or more CPUs. When the processor 1901 is a CPU, the CPU may be a single-core CPU or a multi-core CPU.

[0244] Exemplarily, the processor 1901 in the computer device may be configured to read one or more programs stored in the memory 1902 to perform the following operations:

[0245] Get the first tensor instruction to be executed;

[0246] Based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated for executing the first tensor instruction, dispatch the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs so that the storage-computing integrated memory executes the first tensor instruction;

[0247] The second tensor instruction is a tensor instruction cached in the instruction scheduler except the first tensor instruction.

[0248] It should be noted that although the computer device shown in Figure 19 only shows the processor 1901, the memory 1902, and the bus 1903, during the specific implementation process, those skilled in the art will understand that the computer device also includes other components necessary for normal operation. At the same time, according to specific needs, those skilled in the art will understand that the computer device may also include hardware components that implement other additional functions. In addition, those skilled in the art will understand that the computer device may also include only the components necessary to implement the embodiments of the present application, and does not necessarily include all the components shown in Figure 19.

[0249] The embodiment of the present application also provides a computer-readable storage medium (Memory), which stores a computer program. When the computer program is run, the method of the embodiment shown in Figure 17 above is implemented. It can be understood that the computer-readable storage medium here can include both built-in storage media in the device and, of course, extended storage media supported by the device. The computer-readable storage medium provides a storage space that stores the operating system of the device. In addition, one or more computer programs suitable for being loaded and executed by the processor of the device are also stored in the storage space. It should be noted that the computer-readable storage medium here can be a high-speed RAM or a non-volatile memory, such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.

[0250] An embodiment of the present application also provides a computer program product, which includes: computer program code. When the computer program code is executed by a computer device, the method flow of the embodiment shown in Figure 17 is implemented.

[0251] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0252] It should be understood that the processor mentioned in the embodiments of the present application may be a CPU, or may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0253] It should also be understood that the memory mentioned in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a ROM, a programmable read-only memory (Programmable ROM, PROM), an EPROM, an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) or a flash memory. The volatile memory can be a RAM, which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DR RAM).

[0254] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) is integrated into the processor.

[0255] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0256] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0257] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely exemplary. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0258] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0259] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium.

[0260] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. In the text description of this application, the character " / " generally indicates that the associated objects are in an "or" relationship.

[0261] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0262] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0263] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A storage and computing integrated chip, characterized in that: It includes an instruction memory, an instruction scheduler and at least one storage and computing integrated memory; each of the storage and computing integrated memory includes at least one storage array; The instruction memory is used to obtain a first tensor instruction to be executed; The instruction scheduler is configured to schedule the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated for executing the first tensor instruction, so that the storage-computing integrated memory executes the first tensor instruction; The second tensor instruction is a tensor instruction cached in the instruction scheduler other than the first tensor instruction.

2. The integrated storage and computing chip according to claim 1, characterized in that: In terms of scheduling the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated when executing the first tensor instruction, the instruction scheduler is specifically configured to: When it is determined based on the association relationship that the first tensor instruction does not have a second tensor instruction with a preceding dependency and the target storage array is in an idle state, the first tensor instruction is scheduled to the storage-computing integrated memory to which the target storage array belongs.

3. The integrated storage and computing chip according to claim 1 or 2, characterized in that: Each of the storage-computing integrated memories further includes a control unit; In terms of scheduling the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs, the instruction scheduler is specifically used to: schedule the first tensor instruction to a control unit of the storage-computing integrated memory to which the target storage array belongs; The control unit is configured to, if the target storage array is being operated to execute a third tensor instruction, wait for the third tensor instruction to be completed and then operate the target storage array to execute the first tensor instruction; and if the target storage array is in an idle state, operate the target storage array to execute the first tensor instruction.

4. The integrated storage and computing chip according to claim 3, characterized in that: In terms of operating the target storage array to execute the first tensor instruction, the control unit is specifically configured to: The target storage array is operated based on an instruction type of the first tensor instruction to execute the first tensor instruction.

5. The integrated storage and computing chip according to claim 4, characterized in that: In terms of operating the target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction, the control unit is specifically configured to: If the instruction type is an array control instruction, the storage and calculation functions of the rows or columns of the target storage array are disabled, or the storage and calculation functions of the rows or columns of the target storage array are enabled.

6. The integrated storage and computing chip according to claim 4, characterized in that: Each of the integrated storage and computing memories further includes a near-storage computing circuit; in terms of operating the target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction, the control unit is specifically configured to: If the instruction type is an in-memory calculation instruction, the input tensor is read, and the near-memory calculation circuit is called to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

7. The integrated storage and computing chip according to claim 6, characterized in that: The control unit is further configured to cache states of each processing module in the near memory computing circuit; and in terms of calling the near memory computing circuit to assist in completing the in-memory computation between the input tensor and the first tensor stored in the target storage array, the control unit is specifically configured to: When the first processing module in the near-memory computing circuit is in an idle state, the first processing module is called to assist in completing the in-memory computing of the input tensor and the first tensor stored in the target storage array.

8. The storage and computing integrated chip according to claim 5 or 7, characterized in that: The instruction memory is further used to cache the operation code of the first tensor instruction; the first processing module is determined based on the operation code of the first tensor instruction.

9. The integrated storage and computing chip according to claim 4, characterized in that: Each of the integrated storage and computing memories further includes a near-storage computing circuit; in terms of operating the target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction, the control unit is specifically configured to: If the instruction type is a near memory calculation instruction, then when the near memory calculation circuit has an input tensor, the second tensor stored in the target storage array is read to the near memory calculation circuit, and the near memory calculation circuit is called to complete the near memory calculation of the input tensor and the second tensor; or when the near memory calculation circuit has no input tensor, the second tensor and the third tensor stored in the target storage array are read to the near memory calculation circuit, and the near memory calculation circuit is called to complete the near memory calculation of the second tensor and the third tensor.

10. The storage and computing integrated chip according to claim 9, characterized in that: The control unit is further configured to cache states of each processing module in the near-memory computing circuit; and in calling the near-memory computing circuit to complete the near-memory computing of the input tensor and the second tensor, the control unit is specifically configured to: When the second processing module in the near-memory computing circuit is in an idle state, calling the second processing module to complete the near-memory computing of the input tensor and the second tensor; In calling the near memory calculation circuit to complete the near memory calculation of the second tensor and the third tensor, the control unit is specifically configured to: When the second processing module is in an idle state, the second processing module is called to complete the near memory calculation of the second tensor and the third tensor.

11. The storage and computing integrated chip according to claim 10, characterized in that: The instruction memory is further used to cache the operation code of the first tensor instruction; the second processing module is determined based on the operation code of the first tensor instruction.

12. The storage and computing integrated chip according to any one of claims 6 to 11, characterized in that: The storage and computing integrated chip further includes a cache module, and the cache module includes multiple cache areas; The control unit is further used to write the calculation result of the first tensor instruction into a target cache area among the multiple cache areas, and store the address of the target cache area in association with a target location; the target location is the storage location of the calculation result in the target storage array.

13. The storage and computing integrated chip according to any one of claims 6 to 11, characterized in that: The storage and computing integrated chip further includes a memory output scheduling module; The memory output scheduling module is used to send the calculation result of the first tensor instruction to a central processing unit or a dynamic random access memory.

14. An instruction scheduling method, characterized in that: Applied to a storage-integrated chip, the storage-integrated chip includes an instruction memory, an instruction scheduler, and at least one storage-integrated memory; each of the storage-integrated memory includes at least one storage array; the method includes: The instruction memory obtains a first tensor instruction to be executed; The instruction scheduler schedules the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated for executing the first tensor instruction, so that the storage-computing integrated memory executes the first tensor instruction; The second tensor instruction is a tensor instruction cached in the instruction scheduler other than the first tensor instruction.

15. The method according to claim 14, characterized in that The step of scheduling the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs based on the association relationship between the first tensor instruction and the second tensor instruction and the state of the target storage array to be operated when executing the first tensor instruction includes: When the instruction scheduler determines based on the association relationship that the first tensor instruction does not have a second tensor instruction with a preceding dependency and the target storage array is in an idle state, the first tensor instruction is scheduled to the storage-computing integrated memory to which the target storage array belongs.

16. The method according to claim 14 or 15, characterized in that Each of the storage-computing integrated memories further includes a control unit; and the step of dispatching the first tensor instruction to the storage-computing integrated memory to which the target storage array belongs includes: The instruction scheduler schedules the first tensor instruction to a control unit of the storage-computation integrated memory to which the target storage array belongs; The executing the first tensor instruction includes: If the target storage array is being operated to execute a third tensor instruction, the control unit waits for the third tensor instruction to be completed and then operates the target storage array to execute the first tensor instruction; if the target storage array is in an idle state, the control unit operates the target storage array to execute the first tensor instruction.

17. The method according to claim 16, characterized in that The operating the target storage array to execute the first tensor instruction includes: The control unit operates the target storage array to execute the first tensor instruction based on an instruction type of the first tensor instruction.

18. The method according to claim 17, characterized in that The step of operating the target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction includes: If the instruction type is an array control instruction, the control unit disables the storage and calculation functions of the rows or columns of the target storage array, or enables the storage and calculation functions of the rows or columns of the target storage array.

19. The method according to claim 17, wherein Each of the storage-computation integrated memories further includes a near-memory computing circuit; and operating the target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction includes: If the instruction type is an in-memory calculation instruction, the control unit reads the input tensor and calls the near-memory calculation circuit to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

20. The method according to claim 19, characterized in that The control unit caches states of each processing module in the near memory computing circuit; and calling the near memory computing circuit to assist in completing the in-memory computing of the input tensor and the first tensor stored in the target storage array includes: When the first processing module in the near memory computing circuit is in an idle state, the control unit calls the first processing module to assist in completing the in-memory calculation of the input tensor and the first tensor stored in the target storage array.

21. The method according to claim 20, characterized in that The instruction memory caches an operation code of the first tensor instruction; and the first processing module is determined based on the operation code of the first tensor instruction.

22. The method according to claim 17, wherein Each of the storage-computing integrated memories further includes a near-storage computing circuit; The step of operating the target storage array to execute the first tensor instruction based on the instruction type of the first tensor instruction includes: If the instruction type is a near memory calculation instruction, the control unit reads the second tensor stored in the target storage array to the near memory calculation circuit when the near memory calculation circuit has an input tensor, and calls the near memory calculation circuit to complete the near memory calculation of the input tensor and the second tensor; or if the near memory calculation circuit has no input tensor, reads the second tensor and the third tensor stored in the target storage array to the near memory calculation circuit, and calls the near memory calculation circuit to complete the near memory calculation of the second tensor and the third tensor.

23. The method according to claim 22, characterized in that The control unit caches states of each processing module in the near memory computing circuit; and calling the near memory computing circuit to complete the near memory computing of the input tensor and the second tensor includes: The control unit calls the second processing module in the near-memory computing circuit to complete the near-memory computing of the input tensor and the second tensor when the second processing module is in an idle state; The calling the near memory computing circuit to complete the near memory computing of the second tensor and the third tensor includes: When the second processing module is in an idle state, the control unit calls the second processing module to complete the near memory calculation of the second tensor and the third tensor.

24. The method according to claim 23, wherein The instruction memory caches an operation code of the first tensor instruction; and the second processing module is determined based on the operation code of the first tensor instruction.

25. The method according to claims 19-24, characterized in that The storage-computing integrated chip further includes a cache module, and the cache module includes multiple cache areas; the method further includes: The control unit writes a calculation result of the first tensor instruction into a target cache area among the multiple cache areas, and stores the address of the target cache area in association with a target location; the target location is a storage location of the calculation result in the target storage array.

26. The method according to claims 19-24, characterized in that The integrated storage and computing chip further includes a memory output scheduling module; and the method further includes: The memory output scheduling module sends the calculation result of the first tensor instruction to a central processing unit or a dynamic random access memory.

27. A computer device, characterized in that: It includes a processor and a storage-computing integrated chip according to any one of claims 1 to 12, the processor is configured to send a storage-computing integrated task to the storage-computing integrated chip, the storage-computing integrated chip is configured to execute the storage-computing integrated task and return the execution result of the storage-computing integrated task to the processor.

28. A computer device, characterized in that: The system comprises a processor, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to implement the method according to any one of claims 14 to 26 when executed by the processor.

29. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program for execution by a device, and when the computer program is executed, the method according to any one of claims 14 to 26 is implemented.

30. A computer program product, characterized in that When the computer program product is executed by a device, the device executes the method according to any one of claims 14 to 26.

Images