Operator compiling method and device
By using operator compilation method to generate tensor instruction sets in the storage and computing integrated system, the problems of data handling delay and power consumption under the storage and computing separation architecture are solved, the computing performance and parallelism are improved, and more efficient data processing is achieved .
Patent Information
- Application Number
- CN202311070464.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-08-22
- Publication Date
- 2024-12-03
AI Technical Summary
The existing storage and computing separation architecture leads to data handling delay and transmission power consumption problems in big data application scenarios, and the scalar instruction set processing is low, so it is impossible to fully utilize the advantages of the storage and computing architecture.
The operator compilation method is used to generate a tensor instruction set, and the computing tasks are compiled and executed by the integrated storage and computing system to improve processing parallelism and computing performance.
It significantly improves the computing performance and energy efficiency of the integrated storage and computing system, reduces the frequency of data handling and transmission power consumption, and improves the parallelism of processing and execution efficiency.
Smart Images

Figure CN119065716A_ABST
Abstract
Description
[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on May 31, 2023, with application number 202310639274.4 and invention name “A Data Processing System”, all contents of which are incorporated by reference into this application. Technical Field
[0002] The embodiments of the present application relate to the field of computers, and in particular to an operator compilation method and device. Background Art
[0003] In recent years, with the rapid development of emerging applications such as big data, the explosive growth of information has led to a rapid expansion of computing scale, and computing forms have become increasingly diversified. Multiple application scenarios have put forward higher requirements for the computing performance and storage performance of hardware computing systems. For example, in big data-oriented application scenarios, the storage-computing separation architecture in current mainstream hardware seriously restricts the improvement of performance and energy efficiency, because the processor needs to move data from the memory to the processor through the data bus during operation, and then move the data back to the memory after processing. As the amount of data increases, the delay of this process also increases, even exceeding the read, write and processing delay of the memory and processor itself. In addition, the frequent transfer of large amounts of data between the memory and the processor also brings serious transmission power consumption problems.
[0004] In order to solve the data transfer energy efficiency problem under the storage-computing separation architecture, the storage-computing integrated architecture came into being. The storage-computing integrated architecture greatly reduces the scale and frequency of data transfer between the memory and the processor by implementing embedded computing functions inside the memory chip, so as to significantly improve the computing power and energy efficiency of the computing system. However, most of the instruction sets currently used in the storage-computing integrated architecture are scalar instruction sets. When the memory uses the scalar instruction set to process computing tasks, it needs to process each scalar one by one, resulting in low processing parallelism of computing tasks, and unable to give full play to the advantages of the storage-computing integrated architecture. Summary of the invention
[0005] The embodiment of the present application provides an operator compilation method, in which the storage-computing system can compile a tensor instruction set using the operator compilation method, and use the tensor instruction set to execute the computing task of the storage-computing system, thereby improving the processing parallelism of the computing task. The embodiment of the present application also provides a computing device, a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the operator compilation method.
[0006] In the first aspect, an embodiment of the present application provides an operator compilation method, which can be executed by a storage-computing integrated system, or by a component of the storage-computing integrated system, such as a processor, chip or chip system of the storage-computing integrated system, or by a logic module or software that can implement all or part of the functions of the storage-computing integrated system. The method provided in the first aspect includes: the storage-computing integrated system obtains a compilation operator, and the compilation operator is used to indicate the compilation rules for the source code. The storage-computing integrated system compiles the source code based on the compilation operator to obtain one or more tensor instructions, and the tensor instructions are used to perform computing tasks in the storage-computing integrated system. The tensor instructions include one or more of the following instruction segments: opcode, tensor address, data type, data bit width and tensor length. The storage-computing integrated system stores the tensor instructions in an instruction storage list, and the instruction storage list is used to store compiled tensor instructions.
[0007] In the embodiment of the present application, the storage-computing integrated system can compile the source code based on the compilation operator to obtain tensor instructions, and realize the processing of tensors in the computing tasks by executing the tensor instructions. Compared with the current storage-computing integrated system that uses existing scalar instructions to process computing tasks, the operator compilation method and tensor instruction set provided in the embodiment of the present application improve the processing parallelism of the storage-computing integrated system in processing computing tasks, thereby improving the computing performance of the storage-computing integrated system.
[0008] In some possible implementations, before the storage-computing system compiles the source code based on the compilation operator, the storage-computing system creates a compilation operator table, which is used to store compilation information corresponding to one or more compilation operators, and the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length. In the process of the storage-computing system compiling the source code based on the compilation operator, the storage-computing system directly queries the compilation operator and compilation information in the compilation operator table, and generates tensor instructions corresponding to the source code according to the compilation operator and compilation information.
[0009] In the implementation of the present application, the integrated storage and computing system can pre-store the compilation information in the source code into the compilation operator table, so that during the source code compilation process, the integrated storage and computing system can quickly compile and generate tensor instructions corresponding to the source code information based on the compilation operator table, thereby improving the compilation efficiency of tensor instructions.
[0010] In some possible implementations, in the process of the storage-computing integrated system generating one or more tensor instructions based on the compilation information stored in the operator compilation table, the storage-computing integrated system generates an opcode of the tensor instruction according to the operator number in the compilation operator table, generates a tensor address of the tensor instruction according to the row address, column address and output address in the compilation operator table, and generates the data type, data bit width and tensor length of the tensor instruction according to the data type, data bit width and tensor length in the compilation operator table.
[0011] In the embodiment of the present application, the storage-computing integrated system can quickly compile and generate tensor instructions corresponding to the source code based on the compilation operator table, thereby improving the compilation efficiency of the tensor instructions.
[0012] In some possible implementations, the types of compiled operators include filter operators, sort operators, and aggregation operators. The operator structure of the filter operator includes an opcode, a row address, a column address, an output address, an input data type, and a tensor length. The operator structure of the sort operator includes an opcode, a row address, a column address, an input data type, and a tensor length. The operator structure of the aggregation operator includes an opcode, a row address, a column address, an input data type, and a tensor length.
[0013] In the embodiment of the present application, the storage-computing integrated system provides a variety of compilation operators, which can compile different source codes to obtain tensor instructions, thereby improving the richness of the tensor instruction set.
[0014] In some possible implementations, the instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction. Among them, an address-only instruction is an instruction type in which only tensor addresses participate. A value-only instruction is a tensor instruction with a value. An array control instruction is an instruction for performing an array operation. A near memory circuit control instruction is an instruction for performing a specified near memory circuit operation on an array ID.
[0015] In the embodiment of the present application, the storage and computing integrated system can compile tensor instructions of different instruction types, thereby using tensor instructions of different tensor types to perform a variety of storage and computing operations, thereby improving the applicability to different storage and computing operations.
[0016] In some possible implementations, different types of tensor instructions have different instruction segments. The instruction segment of an instruction in which only addresses are involved includes an opcode, a tensor address, a type, a bit width, and a tensor length. The instruction segment of an instruction in which values are involved includes an opcode, a tensor address, a value, a type, a bit width, and a tensor length. The instruction segment of an array control instruction includes an opcode, a tensor address, a value, and a tensor length. The instruction segment of a near memory circuit control instruction includes an opcode, a storage array identifier, a near memory circuit type, and a near memory circuit state.
[0017] In the embodiment of the present application, the storage and computing integrated system provides tensor instructions of different instruction types and instruction segment formats, thereby improving the feasibility of the solution.
[0018] In some possible implementations, the storage and computing integrated system determines the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list. The instruction association matrix is used to store the dependencies between tensor instructions, and the array status list is used to indicate the status of the array module. The array module is used to execute tensor instructions, and the instruction status includes an executable state and a waiting state.
[0019] In the embodiment of the present application, the storage-computing integrated system can determine the instruction status of the tensor instruction based on the instruction association matrix and the array status list, and then determine the execution order of the tensor instruction, thereby improving the computing efficiency of the storage-computing integrated system.
[0020] In some possible implementations, when the instruction state of the tensor instruction is in an executable state, the storage-computing system sends the tensor instruction to the memory access control unit. The storage-computing system determines an execution operation according to the tensor instruction, and the execution operation includes one or more of the following: a near-memory computing operation, an in-memory computing operation, and a regular read-write operation.
[0021] The tensor instructions generated by the storage-computing integrated system in the embodiment of the present application can be applicable to a variety of storage-computing integrated architectures, thereby improving the applicability of tensor instructions to the storage-computing integrated system.
[0022] In some possible implementations, a tensor instruction set includes a first tensor instruction and a second tensor instruction, and the first tensor instruction and the second tensor instruction are executed in the same array module. The storage-computing integrated system executes the first tensor instruction and the second tensor instruction simultaneously in different execution units of the array module, and the execution unit includes an array and a near storage circuit.
[0023] In the storage-computing-in-one system of the embodiment of the present application, different execution units of the array module can execute different tensor instructions at the same time, thereby improving the execution parallelism of the tensor instructions and the hardware utilization efficiency of the array module, and further improving the computing efficiency of the storage-computing-in-one system.
[0024] In some possible implementations, the dependency between tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency. Among them, read-after-write dependency means that the read operation of one instruction occurs after the write operation of another instruction in concurrent computing. Write-after-write dependency means that the write operation of one instruction occurs after the write operation of another instruction in concurrent computing. Write-after-read dependency means that the write operation of one instruction occurs after the read operation of another instruction in concurrent computing.
[0025] In the embodiment of the present application, the instruction association matrix can store the dependency relationship between multiple tensor instructions, thereby reducing the probability of instruction conflicts during the execution of tensor instructions and improving the computing efficiency of the storage and computing integrated system.
[0026] In some possible implementations, the array status list includes one or more of the following: array module idle state and near memory circuit state. The array module idle state can indicate the idle and busy states of the array module, and the near memory circuit state can indicate the idle states of different operations in the near memory circuit. The storage and computing integrated system can collect the array module idle state and the near memory circuit state through the memory access control unit, and store the array module idle state and the near memory circuit state in the array status list.
[0027] In the embodiment of the present application, the array status list can include the idle state of the array module and the near-memory circuit state, thereby improving the execution efficiency of the storage-computing integrated system in executing tensor instructions.
[0028] In some possible implementations, the instruction storage list includes one or more of the following: an instruction segment of a tensor instruction, a tensor instruction number, a compiled operator number, and an array block identifier.
[0029] In the embodiment of the present application, the storage and computing integrated system stores compiled tensor instructions through an instruction storage list, thereby improving the query efficiency of tensor instructions.
[0030] In the second aspect, an embodiment of the present application provides an operator compilation device, which includes an acquisition unit, a compilation unit and a processing unit. The acquisition unit is used to obtain a compilation operator, and the compilation operator is used to indicate the compilation rules for the source code. The compilation unit is used to compile the source code based on the compilation operator to obtain one or more tensor instructions, and the tensor instructions are used to perform computing tasks in the storage and computing integrated system. The tensor instructions include one or more of the following instruction segments: opcode, tensor address, data type, data bit width and tensor length. The processing unit is used to store the tensor instructions in an instruction storage list, and the instruction storage list is used to store the compiled tensor instructions.
[0031] In some possible implementations, the processing unit is also used to create a compilation operator table, which is used to store compilation information corresponding to one or more compilation operators, and the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width and tensor length; the compilation unit is specifically used to generate one or more tensor instructions based on the compilation information stored in the compilation operator table.
[0032] In some possible implementations, the compilation unit is specifically used to generate an opcode of a tensor instruction according to an operator number in a compiled operator table, generate a tensor address of the tensor instruction according to a row address, a column address, and an output address in the compiled operator table, and generate a data type, data bit width, and tensor length of the tensor instruction according to the data type, data bit width, and tensor length in the compiled operator table.
[0033] In some possible implementations, the instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.
[0034] In some possible implementations, the compilation unit is also used to determine the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list, the instruction association matrix is used to store the dependencies between the tensor instructions, the array status list is used to indicate the status of the array module, the array module is used to execute the tensor instructions, and the instruction status includes an executable state and a waiting state.
[0035] In some possible implementations, the processing unit is further configured to send the tensor instruction to the memory access control unit when the instruction state of the tensor instruction is in an executable state. An execution operation is determined according to the tensor instruction, and the execution operation includes one or more of the following: a near memory calculation operation, an in-memory calculation operation, and a regular read and write operation.
[0036] In some possible implementations, a tensor instruction set includes a first tensor instruction and a second tensor instruction, and the first tensor instruction and the second tensor instruction are executed in the same array module. The processing unit is also used to simultaneously execute the first tensor instruction and the second tensor instruction in different execution units of the array module, and the execution unit includes an array and a near memory circuit.
[0037] In some possible implementations, the dependency relationship between tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.
[0038] In some possible implementations, the array status list includes one or more of the following: array module idle status and near memory circuit status.
[0039] In some possible implementations, the instruction storage list includes one or more of the following: an instruction segment of a tensor instruction, a tensor instruction number, a compiled operator number, and an array block identifier.
[0040] In a third aspect, an embodiment of the present application provides a computing device, comprising a processor, the processor being coupled to a memory, the processor being used to store instructions, and when the instructions are executed by the processor, the computing device executes the method described in the first aspect or any possible implementation method of the first aspect.
[0041] In a fourth aspect, an embodiment of the present application provides a computing device cluster, the computing device cluster includes one or more computing devices, the computing device includes a processor, the processor is coupled to a memory, the processor is used to store instructions, when the instructions are executed by the processor, the computing device cluster executes the method described in the first aspect or any possible implementation method of the first aspect.
[0042] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed, the computer executes the method described in the first aspect or any possible implementation of the first aspect.
[0043] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed, the computer implements the method described in the first aspect or any possible implementation method of the first aspect.
[0044] It can be understood that the beneficial effects that can be achieved by any of the operator compilation devices, computing devices, computing device clusters, computer-readable media or computer program products provided above can refer to the beneficial effects in the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A schematic diagram of the system architecture of a storage-computing integrated system provided in an embodiment of the present application;
[0046] Figure 2 A flowchart of an operator compilation method provided in an embodiment of the present application;
[0047] Figure 3 A schematic diagram of a tensor instruction execution flow provided in an embodiment of the present application;
[0048] Figure 4 A schematic diagram of a compilation operator table provided in an embodiment of the present application;
[0049] Figure 5 A schematic diagram of generating a tensor instruction based on a compilation operator provided in an embodiment of the present application;
[0050] Figure 6 A schematic diagram of a tensor instruction provided in an embodiment of the present application;
[0051] Figure 7 A schematic diagram of another method for generating tensor instructions based on a compilation operator provided in an embodiment of the present application;
[0052] Figure 8 A schematic diagram of an instruction storage list provided in an embodiment of the present application;
[0053] Fig. 9 A schematic diagram of an instruction association matrix provided in an embodiment of the present application;
[0054] Fig.10 A schematic diagram of an array status list provided in an embodiment of the present application;
[0055] Fig.11A schematic diagram of a flow chart for executing a tensor instruction provided in an embodiment of the present application;
[0056] Fig.12 A schematic diagram of the execution status of different execution units in a storage-computing integrated module provided in an embodiment of the present application;
[0057] Fig.13 A schematic diagram of the structure of an operator compilation device provided in an embodiment of the present application;
[0058] Fig.14 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;
[0059] Fig.15 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION
[0060] The embodiments of the present application provide an operator compilation method and device for improving the parallelism of executing computing tasks in a storage-computing integrated system.
[0061] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0062] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0063] First, some terms involved in the embodiments of the present application are introduced to facilitate technical personnel in this field to understand the technical solution.
[0064] The storage-computing integrated architecture is a technical architecture that integrates storage and computing functions. The storage-computing integrated architecture reduces the scale and frequency of data transfer between memory and processor by implementing embedded computing functions inside the memory chip, thereby improving system computing power and energy efficiency.
[0065] Near-memory computing refers to shortening the physical distance between memory and logic processing modules, thereby reducing data transmission latency and power consumption.
[0066] In-memory computing refers to performing calculations inside the memory, thereby eliminating possible data transmission and reducing system latency and power consumption.
[0067] Out of order execution (OOE) is a technique used in high-performance microprocessors to utilize instruction cycles to avoid certain types of latency. The processor can determine the order in which instructions are executed based on the availability of input data. In this way, the processor can avoid waiting for the next program instruction and can process the next instruction that can be executed immediately.
[0068] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.
[0069] See also Figure 1 , Figure 1 A schematic diagram of the system architecture of a storage and computing integrated system provided in an embodiment of the present application. Figure 1 In the example shown, the storage-computing system 10 is a storage-computing task issued by the central processor 20, such as a big data analysis task, a machine learning task, an image processing task, a multimedia encoding and decoding task, and a database query and storage task. The storage-computing system 10 is used to perform storage-computing computing tasks. In the storage-computing system, the embodiment of the present application does not distinguish between storage-computing tasks and computing tasks.
[0070] like Figure 1 As shown, the storage-computing integrated system 10 includes a compiler 101 and a storage-computing integrated chip 102, wherein the storage-computing integrated chip 102 includes a scheduler 1021, a memory access controller unit 1022 and an array module 1023. The compiler 101 and the storage-computing integrated chip 102 are respectively introduced in detail below.
[0071] The compiler 101 is used to receive the storage-computing-in-one task sent by the central processing unit 20, and compile the source code corresponding to the storage-computing-in-one task into instructions, and the instruction format of the instruction is the instruction format that can be directly executed by the storage-computing-in-one chip 102. One or more instructions compiled by the compiler 101 can constitute an instruction set. Since the storage-computing-in-one system 10 can use the instruction set to perform tensor calculations on the storage-computing-in-one chip 102, the instruction set can also be called a storage-computing-in-one instruction set or a tensor instruction set. The compiler 101 is also used to send the compiled tensor instruction set to the storage-computing-in-one chip 102, and the storage-computing-in-one chip 102 stores and executes the tensor instruction set.
[0072] The integrated storage and computing chip 102 is used to implement the processing of storage and computing tasks in the chip. Specifically, the integrated storage and computing chip 102 is used to execute the tensor instruction set compiled by the compiler 101, and complete the storage and computing tasks issued by the central processing unit 20 by executing the tensor instruction set. The integrated storage and computing chip 102 includes a scheduler 1021, a memory access controller unit 1022 and an array module 1023. The specific functions of these parts are introduced below.
[0073] The scheduler 1021 is used to receive the tensor instruction set compiled by the compiler 101 and store the tensor instruction set in the instruction storage list. The scheduler 1021 is also used to determine the instruction state of the tensor instruction based on the instruction association matrix and the array state list. When the instruction state is an executable state, the scheduler 1021 schedules the memory access control unit 1022 to execute the tensor instruction.
[0074] The memory access control unit 1022 is used to specifically execute tensor instructions. Specifically, the memory access control unit 1022 is used to generate a circuit control signal corresponding to the array module 1023 according to the tensor instruction, and use the circuit control signal to control the array module 1023 to perform operations such as storage and calculation. The memory access control unit 1022 can control one or more array modules 1023 to perform operations such as storage and calculation.
[0075] The array module 1023 is used to perform storage-computation tasks. Specifically, the array module 1023 includes a near storage circuit and a storage array. The near storage circuit is used to perform near storage computing operations, and the data after the near storage computing operations are stored in the storage array through the cache. The storage array is used to perform data read and write operations. In addition, the storage array is also used to perform in-memory computing operations, and the results of the in-memory computing operations are stored in the storage array through the cache.
[0076] based on Figure 1 The storage and computing integrated system 10 shown in the figure, the present application also provides an operator compilation method. The operator compilation method of the present application is introduced below in conjunction with an embodiment.
[0077] See also Figure 2 , Figure 2 A flowchart of an operator compilation method provided in an embodiment of the present application. Figure 2 In the example shown, the method includes the following steps:
[0078] S201. The storage and computing integrated system obtains the compilation operator.
[0079] The storage-computing system 10 receives the compilation operator sent by the central processing unit 20, and the compilation operator is used to instruct the compiler 101 on the compilation rules of the source code. Specifically, the compiler 101 can perform lexical analysis, syntax analysis, and code optimization on the source code based on the compilation operator. The storage-computing system 10 can also receive the source code corresponding to the storage-computing task sent by the central processing unit 20, and the compiler 101 can analyze and convert the source code based on the compilation operator to generate specific instructions that the storage-computing system 10 can execute.
[0080] See also Figure 3 , Figure 3 A schematic diagram of a tensor instruction execution flow provided in an embodiment of the present application. Figure 3 In step 1 of the example shown, the storage-computing system 10 receives the compilation operator sent by the central processor 20, and compiles the source code based on the compilation operator. For example, the compilation operator received by the storage-computing system 10 is a screening operator, which can select elements that meet the conditions from the data set according to specific conditions and return them as a new subset.
[0081] S202. The storage-computing integrated system compiles the source code based on the compilation operator to obtain tensor instructions, which are used to execute computing tasks of the storage-computing integrated system.
[0082] After the storage-computing integrated system 10 obtains the source code, it first creates a compilation operator table in the memory, which is used to store compilation information corresponding to one or more compilation operators, and the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length. The storage-computing integrated system 10 stores the compilation information in the source code into the compilation operator table according to the table structure of the compilation operator table.
[0083] In the above compilation operator table, the row address is the address of the tensor after the conditional statement in a source code, and the column address is the first address of the tensor involved in a source code, also known as the first address to be queried. The operator number refers to the identifier corresponding to different compilation operators. The bit width refers to the number of address bits occupied by a source code. The following example introduces the compilation operator table provided in the embodiment of the present application.
[0084] See also Figure 4 , Figure 4 A schematic diagram of source code compilation based on a compilation operator table provided in an embodiment of the present application. Figure 4 In the example shown, table (a) is a compiled operator table created by the storage-computing integrated system 10, and the table entries of the compiled operator table include operator number, row address, column address, output address, data type, data bit width and tensor length.
[0085] The row address is the address of the tensor after the conditional statement in a source code. For example, for the source code "A(A==B)→C", it means to judge whether A is equal to B. If A is equal to B, the comparison result is passed to C. The conditional statement is "Judge whether A is equal to B", so the tensor address after the conditional statement is the address of tensor B. Therefore, the storage and computing system 10 stores the address of tensor B as the row address in the compiled operator table.
[0086] exist Figure 4 In the example shown, the column address is the first address of the tensor involved in a source code, also called the first address to be queried. For another example, in the above source code "A(A==B)→C", the first address of the tensor involved in the source code is the address of tensor A, so the storage-computing system 10 stores the address of tensor A as the column address in the compiled operator table.
[0087] exist Figure 4 In the example shown, the operator number refers to the number corresponding to different compilation operators. For example, in the above source code "A(A==B)→C", the source code is a comparison operation, and the corresponding compilation operator is a filter operator, and the operator number corresponding to the filter operator is "tab1", so the storage and computing system 10 stores "tab1" in the compilation operator table.
[0088] exist Figure 4 In the example shown, the bit width refers to the number of address bits occupied by a source code. For example, in the above source code "A(A==B)→C", the address bits occupied are 100 bits, so the storage and computing system 10 stores "100" in the compilation operator table.
[0089] It is understandable that the compilation operator table created by the storage-computation system 10 can store one or more compilation operators involved in the source code, as well as the compilation information corresponding to the compilation operator. Therefore, the storage-computation system 10 can establish a mapping relationship between the operator number and the compilation information, so as to quickly query the compilation operator table during the source code compilation process and determine the content of the instruction segment of the tensor instruction.
[0090] The following continues to introduce the process of compiling source code based on the compilation operator table after the storage and computing integrated system 10 creates the compilation operator table.
[0091] In the process of generating tensor instructions corresponding to source code based on the compilation operator and the compilation operator table, the storage and computing system 10 first determines each instruction segment of the tensor instruction based on the operator structure of the compilation operator, and determines the specific content of each instruction segment based on the compilation information corresponding to the operator structure. Specifically, the compilation information in the compilation operator table is queried based on the operator number of the compilation operator, and the compilation information to be filled in the corresponding instruction segment is determined, thereby generating the content of each instruction segment of the tensor instruction and obtaining the tensor instruction. The following example introduces the compilation process of the source code.
[0092] Please continue reading Figure 4 ,exist Figure 4 In the example shown, the storage-computing integrated system 10 determines the various instruction segments of the tensor instruction based on the operator structure of the compiled operator, where different compiled operators have different operator structures. For example, the operator structure of the screening operator includes an opcode, a row address, a column address, an output address, an input data type, and a tensor length. The storage-computing integrated system 10 determines the various instruction segments of the tensor instruction based on the screening operator, and the instruction segment of the tensor instruction corresponding to the screening operator includes an opcode, a variable A address, a variable B address, a variable C address, a type, a bit width, and a tensor length. Among them, the variable A address corresponds to the row address of the operator structure, the variable B address corresponds to the address of the operator structure, the variable C address corresponds to the output address of the operator structure, and the type corresponds to the input data type of the operator structure.
[0093] In the example shown in 4, in the process of generating tensor instructions based on the compilation operator, the storage and computing system 10 queries the corresponding row address, column address, output address, data type and tensor length in the compilation operator table based on the operator number of the screening operator, and uses the queried row address, column address, output address, data type and tensor length as the content of the corresponding instruction segment of the tensor instruction. For example, the operator number of the screening operator is "tab1", and the storage and computing system 10 queries the compilation operator table based on "tab1" and finds that the row address is "row1" and the column address is "0x000010". The storage and computing system 10 can obtain the address of variable A and the address of variable B in the instruction segment of the tensor instruction. The storage and computing system 10 can further compile the source code based on other operator structures of the screening operator to obtain a complete tensor instruction.
[0094] The operator structure of the compilation operator in the embodiment of the present application includes one or more of the following: operation code, row address, column address, output address, input data type and tensor length. Among them, different types of compilation operators correspond to different operator structures, and the types of compilation operators are, for example, filter operators, sort operators and aggregation operators. The operator structure of the filtering operator includes an operation code, a row address, a column address, an output address, an input data type and a tensor length. The operator structure of the sorting operator includes an operation code, a row address, a column address, an input data type and a tensor length. The operator structure of the aggregation operator includes an operation code, a row address, a column address, an input data type and a tensor length.
[0095] In the embodiment of the present application, there is a mapping relationship between the operator structure of the compiled operator and the instruction segment of the tensor instruction. For example, the instruction segment of the tensor instruction includes one or more items: opcode, variable A address, variable B address, variable C address, type, bit width and tensor length. The opcode, row address, column address, output address, input data type and tensor length in the operator structure correspond to the opcode, variable A address, variable B address, variable C address, type and tensor length in the instruction segment respectively.
[0096] See also Figure 5 , Figure 5 A schematic diagram of determining the instruction segment of a tensor instruction based on the operator structure of a compiled operator provided in an embodiment of the present application. Figure 5 In the example shown, the integrated storage and computing system 10 generates corresponding tensor instructions based on the operator structure of the filtering operator, the sorting operator, and the aggregation operator.
[0097] exist Figure 5 In the example shown, taking the screening operator as an example, the operator structure of the screening operator includes an opcode, a row address, a column address, an output address, an input data type, and a tensor length. Based on the operator structure of the screening operator, the storage-computing system 10 determines the tensor address by querying the compilation operator table, and further compiles the opcode, variable A address, variable B address, variable C address, type, bit width, and tensor length of the instruction segment of the tensor instruction.
[0098] For example, the opcode of a tensor instruction can be determined as a comparison operation based on the source code "A(A==B)→C", and the corresponding opcode is "0010110", and the address of variable A can be determined as "0x1111" by querying the column address of the compiled operator table. The address of variable B can be determined as "0x1001" by querying the row address of the compiled operator table. The address, type, bit width, and tensor length of variable C can be determined as "0x3442", "0", "01", and "000000101" respectively based on the source code and the operator structure of the compiled operator.
[0099] In the embodiment of the present application, a compilation operator may correspond to one or more different instruction opcodes. Figure 4 In the example shown, the filtering operator includes operation codes such as comparison (compare, cmp), greater than or equal (greater than or equal, gte), greater than (greater than, gt), less than or equal (less than or equal, gt), and less than (Less Than, lt); the sorting operator includes operation codes such as ascending and descending; and the aggregation operator includes operation codes such as sum, avg, and count.
[0100] In the embodiments of the present application, different types of tensor instructions have different instruction segments. Among them, the instruction types of tensor instructions include one or more of the following types: address-only instructions, value-only instructions, array control instructions, and near-memory circuit control instructions. The following specifically introduces the instruction segments corresponding to these instruction types.
[0101] Among them, the instruction only involving the address is the instruction type in which only the tensor address participates. For example, the instruction only involving the address reads A and B from the address of tensor A and the address of tensor B respectively, performs the specified operation, and then writes the result C to the address of tensor C. The instruction segment of the instruction only involving the address includes one or more of the following: opcode, tensor address, type, bit width, and tensor length. Among them, the tensor address can be the address of multiple tensors, and the tensor length refers to the number of elements contained in the tensor.
[0102] A numerical participation instruction is a tensor instruction with a numerical value. For example, a numerical participation instruction reads A from the address of tensor A, performs a specified operation on the value C, and writes the result B to the address of tensor B. The instruction segment of a numerical participation instruction includes one or more of the following: opcode, tensor address, value, type, bit width, and tensor length.
[0103] An array control instruction is an instruction for performing an array operation. For example, an array control instruction reads A from the address of tensor A, and performs a control operation on the array of tensor A addresses corresponding to the specified operation in combination with the value C. The instruction segment of an array control instruction includes one or more of the following: an opcode, a tensor address, a value, and a tensor length.
[0104] The near memory circuit control instruction is an instruction for performing a near memory circuit operation specified by the array ID. The near memory circuit control instruction is specifically determined by the near memory circuit type and near memory circuit state in the instruction. The instruction segment of the near memory circuit control instruction includes one or more of the following: operation code, storage array ID, near memory circuit type, and near memory circuit state.
[0105] See also Figure 6 , Figure 6 A schematic diagram of instruction segments of tensor instructions of different instruction types provided in an embodiment of the present application. Figure 6 In the example shown, the instruction segment of the address-only instruction includes the opcode, tensor A address, tensor B address, tensor C address, type, bit width, and tensor length. The instruction segment of the value-only instruction includes the opcode, tensor A address, tensor B address, value C, type, bit width, and tensor length. The instruction segment of the array control instruction includes the opcode, tensor A address, value C, and tensor length. The instruction segment of the near storage circuit control instruction includes the opcode, storage array ID, near storage circuit type, and near storage circuit state.
[0106] from Figure 6 It can be seen from the example shown that when the type corresponding to the instruction does not require part of the segmentation in the instruction segment, the address or value corresponding to that part of the instruction segment can be set to 0 to indicate skipping. In the above instruction segment, in addition to the tensor address and value, there are also instruction segments for type, bit width, and tensor length. Among them, types include integers and floating-point numbers, integers include numbers from 1 to 16 bits, and floating-point numbers include BF16, FP16, FP32, and FP64, etc. The tensor length is used to identify the extended address length of the tensor address. When the tensor length is 1, the tensor data represented is equal to the scalar data.
[0107] It should be noted that when the storage-computing integrated system 10 compiles source code based on the compiling operator, one or more tensor instructions can be compiled from a broken source code based on different compiling operators.
[0108] See also Figure 7 , Figure 7 A schematic diagram of obtaining a tensor instruction based on a compilation operator compilation provided in an embodiment of the present application. Figure 7 In the example shown, the storage-computing system 10 compiles one or more tensor instructions based on a single compilation operator. For example, the storage-computing system 10 compiles "select*from t where ta>10" based on the screening operator to obtain one tensor instruction, namely, tensor instruction 0. For another example, the storage-computing system 10 compiles "select a+b*c from t" based on the projection operator to obtain two tensor instructions, namely, tensor instruction 1 and tensor instruction 2.
[0109] exist Figure 7In the example shown, a single source code command can be compiled into a combination of a single or multiple tensor storage instructions. For example, the three operators in the table correspond to different source code commands, the filter operator is used to take numbers greater than 10 in order, the projection operator is used to calculate |a+b*c|, and the ascending sort operator is used for ascending sorting. The filter operator corresponds to command 1, the projection operator corresponds to command 1 and command 2, and the ascending sort operator corresponds to command 3. Among them, command 0 and command 2 can be implemented using a single instruction. Command 1 and command 3 both require multiple instructions to implement.
[0110] S203. The storage and computing integrated system stores the tensor instructions into the instruction storage list.
[0111] After the storage and computing system 10 compiles the tensor, it stores the tensor instruction in the instruction storage list. The instruction storage list includes one or more of the following: the instruction segment of the tensor instruction, the tensor instruction number, the compilation operator number and the array block identifier. The instruction storage list is described in detail below.
[0112] See also Figure 8 , Figure 8 A schematic diagram of an instruction storage list provided in an embodiment of the present application. Figure 8 In the example shown, the storage-computing integrated system 10 generates tensor instructions in the order in which they are stored in the instruction storage list, and the instructions first stored in the instruction storage list will be executed first. In addition to storing the instruction segments of the tensor instructions, the instruction storage list also stores the tensor instruction number, compiled operator number, read-write address, array block identifier, and read-write coordinate position of the tensor instruction.
[0113] For example, the instruction storage column stores the instruction segment of a tensor instruction with instruction label "0" and compiled operator number "0". The read and write addresses of the tensor instruction are "[0x0011, 0x0104]" and "[0x0111]" respectively, the array block identifier of the storage array operated by the tensor instruction is "0", and the read and write coordinate positions of the storage array operated by the tensor instruction are "[(1,4,1,8),(2,3,4,7)]" and "[(0,-1,0,-1)]" respectively.
[0114] Please continue reading Figure 3 ,exist Figure 3 In step 2 of the example shown, after the storage-computing integrated system 10 compiles the source code based on the compilation operator to generate tensor instructions, the tensor instructions are stored in an instruction storage list.
[0115] S204. The storage and computing integrated system executes tensor instructions in the instruction storage list based on the instruction association matrix and array status list.
[0116] The storage and computing integrated system 10 executes the tensor instructions in the instruction storage list based on the instruction association matrix and the array state list. Specifically, the storage and computing integrated system 10 determines the instruction state of the tensor instructions in the instruction storage list according to the instruction association matrix and the array state list, and the instruction state includes an executable state and a waiting state. Among them, the instruction association matrix is used to store the dependencies between tensor instructions, the array state list is used to indicate the state of the array module, and the array module is used to execute the tensor instruction.
[0117] Please continue reading Figure 3 ,exist Figure 3 In steps 2 to 7 of the example shown, the storage-computing integrated system 10 decodes the tensor instruction after obtaining the compiled tensor instruction, and stores the decoded tensor instruction in the instruction storage list. The storage-computing integrated system 10 determines whether the tensor instruction is executable based on the instruction association matrix and the array status list. If the tensor instruction is executable, the tensor instruction is sent to the memory access control unit. If the tensor instruction is not executable, it continues to wait for execution in the instruction storage list.
[0118] The instruction association matrix and array status list are described in detail below.
[0119] In the embodiment of the present application, the instruction association matrix is used to store the association relationship between the instructions in the instruction storage list. The instruction association matrix is a matrix with the same length and width as the number of tensor instructions in the instruction storage list, and the matrix elements represent the instruction dependency. The dependency relationship between tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.
[0120] Among them, read-after-write dependency means that in concurrent computing, the read operation of one instruction occurs after the write operation of another instruction, that is, one instruction can only read a certain shared data after the other instruction completes writing the shared data. Write-after-write dependency means that in concurrent computing, the write operation of one instruction occurs after the write operation of another instruction, that is, one instruction can only modify a certain shared data after the other instruction completes writing the shared data. Write-after-read dependency means that in concurrent computing, the write operation of one instruction occurs after the read operation of another instruction, that is, one instruction can only modify a certain shared data after the other instruction completes reading the shared data.
[0121] See also Fig. 9 , Fig. 9 A schematic diagram of an instruction association matrix provided in an embodiment of the present application. Fig. 9In the example shown, the value of (i, j) in the matrix element of the instruction association matrix represents the conflict relationship between the i-th instruction and the j-th instruction, where 0 indicates no address conflict between the two, 1 indicates a write after read (WAR) dependency, 2 indicates a write after write (WAW) dependency, and 3 indicates a read after write (RAW) dependency. For example, instruction 1 depends on instruction 0 after read, instruction 2 depends on instruction 1 after write, instruction 3 depends on instruction 1 after write, and instruction 3 depends on instruction 2 after write.
[0122] In the embodiment of the present application, the array status list is used to query the status of the storage array and the status of the near storage circuit, so as to determine whether the tensor instruction can be sent to the corresponding storage array or near storage circuit for execution. The array status list includes one or more of the following: array module idle state and near storage circuit state.
[0123] See also Fig.10 , Fig.10 A schematic diagram of an array status list provided in an embodiment of the present application. Fig.10 In the array status list shown, the array status list displays the status of four storage arrays numbered "CB1", "CB3", "CB4" and "CB4", wherein an array status of "1" indicates that the array is busy, and an array status of "0" indicates that the array is idle. The array status list can also record the status of the near memory circuit, for example, the array status list can also record the status of the adder in the near memory circuit or the status of the XOR operator in the near memory circuit.
[0124] The storage-computing integrated system 10 determines the instruction state of the tensor instruction in the instruction storage list according to the instruction association matrix and the array state list. Specifically, when the tensor instruction to be executed satisfies the dependency of the instruction association matrix, and the storage array or near storage circuit operated by the tensor instruction is in an idle state, the instruction state of the tensor instruction is an executable state. In the embodiment of the present application, the storage-computing integrated system 10 sends a tensor instruction to the memory access control unit based on the instruction state of the tensor instruction, which is also called out-of-order execution.
[0125] When the instruction state of the tensor instruction is in an executable state, the tensor instruction is sent to the memory access control unit. The storage and computing integrated system 10 determines the execution operation according to the tensor instruction, and the execution operation includes one or more of the following: near memory computing operation, in-memory computing operation and conventional read and write operation. Among them, the near memory computing operation refers to the operation of reading the required data through the array or cache, and then calling the corresponding near memory circuit to complete the calculation. The in-memory computing operation refers to the direct use of the storage array for calculation. Conventional read and write operations refer to read and write operations on the storage array. The computational parallelism of tensor instructions in the above-mentioned different storage and computing types is different.
[0126] The following introduces several operations of the storage and computing system 10 to execute tensor instructions with examples.
[0127] See also Fig.11 , Fig.11 A schematic diagram of a memory access control unit performing a storage-computation-in-one operation provided in an embodiment of the present application. Fig.11 In the example shown, after receiving the tensor instruction, the memory access control unit 1022 generates a circuit control signal corresponding to the storage-computation integrated array model group 1023, and the storage-computation integrated array model group 1023 determines the type of storage-computation operation based on the control circuit signal.
[0128] exist Fig.11 In the example shown, when the type of storage-computation operation is a near-memory computing operation, the array module 1023 reads the required data via the array or cache, and then calls the corresponding near-memory circuit to complete the computing operation. During the near-memory computing process, the array module 1023 in the storage-computation integrated chip 102 performs a single-row array read, and then transmits it to the near-memory circuit via the cache.
[0129] exist Fig.11 In the example shown, when the type of the storage operation is an in-memory operation, the array module 1023 directly uses the storage array to perform the calculation operation, for example, performing multi-row array multiplication and addition operations in the storage array, and writing the calculation results to the storage array via the cache. When the type of the storage operation is a regular read and write operation, the array module 1023 performs read and write operations on the data in each data unit of the storage array, and transmits the data via the cache.
[0130] In the implementation of the present application, the first tensor instruction and the second tensor instruction are respectively and simultaneously executed in different execution units of the array module 1023, the different execution units include an array and a near storage circuit, and the first tensor instruction and the second tensor instruction are different tensor instructions. The following is an introduction with examples.
[0131] See also Fig.12 , Fig.12 A schematic diagram of different tensor instructions executed in a storage-computation-in-one array model provided in an embodiment of the present application. Fig.12In the example shown, Figure (a) is a schematic diagram of 4 different tensor instructions, and Figure (b) is a schematic diagram of the execution status of different tensor instructions at different times. Among the 4 tensor instructions shown in Figure (a), instruction 0, instruction 1, and instruction 2 are all distributed in an array module and cannot be read and written in parallel. Instruction 3 is distributed in another array module, and its processing process can be parallel to instruction 0, instruction 1, and instruction 2. At the same time, there is a read-after-write conflict between instruction 1 and instruction 0, so instruction 1 needs to wait for instruction 0 to be written before it can be read.
[0132] As can be seen from the example shown in Figure (b), during the instruction execution process, instruction 0 and instruction 1 first enter the scheduling in parallel, and the array module 1023 recognizes the read-after-write RAW dependency between the two, so instruction 1 starts waiting for the completion of instruction 0. During this period, instruction 2 and instruction 3 are also entered into the scheduling. Since instruction 3 is on another array module and does not conflict with other instructions, it can be executed without restriction.
[0133] From the example shown in Figure (b), it can be seen that the array and near memory circuit of the array module 1023 can respectively and simultaneously execute instruction 0 and instruction 2. For example, when the near memory circuit of the array module 1023 is calculating, the storage array is in an idle state. Therefore, when instruction 0 is performing near memory circuit calculation operation, instruction 2 can simultaneously perform a read operation.
[0134] In addition, it can be seen from the example shown in Figure (b) that since instruction 2 is a max instruction, the reading part is divided into two steps, namely, identifying the maximum value and reading the maximum value. After completing the first step of reading, instruction 0 completes the wait for instruction 2, and instruction 0 occupies the array to complete the write operation, and then instruction 2 continues the read operation. Therefore, instruction 1 can complete the write operation on the idle array only after instruction 2 completes the write.
[0135] It can be seen from the above embodiments that the storage-computing integrated system in the embodiments of the present application can generate a tensor instruction set based on the compiled operator, and can process tensor objects through the tensor instruction set, thereby realizing storage array-level operations on the storage-computing integrated system, improving the processing parallelism of the storage-computing integrated system in processing computing tasks, and thereby improving the computing performance of the storage-computing integrated system.
[0136] Based on the above method embodiment, the embodiment of the present application further provides an operator compilation device. The operator compilation device provided by the embodiment of the present application is specifically introduced below.
[0137] See also Fig.13 , Fig.13 A schematic diagram of the structure of an operator compilation device provided in an embodiment of the present application. Fig.13In the example shown, the operator compilation device 1300 is used to implement the various steps performed by the storage-computation-in-one system in the above-mentioned embodiments. The operator compilation device 1300 includes an acquisition unit 1301 , a compilation unit 1302 and a processing unit 1303 .
[0138] The acquisition unit 1301 is used to obtain the compilation operator, which is used to indicate the compilation rules for the source code. The compilation unit 1302 is used to compile the source code based on the compilation operator to obtain one or more tensor instructions. The tensor instructions are used to perform computing tasks in the storage and computing system. The tensor instructions include one or more of the following instruction segments: operation code, tensor address, data type, data bit width and tensor length. The processing unit 1303 is used to store the tensor instructions in the instruction storage list, and the instruction storage list is used to store the compiled tensor instructions.
[0139] In some possible implementations, the processing unit 1303 is further configured to create a compilation operator table, the compilation operator table is used to store compilation information corresponding to one or more compilation operators, and the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length. The compilation unit 1302 is specifically configured to generate one or more tensor instructions based on the compilation information stored in the compilation operator table.
[0140] In some possible implementations, the compilation unit 1302 is specifically used to generate an opcode of a tensor instruction according to an operator number in a compiled operator table, generate a tensor address of the tensor instruction according to a row address, a column address, and an output address in the compiled operator table, and generate a data type, data bit width, and tensor length of the tensor instruction according to the data type, data bit width, and tensor length in the compiled operator table.
[0141] In some possible implementations, the instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.
[0142] In some possible implementations, the compilation unit 1302 is also used to determine the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list, the instruction association matrix is used to store the dependencies between the tensor instructions, the array status list is used to indicate the status of the array module, the array module is used to execute the tensor instructions, and the instruction status includes an executable state and a waiting state.
[0143] In some possible implementations, the processing unit 1303 is further configured to send the tensor instruction to the memory access control unit when the instruction state of the tensor instruction is in an executable state. An execution operation is determined according to the tensor instruction, and the execution operation includes one or more of the following: a near memory calculation operation, an in-memory calculation operation, and a conventional read and write operation.
[0144] In some possible implementations, the tensor instruction set includes a first tensor instruction and a second tensor instruction, and the first tensor instruction and the second tensor instruction are executed in the same array module. The processing unit 1303 is also used to simultaneously execute the first tensor instruction and the second tensor instruction in different execution units of the array module, and the execution unit includes an array and a near memory circuit.
[0145] In some possible implementations, the dependency relationship between tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.
[0146] In some possible implementations, the array status list includes one or more of the following: array module idle status and near memory circuit status.
[0147] In some possible implementations, the instruction storage list includes one or more of the following: an instruction segment of a tensor instruction, a tensor instruction number, a compiled operator number, and an array block identifier.
[0148] It should be understood that the division of the units in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And the units in the device can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some units can also be implemented in the form of software calling through processing elements, and some units can be implemented in the form of hardware. For example, each unit can be a separately established processing element, or it can be integrated in a certain chip of the device. In addition, it can also be stored in the memory in the form of a program, and called and executed by a certain processing element of the device. The function of the unit. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element described here can also be a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by an integrated logic circuit of hardware in the processor element or in the form of software calling through a processing element.
[0149] It is worth noting that, for the above method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required for the present application.
[0150] Other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with the fact that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.
[0151] See also Fig.14 , Fig.14 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. Fig.14 As shown, the computing device 1400 includes: a processor 1401, a memory 1402, a communication interface 1403 and a bus 1404. The processor 1401, the memory 1402 and the communication interface 1403 are coupled via a bus (not marked in the figure). The memory 1402 stores instructions. When the execution instructions in the memory 1402 are executed, the computing device 1400 executes the method executed by the storage and computing system in the above method embodiment.
[0152] The computing device 1400 may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), or a combination of at least two of these integrated circuit forms. For another example, when a unit in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0153] The processor 1401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.
[0154] The memory 1402 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).
[0155] The memory 1402 stores executable program codes, and the processor 1401 executes the executable program codes to respectively implement the functions of the aforementioned acquisition unit 1301, processing unit 1302, and processing unit 1303, thereby implementing the aforementioned routing table compression method. That is, the memory 1402 stores instructions for executing the aforementioned routing table compression method.
[0156] The communication interface 1403 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1400 and other devices or a communication network.
[0157] In addition to the data bus, the bus 1404 may also include a power bus, a control bus, and a status signal bus. The bus may be a peripheral component interconnect express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus may be divided into an address bus, a data bus, a control bus, etc.
[0158] See also Fig.15 , Fig.15 A schematic diagram of a computing device cluster provided in an embodiment of the present application. Fig.15 As shown, the computing device cluster 1500 includes at least one computing device 1400 .
[0159] like Fig.15 As shown, the computing device cluster 1500 includes at least one computing device 1400. The memory 1402 in one or more computing devices 1400 in the computing device cluster 1500 may store the same instructions for executing the above-mentioned routing table compression method.
[0160] In some possible implementations, the memory 1402 of one or more computing devices 1400 in the computing device cluster 1500 may also respectively store partial instructions for executing the above-mentioned routing table compression method. In other words, the combination of one or more computing devices 1400 may jointly execute instructions for executing the above-mentioned routing table compression method.
[0161] It should be noted that the memory 1402 in different computing devices 1400 in the computing device cluster 1500 can store different instructions, which are respectively used to execute part of the functions of the above-mentioned operator compilation device. That is, the instructions stored in the memory 1402 in different computing devices 1400 can implement the functions of one or more modules in the acquisition unit 1301, the compilation unit 1302 and the processing unit 1303.
[0162] In some possible implementations, one or more computing devices 1400 in the computing device cluster 1500 may be connected via a network, which may be a wide area network or a local area network.
[0163] In another embodiment of the present application, a computer-readable storage medium is provided, in which computer execution instructions are stored. When the processor of the device executes the computer execution instructions, the device executes the method executed by the storage-computing integrated system in the above method embodiment.
[0164] In another embodiment of the present application, a computer program product is further provided, the computer program product comprising computer executable instructions, the computer executable instructions being stored in a computer readable storage medium. When a processor of a device executes the computer executable instructions, the device executes the method executed by the storage-computing-in-one system in the above method embodiment.
[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0166] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0167] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0168] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.
Claims
1. An operator compilation method, characterized in that: Applied to a storage-computing integrated system, the method includes: Obtaining a compilation operator, where the compilation operator is used to indicate a compilation rule of a source code; Compiling the source code based on the compilation operator to obtain one or more tensor instructions, where the tensor instructions are used to execute computing tasks in the storage-computing-in-one system, and the tensor instructions include one or more of the following instruction segments: an operation code, a tensor address, a data type, a data bit width, and a tensor length; The tensor instruction is stored in an instruction storage list, where the instruction storage list is used to store compiled tensor instructions.
2. The method according to claim 1, characterized in that: Before compiling the source code based on the compilation operator, the method further includes: Create a compilation operator table, where the compilation operator table is used to store compilation information corresponding to one or more of the compilation operators, where the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length; The compiling the source code based on the compiling operator comprises: One or more tensor instructions are generated based on the compilation information stored in the compilation operator table.
3. The method according to claim 2, characterized in that The generating one or more tensor instructions based on the compilation information stored in the operator compilation table includes: Generate an operation code for the tensor instruction according to the operator number in the compiled operator table; Generate a tensor address of the tensor instruction according to the row address, column address and output address in the compiled operator table; The data type, data bit width and tensor length of the tensor instruction are generated respectively according to the data type, data bit width and tensor length in the compiled operator table.
4. The method according to any one of claims 1 to 3, characterized in that The instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.
5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The instruction status of the tensor instructions in the instruction storage list is determined according to the instruction association matrix and the array status list, the instruction association matrix is used to store the dependencies between the tensor instructions, the array status list is used to indicate the status of the array module, the array module is used to execute the tensor instructions, and the instruction status includes an executable state and a waiting state.
6. The method according to claim 5, characterized in that The method further comprises: When the instruction state of the tensor instruction is in the executable state, sending the tensor instruction to a memory access control unit; An execution operation is determined according to the tensor instruction, where the execution operation includes one or more of the following: a near-memory computing operation, an in-memory computing operation, and a regular read-write operation.
7. The method according to any one of claims 1 to 6, characterized in that The tensor instruction set includes a first tensor instruction and a second tensor instruction, wherein the first tensor instruction and the second tensor instruction are executed in the same array module, and the method further includes: The first tensor instruction and the second tensor instruction are respectively and simultaneously executed in different execution units of the array module, and the execution unit includes an array and a near memory circuit.
8. The method according to any one of claims 5 to 7, characterized in that The dependency relationship between the tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.
9. The method according to any one of claims 5 to 8, characterized in that The array status list includes one or more of the following: array module idle status and near memory circuit status.
10. The method according to any one of claims 5 to 9, characterized in that The instruction storage list includes one or more of the following: an instruction segment of the tensor instruction, a tensor instruction number, a compilation operator number, and an array block identifier.
11. An operator compilation device, characterized in that: The device comprises: An acquisition unit, used for acquiring a compilation operator, where the compilation operator is used for indicating a compilation rule for a source code; A compiling unit, configured to compile the source code based on the compiling operator to obtain one or more tensor instructions, wherein the tensor instructions are used to execute computing tasks in the storage-computing-in-one system, and the tensor instructions include one or more of the following instruction segments: an operation code, a tensor address, a data type, a data bit width, and a tensor length; A processing unit is used to store the tensor instruction into an instruction storage list, where the instruction storage list is used to store compiled tensor instructions.
12. The device according to claim 11, characterized in that The processing unit is also used for: Create a compilation operator table, where the compilation operator table is used to store compilation information corresponding to one or more of the compilation operators, where the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length; The compilation unit is specifically used for: One or more tensor instructions are generated based on the compilation information stored in the compilation operator table.
13. The device according to claim 12, characterized in that The compilation unit is specifically used for: Generate an operation code for the tensor instruction according to the operator number in the compiled operator table; Generate a tensor address of the tensor instruction according to the row address, column address and output address in the compiled operator table; The data type, data bit width and tensor length of the tensor instruction are generated respectively according to the data type, data bit width and tensor length in the compiled operator table.
14. The device according to any one of claims 11 to 13, characterized in that The instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.
15. The device according to any one of claims 11 to 14, characterized in that The compilation unit is also used to: The instruction status of the tensor instructions in the instruction storage list is determined according to the instruction association matrix and the array status list, the instruction association matrix is used to store the dependencies between the tensor instructions, the array status list is used to indicate the status of the array module, the array module is used to execute the tensor instructions, and the instruction status includes an executable state and a waiting state.
16. The device according to claim 15, characterized in that The processing unit is also used for: When the instruction state of the tensor instruction is in the executable state, sending the tensor instruction to a memory access control unit; An execution operation is determined according to the tensor instruction, where the execution operation includes one or more of the following: a near-memory computing operation, an in-memory computing operation, and a regular read-write operation.
17. The device according to any one of claims 11 to 16, characterized in that The tensor instruction set includes a first tensor instruction and a second tensor instruction, wherein the first tensor instruction and the second tensor instruction are executed in the same array module, and the processing unit is further configured to: The first tensor instruction and the second tensor instruction are respectively and simultaneously executed in different execution units of the array module, and the execution unit includes an array and a near memory circuit.
18. The device according to any one of claims 15 to 17, characterized in that The dependency relationship between the tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.
19. The device according to any one of claims 15 to 18, characterized in that The array status list includes one or more of the following: array module idle status and near memory circuit status.
20. The device according to any one of claims 15 to 19, characterized in that The instruction storage list includes one or more of the following: an instruction segment of the tensor instruction, a tensor instruction number, a compilation operator number, and an array block identifier.
21. A computing device, characterized in that The electronic device comprises a processor coupled to a memory, wherein the processor is used to store instructions. When the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 10.
22. A computing device cluster, characterized in that: The system comprises at least one computing device, wherein the computing device comprises a processor, wherein the processor is coupled to a memory, and the processor is used to store instructions. When the instructions are executed by the processor, the computing device cluster executes the method according to any one of claims 1 to 10.
23. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed, the computer is caused to perform the method according to any one of claims 1 to 10.
24. A computer program product, comprising instructions, characterized in that: When the instructions are executed, the computer implements the method according to any one of claims 1 to 10.
Citation Information
Cited By
Test instruction generation method and device, electronic equipment, medium and product
CN120973695A
Operator compiling method and apparatus
EP4697162A1
Operator compiling method and apparatus
WO2024244856A1
Compute-in-memory chip, instruction scheduling method, and related apparatus
WO2025190264A1
Control method, control apparatus, in-memory computing system, and electronic device
WO2026200589A1
