Operator compiling method and apparatus

By using operator compilation method to generate tensor instruction sets in the storage and computing integrated system, the problems of data handling delay and power consumption under the storage and computing separation architecture are solved, the computing performance and parallelism are improved, and more efficient data processing is achieved .

WO2024244856A1PCT designated stage expired Publication Date: 2024-12-05HUAWEI TECH CO LTD

Patent Information

Application Number
PCT/CN2024/090157
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-08-22
Filing Date
2024-04-26
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The existing storage and computing separation architecture leads to data handling delay and transmission power consumption problems in big data application scenarios, and the scalar instruction set processing is low, so it is impossible to fully utilize the advantages of the storage and computing architecture.

Method used

The operator compilation method is used to compile the source code into a tensor instruction set, and the calculation tasks of the integrated storage and computing system are executed through the tensor instruction set, improving the parallelism of processing, thereby improving the computing performance.

Benefits of technology

The computing performance and energy efficiency of the integrated storage and computing system are significantly improved through the tensor instruction set, the data handling frequency and transmission power consumption are reduced, and the processing parallelism is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024090157_05122024_PF_FP_ABST
    Figure CN2024090157_05122024_PF_FP_ABST
Patent Text Reader

Abstract

An operator compiling method and apparatus, which are used for implementing tensor data processing of an in-memory computing architecture, and improving the computing task processing parallelism of the in-memory computing architecture. The method comprises: acquiring a compilation operator, which is used for indicating a compilation rule for source codes; compiling the source codes on the basis of the compilation operator, so as to obtain one or more tensor instructions, wherein the tensor instructions are used for executing computing tasks in an in-memory computing system, and each tensor instruction comprises one or more of the following instruction segments: an operation code, a tensor address, a data type, a data bit width and a tensor length; and storing the tensor instructions in an instruction storage list, which is used for storing compiled tensor instructions.
Need to check novelty before this filing date? Find Prior Art

Description

Operator compilation method and device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on May 31, 2023, with application number 202310639274.4 and invention name “A data processing system”, and claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on August 22, 2023, with application number 202311070464.5 and invention name “An operator compilation method and device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the field of computers, and more particularly to a method and apparatus for compiling an operator. Background Art

[0003] In recent years, with the rapid development of emerging applications such as big data, the explosive growth of information has led to a rapid expansion in computing scale, and computing forms have become increasingly diversified. The diverse application scenarios have placed higher demands on the computing and storage performance of hardware computing systems. For example, in big data applications, the storage-computing separation architecture of current mainstream hardware severely restricts performance and energy efficiency improvements. This is because the processor needs to move data from memory to the processor via the data bus during operations, and then move the data back to memory after processing. As the amount of data increases, the delay in this process also increases, even exceeding the read, write, and processing delays of the memory and processor itself. In addition, the frequent transfer of large amounts of data between memory and processors also brings serious transmission power consumption issues.

[0004] To address the data transfer energy efficiency issues inherent in the separate memory and compute architecture, the integrated memory and compute architecture emerged. This architecture significantly reduces the scale and frequency of data transfers between memory and processors by implementing embedded computing capabilities within the memory chip, significantly improving the computing power and energy efficiency of the computing system. However, most instruction sets currently used in integrated memory and compute architectures are scalar instruction sets. When memory uses scalar instruction sets to process computing tasks, it must process each scalar individually, resulting in low parallelism in the processing of computing tasks and failing to fully leverage the advantages of the integrated memory and compute architecture.

[0005] Summary of the Invention

[0006] The present application provides an operator compilation method, wherein a storage-computing system can use the operator compilation method to compile a tensor instruction set, and use the tensor instruction set to execute computing tasks of the storage-computing system, thereby improving the processing parallelism of the computing tasks. The present application also provides a computing device, a computing device cluster, a computer-readable storage medium, and a computer program product corresponding to the operator compilation method.

[0007] In the first aspect, an embodiment of the present application provides an operator compilation method, which can be executed by a storage-computing integrated system, or by a component of the storage-computing integrated system, such as a processor, chip or chip system of the storage-computing integrated system, or by a logic module or software that can realize all or part of the functions of the storage-computing integrated system. The method provided in the first aspect includes: the storage-computing integrated system obtains a compilation operator, and the compilation operator is used to indicate the compilation rules for the source code. The storage-computing integrated system compiles the source code based on the compilation operator to obtain one or more tensor instructions, and the tensor instructions are used to execute computing tasks in the storage-computing integrated system. The tensor instructions include one or more of the following instruction segments: opcode, tensor address, data type, data bit width and tensor length. The storage-computing integrated system stores the tensor instructions in an instruction storage list, and the instruction storage list is used to store compiled tensor instructions.

[0008] In the embodiment of the present application, the storage-computing integrated system can compile the source code based on the compilation operator to obtain tensor instructions, and realize the processing of tensors in the computing task by executing the tensor instructions. Compared with the current storage-computing integrated system that uses existing scalar instructions to process computing tasks, the operator compilation method and tensor instruction set provided in the embodiment of the present application improve the processing parallelism of the storage-computing integrated system in processing computing tasks, thereby improving the computing performance of the storage-computing integrated system.

[0009] In some possible implementations, before the integrated storage-computing system compiles source code based on a compilation operator, the integrated storage-computing system creates a compilation operator table. The compilation operator table is used to store compilation information corresponding to one or more compilation operators. The compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length. During the process of compiling the source code based on the compilation operator, the integrated storage-computing system directly queries the compilation operator and compilation information in the compilation operator table and generates tensor instructions corresponding to the source code based on the compilation operator and compilation information.

[0010] In the implementation of this application, the storage-computing integrated system can pre-store the compilation information in the source code into the compilation operator table, so that during the source code compilation process, the storage-computing integrated system can quickly compile and generate tensor instructions corresponding to the source code information based on the compilation operator table, thereby improving the compilation efficiency of tensor instructions.

[0011] In some possible implementations, in the process of the storage-computing integrated system generating one or more tensor instructions based on the compilation information stored in the operator compilation table, the storage-computing integrated system generates the opcode of the tensor instruction according to the operator number in the compilation operator table, generates the tensor address of the tensor instruction according to the row address, column address and output address in the compilation operator table, and generates the data type, data bit width and tensor length of the tensor instruction according to the data type, data bit width and tensor length in the compilation operator table.

[0012] In the embodiment of the present application, the storage and computing integrated system can quickly compile and generate tensor instructions corresponding to the source code based on the compilation operator table, thereby improving the compilation efficiency of the tensor instructions.

[0013] In some possible implementations, the types of compiled operators include filter operators, sort operators, and aggregation operators. The operator structure of a filter operator includes an opcode, a row address, a column address, an output address, an input data type, and a tensor length. The operator structure of a sort operator includes an opcode, a row address, a column address, an input data type, and a tensor length. The operator structure of an aggregation operator includes an opcode, a row address, a column address, an input data type, and a tensor length.

[0014] In the embodiment of the present application, the storage-computing integrated system provides a variety of compilation operators, which can compile different source codes to obtain tensor instructions, thereby improving the richness of the tensor instruction set.

[0015] In some possible implementations, the instruction type includes one or more of the following: an address-only instruction, a value-involved instruction, an array control instruction, and a near memory circuit control instruction. An address-only instruction is an instruction type that only involves tensor addresses. A value-involved instruction is a tensor instruction that also includes a value. An array control instruction is an instruction that performs an array operation. A near memory circuit control instruction is an instruction that performs a near memory circuit operation specified by an array ID.

[0016] In the embodiment of the present application, the storage and computing integrated system can compile tensor instructions of different instruction types, thereby using tensor instructions of different tensor types to perform multiple storage and computing operations, thereby improving the applicability to different storage and computing operations.

[0017] In some possible implementations, different types of tensor instructions have different instruction segments. The instruction segment of an instruction involving only addresses includes an opcode, tensor address, type, bit width, and tensor length. The instruction segment of an instruction involving only values ​​includes an opcode, tensor address, value, type, bit width, and tensor length. The instruction segment of an array control instruction includes an opcode, tensor address, value, and tensor length. The instruction segment of a near memory circuit control instruction includes an opcode, storage array identifier, near memory circuit type, and near memory circuit status.

[0018] In the embodiment of the present application, the storage and computing integrated system provides tensor instructions and instruction segment formats of different instruction types, thereby improving the feasibility of the solution.

[0019] In some possible implementations, the storage and computing integrated system determines the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list. The instruction association matrix is ​​used to store the dependencies between tensor instructions, and the array status list is used to indicate the status of the array module. The array module is used to execute tensor instructions, and the instruction status includes an executable state and a waiting state.

[0020] In the embodiment of the present application, the storage and computing integrated system can determine the instruction status of the tensor instruction based on the instruction association matrix and the array status list, and then determine the execution order of the tensor instruction, thereby improving the computing efficiency of the storage and computing integrated system.

[0021] In some possible implementations, when the tensor instruction's instruction state is in an executable state, the integrated storage-computing system sends the tensor instruction to the memory access control unit. The integrated storage-computing system determines an execution operation based on the tensor instruction, where the execution operation includes one or more of the following: a near-memory computation operation, an in-memory computation operation, and a regular read / write operation.

[0022] The tensor instructions generated by the storage-computing integrated system in the embodiment of the present application can be applied to a variety of storage-computing integrated architectures, thereby improving the applicability of tensor instructions to the storage-computing integrated system.

[0023] In some possible implementations, the tensor instruction set includes a first tensor instruction and a second tensor instruction, and the first tensor instruction and the second tensor instruction are executed in the same array module. The storage and computing integrated system executes the first tensor instruction and the second tensor instruction simultaneously in different execution units of the array module, and the execution unit includes an array and a near storage circuit.

[0024] In the storage-computing integrated system of the embodiment of the present application, different execution units of the array module can execute different tensor instructions simultaneously, thereby improving the execution parallelism of the tensor instructions and the hardware utilization efficiency of the array module, and further improving the computing efficiency of the storage-computing integrated system.

[0025] In some possible implementations, the dependencies between tensor instructions include one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency. A read-after-write dependency means that a read operation of one instruction occurs after a write operation of another instruction in a concurrent computation. A write-after-write dependency means that a write operation of one instruction occurs after a write operation of another instruction in a concurrent computation. A write-after-read dependency means that a write operation of one instruction occurs after a read operation of another instruction in a concurrent computation.

[0026] In the embodiment of the present application, the instruction association matrix can store the dependency relationship between multiple tensor instructions, thereby reducing the probability of instruction conflicts during the execution of tensor instructions and improving the computing efficiency of the storage and computing integrated system.

[0027] In some possible implementations, the array status list includes one or more of the following: array module idle status and near memory circuit status. The array module idle status can indicate the idle and busy status of the array module, and the near memory circuit status can indicate the idle status of different operations in the near memory circuit. The storage and computing system can collect the array module idle status and near memory circuit status via the memory access control unit and store the array module idle status and near memory circuit status in the array status list.

[0028] In the embodiment of the present application, the array status list can include the idle state of the array module and the near-memory circuit state, thereby improving the execution efficiency of the storage and computing integrated system in executing tensor instructions.

[0029] In some possible implementations, the instruction storage list includes one or more of the following: an instruction segment of a tensor instruction, a tensor instruction number, a compilation operator number, and an array block identifier.

[0030] In the embodiment of the present application, the storage and computing integrated system stores compiled tensor instructions through an instruction storage list, thereby improving the query efficiency of tensor instructions.

[0031] In a second aspect, an embodiment of the present application provides an operator compilation device, which includes an acquisition unit, a compilation unit, and a processing unit. The acquisition unit is used to obtain a compilation operator, and the compilation operator is used to indicate the compilation rules for the source code. The compilation unit is used to compile the source code based on the compilation operator to obtain one or more tensor instructions. The tensor instructions are used to perform computing tasks in the storage and computing integrated system. The tensor instructions include one or more of the following instruction segments: opcode, tensor address, data type, data bit width, and tensor length. The processing unit is used to store the tensor instructions in an instruction storage list, and the instruction storage list is used to store the compiled tensor instructions.

[0032] In some possible implementations, the processing unit is also used to create a compilation operator table, which is used to store compilation information corresponding to one or more compilation operators, and the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width and tensor length; the compilation unit is specifically used to generate one or more tensor instructions based on the compilation information stored in the compilation operator table.

[0033] In some possible implementations, the compilation unit is specifically used to generate an opcode of a tensor instruction based on the operator number in the compilation operator table, generate a tensor address of the tensor instruction based on the row address, column address and output address in the compilation operator table, and generate a data type, data bit width and tensor length of the tensor instruction based on the data type, data bit width and tensor length in the compilation operator table.

[0034] In some possible implementations, the instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.

[0035] In some possible implementations, the compilation unit is also used to determine the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list. The instruction association matrix is ​​used to store the dependencies between tensor instructions, and the array status list is used to indicate the status of the array module. The array module is used to execute tensor instructions, and the instruction status includes an executable state and a waiting state.

[0036] In some possible implementations, the processing unit is further configured to send the tensor instruction to the memory access control unit when the instruction state of the tensor instruction is in an executable state. An execution operation is determined based on the tensor instruction, where the execution operation includes one or more of the following: a near-memory computation operation, an in-memory computation operation, and a regular read / write operation.

[0037] In some possible implementations, the tensor instruction set includes a first tensor instruction and a second tensor instruction, and the first tensor instruction and the second tensor instruction are executed in the same array module. The processing unit is also used to simultaneously execute the first tensor instruction and the second tensor instruction in different execution units of the array module, and the execution unit includes an array and a near memory circuit.

[0038] In some possible implementations, the dependency relationship between tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.

[0039] In some possible implementations, the array status list includes one or more of the following: an array module idle state and a near memory circuit state.

[0040] In some possible implementations, the instruction storage list includes one or more of the following: an instruction segment of a tensor instruction, a tensor instruction number, a compilation operator number, and an array block identifier.

[0041] In a third aspect, an embodiment of the present application provides a computing device, comprising a processor coupled to a memory, the processor being used to store instructions. When the instructions are executed by the processor, the computing device executes the method described in the first aspect or any possible implementation of the first aspect.

[0042] In a fourth aspect, an embodiment of the present application provides a computing device cluster, which includes one or more computing devices, each of which includes a processor coupled to a memory, and the processor is used to store instructions. When the instructions are executed by the processor, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect.

[0043] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0044] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed, the computer implements the method described in the first aspect or any possible implementation method of the first aspect.

[0045] It can be understood that the beneficial effects that can be achieved by any of the operator compilation devices, computing devices, computing device clusters, computer-readable media or computer program products provided above can refer to the beneficial effects in the corresponding methods and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of the system architecture of a storage and computing integrated system provided in an embodiment of the present application;

[0047] FIG2 is a schematic diagram of a flow chart of an operator compilation method provided in an embodiment of the present application;

[0048] FIG3 is a schematic diagram of a tensor instruction execution flow provided in an embodiment of the present application;

[0049] FIG4 is a schematic diagram of a compilation operator table provided in an embodiment of the present application;

[0050] FIG5 is a schematic diagram of generating a tensor instruction based on a compilation operator according to an embodiment of the present application;

[0051] FIG6 is a schematic diagram of a tensor instruction provided in an embodiment of the present application;

[0052] FIG7 is a schematic diagram of another method for generating tensor instructions based on a compilation operator according to an embodiment of the present application;

[0053] FIG8 is a schematic diagram of an instruction storage list provided in an embodiment of the present application;

[0054] FIG9 is a schematic diagram of an instruction association matrix provided in an embodiment of the present application;

[0055] FIG10 is a schematic diagram of an array status list provided in an embodiment of the present application;

[0056] FIG11 is a schematic diagram of a flow chart of executing a tensor instruction according to an embodiment of the present application;

[0057] FIG12 is a schematic diagram of the execution status of different execution units in a storage-computing integrated module provided in an embodiment of the present application;

[0058] FIG13 is a schematic diagram of the structure of an operator compilation device provided in an embodiment of the present application;

[0059] FIG14 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0060] FIG15 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The embodiments of the present application provide an operator compilation method and apparatus for improving the parallelism of computing task execution in a storage-computing integrated system.

[0062] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0063] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0064] First, some terms involved in the embodiments of the present application are introduced to facilitate those skilled in the art to understand the technical solutions.

[0065] A storage-computing integrated architecture is a technology that integrates storage and computing functions. By implementing embedded computing functions within memory chips, this architecture reduces the size and frequency of data transfers between memory and processors, thereby improving system computing power and energy efficiency.

[0066] Near-memory computing refers to shortening the physical distance between memory and logic processing modules, thereby reducing data transmission delay and power consumption.

[0067] In-memory computing refers to performing calculations inside the memory, eliminating possible data transmission and reducing system latency and power consumption.

[0068] Out-of-order execution (OOE) is a technique used in high-performance microprocessors to utilize instruction cycles to avoid certain types of latency. OOO allows the processor to execute instructions in a specific order based on the availability of input data. This allows the processor to avoid waiting for the next program instruction and instead process the next instruction that can be executed immediately.

[0069] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the present application is introduced below with reference to the accompanying drawings.

[0070] Please refer to Figure 1, which is a schematic diagram of the system architecture of a storage-computing integrated system provided in an embodiment of the present application. In the example shown in Figure 1, the storage-computing integrated system 10 is a storage-computing integrated task issued by the central processing unit 20, such as big data analysis tasks, machine learning tasks, image processing tasks, multimedia encoding and decoding tasks, and database query and storage tasks. The storage-computing integrated system 10 is used to execute storage-computing integrated computing tasks. In the storage-computing integrated system, the embodiment of the present application does not distinguish between storage-computing integrated tasks and computing tasks.

[0071] As shown in Figure 1, the storage-computing integrated system 10 includes a compiler 101 and a storage-computing integrated chip 102, wherein the storage-computing integrated chip 102 includes a scheduler 1021, a memory access controller unit 1022, and an array module 1023. The compiler 101 and the storage-computing integrated chip 102 are respectively described in detail below.

[0072] The compiler 101 is used to receive the storage-computing-in-one task sent by the central processing unit 20, and compile the source code corresponding to the storage-computing-in-one task into instructions. The instruction format of the instruction is the instruction format that can be directly executed by the storage-computing-in-one chip 102. The one or more instructions compiled by the compiler 101 can constitute an instruction set. Since the storage-computing-in-one system 10 can use the instruction set to perform tensor calculations on the storage-computing-in-one chip 102, the instruction set can also be called a storage-computing-in-one instruction set or a tensor instruction set. The compiler 101 is also used to send the compiled tensor instruction set to the storage-computing-in-one chip 102, and the storage-computing-in-one chip 102 stores and executes the tensor instruction set.

[0073] The integrated memory and computing chip 102 is used to implement the processing of integrated memory and computing tasks within the chip. Specifically, the integrated memory and computing chip 102 is used to execute the tensor instruction set compiled by the compiler 101, and completes the integrated memory and computing tasks issued by the central processing unit 20 by executing the tensor instruction set. The integrated memory and computing chip 102 includes a scheduler 1021, a memory access controller unit 1022, and an array module 1023. The specific functions of these components are described below.

[0074] The scheduler 1021 is configured to receive the tensor instruction set compiled by the compiler 101 and store the tensor instruction set in an instruction storage list. The scheduler 1021 is also configured to determine the instruction state of the tensor instruction based on the instruction affinity matrix and the array state list. When the instruction state is executable, the scheduler 1021 schedules the memory access control unit 1022 to execute the tensor instruction.

[0075] The memory access control unit 1022 is used to execute tensor instructions. Specifically, the memory access control unit 1022 generates circuit control signals corresponding to the array module 1023 based on the tensor instructions, and uses these circuit control signals to control the array module 1023 to perform operations such as storage and calculation. The memory access control unit 1022 can control one or more array modules 1023 to perform operations such as storage and calculation.

[0076] Array module 1023 is used to perform integrated storage and computing tasks. Specifically, array module 1023 includes near-memory circuitry and a storage array. The near-memory circuitry is used to perform near-memory computing operations and store the resulting data in the storage array via cache. The storage array is used for data read and write operations. In addition, the storage array is used to perform in-memory computing operations and store the results of these operations in the storage array via cache.

[0077] Based on the storage-computing integrated system 10 shown in Figure 1 , the present application further provides an operator compilation method. The operator compilation method of the present application is described below in conjunction with an embodiment.

[0078] Please refer to Figure 2, which is a flow chart of an operator compilation method provided in an embodiment of the present application. In the example shown in Figure 2, the method includes the following steps:

[0079] S201. The storage and computing integrated system obtains the compilation operator.

[0080] The storage-computing system 10 receives the compilation operator sent by the central processing unit 20. The compilation operator is used to instruct the compiler 101 on the compilation rules of the source code. Specifically, the compiler 101 can perform lexical analysis, syntax analysis, and code optimization on the source code based on the compilation operator. The storage-computing system 10 can also receive the source code corresponding to the storage-computing task sent by the central processing unit 20. The compiler 101 can analyze and convert the source code based on the compilation operator to generate specific instructions that the storage-computing system 10 can execute.

[0081] Please refer to Figure 3, which is a flowchart of a tensor instruction execution provided by an embodiment of the present application. In step 1 of the example shown in Figure 3, the storage-computing system 10 receives the compilation operator sent by the central processing unit 20 and compiles the source code based on the compilation operator. For example, the compilation operator received by the storage-computing system 10 is a filtering operator, which can select elements that meet the conditions from the data set according to specific conditions and return them as a new subset.

[0082] S202. The storage-computing system compiles the source code based on the compilation operator to obtain tensor instructions, which are used to execute computing tasks of the storage-computing system.

[0083] After acquiring the source code, the integrated storage and computing system 10 first creates a compilation operator table in memory. This compilation operator table is used to store compilation information corresponding to one or more compilation operators. The compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length. The integrated storage and computing system 10 stores the compilation information in the source code in the compilation operator table according to the table structure of the compilation operator table.

[0084] In the compilation operator table described above, the row address is the address of the tensor following the conditional statement in a source code line, and the column address is the first address of the tensor involved in the source code line, also known as the first address to be queried. The operator number is the identifier corresponding to different compilation operators. The bit width refers to the number of address bits occupied by a source code line. The following example describes the compilation operator table provided in an embodiment of the present application.

[0085] Please refer to Figure 4, which is a schematic diagram of source code compilation based on a compilation operator table provided in an embodiment of the present application. In the example shown in Figure 4, Table (a) is a compilation operator table created by the storage and computing integrated system 10. The table entries of the compilation operator table include operator number, row address, column address, output address, data type, data bit width, and tensor length.

[0086] The row address is the address of the tensor following a conditional statement in the source code. For example, the source code "A(A==B)→C" indicates that A is equal to B. If A is equal to B, the comparison result is passed to C. The conditional statement is "determine whether A is equal to B." Therefore, the tensor address following the conditional statement is the address of tensor B. Therefore, the storage-computing system 10 stores the address of tensor B as the row address in the compiled operator table.

[0087] In the example shown in FIG4 , the column address is the first address of the tensor involved in a source code, also known as the first address to be queried. For another example, in the source code "A(A==B)→C" above, the first address of the tensor involved in the source code is the address of tensor A. Therefore, the storage-computing system 10 stores the address of tensor A as the column address in the compiled operator table.

[0088] In the example shown in Figure 4, operator numbers refer to the numbers corresponding to different compilation operators. For example, in the source code "A(A==B)→C", the source code is a comparison operation, and the corresponding compilation operator is a filter operator. The operator number corresponding to the filter operator is "tab1". Therefore, the storage and computing system 10 stores "tab1" in the compilation operator table.

[0089] In the example shown in FIG4 , bit width refers to the number of address bits occupied by a source code line. For example, in the source code “A(A==B)→C”, the address bits occupied are 100. Therefore, the storage and computing system 10 stores “100” in the compilation operator table.

[0090] It is understood that the compilation operator table created by the storage-computing system 10 can store one or more compilation operators involved in the source code, as well as the compilation information corresponding to the compilation operators. Therefore, the storage-computing system 10 can establish a mapping relationship between operator numbers and compilation information, thereby quickly querying the compilation operator table during the source code compilation process to determine the content of the instruction segment of the tensor instruction.

[0091] The following continues to introduce the process of compiling source code based on the compilation operator table after the storage and computing integrated system 10 creates the compilation operator table.

[0092] In the process of generating tensor instructions corresponding to the source code based on the compilation operator and the compilation operator table, the storage and computing system 10 first determines the various instruction segments of the tensor instruction based on the operator structure of the compilation operator, and determines the specific content of each instruction segment based on the compilation information corresponding to the operator structure. Specifically, based on the operator number of the compilation operator, the compilation information in the compilation operator table is queried to determine the compilation information to be filled in the corresponding instruction segment, thereby generating the content of each instruction segment of the tensor instruction and obtaining the tensor instruction. The following example introduces the compilation process of the source code.

[0093] Please continue to refer to Figure 4. In the example shown in Figure 4, the storage and computing integrated system 10 determines the various instruction segments of the tensor instruction based on the operator structure of the compilation operator, where different compilation operators have different operator structures. For example, the operator structure of the screening operator includes an opcode, a row address, a column address, an output address, an input data type, and a tensor length. The storage and computing integrated system 10 determines the various instruction segments of the tensor instruction based on the screening operator, and the instruction segment of the tensor instruction corresponding to the screening operator includes an opcode, a variable A address, a variable B address, a variable C address, a type, a bit width, and a tensor length. Among them, the variable A address corresponds to the row address of the operator structure, the variable B address corresponds to the address of the operator structure, the variable C address corresponds to the output address of the operator structure, and the type corresponds to the input data type of the operator structure.

[0094] In the example shown in Figure 4, in the process of generating tensor instructions based on the compilation operator, the storage and computing system 10 queries the corresponding row address, column address, output address, data type and tensor length in the compilation operator table based on the operator number of the screening operator, and uses the queried row address, column address, output address, data type and tensor length as the content of the corresponding instruction segment of the tensor instruction. For example, the operator number of the screening operator is "tab1", and the storage and computing system 10 queries the compilation operator table based on "tab1" and finds that the row address is "row1" and the column address is "0x000010". The storage and computing system 10 can obtain the address of variable A and the address of variable B in the instruction segment of the tensor instruction. The storage and computing system 10 can further compile the source code based on other operator structures of the screening operator to obtain a complete tensor instruction.

[0095] In the embodiment of the present application, the operator structure of the compilation operator includes one or more of the following: opcode, row address, column address, output address, input data type and tensor length. Among them, different compilation operator types correspond to different operator structures, and the types of compilation operators are, for example, filter operators, sort operators and aggregation operators. The operator structure of the filtering operator includes opcode, row address, column address, output address, input data type and tensor length. The operator structure of the sorting operator includes opcode, row address, column address, input data type and tensor length. The operator structure of the aggregation operator includes opcode, row address, column address, input data type and tensor length.

[0096] In the embodiment of the present application, there is a mapping relationship between the operator structure of the compiled operator and the instruction segment of the tensor instruction. For example, the instruction segment of the tensor instruction includes one or more items: opcode, variable A address, variable B address, variable C address, type, bit width and tensor length. The opcode, row address, column address, output address, input data type and tensor length in the operator structure correspond to the opcode, variable A address, variable B address, variable C address, type and tensor length in the instruction segment respectively.

[0097] Please refer to Figure 5, which is a schematic diagram of an embodiment of the present application providing an example of determining the instruction segment of a tensor instruction based on the operator structure of a compiled operator. In the example shown in Figure 5, the storage-computing system 10 generates corresponding tensor instructions based on the operator structures of the filtering operator, the sorting operator, and the aggregation operator.

[0098] In the example shown in FIG5 , taking the filter operator as an example, the filter operator's operator structure includes an opcode, a row address, a column address, an output address, an input data type, and a tensor length. Based on the filter operator's operator structure, the storage-computing system 10 determines the tensor address by querying the compiled operator table, and further compiles the instruction segment of the tensor instruction to obtain the opcode, variable A address, variable B address, variable C address, type, bit width, and tensor length.

[0099] For example, the opcode of a tensor instruction can be determined as a comparison operation based on the source code "A(A==B)→C", with the corresponding opcode being "0010110". The address of variable A can be determined as "0x1111" by querying the column address of the compiled operator table. The address of variable B can be determined as "0x1001" by querying the row address of the compiled operator table. The address, type, bit width, and tensor length of variable C can be determined as "0x3442", "0", "01", and "000000101" respectively based on the source code and the operator structure of the compiled operator.

[0100] In the embodiment of the present application, a compilation operator may correspond to one or more different instruction opcodes. For example, in the example shown in FIG4 , the screening operator includes opcodes such as compare (cmp), greater than or equal (gte), greater than (gt), less than or equal (gt), and less than (lt), the sorting operator includes opcodes such as ascending and descending, and the aggregation operator includes opcodes such as sum, avg, and count.

[0101] In the embodiments of the present application, different types of tensor instructions have different instruction segments. The instruction types of tensor instructions include one or more of the following: address-only instructions, value-only instructions, array control instructions, and near-memory circuit control instructions. The following details the instruction segments corresponding to these instruction types.

[0102] Among them, an address-only instruction is an instruction type that only involves tensor addresses. For example, an address-only instruction reads A and B from the addresses of tensors A and B, respectively, performs the specified operation, and then writes the result C to the address of tensor C. The instruction segment of an address-only instruction includes one or more of the following: opcode, tensor address, type, bit width, and tensor length. A tensor address can be the addresses of multiple tensors, and the tensor length refers to the number of elements contained in the tensor.

[0103] A value-participating instruction is a tensor instruction that contains a value. For example, a value-participating instruction reads A from address Tensor A, performs a specified operation on it with value C, and then writes the result B to address Tensor B. The instruction segment of a value-participating instruction includes one or more of the following: opcode, tensor address, value, type, bit width, and tensor length.

[0104] Array control instructions are instructions for performing array operations. For example, an array control instruction reads A from tensor A's address and, in combination with value C, performs a control operation on the array corresponding to tensor A's address. The instruction segment of an array control instruction includes one or more of the following: an opcode, a tensor address, a value, and a tensor length.

[0105] A local memory circuit control instruction performs a local memory circuit operation on a specific array ID. The local memory circuit type and local memory circuit status of the instruction determine the local memory circuit type and local memory circuit status. The instruction segment of a local memory circuit control instruction includes one or more of the following: an opcode, a memory array ID, a local memory circuit type, and a local memory circuit status.

[0106] Please refer to Figure 6, which is a schematic diagram of the instruction segment of a tensor instruction of different instruction types provided in an embodiment of the present application. In the example shown in Figure 6, the instruction segment of the instruction in which only the address participates includes the opcode, tensor A address, tensor B address, tensor C address, type, bit width and tensor length. The instruction segment of the instruction in which the value participates includes the opcode, tensor A address, tensor B address, value C, type, bit width and tensor length. The instruction segment of the array control instruction includes the opcode, tensor A address, value C and tensor length. The instruction segment of the near memory circuit control instruction includes the opcode, storage array ID, near memory circuit type and near memory circuit status.

[0107] As can be seen from the example shown in Figure 6, when the type corresponding to the instruction does not require part of the segmentation in the instruction segment, the address or value corresponding to that part of the instruction segment can be set to 0 to indicate skipping. In the above instruction segment, in addition to the tensor address and value, there are also instruction segments for type, bit width, and tensor length. Among them, types include integers and floating-point numbers, integers include numbers from 1 to 16 bits, and floating-point numbers include BF16, FP16, FP32, and FP64, etc. The tensor length is used to identify the extended address length of the tensor address. When the tensor length is 1, the tensor data represented is equal to the scalar data.

[0108] It should be noted that when the storage-computing system 10 compiles source code based on the compilation operator, one or more tensor instructions can be compiled from a broken source code based on different compilation operators.

[0109] Please refer to Figure 7, which is a schematic diagram of a tensor instruction obtained by compiling based on a compilation operator according to an embodiment of the present application. In the example shown in Figure 7, the storage-computing integrated system 10 compiles one or more tensor instructions based on a single compilation operator. For example, the storage-computing integrated system 10 compiles "select*from t where ta>10" based on the screening operator to obtain one tensor instruction, namely tensor instruction 0. For another example, the storage-computing integrated system 10 compiles "select a+b*c from t" based on the projection operator to obtain two tensor instructions, namely tensor instruction 1 and tensor instruction 2.

[0110] In the example shown in Figure 7, a single source code command can be compiled into a combination of a single or multiple tensor storage instructions. For example, the three operators in the table correspond to different source code commands. The filter operator is used to select numbers greater than 10 in order, the projection operator is used to calculate |a+b*c|, and the ascending sort operator is used to sort in ascending order. The filter operator corresponds to command 1, the projection operator corresponds to command 1 and command 2, and the ascending sort operator corresponds to command 3. Among them, command 0 and command 2 can be implemented using a single instruction. Command 1 and command 3 both require multiple instructions to implement.

[0111] S203. The storage and computing integrated system stores the tensor instruction into the instruction storage list.

[0112] After compiling the tensor, the storage and computing system 10 stores the tensor instruction in an instruction storage list. The instruction storage list includes one or more of the following: the tensor instruction's instruction segment, the tensor instruction number, the compiled operator number, and the array block identifier. The instruction storage list is described in detail below.

[0113] Please refer to Figure 8, which is a schematic diagram of an instruction storage list provided in an embodiment of the present application. In the example shown in Figure 8, the storage and computing system 10 generates tensor instructions in the order in which they are stored in the instruction storage list. The instructions stored first in the instruction storage list will be executed first. In addition to storing the instruction segment of the tensor instruction, the instruction storage list also stores the tensor instruction number, compiled operator number, read and write address, array block identifier, and read and write coordinate position of the tensor instruction.

[0114] For example, the instruction storage column stores the instruction segment of a tensor instruction with instruction label "0" and compilation operator number "0". The read and write addresses of the tensor instruction are "[0x0011, 0x0104]" and "[0x0111]" respectively. The array block identifier of the storage array on which the tensor instruction operates is "0", and the read and write coordinate positions of the storage array on which the tensor instruction operates are "[(1,4,1,8),(2,3,4,7)]" and "[(0,-1,0,-1)]" respectively.

[0115] Please continue to refer to FIG3 . In step 2 of the example shown in FIG3 , after the storage-computing integrated system 10 compiles the source code based on the compilation operator to generate tensor instructions, the tensor instructions are stored in the instruction storage list.

[0116] S204. The storage and computing integrated system executes tensor instructions in the instruction storage list based on the instruction association matrix and array status list.

[0117] The storage and computing system 10 executes the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list. Specifically, the storage and computing system 10 determines the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list. The instruction status includes an executable state and a waiting state. Among them, the instruction association matrix is ​​used to store the dependencies between tensor instructions, and the array status list is used to indicate the state of the array module, which is used to execute the tensor instructions.

[0118] Please continue to refer to Figure 3. In steps 2 to 7 of the example shown in Figure 3, the storage and computing integrated system 10 decodes the tensor instruction after obtaining the compiled tensor instruction, and stores the decoded tensor instruction in the instruction storage list. The storage and computing integrated system 10 determines whether the tensor instruction is executable based on the instruction association matrix and the array status list. If the tensor instruction is executable, the tensor instruction is sent to the memory access control unit. If the tensor instruction is not executable, it continues to wait for execution in the instruction storage list.

[0119] The following is a detailed introduction to the instruction association matrix and array status list.

[0120] In the embodiments of the present application, the instruction dependency matrix is ​​used to store the relationships between instructions in the instruction storage list. The instruction dependency matrix is ​​a matrix with the same length and width as the number of tensor instructions in the instruction storage list, and the matrix elements represent the instruction dependencies. The dependencies between tensor instructions include one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.

[0121] Among them, read-after-write dependency refers to the situation in which the read operation of one instruction occurs after the write operation of another instruction in concurrent computing, that is, one instruction can only read a certain shared data after the other instruction completes writing the shared data. Write-after-write dependency refers to the situation in which the write operation of one instruction occurs after the write operation of another instruction in concurrent computing, that is, one instruction can only modify a certain shared data after the other instruction completes writing the shared data. Write-after-read dependency refers to the situation in which the write operation of one instruction occurs after the read operation of another instruction in concurrent computing, that is, one instruction can only modify a certain shared data after the other instruction completes reading the shared data.

[0122] Please refer to Figure 9, which is a schematic diagram of an instruction association matrix provided in an embodiment of the present application. In the example shown in Figure 9, the value of (i, j) in the matrix element in the instruction association matrix represents the conflict relationship between the i-th instruction and the j-th instruction, where 0 indicates that there is no address conflict between the two, 1 indicates a write after read (WAR) dependency, 2 indicates a write after write (WAW) dependency, and 3 indicates a read after write (RAW) dependency. For example, instruction 1 depends on instruction 0 after read, instruction 2 depends on instruction 1 after write, instruction 3 depends on instruction 1 after write, and instruction 3 depends on instruction 2 after write.

[0123] In the embodiment of the present application, the array status list is used to query the status of the storage array and the status of the near memory circuit, thereby determining whether a tensor instruction can be sent to the corresponding storage array or near memory circuit for execution. The array status list includes one or more of the following: the idle state of the array module and the near memory circuit status.

[0124] Please refer to Figure 10, which is a schematic diagram of an array status list provided in an embodiment of the present application. In the array status list shown in Figure 10, the array status list displays the status of four storage arrays numbered "CB1," "CB3," "CB4," and "CB5," respectively. An array status of "1" indicates that the array is busy, and an array status of "0" indicates that the array is idle. The array status list can also record the status of the near memory circuit. For example, the array status list can also record the status of the adder in the near memory circuit or the status of the XOR operator in the near memory circuit.

[0125] The storage-computing integrated system 10 determines the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list. Specifically, when the tensor instruction to be executed satisfies the dependency relationship of the instruction association matrix and the storage array or near storage circuit on which the tensor instruction operates is in an idle state, the instruction status of the tensor instruction is an executable state. In the embodiment of the present application, the storage-computing integrated system 10 sends tensor instructions to the memory access control unit based on the instruction status of the tensor instructions, which is also called out-of-order execution.

[0126] When the instruction state of the tensor instruction is in the executable state, the tensor instruction is sent to the memory access control unit. The storage and computing integrated system 10 determines the execution operation according to the tensor instruction, and the execution operation includes one or more of the following: near memory computing operation, in-memory computing operation and conventional read and write operation. Among them, the near memory computing operation refers to the operation of reading the required data through the array or cache, and then calling the corresponding near memory circuit to complete the calculation. The in-memory computing operation refers to the operation of directly using the storage array to perform calculations. Conventional read and write operations refer to reading and writing operations on the storage array. The computational parallelism of the tensor instructions in the above different storage and computing types is different.

[0127] The following introduces several operations of the storage and computing system 10 to execute tensor instructions with examples.

[0128] Please refer to Figure 11, which is a schematic diagram of a memory access control unit performing a memory-computation-integrated operation according to an embodiment of the present application. In the example shown in Figure 11, after receiving a tensor instruction, the memory access control unit 1022 generates a circuit control signal corresponding to the memory-computation-integrated array model group 1023, and the memory-computation-integrated array model group 1023 determines the type of memory-computation operation based on the control circuit signal.

[0129] In the example shown in Figure 11, when the type of storage-computation operation is a near-memory calculation operation, the array module 1023 reads the required data from the array or cache, and then calls the corresponding near-memory circuit to complete the calculation operation. During the near-memory calculation process, the array module 1023 in the storage-computation integrated chip 102 reads a single row of the array and then transmits it to the near-memory circuit via the cache.

[0130] In the example shown in Figure 11, when the type of storage operation is an in-memory operation, the array module 1023 directly uses the storage array to perform the operation, for example, performing multiplication and addition operations on multiple rows of the storage array, and writing the calculation results to the storage array via the cache. When the type of storage operation is a regular read / write operation, the array module 1023 performs read and write operations on the data in each data unit of the storage array and transfers the data via the cache.

[0131] In the implementation of the present application, the first tensor instruction and the second tensor instruction are respectively and simultaneously executed in different execution units of the array module 1023, the different execution units including the array and the near memory circuit, and the first tensor instruction and the second tensor instruction are different tensor instructions. The following is an example to introduce.

[0132] Please refer to Figure 12, which is a schematic diagram of the execution of different tensor instructions in a storage-computing integrated array model provided by an embodiment of the present application. In the example shown in Figure 12, Figure (a) is a schematic diagram of 4 different tensor instructions, and Figure (b) is a schematic diagram of the execution status of different tensor instructions at different times. Among the 4 tensor instructions shown in Figure (a), instruction 0, instruction 1 and instruction 2 are all distributed in one array module and cannot be read and written in parallel. Instruction 3 is distributed in another array module, and its processing process and instruction 0, instruction 1 and instruction 2 can be parallel. At the same time, there is a read-after-write conflict between instruction 1 and instruction 0, so instruction 1 needs to wait for instruction 0 to be written before it can be read.

[0133] As shown in the example in Figure (b), during instruction execution, instructions 0 and 1 are first scheduled in parallel. Array module 1023 identifies a read-after-write (RAW) dependency between the two, so instruction 1 begins waiting for the completion of instruction 0. During this time, instructions 2 and 3 are also scheduled. Since instruction 3 is on another array module and does not conflict with other instructions, it can execute unrestricted.

[0134] As can be seen from the example shown in Figure (b), the array and near memory circuits of array module 1023 can respectively execute instruction 0 and instruction 2 simultaneously. For example, when the near memory circuit of array module 1023 is performing calculations, the storage array is in an idle state. Therefore, while instruction 0 is performing near memory calculations, instruction 2 can simultaneously perform read operations.

[0135] Furthermore, as shown in the example in Figure (b), since instruction 2 is a max instruction, the read operation is divided into two steps: identifying the maximum value and reading the maximum value. After completing the first read step, instruction 0 waits for instruction 2, occupying the array to complete the write operation, and then instruction 2 continues the read operation. Therefore, instruction 1 can complete the write operation on the idle array only after instruction 2 completes the write operation.

[0136] It can be seen from the above embodiments that the storage-computing integrated system in the embodiments of the present application can generate a tensor instruction set based on the compilation operator, and can process tensor objects through the tensor instruction set, thereby realizing storage array-level operations on the storage-computing integrated system, improving the processing parallelism of the storage-computing integrated system in processing computing tasks, and thereby improving the computing performance of the storage-computing integrated system.

[0137] Based on the above method embodiment, the embodiment of the present application further provides an operator compilation device. The operator compilation device provided by the embodiment of the present application is described in detail below.

[0138] Please refer to Figure 13, which is a schematic diagram of the structure of an operator compilation device provided in an embodiment of the present application. In the example shown in Figure 13, the operator compilation device 1300 is used to implement the various steps performed by the storage-computation integrated system in each of the above embodiments. The operator compilation device 1300 includes an acquisition unit 1301, a compilation unit 1302, and a processing unit 1303.

[0139] The acquisition unit 1301 is used to obtain a compilation operator, which is used to indicate the compilation rules for the source code. The compilation unit 1302 is used to compile the source code based on the compilation operator to obtain one or more tensor instructions. The tensor instructions are used to perform computing tasks in the storage and computing integrated system. The tensor instructions include one or more of the following instruction segments: opcode, tensor address, data type, data bit width, and tensor length. The processing unit 1303 is used to store the tensor instructions in an instruction storage list, which is used to store compiled tensor instructions.

[0140] In some possible implementations, processing unit 1303 is further configured to create a compilation operator table, which is configured to store compilation information corresponding to one or more compilation operators. The compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length. Compilation unit 1302 is specifically configured to generate one or more tensor instructions based on the compilation information stored in the compilation operator table.

[0141] In some possible implementations, the compilation unit 1302 is specifically used to generate an opcode for a tensor instruction based on the operator number in the compilation operator table, generate a tensor address for the tensor instruction based on the row address, column address, and output address in the compilation operator table, and generate a data type, data bit width, and tensor length for the tensor instruction based on the data type, data bit width, and tensor length in the compilation operator table.

[0142] In some possible implementations, the instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.

[0143] In some possible implementations, the compilation unit 1302 is also used to determine the instruction status of the tensor instructions in the instruction storage list based on the instruction association matrix and the array status list. The instruction association matrix is ​​used to store the dependencies between tensor instructions, and the array status list is used to indicate the status of the array module. The array module is used to execute tensor instructions, and the instruction status includes an executable state and a waiting state.

[0144] In some possible implementations, the processing unit 1303 is further configured to send the tensor instruction to the memory access control unit when the instruction state of the tensor instruction is in an executable state. An execution operation is determined based on the tensor instruction, where the execution operation includes one or more of the following: a near-memory computation operation, an in-memory computation operation, and a regular read / write operation.

[0145] In some possible implementations, the tensor instruction set includes a first tensor instruction and a second tensor instruction, and the first tensor instruction and the second tensor instruction are executed in the same array module. The processing unit 1303 is also used to simultaneously execute the first tensor instruction and the second tensor instruction in different execution units of the array module, and the execution unit includes an array and a near memory circuit.

[0146] In some possible implementations, the dependency relationship between tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.

[0147] In some possible implementations, the array status list includes one or more of the following: an array module idle state and a near memory circuit state.

[0148] In some possible implementations, the instruction storage list includes one or more of the following: an instruction segment of a tensor instruction, a tensor instruction number, a compilation operator number, and an array block identifier.

[0149] It should be understood that the division of units in the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, the units in the device can all be implemented in the form of software called through processing elements; or they can all be implemented in the form of hardware; or some units can be implemented in the form of software called through processing elements, and some units can be implemented in the form of hardware. For example, each unit can be a separately established processing element, or it can be integrated into a certain chip of the device. In addition, it can also be stored in the memory in the form of a program, called by a certain processing element of the device and perform the function of the unit. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element described here can also be a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by the integrated logic circuit of the hardware in the processor element or in the form of software called through the processing element.

[0150] It is worth noting that, for the sake of simplicity of description, the above method embodiments are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited to the order of the actions described. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required for this application.

[0151] Other reasonable step combinations that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0152] Please refer to Figure 14, which is a schematic diagram of the structure of a computing device provided in an embodiment of the present application. As shown in Figure 14, the computing device 1400 includes: a processor 1401, a memory 1402, a communication interface 1403, and a bus 1404. The processor 1401, the memory 1402, and the communication interface 1403 are coupled via a bus (not labeled in the figure). The memory 1402 stores instructions. When the execution instructions in the memory 1402 are executed, the computing device 1400 executes the method performed by the storage and computing system in the above method embodiment.

[0153] The computing device 1400 may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), one or more digital signal processors (DSPs), one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms. For example, when a unit in the apparatus can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For example, these units may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0154] The processor 1401 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0155] Memory 1402 may be volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0156] Memory 1402 stores executable program code, and processor 1401 executes the executable program code to respectively implement the functions of the aforementioned acquisition unit 1301, processing unit 1302, and processing unit 1303, thereby implementing the aforementioned routing table compression method. In other words, memory 1402 stores instructions for executing the aforementioned routing table compression method.

[0157] The communication interface 1403 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1400 and other devices or a communication network.

[0158] In addition to the data bus, bus 1404 may also include a power bus, a control bus, and a status signal bus. The bus may be a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a unified bus (Ubus or UB), a Compute Express Link (CXL), or a Cache Coherent Interconnect for Accelerators (CCIX). Buses can be categorized as address buses, data buses, and control buses.

[0159] Please refer to Figure 15 , which is a schematic diagram of a computing device cluster provided in an embodiment of the present application. As shown in Figure 15 , the computing device cluster 1500 includes at least one computing device 1400 .

[0160] As shown in Figure 15, the computing device cluster 1500 includes at least one computing device 1400. The memory 1402 in one or more computing devices 1400 in the computing device cluster 1500 may store the same instructions for executing the above-mentioned routing table compression method.

[0161] In some possible implementations, the memory 1402 of one or more computing devices 1400 in the computing device cluster 1500 may also store partial instructions for executing the above-mentioned routing table compression method. In other words, the combination of one or more computing devices 1400 can jointly execute the instructions for executing the above-mentioned routing table compression method.

[0162] It should be noted that the memory 1402 in different computing devices 1400 in the computing device cluster 1500 can store different instructions, each for executing a portion of the functions of the operator compilation apparatus described above. In other words, the instructions stored in the memory 1402 in different computing devices 1400 can implement the functions of one or more modules in the acquisition unit 1301, the compilation unit 1302, and the processing unit 1303.

[0163] In some possible implementations, one or more computing devices 1400 in the computing device cluster 1500 may be connected via a network, which may be a wide area network or a local area network.

[0164] In another embodiment of the present application, a computer-readable storage medium is provided, in which computer execution instructions are stored. When the processor of the device executes the computer execution instructions, the device executes the method executed by the storage and computing integrated system in the above method embodiment.

[0165] In another embodiment of the present application, a computer program product is provided, comprising computer-executable instructions stored in a computer-readable storage medium. When a processor of a device executes the computer-executable instructions, the device performs the method performed by the storage-computing integrated system in the above-described method embodiment.

[0166] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0167] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0168] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0169] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0170] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

Claims

1. An operator compilation method, characterized in that: Applied to a storage-computing integrated system, the method includes: Obtaining a compilation operator, where the compilation operator is used to indicate a compilation rule of a source code; Compiling the source code based on the compilation operator to obtain one or more tensor instructions, where the tensor instructions are used to execute computing tasks in the storage-computing-in-one system, and the tensor instructions include one or more of the following instruction segments: an operation code, a tensor address, a data type, a data bit width, and a tensor length; The tensor instruction is stored in an instruction storage list, where the instruction storage list is used to store compiled tensor instructions.

2. The method according to claim 1, characterized in that Before compiling the source code based on the compilation operator, the method further includes: Create a compilation operator table, where the compilation operator table is used to store compilation information corresponding to one or more of the compilation operators, where the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length; The compiling the source code based on the compiling operator comprises: One or more tensor instructions are generated based on the compilation information stored in the compilation operator table.

3. The method according to claim 2, characterized in that The generating one or more tensor instructions based on the compilation information stored in the operator compilation table includes: Generate an operation code for the tensor instruction according to the operator number in the compiled operator table; Generate a tensor address of the tensor instruction according to the row address, column address and output address in the compiled operator table; The data type, data bit width and tensor length of the tensor instruction are generated respectively according to the data type, data bit width and tensor length in the compiled operator table.

4. The method according to any one of claims 1 to 3, characterized in that The instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The instruction status of the tensor instructions in the instruction storage list is determined according to the instruction association matrix and the array status list, the instruction association matrix is ​​used to store the dependencies between the tensor instructions, the array status list is used to indicate the status of the array module, the array module is used to execute the tensor instructions, and the instruction status includes an executable state and a waiting state.

6. The method according to claim 5, characterized in that The method further comprises: When the instruction state of the tensor instruction is in the executable state, sending the tensor instruction to a memory access control unit; An execution operation is determined according to the tensor instruction, where the execution operation includes one or more of the following: a near-memory computing operation, an in-memory computing operation, and a regular read-write operation.

7. The method according to any one of claims 1 to 6, characterized in that The tensor instruction set includes a first tensor instruction and a second tensor instruction, wherein the first tensor instruction and the second tensor instruction are executed in the same array module, and the method further includes: The first tensor instruction and the second tensor instruction are respectively and simultaneously executed in different execution units of the array module, and the execution unit includes an array and a near memory circuit.

8. The method according to any one of claims 5 to 7, characterized in that The dependency relationship between the tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.

9. The method according to any one of claims 5 to 8, characterized in that The array status list includes one or more of the following: array module idle status and near memory circuit status.

10. The method according to any one of claims 5 to 9, characterized in that The instruction storage list includes one or more of the following: an instruction segment of the tensor instruction, a tensor instruction number, a compilation operator number, and an array block identifier.

11. An operator compilation device, characterized in that: The device comprises: An acquisition unit, used for acquiring a compilation operator, where the compilation operator is used for indicating a compilation rule for a source code; A compiling unit, configured to compile the source code based on the compiling operator to obtain one or more tensor instructions, wherein the tensor instructions are used to execute computing tasks in the storage-computing-in-one system, and the tensor instructions include one or more of the following instruction segments: an operation code, a tensor address, a data type, a data bit width, and a tensor length; A processing unit, configured to store the tensor instruction into an instruction storage list, wherein the instruction storage list is used to store the compiled tensor instruction.

12. The device according to claim 11, characterized in that The processing unit is also used for: Create a compilation operator table, where the compilation operator table is used to store compilation information corresponding to one or more of the compilation operators, where the compilation information includes one or more of the following: operator number, row address, column address, output address, data type, data bit width, and tensor length; The compilation unit is specifically used for: One or more tensor instructions are generated based on the compilation information stored in the compilation operator table.

13. The device according to claim 12, characterized in that The compilation unit is specifically used for: Generate an operation code for the tensor instruction according to the operator number in the compiled operator table; Generate a tensor address of the tensor instruction according to the row address, column address and output address in the compiled operator table; The data type, data bit width and tensor length of the tensor instruction are generated respectively according to the data type, data bit width and tensor length in the compiled operator table.

14. The device according to any one of claims 11 to 13, characterized in that The instruction type includes one or more of the following: an address-only instruction, a value-only instruction, an array control instruction, and a near memory circuit control instruction.

15. The device according to any one of claims 11 to 14, characterized in that The compilation unit is also used to: The instruction status of the tensor instructions in the instruction storage list is determined according to the instruction association matrix and the array status list, the instruction association matrix is ​​used to store the dependencies between the tensor instructions, the array status list is used to indicate the status of the array module, the array module is used to execute the tensor instructions, and the instruction status includes an executable state and a waiting state.

16. The device according to claim 15, characterized in that The processing unit is also used for: When the instruction state of the tensor instruction is in the executable state, sending the tensor instruction to a memory access control unit; An execution operation is determined according to the tensor instruction, where the execution operation includes one or more of the following: a near-memory computing operation, an in-memory computing operation, and a regular read-write operation.

17. The device according to any one of claims 11 to 16, characterized in that The tensor instruction set includes a first tensor instruction and a second tensor instruction, wherein the first tensor instruction and the second tensor instruction are executed in the same array module, and the processing unit is further configured to: The first tensor instruction and the second tensor instruction are respectively and simultaneously executed in different execution units of the array module, and the execution unit includes an array and a near memory circuit.

18. The device according to any one of claims 15 to 17, characterized in that The dependency relationship between the tensor instructions includes one or more of the following: read-after-write dependency, write-after-write dependency, and write-after-read dependency.

19. The device according to any one of claims 15 to 18, characterized in that The array status list includes one or more of the following: array module idle status and near memory circuit status.

20. The device according to any one of claims 15 to 19, characterized in that The instruction storage list includes one or more of the following: an instruction segment of the tensor instruction, a tensor instruction number, a compilation operator number, and an array block identifier.

21. A computing device, characterized in that The electronic device comprises a processor coupled to a memory, wherein the processor is used to store instructions. When the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 10.

22. A computing device cluster, characterized in that: The system comprises at least one computing device, wherein the computing device comprises a processor, wherein the processor is coupled to a memory, and the processor is used to store instructions. When the instructions are executed by the processor, the computing device cluster executes the method according to any one of claims 1 to 10.

23. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed, the computer is caused to perform the method according to any one of claims 1 to 10.

24. A computer program product, comprising instructions, characterized in that: When the instructions are executed, the computer implements the method according to any one of claims 1 to 10.

Citation Information

Patent Citations

  • Operator compiling method and device

    CN119065716A

  • Instruction processing method and device and related product

    CN111949317A

  • Compiling method and device, computing equipment and medium

    CN112328227A

  • Neural network compiling method for storage and calculation integrated platform

    CN112465108A

  • Systems and methods for generation of sparse code for convolutional neural networks

    US20210182676A1

Cited By

  • In-storage calculation module, in-storage calculation method, equipment and storage medium

    CN120540714A