Method for compiling operator, device for compiling operator, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention discloses an operator compiling method, an operator compiling device, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining an intermediate representation of a to-be-compiled operator, wherein the intermediate representation of the to-be-compiled operator at least comprises an intermediate representation of a shared memory and an intermediate representation of an interactive operator; based on the size of the shared memory and the intermediate representation of the interactive operator, configuring operator rearrangement granularity in the intermediate representation of the shared memory to generate the configured intermediate representation of the shared memory; and based on the configured intermediate representation of the shared memory, updating the intermediate representation of the operator to be compiled, and compiling the intermediate representation of the operator to be compiled to generate an assembly code corresponding to the operator to be compiled. According to the embodiment of the invention, the operator development efficiency is effectively improved, the error probability is reduced, and the performance of the operator is optimized, so that the operator is more adaptive to an artificial intelligence chip.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more particularly to a method for compiling operators, an apparatus for compiling operators, an electronic device, and a computer-readable storage medium. Background Art

[0002] Artificial intelligence chips are developing rapidly and have been widely used in various fields, such as intelligent driving, medical diagnosis, smart home, etc. The main feature of artificial intelligence chips is that they can efficiently process neural network models to support complex computing tasks. Specifically, an artificial intelligence chip can implement the inference process / training process of a neural network model by executing a series of operators. Operators may involve basic operations in a neural network, such as convolution, pooling, matrix multiplication, and activation functions. Artificial intelligence chips are designed to efficiently process these operators, and through parallel computing and hardware acceleration technologies, significantly improve the training and inference efficiency of neural networks.

[0003] Currently, it has been considered to use an assembly language operator generation program to develop and optimize operators, so as to convert the logic and functions of operators into an assembly instruction set, enabling the execution of operators to adapt to the hardware architecture of artificial intelligence chips. For example, enabling each operator to adapt to the parallel processing framework of a general-purpose parallel graphics processing unit (GPGPU). However, the traditional method of using an assembly language operator generation program to develop and optimize operators still has technical problems such as reduced parallelism of data access, high development difficulty, and difficulty in debugging and verification.

[0004] Therefore, it is necessary to improve the current operator generation and optimization scheme to improve the operator development efficiency, reduce the error probability, and at the same time optimize the performance of operators to make them more suitable for artificial intelligence chips. Summary of the Invention

[0005] Embodiments of the present disclosure provide a method for compiling operators, an apparatus for compiling operators, an electronic device, and a computer-readable storage medium.

[0006] An embodiment of the present disclosure provides a method for compiling an operator. The method includes: obtaining an intermediate representation of the operator to be compiled, where the intermediate representation of the operator to be compiled at least includes an intermediate representation of shared memory and an intermediate representation of an interaction operator. Among them, the intermediate representation of the shared memory at least includes an attribute value indicating the size of the shared memory, and the interaction operator is associated with data interaction between video memory and the shared memory; based on the size of the shared memory and the intermediate representation of the interaction operator, configure an operator rearrangement granularity in the intermediate representation of the shared memory to generate the configured intermediate representation of the shared memory; based on the configured intermediate representation of the shared memory, update the intermediate representation of the operator to be compiled, and compile the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled.

[0007] For example, the configuring an operator rearrangement granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interaction operator includes: determining an execution granularity of the interaction operator based on the intermediate representation of the interaction operator, and configuring an attribute value of the operator rearrangement granularity or the size of the shared memory processed once in the intermediate representation of the shared memory based on the size of the shared memory and the execution granularity of the interaction operator.

[0008] For example, the method for compiling an operator further includes: configuring an attribute value of the operator rearrangement granularity in the intermediate representation of the interaction operator to generate the configured intermediate representation of the interaction operator; or configuring the size of the data processed once in the intermediate representation of the interaction operator based on the configured intermediate representation of the shared memory to generate the configured intermediate representation of the interaction operator.

[0009] For example, the updating the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory includes: updating the intermediate representation of the operator to be compiled based on the configured intermediate representation of the interaction operator and the configured intermediate representation of the shared memory.

[0010] For example, the intermediate representation of the operator to be compiled includes the intermediate representation of the operator for splitting shared memory or the intermediate representation of the operator for reusing shared memory. Compiling the intermediate representation of the operator to be compiled includes: creating a dedicated intermediate representation of the shared memory according to the intermediate representation of the shared memory, where the dedicated intermediate representation of the shared memory is specific to the intermediate representation of the operator for splitting shared memory or the intermediate representation of the operator for reusing shared memory; in response to determining that the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory has been configured, not changing the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory; and in response to determining that the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory has not been configured, configuring the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory as the attribute value of the operator rearrangement granularity in the intermediate representation of the shared memory.

[0011] For example, the intermediate representation of the operator to be compiled includes the intermediate representation of the operator for using shared memory. Compiling the intermediate representation of the operator to be compiled includes: configuring the attribute value of the operator rearrangement granularity of the shared memory for the intermediate representation of the operator for using shared memory, so that the corresponding assembly instruction block of the intermediate representation of the operator for using shared memory includes the attribute value of the operator rearrangement granularity of the shared memory.

[0012] For example, obtaining the intermediate representation of the operator to be compiled includes: traversing and analyzing the intermediate representation of the operator to be compiled through an analysis module to obtain the intermediate representation of the interactive operator.

[0013] For example, an embodiment of the present disclosure provides a device for compiling an operator. The device includes: a first module configured to: obtain the intermediate representation of the operator to be compiled, where the intermediate representation of the operator to be compiled at least includes the intermediate representation of the shared memory and the intermediate representation of the interactive operator, and where the intermediate representation of the shared memory at least includes an attribute value indicating the size of the shared memory, and the interactive operator is associated with data interaction between the video memory and the shared memory; a second module configured to: configure the operator rearrangement granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interactive operator to generate the configured intermediate representation of the shared memory; a third module configured to: update the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory; and a fourth module configured to: compile the intermediate representation of the operator to be compiled to generate the assembly code corresponding to the operator to be compiled.

[0014] An embodiment of the present disclosure provides an electronic device, which includes: a processor; and a memory. Wherein, a computer-executable program is stored in the memory, and when the computer-executable program is executed by the processor, the method for compiling an operator described above is executed.

[0015] An embodiment of the present disclosure provides a computer-readable storage medium, on which a computer instruction is stored, and when the computer instruction is executed by a processor, the method for compiling an operator described above is implemented.

[0016] According to another aspect of the present disclosure, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above aspects or various alternative implementation manners of the above aspects.

[0017] Currently, the traditional method of using an assembly language operator to generate a program to develop and optimize an operator still has technical problems such as reduced parallelism of data access, high development difficulty, and difficulty in debugging and verification.

[0018] For this reason, the embodiment of the present disclosure realizes an automatic optimization of the segmentation and reuse strategy of the shared memory by automatically configuring the operator rearrangement granularity in the intermediate representation of the shared memory, so as to improve the operator development efficiency and performance, reduce the burden on developers, reduce configuration errors, and improve the adaptability of the artificial intelligence chip. Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for the description of the embodiments will be briefly introduced below. The drawings in the following description are only exemplary embodiments of the present disclosure.

[0020] Figure 1 A flowchart of the method for compiling an operator according to an embodiment of the present disclosure is shown.

[0021] Figure 2 An example of a plurality of operators to be compiled that make up a neural network model according to an embodiment of the present disclosure is shown.

[0022] Figure 3 A schematic diagram of the intermediate representation of an operator to be compiled according to an embodiment of the present disclosure is shown.

[0023] Figure 4 A schematic diagram of the pseudo-code of the intermediate representation of the shared memory according to an embodiment of the present disclosure is shown.

[0024] Figure 5A schematic diagram showing a method for compiling operators according to an embodiment of the present disclosure.

[0025] Figure 6 A schematic diagram showing a device for compiling operators according to an embodiment of the present disclosure.

[0026] Figure 7 An architecture diagram showing a computing device according to an embodiment of the present disclosure.

[0027] Figure 8 A schematic diagram showing a computer-readable storage medium according to an embodiment of the present disclosure. Detailed implementation manners

[0028] In order to make the objectives, technical solutions and advantages of the present disclosure more apparent, exemplary embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all embodiments of the present disclosure. It should be understood that the present disclosure is not limited by the exemplary embodiments described herein.

[0029] In this specification and the accompanying drawings, steps and elements having substantially the same or similar steps and elements are denoted by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. At the same time, in the description of the present disclosure, terms such as "first", "second", etc. are only used for distinguishing descriptions, and cannot be understood as indicating or implying relative importance or order.

[0030] To facilitate the description of the present disclosure, the following concepts related to the present disclosure are introduced.

[0031] Optionally, various models involved in the embodiments of the present disclosure can be artificial intelligence models, especially neural network models based on artificial intelligence. Generally, a neural network model based on artificial intelligence is implemented as an acyclic graph, where neurons are arranged in different layers. Generally, a neural network model includes an input layer and an output layer, and the input layer and the output layer are separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for the output generated in the output layer. Network nodes (i.e., neurons) are fully connected to nodes in adjacent layers via edges, and there are no edges between nodes within each layer. The data received at the nodes of the input layer of the neural network is propagated to the nodes of the output layer via any one of the hidden layer, activation layer, pooling layer, convolutional layer, etc. The input and output of the neural network model can take various forms, and the present disclosure does not limit this.

[0032] Artificial intelligence models are usually composed of individual computing modules, which are operators (abbreviated as Op). An operator is a mapping O: X → X' from one function space to another; generally speaking, performing a certain operation on any function can be considered an operator. An operator is different from traditional physical operators, and it is a representation method of a mapping relationship. Elements of a certain vector space generate another element in another vector space (which may also be the same vector space) through this mapping relationship. Specifically, in the fields of deep learning and artificial intelligence, an operator can also refer to a function that performs a certain mathematical operation or transformation on a tensor.

[0033] Common operators include: Convolution, Pooling, Fully Connected, Activation Function, Normalization, Loss Function, etc. These operators form the basic computing units of the deep neural network model through combination and stacking. An artificial intelligence chip can implement the inference process / training process of the neural network model by executing a series of operators.

[0034] Currently, it has been considered to use an assembly language operator generation program to develop and optimize operators, so as to convert the logic and functions of operators into assembly instructions, enabling operators to adapt to the hardware architecture of artificial intelligence chips. For example, it can adapt to the parallel processing framework of general-purpose parallel graphics processing units (GPGPUs).

[0035] During the process of using an assembly language operator generation program to develop and optimize operators, developers often need to use shared memory to store temporary data. Shared memory is a high-speed cache shared among threads in a GPU, which can accelerate the efficiency of data access. Shared memory can be divided into multiple memory units of equal size that can be accessed simultaneously, and these memory modules are also called Banks.

[0036] Although using shared memory has many advantages, using shared memory to store temporary data may also lead to Bank conflicts. Specifically, when multiple threads access different addresses of the same Bank simultaneously, a Bank conflict will occur. This conflict will reduce the parallelism of data access, force access operations to be serialized, thereby increasing latency and reducing the performance of operators.

[0037] In addition, for some complex operators, shared memory often needs to be sliced and reused multiple times, which increases the development difficulty, is prone to introducing errors, and is difficult to debug and verify. For example, in the matrix multiplication operator, the input tensor is usually divided into multiple small blocks, loaded into the shared memory sequentially for calculation, and after the calculation is completed, the result is written back to the global memory, and then the same shared memory is reused to load the next set of data. This slicing and reuse strategy requires developers to manually and precisely specify the boundaries of each slice, the loading order, and the reuse timing.

[0038] This way of manually specifying the shared memory access strategy by developers often leads to errors or performance degradation. For example, in the convolution operator, if the slicing is too fine, it may lead to frequent memory synchronization overhead and reduce performance; if the slicing is too coarse, it may exceed the capacity of the shared memory and trigger an overflow error.

[0039] Therefore, it is necessary to improve the current operator generation and optimization scheme to improve the operator development efficiency, reduce the probability of errors, optimize the performance of the operator, and make the assembly code corresponding to the operator more adaptable to the artificial intelligence chip.

[0040] To this end, according to an embodiment of the present disclosure, a method for compiling an operator is proposed, including: obtaining an intermediate representation of the operator to be compiled, where the intermediate representation (Intermediate Representation, IR) of the operator to be compiled at least includes an intermediate representation of the shared memory and an intermediate representation of the interaction operator. Among them, the intermediate representation of the shared memory at least includes an attribute value indicating the size of the shared memory, and the interaction operator is associated with the data interaction between the video memory and the shared memory; based on the size of the shared memory and the intermediate representation of the interaction operator, configure an attribute value indicating the operator rearrangement granularity (swizzleGran) in the intermediate representation of the shared memory to generate the configured intermediate representation of the shared memory; based on the configured intermediate representation of the shared memory, update the intermediate representation of the operator to be compiled, and compile the intermediate representation of the operator to be compiled to generate the assembly code corresponding to the operator to be compiled.

[0041] Among them, the intermediate representation of the operator refers to an abstract description form between the source code and the target machine code, which can maintain the stability of the code structure in different optimization stages and provides room for hardware-specific optimization. The operator rearrangement granularity refers to the granularity size of the operation when data rearrangement or recombination is performed in the shared memory. The embodiment of the present disclosure realizes the optimization of the shared memory access strategy by automatically configuring the operator rearrangement granularity in the intermediate representation of the shared memory, thereby improving the operator development efficiency and performance, reducing the burden on developers, reducing configuration errors, and improving the adaptability of the artificial intelligence chip.

[0042] In an optional aspect, embodiments of the present disclosure automatically determine the attribute value of the operator rearrangement granularity suitable for the shared memory by jointly analyzing the size of the shared memory and the intermediate representation of the interaction operator, and configure the operator rearrangement granularity in the intermediate representation of the shared memory, without the developer manually specifying the operator rearrangement granularity. This not only reduces potential errors caused by manual configuration, but also improves the execution efficiency of the operator on the artificial intelligence chip by optimizing the data access pattern.

[0043] In another optional aspect, embodiments of the present disclosure can be adapted to the situation where the hardware in the artificial intelligence chip automatically deduces the operator rearrangement granularity. Using the hardware to automatically deduce the operator rearrangement granularity can make the determination process of the rearrangement granularity more efficient, reduce the interference of human operations on the operator performance, and improve the stability of the overall development process.

[0044] Specific embodiments of the present disclosure are described in detail with reference to the following respective drawings.

[0045] Figure 1 A flowchart of a method 10 for compiling an operator according to an embodiment of the present disclosure is shown.

[0046] For example, as Figure 1 shown, at least one embodiment of the present disclosure provides a method 10 for compiling an operator, which includes the following operations S110 to S140. Of course, the method 10 according to the embodiments of the present disclosure may further include more or fewer operations or steps, and the present disclosure is not limited thereto.

[0047] In operation S110, obtain the intermediate representation of the operator to be compiled, where the intermediate representation of the operator to be compiled at least includes the intermediate representation of the shared memory and the intermediate representation of the interaction operator, and where the intermediate representation of the shared memory at least includes an attribute value indicating the size of the shared memory, and the interaction operator is associated with the data interaction between the video memory and the shared memory.

[0048] In operation S120, based on the size of the shared memory and the intermediate representation of the interaction operator, configure the operator rearrangement granularity in the intermediate representation of the shared memory to generate the configured intermediate representation of the shared memory.

[0049] In operation S130, update the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory.

[0050] In operation S140, compile the intermediate representation of the operator to be compiled to generate the assembly code corresponding to the operator to be compiled.

[0051] Embodiments of the present disclosure optimize the memory access strategy of the shared memory by automatically configuring the operator rearrangement granularity in the intermediate representation of the shared memory, thereby improving the operator development efficiency and performance, reducing the burden on developers, reducing configuration errors, and improving the adaptability of artificial intelligence chips.

[0052] In an optional aspect, embodiments of the present disclosure jointly analyze the size of the shared memory and the intermediate representation of the interacting operators to automatically determine the attribute value of the appropriate operator rearrangement granularity of the shared memory or the size of the shared memory processed at one time, and configure the operator rearrangement granularity in the intermediate representation of the shared memory without the developer manually specifying the operator rearrangement granularity. This not only reduces potential errors caused by manual configuration but also improves the execution efficiency of the operator on the artificial intelligence chip by optimizing the data access pattern.

[0053] In another optional aspect, embodiments of the present disclosure can be adapted to the case where the hardware in the artificial intelligence chip automatically deduces the operator rearrangement granularity. Using the hardware to automatically deduce the operator rearrangement granularity can make the determination process of the rearrangement granularity more efficient, reduce the interference of human operations on the operator performance, and improve the stability of the overall development process.

[0054] Next, refer to Figure 2 to further illustrate the operator to be compiled according to the embodiments of the present disclosure. Figure 2 FIG. shows an example of multiple operators to be compiled that make up a neural network model according to the embodiments of the present disclosure.

[0055] Optionally, the operator to be compiled can be any one of the multiple operators in the neural network model. As Figure 2 shown, the neural network model can include multiple nodes, where each node (or also referred to as an operator) represents a basic computational operation, such as matrix multiplication or activation function. The edges between the nodes indicate the data flow direction, defining the dependency relationship and execution order between the operations. The topological structure of the computational graph reflects the organization of these nodes and edges and determines how to efficiently execute the computational task. The input node serves as the starting point of the computation, accepting the input data of the model, while the output node produces the final computational result. In Figure 2 , the TMA (tensor read), conv2d (two-dimensional convolution), add (addition), relu (rectified linear unit), and concat (concatenation) nodes may all be operators to be compiled, and the input data is calculated through each operator in the direction of the edge, finally obtaining the output of the neural network model.

[0056] Optionally, the operator to be compiled is converted into assembly code for a specific hardware (such as a Graphics Processing Unit (GPU), a Tensor Processing Unit (TPU), a Neural Network Processing Unit (NPU), etc.) through an assembly language operator generation program. The assembly language operator generation program can generate efficient execution code according to the characteristics of the operator. For example, in a Convolutional Neural Network (CNN), the conv2d operator is used to extract image features, where the input tensor may be image data and the output tensor is the feature map; the add operator may be used to implement a residual connection, where the input tensors are the calculation results from two branches and the output tensor is their sum; the relu operator is used to introduce non-linearity, where the input tensor is the result of the previous layer and the output tensor is the value after being processed by the activation function; the concat operator is used to concatenate multiple feature maps, where the input tensors are multiple feature maps and the output tensor is the concatenated result.

[0057] Optionally, during the process of generating the assembly code corresponding to the operator to be compiled through the assembly language operator generation program, the impact of the characteristics of the artificial intelligence chip on the compilation of the operator also needs to be considered. For example, an artificial intelligence chip may have two storage areas, shared memory and video memory. The shared memory has a fast access speed but a small capacity, while the video memory has a large capacity but a slow access speed. The operator to be compiled needs to optimize the data access pattern according to the characteristics of the shared memory and the video memory. For example, the TMA operator loads data from the video memory to the shared memory, and then the conv2d operator loads data from the shared memory for calculation, while the add operator may directly operate on the intermediate results in the shared memory. When generating code through the assembly language operator generation program, the calculation logic and memory allocation will be adjusted according to the characteristics of the input and output tensors of the operator and the hardware architecture of the chip to improve the execution efficiency.

[0058] Optionally, during the process of generating the assembly code corresponding to the operator to be compiled through the assembly language operator generation program, the method 10 in the embodiments of the present disclosure can be adopted to optimize the data access pattern by using the attribute value of the operator rearrangement granularity. Thus, the optimization of the shared memory access strategy is achieved, thereby improving the operator development efficiency and performance, reducing the burden on developers, reducing configuration errors, and improving the adaptability of the artificial intelligence chip.

[0059] Next, reference is made to Figure 3 to further illustrate the intermediate representation 302 of the operator to be compiled according to the embodiments of the present disclosure. Figure 3 A schematic diagram of the intermediate representation 302 of the operator to be compiled according to the embodiments of the present disclosure is shown.

[0060] Optionally, the intermediate representation refers to an abstract description form between the source code and the target machine code, which can maintain the stability of the code structure during different optimization phases and provides room for hardware-specific optimizations. In an automatic code generation system such as an assembly language operator generation program, the intermediate representation serves as the basis for optimization passes and code transformation, enabling the assembly language operator generation program to generate highly efficient executable code for a specific artificial intelligence chip. During the development of an operator, the intermediate representation usually includes the operator's logical structure, data dependency relationships, and storage resource information, etc., which can effectively describe the organizational state of the operator before it is actually executed by the hardware.

[0061] For example, taking Figure 3 as an example for illustration. The intermediate representation 302 of the operator to be compiled may include multiple intermediate representations at different levels and with different functions, which can be respectively used to define or describe the computational logic description inside the operator, memory access patterns, parallelization strategies, specific hardware optimization requirements, operator fusion and scheduling information, etc. The present disclosure does not limit this.

[0062] For example, the intermediate representation 302 of the operator to be compiled may at least include: the intermediate representation 304 of shared memory, the intermediate representation 306 of interaction operators, the intermediate representation 308 of computational operations, etc.

[0063] For example, the intermediate representation 304 of shared memory is an intermediate representation that describes the cache or programmable shared memory area in an artificial intelligence chip, which is shown as the SharedTensor class. In many artificial intelligence accelerator architectures, shared memory is a cache that can be concurrently accessed by multiple processing units and is used to temporarily store intermediate computational results to reduce access to the video memory. The intermediate representation 304 of shared memory includes attribute information such as memory size and user data type. Before the operation S120 is executed, the attribute value of the operator rearrangement granularity in the intermediate representation of shared memory can be empty, and after the operation S120 is executed, the attribute value of the operator rearrangement granularity can be configured to a specific value. Of course, the present disclosure is not limited thereto.

[0064] An interaction operator refers to any operator in the operator to be compiled that involves data interaction between the video memory and the shared memory. It may either be an operator specifically for data interaction between the video memory and the shared memory, or an operator for computing tensor data, but involves operations such as tensor data loading, storage, and synchronization. The interaction operator can also be presented in the intermediate representation, such as Figure 3 the intermediate representation 306 of the interaction operator in

[0065] For example, as Figure 3As shown, during the calculation process, the interaction operator may involve loading data from the video memory to the shared memory (such as the operator Global2Shared), or writing the data in the shared memory back to the video memory (such as the operator Shared2Global), or synchronizing the data in the shared memory with the data in the video memory (not shown) to adapt to the architecture of the artificial intelligence chip and improve the technical efficiency. For example, the intermediate representation of the interaction operator may also define the logical structure and data dependency relationships during the data interaction process to optimize memory access. The present disclosure is not limited thereto.

[0066] The intermediate representation 308 of the calculation operation is the intermediate representation of the operator for a specific calculation process. Taking the intermediate representation of the multiplication operation (matmul) as an example, it is used to calculate the product of two matrices, for example, C = A * B, where the shape of A is M * K, the shape of B is K * N, and the shape of C is M * N. In the artificial intelligence chip, to accelerate this operation, usually, part of the data of A and B is loaded into the shared memory, and then multiple computing units calculate different parts of C in parallel. Thus, the intermediate representation 308 of the calculation operation may implement the above process by calling the intermediate representation 304 of the shared memory and the intermediate representation 306 of the interaction operator described above. Of course, the present disclosure is not limited thereto.

[0067] Next, refer to Figure 4 to further illustrate the intermediate representation 304 of the shared memory according to an embodiment of the present disclosure. Figure 4 FIG. shows a schematic diagram of the pseudo-code of the intermediate representation 304 of the shared memory according to an embodiment of the present disclosure.

[0068] As Figure 4 shown, the intermediate representation 304 of the shared memory is a specific class for encapsulating the relevant information and operations of the shared memory. For example, the intermediate representation 304 of the shared memory includes the following multiple public functions that can be called / accessed by external objects: the constructor SharedTensor(const Shape user_shape, DataType user_dataType), the function getSize() for returning the memory size, the function setSwizzleRefShape(const Shape user_swizzleRefShape) for configuring the rearrangement shape of the operator, the function getSwizzleRefShape() for returning the rearrangement shape of the operator, the function setSwizzleGran(int user_swizzleGranIn) for configuring the rearrangement granularity of the operator, the function getSwizzleGran() for returning the rearrangement granularity of the operator, and the function isSwizzle() const for checking whether the operator rearrangement function is enabled.

[0069] As Figure 4 shown, in addition to the attribute value of the operator rearrangement granularity that needs to be configured, the intermediate representation 304 (SharedTensor) of the shared memory also includes the shape (user_shape) and user data type (user_dataType) information of the user data that needs to be configured in the initialization stage. Specifically, the externally accessible constructor SharedTensor(const Shape user_shape, DataType user_dataType) can initialize the shape and data type of the shared memory. For example, the user_shape parameter specifies the shape of the shared memory that the user data needs to use, such as 16x16; the user_dataType parameter specifies the data type of the user data, such as floating point number. Then, the external object can obtain the size Size of the shared memory of the intermediate representation 304 (SharedTensor) of the shared memory through the function getSize(), that is, the size Size of the shared memory of the current hardware.

[0070] Next, the function setSwizzleRefShape(const Shape user_swizzleRefShape) that can be accessed by external objects and is used to configure the operator rearrangement shape can be used to configure the reference shape of the operator rearrangement granularity in the intermediate representation 304 of the shared memory. The reference shape of the operator rearrangement granularity is also the size of the shared memory processed in a single time. The function setSwizzleRefShape can also be used as an interface to enable the hardware automatic derivation function. After the hardware automatic derivation function is started, the hardware can automatically derive the operator rearrangement granularity according to the reference shape of the operator rearrangement granularity (that is, the size of the shared memory processed in a single time). After configuration, the isSwizzle flag is set to true, indicating that the operator rearrangement function is enabled. Further, the external object can also obtain the reference shape swizzleRefShape of the operator rearrangement granularity in the intermediate representation 304 of the shared memory through the function getSwizzleRefShape().

[0071] The external object can also allow the intermediate representation 304 of the shared memory to enable the analysis module to directly specify the attribute value of the operator rearrangement granularity swizzleGran through the function setSwizzleGran. For example, if swizzleGran = 16, it means that the data is rearranged in units of 16 Byte blocks. After configuration, the isSwizzle flag is set to true. Further, the attribute value of the operator rearrangement granularity of the intermediate representation 304 of the shared memory can also be obtained through the method getSwizzleGran().

[0072] In addition, an external object can also determine whether the operator rearrangement function is enabled for the intermediate representation 304 of the shared memory through isSwizzle(). The intermediate representation 304 of the shared memory may further include other functions or capabilities that can be called by external objects, which are not described one by one in this disclosure.

[0073] Optionally, the intermediate representation 304 of the shared memory includes multiple private parameters that cannot be called / accessed by external objects: tensor_size, swizzleRefShape, swizzleGran, isSwizzle. These parameters can only be indirectly modified through the various public functions described above to protect the integrity and security of the data.

[0074] Reference Figure 4 The intermediate representation 304 of the shared memory according to the embodiments of the present disclosure can be used to support efficient computing operations in an artificial intelligence chip. The intermediate representation 304 of the shared memory according to the embodiments of the present disclosure combines the characteristics of operators and shared memory, providing a flexible operator rearrangement configuration method. Whether it is automatically deduced by the hardware through the reference shape or the operator rearrangement granularity is directly specified, the analysis module can optimize the data access pattern, thereby improving the computing performance. This intermediate representation 304 is applicable to various computing tasks and can fully utilize the hardware acceleration capabilities.

[0075] Next, reference Figure 5 is made to further illustrate some optional details of the method 10 according to the embodiments of the present disclosure. Figure 5 A schematic diagram showing a method for compiling an operator according to an embodiment of the present disclosure. It should be noted that the method 10 according to the embodiments of the present disclosure can be fully executed by a device equipped with any assembly language operator generation program, or can be cooperatively executed by a device equipped with other analysis module devices and a device equipped with any assembly language operator generation program. The present disclosure is not limited thereto.

[0076] In operation S110, an intermediate representation of the operator to be compiled is obtained. The intermediate representation of the operator to be compiled includes at least an intermediate representation of the shared memory and an intermediate representation of the interaction operator. Among them, the intermediate representation of the shared memory includes at least an attribute value indicating the size of the shared memory, and the interaction operator is associated with data interaction between the video memory and the shared memory.

[0077] Previously, reference has been made to Figures 2 to 4The intermediate representation of the operator to be compiled, the intermediate representation of the shared memory, and the intermediate representation of the interactive operator are described in detail, and will not be elaborated herein. Optionally, in operation S110, the intermediate representation 304 of the shared memory can be determined by directly searching for the class SharedTensor in the intermediate representation of the operator to be compiled. By searching for this SharedTensor class, the intermediate representation of the shared memory can be quickly located.

[0078] However, as mentioned above, the intermediate representation of the interactive operator may involve multiple cases. Therefore, in an alternative embodiment, an analysis module (pass) may be used to traverse and analyze the intermediate representation. The analysis module can be a module in the assembly language operator generation program or another module independent of the assembly language operator generation program, and is used to perform specific analysis or transformation on the intermediate representation. For example, the analysis module can be used to traverse and analyze the intermediate representation of the operator to be compiled to identify and extract all operators involved in the interaction between the shared memory and the external video memory, and these operators will be used as the intermediate representation of the interactive operator. For example, the analysis module can find all the code that reads data from the video memory to the shared memory, or writes the data in the shared memory back to the video memory, or synchronizes the data in the shared memory with the data in the video memory by scanning each operation (method) or function in the intermediate representation of the operator to be compiled, and mark the operator with this code as the intermediate representation of the interactive operator.

[0079] As a specific example, the Tensor Memory Access (TMA) is used as an instance of the intermediate representation of the interactive operator. TMA is a general intermediate representation used to represent data read operations, especially data transfer between the shared memory and the external video memory. During the compilation of the artificial intelligence chip, TMA can represent a specific data read operation, such as reading tensor data from the video memory and loading it into the shared memory. The design of TMA enables the assembly language operator generation program to more effectively optimize the data access pattern, thereby improving the overall computing performance. In one embodiment, TMA may include information such as the source address (location in the video memory), the target address (location in the shared memory), and the data size, and these information are used to generate the intermediate representation of the interactive operator. Of course, the present disclosure is not limited thereto.

[0080] In operation S120, based on the size of the shared memory and the intermediate representation of the interactive operator, an operator rearrangement granularity is configured in the intermediate representation of the shared memory to generate the configured intermediate representation of the shared memory.

[0081] Optionally, the attribute value of the operator rearrangement granularity may be affected by various factors. For example, the hardware structure of the artificial intelligence chip, the size of the shared memory, the execution logic of each operation in the interactive operator, the control logic of concurrent access, and so on. For example, if the shared memory is large, a larger operator rearrangement granularity can be selected to reduce the number of accesses to the global video memory. If the shared memory is small, a smaller operator rearrangement granularity is required to ensure that the data block can be fully loaded into the shared memory and avoid frequent memory swapping.

[0082] As another example, if the intermediate representation of the interactive operator shows that it may be necessary to access the shared memory continuously multiple times within a predetermined time period, the operator rearrangement granularity may be set to be approximately the same as the access range. For example, when an interactive operator may need to continuously access 128 bytes of data, the operator rearrangement granularity can be set to 128 bytes. Of course, limited by the size of the shared memory, the operator rearrangement granularity may also be set to 64 bytes to achieve a balance between meeting the size limit of the shared memory and meeting the execution performance of the intermediate representation of the interactive operator.

[0083] As yet another example, if there are multiple interactive operators that may need to concurrently access the shared memory simultaneously, the operator rearrangement granularity may need to take into account the number of concurrent threads to set a relatively balanced operator rearrangement granularity to evenly allocate the shared memory to each interactive operator. Of course, the present disclosure is not limited thereto.

[0084] In addition, in an optional embodiment of the present disclosure, in operation S120, based on the intermediate representation of the interactive operator, the execution granularity of the interactive operator is determined, and based on the size of the shared memory and the execution granularity of the interactive operator, the attribute value of the operator rearrangement granularity or the size of the shared memory processed each time is configured in the intermediate representation of the shared memory. Of course, the present disclosure is not limited thereto.

[0085] Optionally, during the compilation and execution of the artificial intelligence chip, the execution granularity of the interactive operator refers to the amount of data or the size of the data block processed at one time by the hardware or software when executing the interactive operator. For example, in the operation of transferring data from the video memory to the shared memory, the execution granularity may represent the number of bytes read or written at one time. The execution granularity of the interactive operator and the operator rearrangement granularity will jointly affect the memory access efficiency. Different interactive operators may have different execution granularities. Therefore, in an optional embodiment of the present disclosure, correlation analysis of the execution granularity of the interactive operator and the operator rearrangement granularity is also considered to determine the attribute value of the operator rearrangement granularity.

[0086] Optionally, the rearrangement granularity of the shared memory can be derived based on the size of the shared memory and the execution granularity of the interaction operator, and configured in the SwizzleGran attribute of the shared memory through the aforementioned method setSwizzleGran(int user_swizzleGranIn). Specifically, the operator rearrangement granularity usually needs to be coordinated with the execution granularity to ensure the efficiency of data access. For example, assuming that the intermediate representation of the interaction operator specifies that 32 * 8 = 256 bytes (Byte) of data are processed at a time (8 represents the number of channels), then the operator rearrangement granularity of the shared memory can be set to 32 bytes and recorded in the SwizzleGran attribute of the SharedTensor object. In this way, the assembly language operator generation program and the hardware can be optimized according to the operator rearrangement granularity when processing data.

[0087] Another way is to store the single - processing size of the shared memory and pass it to the hardware, which automatically analyzes and determines the appropriate rearrangement granularity. For example, when the TMA operator is an interaction operator, the intermediate representation of the TMA operator may contain information that the single - processing size is 64 * 8 bytes. The information with a single - processing size of 64 * 8 bytes can be configured in the Shape attribute of the SharedTensor object through the aforementioned method setSwizzleRefShape(const Shape user_swizzleRefShape). During the compilation of the subsequent interaction operator, the assembly language operator generation program configures this size on the assembly language instruction, and the subsequent hardware uses this information during the execution of the instruction to automatically derive and execute the operator rearrangement granularity. This method reduces the burden on the assembly language operator generation program and allows the hardware to adjust flexibly.

[0088] Optionally, the attribute value of the operator rearrangement granularity can also be configured in the intermediate representation of the interaction operator to generate the configured intermediate representation of the interaction operator; or based on the configured intermediate representation of the shared memory, the size of the data processed at a time can be configured in the intermediate representation of the interaction operator to generate the configured intermediate representation of the interaction operator. Specifically, the intermediate representation of all interaction operators can include the SwizzleGran attribute or the SwizzleRefShape attribute for recording the operator rearrangement granularity. For example, in one embodiment, the intermediate representation of TMA not only specifies that the single - processing size is 128 * 64 bytes, but also configures SwizzleRefShape to be equal to 128 * 64 bytes through the analysis module. This attribute value is written into the intermediate representation of TMA during the compilation process for subsequent optimization and hardware execution.

[0089] Optionally, either the attribute value of the operator rearrangement granularity in the shared memory or the attribute value of the operator rearrangement granularity in the interactive operator may dynamically change during the execution process of the operator to be compiled. For example, the attribute value of the operator rearrangement granularity may match a certain time period or a certain operation and change with the change of the operation. The present disclosure is not limited thereto.

[0090] In operation S130, based on the intermediate representation of the configured shared memory, update the intermediate representation of the operator to be compiled.

[0091] Specifically, as the analysis module or the compilation module configures the attribute value of the operator rearrangement granularity in the intermediate representation of the shared memory, the intermediate representation of the operator to be compiled also needs to be adaptively updated. During the process of generating assembly code using the intermediate representation of the operator to be compiled, if any operator in the intermediate representation of the operator to be compiled involves any operation on the shared memory, the assembly language operator generation program may configure the shared memory operator rearrangement granularity attribute into the corresponding instruction, thereby improving the data access efficiency.

[0092] Optionally, in an embodiment of the present disclosure, the intermediate representation of the operator to be compiled may also be updated based on both the configured intermediate representation of the interactive operator and the configured intermediate representation of the shared memory. Of course, the present disclosure is not limited thereto.

[0093] In operation S140, compile the intermediate representation of the operator to be compiled to generate the assembly code corresponding to the operator to be compiled.

[0094] In operation S140, the assembly language operator generation program may convert the intermediate representation into an instruction set executable by hardware (GPU, TPU, NPU) to ensure that the hardware can execute the computing task as expected. More specifically, during the process of generating assembly code using the intermediate representation of the operator to be compiled, since the intermediate representation of the operator involving any operation on the shared memory includes the operator rearrangement granularity attribute, the assembly language operator generation program can configure the shared memory operator rearrangement granularity attribute into the corresponding hardware instruction (for example, assembly code) so that the hardware can access the shared memory by means of operator rearrangement, thereby improving the data access efficiency. Of course, the present disclosure is not limited thereto.

[0095] During the process of generating assembly code using IR, if the IR is an IR using shared memory, configure the rearrangement granularity attribute of the shared memory to the IR, so that the hardware accesses the shared memory by means of operator rearrangement, improving performance while ensuring correctness.

[0096] Optionally, each operator in the operator to be compiled can be divided into two categories: operators that use shared memory and operators that do not use shared memory. For the intermediate representation of those operators that use shared memory, operation S140 further includes: configuring an attribute value of the rearrangement granularity of the operator using shared memory for the intermediate representation of the operator using shared memory, so that the corresponding assembly instruction of the intermediate representation of the operator using shared memory includes the attribute value of the rearrangement granularity of the operator using shared memory. Of course, the present disclosure is not limited thereto.

[0097] For example, when generating assembly code, assume that the intermediate representation of an operator that uses shared memory is configured with an operator rearrangement granularity of 16 bytes (where the number of channels is 8) or a shared memory size of 128 * 64 for single processing. The assembly language operator generation program will embed this operator rearrangement granularity attribute in the corresponding load instruction. When the hardware executes this instruction, it will load data into the shared memory in units of 128 bytes, reducing the latency of memory access. Similarly, for the store instruction, the assembly language operator generation program can also optimize the mode of writing data from the shared memory back to the video memory by configuring the operator rearrangement granularity. Similarly, the alu instruction can also be configured with the operator rearrangement granularity attribute. In this way, the assembly code can make full use of the hardware characteristics of the artificial intelligence chip to improve the computing performance while ensuring correctness.

[0098] In some embodiments of the present disclosure, the intermediate representation of the operator to be compiled may include an intermediate representation for slicing (view) shared memory or an intermediate representation for reusing (make tensor) shared memory. "Slicing" shared memory means dividing a shared memory area into multiple smaller sub-areas so that multiple processes or threads can access different parts simultaneously. And "reusing" shared memory means reusing the same shared memory area among multiple processes or tasks to reduce the overhead of memory allocation and improve memory utilization efficiency. In both cases, the existing intermediate representation of the shared memory may no longer be adapted to the sliced or reused shared memory, and new dedicated intermediate representations need to be created for these sliced or reused shared memories.

[0099] In operation S140, for the case of the intermediate representation of the operator for splitting shared memory or the intermediate representation of the operator for reusing shared memory, the assembly language operator generation program will create a dedicated intermediate representation of the shared memory for the intermediate representation of the operator for splitting shared memory or the intermediate representation of the operator for reusing shared memory according to the existing intermediate representation SharedTensor of the shared memory. The dedicated intermediate representation of the shared memory is specific to the intermediate representation of the operator for splitting shared memory or the intermediate representation of the operator for reusing shared memory. These dedicated intermediate representations can also be named SharedTensor. Of course, the present disclosure is not limited thereto.

[0100] These dedicated intermediate representations of the shared memory also need to configure the attribute value of the operator rearrangement granularity. Operation S140 may further include: in response to determining that the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory has been configured, not changing the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory; and in response to determining that the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory has not been configured, configuring the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory as the attribute value of the operator rearrangement granularity in the intermediate representation of the shared memory. Of course, the present disclosure is not limited thereto.

[0101] Specifically, since the dedicated intermediate representation of the shared memory may have been declared in the intermediate representation of the operator to be compiled, during the traversal of the previous analysis module, the operator rearrangement attribute of this type of dedicated intermediate representation may have been configured. In this case, for the operator rearrangement granularity attribute of this type of dedicated intermediate representation of the shared memory, the assembly language operator generation program will no longer change it. If the operator rearrangement attribute of this type of dedicated intermediate representation has not been configured during the traversal of the previous analysis module, then the operator rearrangement attribute of this type of dedicated intermediate representation is configured as the attribute value of the operator rearrangement granularity in the intermediate representation of the shared memory, that is, inherit the attribute value of the operator rearrangement granularity in the existing dedicated intermediate representation of the shared memory. In this way, the consistency of the memory access pattern can be maintained. Of course, the present disclosure is not limited thereto.

[0102] In summary, the embodiment of the present disclosure realizes the automatic optimization of the memory access strategy of the shared memory by automatically configuring the operator rearrangement granularity in the intermediate representation of the shared memory, so as to improve the operator development efficiency and performance, reduce the burden on developers, reduce configuration errors, and improve the adaptability of the artificial intelligence chip.

[0103] Figure 6 FIG. shows a schematic diagram of a device 60 for compiling an operator according to an embodiment of the present disclosure.

[0104] As Figure 6As shown, the apparatus 60 for compiling an operator according to an embodiment of the present disclosure includes a first module 61, a second module 62, a third module 63, and a fourth module 64.

[0105] The first module 61 may be configured to: obtain an intermediate representation of an operator to be compiled, where the intermediate representation of the operator to be compiled at least includes an intermediate representation of shared memory and an intermediate representation of an interaction operator. Among them, the intermediate representation of the shared memory at least includes an attribute value indicating the size of the shared memory, and the interaction operator is associated with data interaction between video memory and the shared memory.

[0106] The second module 62 is configured to: based on the size of the shared memory and the intermediate representation of the interaction operator, configure an operator rearrangement granularity in the intermediate representation of the shared memory to generate the configured intermediate representation of the shared memory.

[0107] The third module 63 is configured to: based on the configured intermediate representation of the shared memory, update the intermediate representation of the operator to be compiled.

[0108] The fourth module 64 is configured to: compile the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled.

[0109] In addition, an embodiment of the present disclosure further provides an electronic device, including: a processor; and a memory, where a computer-executable program is stored in the memory, and when the computer-executable program is executed by the processor, the method for compiling an operator described above is executed.

[0110] The processor in the embodiment of the present disclosure may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, operations, and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc., and may be of the X86 architecture or the ARM architecture.

[0111] In general, the various exemplary embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device. When aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented as non-limiting examples in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof.

[0112] For example, the method or apparatus according to the embodiments of the present disclosure may also be implemented by means of Figure 7 the architecture of the computing device 3000 shown. As Figure 7 shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, input / output components 3060, a hard disk 3070, etc. The storage devices in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for the processing and / or communication of the methods provided by the present disclosure and the program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 7 the architecture shown is only exemplary, and when implementing different devices, one or more components shown in the Figure 7 computing device may be omitted according to actual needs.

[0113] According to another aspect of the present disclosure, a computer-readable storage medium is also provided. Figure 8 A schematic diagram of the storage medium 4000 according to the present disclosure is shown.

[0114] As Figure 8As shown, computer-readable instructions 4010 are stored on the computer-readable storage medium 4020. When the computer-readable instructions 4010 are run by a processor, the methods according to the embodiments of the present disclosure described with reference to the above figures can be executed. The computer-readable storage medium in the embodiments of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM). It should be noted that the memories for the methods described herein are intended to include, but are not limited to, these and any other suitable types of memories. It should be noted that the memories for the methods described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0115] Embodiments of the present disclosure also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods according to the embodiments of the present disclosure.

[0116] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0117] Generally speaking, the various exemplary embodiments of the present disclosure can be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, a microprocessor, or other computing devices. When aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof.

[0118] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art should understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A method for compiling an operator, characterized in that, The method includes: Obtaining an intermediate representation of an operator to be compiled, where the intermediate representation of the operator to be compiled at least includes an intermediate representation of shared memory and an intermediate representation of an interaction operator. Among them, the intermediate representation of the shared memory at least includes an attribute value indicating the size of the shared memory, and the interaction operator is associated with data interaction between video memory and the shared memory; Based on the size of the shared memory and the intermediate representation of the interaction operator, configuring an operator rearrangement granularity in the intermediate representation of the shared memory to generate the configured intermediate representation of the shared memory; Based on the configured intermediate representation of the shared memory, updating the intermediate representation of the operator to be compiled, and Compiling the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled.

2. The method according to claim 1, wherein Among them, The configuring an operator rearrangement granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interaction operator includes: Determining an execution granularity of the interaction operator based on the intermediate representation of the interaction operator, and Based on the size of the shared memory and the execution granularity of the interaction operator, configuring an attribute value of the operator rearrangement granularity or the size of the shared memory processed once in the intermediate representation of the shared memory.

3. The method according to claim 1, wherein The method further includes: Configuring an attribute value of the operator rearrangement granularity in the intermediate representation of the interaction operator to generate the configured intermediate representation of the interaction operator; or Based on the configured intermediate representation of the shared memory, configuring the size of the data processed once in the intermediate representation of the interaction operator to generate the configured intermediate representation of the interaction operator.

4. The method according to claim 3, characterized in that Among them, The updating the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory includes: Updating the intermediate representation of the operator to be compiled based on the configured intermediate representation of the interaction operator and the configured intermediate representation of the shared memory.

5. The method according to claim 1, wherein Among them, The intermediate representation of the operator to be compiled includes an intermediate representation of an operator for splitting shared memory or an intermediate representation of an operator for reusing shared memory. The compiling the intermediate representation of the operator to be compiled includes: Creating a dedicated intermediate representation of the shared memory according to the intermediate representation of the shared memory, where the dedicated intermediate representation of the shared memory is specific to the intermediate representation of the operator for splitting shared memory or the intermediate representation of the operator for reusing shared memory; In response to determining that the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory has been configured, not changing the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory; and In response to determining that the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory has not been configured, configuring the attribute value of the operator rearrangement granularity in the dedicated intermediate representation of the shared memory as the attribute value of the operator rearrangement granularity in the intermediate representation of the shared memory.

6. The method according to claim 1, wherein Among them, The intermediate representation of the operator to be compiled includes an intermediate representation of an operator for using shared memory. The compiling the intermediate representation of the operator to be compiled includes: Configure the attribute value of the operator rearrangement granularity of the shared memory for the operator intermediate representation using the shared memory, so that the corresponding assembly instruction block of the operator intermediate representation using the shared memory includes the attribute value of the operator rearrangement granularity of the shared memory.

7. The method according to claim 1, wherein Wherein, The obtaining of the intermediate representation of the operator to be compiled includes: Traverse and analyze the intermediate representation of the operator to be compiled through an analysis module to obtain the intermediate representation of the interaction operator.

8. An apparatus for compiling operators, characterized in that The device includes: A first module configured to: obtain the intermediate representation of the operator to be compiled, the intermediate representation of the operator to be compiled at least including the intermediate representation of the shared memory and the intermediate representation of the interaction operator, wherein the intermediate representation of the shared memory at least includes an attribute value indicating the size of the shared memory, and the interaction operator is associated with data interaction between the video memory and the shared memory; A second module configured to: configure the operator rearrangement granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interaction operator to generate the configured intermediate representation of the shared memory; A third module configured to: update the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory; and A fourth module configured to: compile the intermediate representation of the operator to be compiled to generate the assembly code corresponding to the operator to be compiled.

9. An electronic device, characterized in that, The electronic device includes: A processor; and A memory, wherein the memory stores a computer-executable program, and when the computer-executable program is executed by the processor, the method for compiling an operator according to any one of claims 1-7 is executed.

10. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed by the processor, the method for compiling an operator according to any one of claims 1-7 is implemented.

Images