Method for compiling an operator, device for compiling an operator, electronic device, and computer-readable storage medium

Abstract

The embodiments of the present disclosure disclose a method for compiling operators, an apparatus for compiling operators, an electronic device, and a computer-readable storage medium. The method includes: obtaining an intermediate representation of the operator to be compiled, the intermediate representation of the operator to be compiled including at least an intermediate representation of a shared memory and an intermediate representation of an interactive operator; configuring the operator reordering granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interactive operator to generate the configured intermediate representation of the shared memory; updating the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory, and compiling the intermediate representation of the operator to be compiled to generate the assembly code corresponding to the operator to be compiled. The embodiments of the present disclosure effectively improve the efficiency of operator development, reduce the probability of errors, and optimize the performance of the operator, making it more suitable for artificial intelligence chips.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more particularly to a method for compiling an operator, an apparatus for compiling an operator, an electronic device, and a computer-readable storage medium. Background Art

[0002] Artificial intelligence chips are developing rapidly and have been widely used in various fields, such as intelligent driving, medical diagnosis, and smart homes. A key feature of AI chips is their ability to efficiently process neural network models to support complex computing tasks. Specifically, AI chips implement the inference and training of neural network models by executing a series of operators. These operators may involve basic neural network operations such as convolution, pooling, matrix multiplication, and activation functions. AI chips are designed to efficiently process these operators, significantly improving the efficiency of neural network training and inference through parallel computing and hardware acceleration technologies.

[0003] Currently, the use of assembly language operator generators to develop and optimize operators has been considered. This converts the operator's logic and functionality into assembly instruction sets, allowing the operator's execution to adapt to the hardware architecture of AI chips. For example, this allows each operator to adapt to the parallel processing framework of a general-purpose parallel graphics processing unit (GPGPU). However, this traditional approach of developing and optimizing operators using assembly language operator generators still suffers from technical issues such as reduced data access parallelism, increased development difficulty, and difficulty in debugging and verification.

[0004] Therefore, it is necessary to improve the current operator generation and optimization schemes to increase operator development efficiency, reduce the probability of errors, and optimize the performance of operators to make them more suitable for artificial intelligence chips. Summary of the Invention

[0005] Embodiments of the present disclosure provide a method for compiling an operator, an apparatus for compiling an operator, an electronic device, and a computer-readable storage medium.

[0006] An embodiment of the present disclosure provides a method for compiling an operator, the method comprising: obtaining an intermediate representation of an operator to be compiled, the intermediate representation of the operator to be compiled comprising at least an intermediate representation of a shared memory and an intermediate representation of an interactive operator, wherein the intermediate representation of the shared memory comprises at least an attribute value indicating the size of the shared memory, and the interactive operator is associated with data interaction between a video memory and the shared memory; based on the size of the shared memory and the intermediate representation of the interactive operator, configuring an operator reordering granularity in the intermediate representation of the shared memory to generate a configured intermediate representation of the shared memory; based on the configured intermediate representation of the shared memory, updating the intermediate representation of the operator to be compiled, and compiling the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled.

[0007] For example, configuring the operator reordering granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interacting operator includes: determining the execution granularity of the interacting operator based on the intermediate representation of the interacting operator, and configuring the attribute value of the operator reordering granularity or the size of the shared memory for single processing in the intermediate representation of the shared memory based on the size of the shared memory and the execution granularity of the interacting operator.

[0008] For example, the method of compiling operators also includes: configuring the attribute value of the operator reordering granularity in the intermediate representation of the interaction operator to generate the configured intermediate representation of the interaction operator; or configuring the data size of a single processing in the intermediate representation of the interaction operator based on the configured intermediate representation of the shared memory to generate the configured intermediate representation of the interaction operator.

[0009] For example, updating the intermediate representation of the operator to be compiled based on the intermediate representation of the shared memory after the configuration includes: updating the intermediate representation of the operator to be compiled based on the intermediate representation of the interactive operator after the configuration and the intermediate representation of the shared memory after the configuration.

[0010] For example, the intermediate representation of the operator to be compiled includes an intermediate representation of an operator for splitting shared memory or an intermediate representation of an operator for reusing shared memory, and compiling the intermediate representation of the operator to be compiled includes: creating a dedicated intermediate representation of the shared memory based on the intermediate representation of the shared memory, wherein the dedicated intermediate representation of the shared memory is specific to the intermediate representation of the operator for splitting shared memory or the intermediate representation of the operator for reusing shared memory; in response to determining that the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory has been configured, not changing the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory; and in response to determining that the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory has not been configured, configuring the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory to the attribute value of the operator reordering granularity in the intermediate representation of the shared memory.

[0011] For example, the intermediate representation of the operator to be compiled includes an intermediate representation of an operator for using shared memory, and compiling the intermediate representation of the operator to be compiled includes: configuring the attribute value of the operator reordering granularity of the shared memory to the intermediate representation of the operator for using shared memory, so that the corresponding assembly instruction block of the intermediate representation of the operator for using shared memory includes the attribute value of the operator reordering granularity of the shared memory.

[0012] For example, obtaining the intermediate representation of the operator to be compiled includes: performing traversal analysis on the intermediate representation of the operator to be compiled by an analysis module to obtain the intermediate representation of the interactive operator.

[0013] For example, an embodiment of the present disclosure provides a device for compiling operators, the device comprising: a first module, configured to: obtain an intermediate representation of the operator to be compiled, the intermediate representation of the operator to be compiled comprising at least an intermediate representation of a shared memory and an intermediate representation of an interactive operator, wherein the intermediate representation of the shared memory comprises at least an attribute value indicating the size of the shared memory, and the interactive operator is associated with data interaction between the video memory and the shared memory; a second module, configured to: configure an operator reordering granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interactive operator to generate a configured intermediate representation of the shared memory; a third module, configured to: update the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory; and a fourth module, configured to: compile the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled.

[0014] An embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory, wherein the memory stores a computer executable program, and when the processor executes the computer executable program, the above-mentioned method of compiling an operator is executed.

[0015] An embodiment of the present disclosure provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed by a processor, the above-mentioned method of compiling an operator is implemented.

[0016] According to another aspect of the present disclosure, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable medium and executes the computer instructions, causing the computer device to perform the methods provided in the various aspects or various optional implementations of the various aspects described above.

[0017] Currently, the traditional method of using assembly language operator generation programs to develop and optimize operators still has technical problems such as reduced parallelism of data access, high development difficulty, and difficulty in debugging and verification.

[0018] To this end, the embodiments of the present disclosure automatically optimize the shared memory segmentation and reuse strategy by automatically configuring the operator reordering granularity in the intermediate representation of the shared memory, so as to improve the operator development efficiency and performance, reduce the burden on developers, reduce configuration errors and improve the adaptability of artificial intelligence chips. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for describing the embodiments. The drawings described below are only exemplary embodiments of the present disclosure.

[0020] Figure 1 A flowchart of a method for compiling an operator according to an embodiment of the present disclosure is shown.

[0021] Figure 2 An example of multiple operators to be compiled that constitute a neural network model according to an embodiment of the present disclosure is shown.

[0022] Figure 3 A schematic diagram of an intermediate representation of an operator to be compiled according to an embodiment of the present disclosure is shown.

[0023] Figure 4 A schematic diagram showing pseudo code of an intermediate representation of a shared memory according to an embodiment of the present disclosure is shown.

[0024] Figure 5A schematic diagram illustrating a method for compiling operators according to an embodiment of the present disclosure is shown.

[0025] Figure 6 is a schematic diagram illustrating an apparatus for compiling an operator according to an embodiment of the present disclosure.

[0026] Figure 7 is an architectural diagram illustrating a computing device according to an embodiment of the present disclosure.

[0027] Figure 8 is a schematic diagram illustrating a computer-readable storage medium according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present disclosure more apparent, the exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0029] In this specification and the accompanying drawings, substantially the same or similar steps and elements are denoted by the same or similar reference numerals, and repeated descriptions of these steps and elements will be omitted. At the same time, in the description of the present disclosure, the terms "first", "second", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance or ranking.

[0030] To facilitate description of the present disclosure, concepts related to the present disclosure are introduced below.

[0031] Optionally, the various models involved in the embodiments of the present disclosure may be artificial intelligence models, in particular artificial intelligence-based neural network models. Typically, artificial intelligence-based neural network models are implemented as acyclic graphs in which neurons are arranged in different layers. Typically, a neural network model includes an input layer and an output layer, which are separated by at least one hidden layer. The hidden layer transforms the input received by the input layer into a representation useful for the output generated in the output layer. Network nodes (i.e., neurons) are fully connected to nodes in adjacent layers via edges, and there are no edges between nodes in each layer. Data received at the nodes of the input layer of the neural network is propagated to the nodes of the output layer via any one of the hidden layers, activation layers, pooling layers, convolutional layers, etc. The input and output of the neural network model can take various forms, and the present disclosure does not limit this.

[0032] Artificial intelligence models are typically composed of computational modules, known as operators (Op). An operator is a mapping O: X → X' from one function space to another. Broadly speaking, any operation on a function can be considered an operator. Operators, unlike traditional physical operators, represent a mapping relationship. Elements in one vector space produce elements in another vector space (possibly the same vector space) through this mapping relationship. Specifically, in the fields of deep learning and artificial intelligence, operators can refer to functions that perform mathematical operations or transformations on tensors.

[0033] Common operators include convolution, pooling, fully connected, activation function, normalization, loss function, and so on. These operators, when combined and stacked, form the basic computing units of deep neural network models. AI chips can implement the inference and training processes of neural network models by executing a series of operators.

[0034] Currently, consideration has been given to using assembly language operator generation programs to develop and optimize operators, so as to convert the logic and functions of the operators into assembly instructions, so that the operators can adapt to the hardware architecture of artificial intelligence chips, for example, the parallel processing framework of general-purpose parallel graphics processing units (GPGPUs).

[0035] When developing and optimizing operators using assembly language operator generators, developers often need to use shared memory to store temporary data. Shared memory is a high-speed cache shared between threads on a GPU, which accelerates data access. Shared memory can be divided into multiple, equally sized, simultaneously accessible memory units, known as banks.

[0036] While using shared memory has many advantages, storing temporary data in it can also lead to bank conflicts. Specifically, bank conflicts occur when multiple threads simultaneously access different addresses in the same bank. These conflicts reduce the parallelism of data access and force access operations to be serialized, increasing latency and reducing operator performance.

[0037] Furthermore, for certain complex operators, shared memory often needs to be split and reused multiple times, which increases development complexity, easily introduces errors, and makes debugging and verification difficult. For example, in a matrix multiplication operator, the input tensor is typically divided into multiple small blocks, which are sequentially loaded into shared memory for calculation. After the calculation is complete, the results are written back to global memory, and the same shared memory is reused to load the next set of data. This split and reuse strategy requires developers to manually and precisely specify the boundaries of each split, the loading order, and the timing of reuse.

[0038] This method of manually specifying shared memory access policies by developers often leads to errors or performance degradation. For example, in the convolution operator, if the partitioning is too fine, it may lead to frequent memory synchronization overhead and degrade performance; if the partitioning is too coarse, it may exceed the capacity of the shared memory and trigger an overflow error.

[0039] Therefore, it is necessary to improve the current operator generation and optimization schemes to improve operator development efficiency, reduce the probability of errors, and optimize the performance of operators, so that the assembly code corresponding to the operator is more suitable for artificial intelligence chips.

[0040] To this end, according to an embodiment of the present disclosure, a method for compiling operators is proposed, including: obtaining an intermediate representation of the operator to be compiled, the intermediate representation (IR) of the operator to be compiled including at least an intermediate representation of a shared memory and an intermediate representation of an interactive operator, wherein the intermediate representation of the shared memory includes at least an attribute value indicating the size of the shared memory, and the interactive operator is associated with data interaction between the video memory and the shared memory; based on the size of the shared memory and the intermediate representation of the interactive operator, configuring an attribute value indicating an operator swizzleGran in the intermediate representation of the shared memory to generate a configured intermediate representation of the shared memory; based on the configured intermediate representation of the shared memory, updating the intermediate representation of the operator to be compiled, and compiling the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled.

[0041] Among them, the intermediate representation of the operator refers to an abstract description form between the source code and the target machine code, which can maintain the stability of the code structure in different optimization stages and provide space for hardware-specific optimization. The operator reordering granularity refers to the granularity of the operation when rearranging or reorganizing data in shared memory. The embodiment of the present disclosure optimizes the shared memory access strategy by automatically configuring the operator reordering granularity in the intermediate representation of the shared memory, thereby improving the efficiency and performance of operator development, reducing the burden on developers, reducing configuration errors and improving the adaptability of artificial intelligence chips.

[0042] In an optional aspect, the disclosed embodiments automatically determine the appropriate shared memory operator reordering granularity by jointly analyzing the shared memory size and the intermediate representation of interacting operators. This granularity is then configured in the shared memory intermediate representation, eliminating the need for developers to manually specify the operator reordering granularity. This not only reduces potential errors caused by manual configuration but also improves operator execution efficiency on AI chips by optimizing data access patterns.

[0043] In another optional aspect, the disclosed embodiments can be adapted for use in AI chips using hardware to automatically derive operator reordering granularity. This can make the reordering granularity determination process more efficient, reduce the impact of manual intervention on operator performance, and improve the stability of the overall development process.

[0044] Various embodiments of the present disclosure are described in detail through the following drawings.

[0045] Figure 1 A flowchart of a method 10 for compiling operators according to an embodiment of the present disclosure is shown.

[0046] For example, Figure 1 As shown, at least one embodiment of the present disclosure provides a method 10 for compiling an operator, which includes the following operations S110 to S140. Of course, the method 10 according to the embodiment of the present disclosure may also include more or fewer operations or steps, and the present disclosure is not limited thereto.

[0047] In operation S110, an intermediate representation of the operator to be compiled is obtained, where the intermediate representation of the operator to be compiled includes at least an intermediate representation of the shared memory and an intermediate representation of the interactive operator, wherein the intermediate representation of the shared memory includes at least a property value indicating the size of the shared memory, and the interactive operator is associated with data interaction between the video memory and the shared memory.

[0048] In operation S120 , based on the size of the shared memory and the intermediate representation of the interacting operators, an operator reordering granularity is configured in the intermediate representation of the shared memory to generate a configured intermediate representation of the shared memory.

[0049] In operation S130 , the intermediate representation of the operator to be compiled is updated based on the configured intermediate representation of the shared memory.

[0050] In operation S140 , the intermediate representation of the operator to be compiled is compiled to generate assembly code corresponding to the operator to be compiled.

[0051] The disclosed embodiments optimize the shared memory access strategy by automatically configuring the operator reordering granularity in the intermediate representation of the shared memory, thereby improving operator development efficiency and performance, reducing the burden on developers, reducing configuration errors, and improving the adaptability of artificial intelligence chips.

[0052] In an optional aspect, the disclosed embodiments automatically determine the appropriate shared memory operator reordering granularity or the size of the shared memory for a single transaction by jointly analyzing the shared memory size and the intermediate representation of the interacting operators. The granularity is then configured in the shared memory intermediate representation, eliminating the need for developers to manually specify the operator reordering granularity. This not only reduces potential errors caused by manual configuration but also improves the execution efficiency of operators on AI chips by optimizing data access patterns.

[0053] In another optional aspect, the disclosed embodiments can be adapted for use in AI chips using hardware to automatically derive operator reordering granularity. This can make the reordering granularity determination process more efficient, reduce the impact of manual intervention on operator performance, and improve the stability of the overall development process.

[0054] Next, refer to Figure 2 To further illustrate the operator to be compiled according to the embodiment of the present disclosure. Figure 2 An example of multiple operators to be compiled that constitute a neural network model according to an embodiment of the present disclosure is shown.

[0055] Optionally, the operator to be compiled can be any one of the multiple operators in the neural network model. Figure 2 As shown in the figure, a neural network model can include multiple nodes, where each node (also called an operator) represents a basic computing operation, such as matrix multiplication or activation function. The edges between nodes indicate the flow of data and define the dependencies and execution order between operations. The topological structure of the computational graph reflects the organization of these nodes and edges and determines how to perform computational tasks efficiently. The input node serves as the starting point of the calculation and accepts the input data of the model, while the output node produces the final calculation result. Figure 2 In the example, TMA (tensor read), conv2d (two-dimensional convolution), add (addition), relu (rectified linear unit), and concat (connection) nodes may all be operators to be compiled. The input data is calculated by each operator along the direction of the edge, and finally the output of the neural network model is obtained.

[0056] Optionally, operators to be compiled are converted into assembly code for specific hardware (such as graphics processing units (GPUs), tensor processing units (TPUs), neural network processing units (NPUs), etc.) through an assembly language operator generator. The assembly language operator generator generates efficient execution code based on the characteristics of the operator. For example, in a convolutional neural network (CNN), the conv2d operator is used to extract image features, where its input tensor may be image data and its output tensor is a feature map. The add operator may be used to implement a residual connection, where its input tensor comes from the computation results of two branches and its output tensor is their sum. The relu operator is used to introduce nonlinearity, where its input tensor is the result of the previous layer and its output tensor is the value processed by the activation function. The concat operator is used to concatenate multiple feature maps, where its input tensor is multiple feature maps and its output tensor is the concatenated result.

[0057] Optionally, when generating the assembly code corresponding to the operator to be compiled using an assembly language operator generator, the impact of the characteristics of the AI ​​chip on the compilation of the operator also needs to be considered. For example, an AI chip may have two storage areas: shared memory and video memory. Shared memory has fast access speed but small capacity, while video memory has large capacity but slow access speed. The operator to be compiled needs to optimize the data access mode based on the characteristics of shared memory and video memory. For example, the TMA operator loads data from video memory to shared memory, and then the conv2d operator loads data from shared memory for calculation, while the add operator may operate on intermediate results directly in shared memory. When generating code using the assembly language operator generator, the calculation logic and memory allocation will be adjusted according to the input and output tensor characteristics of the operator and the hardware architecture of the chip to improve execution efficiency.

[0058] Optionally, when generating the assembly code corresponding to the operator to be compiled using the assembly language operator generation program, method 10 according to the embodiment of the present disclosure can be used to optimize the data access pattern by utilizing the attribute value of the operator reordering granularity. This optimizes the shared memory access strategy, thereby improving operator development efficiency and performance, reducing the burden on developers, reducing configuration errors, and improving the adaptability of artificial intelligence chips.

[0059] Next, refer to Figure 3 To further illustrate the intermediate representation 302 of the operator to be compiled according to an embodiment of the present disclosure. Figure 3 FIG. 3 is a schematic diagram showing an intermediate representation 302 of an operator to be compiled according to an embodiment of the present disclosure.

[0060] Alternatively, an intermediate representation (IR) is an abstract description between source code and target machine code that maintains the stability of the code structure during different optimization phases, providing space for hardware-specific optimizations. In automatic code generation systems such as assembly language operator generators, IR serves as the basis for optimization delivery and code conversion, enabling assembly language operator generators to generate efficient execution code for specific AI chips. During operator development, IR typically includes information such as the operator's logical structure, data dependencies, and storage resource information, effectively describing the operator's organizational state before it is actually executed by hardware.

[0061] For example, Figure 3 The intermediate representation 302 of the operator to be compiled may include multiple intermediate representations at different levels responsible for different functions. These intermediate representations can be used to define or describe the operator's internal computational logic description, memory access mode, parallelization strategy, specific hardware optimization requirements, operator fusion and scheduling information, etc., which is not limited in this disclosure.

[0062] For example, the intermediate representation 302 of the operator to be compiled may include at least: an intermediate representation 304 of a shared memory, an intermediate representation 306 of an interactive operator, an intermediate representation 308 of a computing operation, and the like.

[0063] For example, the intermediate representation 304 of shared memory is an intermediate representation that describes a cache or programmable shared memory area in an artificial intelligence chip, which is shown as a SharedTensor class. In many artificial intelligence accelerator architectures, shared memory is a cache that can be accessed concurrently by multiple processing units, which is used to temporarily store intermediate calculation results and reduce access to video memory. The intermediate representation 304 of shared memory includes attribute information such as memory size and user data type. Before operation S120 is executed, the attribute value of the operator reordering granularity in the intermediate representation of shared memory can be empty. After operation S120 is executed, the attribute value of the operator reordering granularity can be configured to a specific value. Of course, the present disclosure is not limited to this.

[0064] Interaction operators refer to any operators involved in data interaction between video memory and shared memory among the operators to be compiled. They may be operators specifically used for data interaction between video memory and shared memory, or they may be operators used to calculate tensor data, but involve loading operations, storage operations, synchronization operations, etc. of tensor data. Interaction operators can also be presented using intermediate representations, for example Figure 3 The intermediate representation 306 of the interaction operator in . Of course, the present disclosure is not limited to this.

[0065] For example, if Figure 3As shown, during the computation process, interactive operators may involve loading data from video memory into shared memory (e.g., the operator Global2Shared), writing data from shared memory back to video memory (e.g., the operator Shared2Global), or synchronizing data from shared memory with data in video memory (not shown) to adapt to the architecture of the AI ​​chip and improve technical efficiency. For example, the intermediate representation of the interactive operator may also define the logical structure and data dependencies during the data interaction process to optimize memory access. This disclosure is not limited to this.

[0066] The intermediate representation 308 of the computing operation is an intermediate representation of the operator used in a specific computing process. Taking the intermediate representation of the multiplication operation (matmul) as an example, it is used to calculate the product of two matrices, such as C=A*B, where the shape of A is M*K, the shape of B is K*N, and the shape of C is M*N. In artificial intelligence chips, in order to speed up this operation, part of the data of A and B is usually loaded into shared memory, and then different parts of C are calculated in parallel by multiple computing units. Therefore, the intermediate representation 308 of the computing operation may implement the above process by calling the intermediate representation 304 of the shared memory and the intermediate representation 306 of the interactive operator described above. Of course, the present disclosure is not limited to this.

[0067] Next, refer to Figure 4 To further illustrate the shared memory intermediate representation 304 according to an embodiment of the present disclosure. Figure 4 A schematic diagram showing pseudo code of the shared memory intermediate representation 304 according to an embodiment of the present disclosure is shown.

[0068] like Figure 4 As shown, the shared memory intermediate representation 304 is a specific class that encapsulates shared memory-related information and operations. For example, the shared memory intermediate representation 304 includes the following public functions that can be called / accessed by external objects: the constructor SharedTensor(const Shape user_shape, DataType user_dataType), the function getSize() for returning the memory size, the function setSwizzleRefShape(const Shape user_swizzleRefShape) for configuring the operator swizzle shape, the function getSwizzleRefShape() for returning the operator swizzle shape, the function setSwizzleGran(int user_swizzleGranIn) for configuring the operator swizzle granularity, the function getSwizzleGran() for returning the operator swizzle granularity, and the function isSwizzle() const for checking whether the operator swizzle function is enabled.

[0069] like Figure 4 As shown, in addition to the properties that require configuration for the operator reordering granularity, the shared memory intermediate representation 304 (SharedTensor) also includes information about the user data shape (user_shape) and user data type (user_dataType), which must be configured during initialization. Specifically, the externally accessible constructor SharedTensor(constShape user_shape, DataType user_dataType) initializes the shared memory shape and data type. For example, the user_shape parameter specifies the shared memory shape used for user data, such as 16x16; the user_dataType parameter specifies the user data data type, such as a floating-point number. External objects can then use the getSize() function to obtain the shared memory size of the shared memory intermediate representation 304 (SharedTensor), which is the shared memory size of the current hardware.

[0070] Next, the function setSwizzleRefShape(const Shape user_swizzleRefShape), accessible to external objects for configuring the operator swizzle shape, can be used to configure the reference shape of the operator swizzle granularity in the shared memory intermediate representation 304. The reference shape of the operator swizzle granularity is also the size of the shared memory processed in a single transaction. The function setSwizzleRefShape also serves as an interface for enabling the hardware automatic derivation function. After enabling the hardware automatic derivation function, the hardware can automatically derive the operator swizzle granularity based on the reference shape of the operator swizzle granularity (i.e., the size of the shared memory processed in a single transaction). After configuration, the isSwizzle flag is set to true, indicating that the operator swizzle function is enabled. Furthermore, external objects can obtain the reference shape swizzleRefShape of the operator swizzle granularity in the shared memory intermediate representation 304 using the function getSwizzleRefShape(), which returns the operator swizzle shape.

[0071] External objects can also use the setSwizzleGran function to allow the analysis module to directly specify the operator reordering granularity (swizzleGran) in the shared memory intermediate representation 304. For example, if swizzleGran = 16, data is reordered in 16-byte blocks. After configuration, the isSwizzle flag is set to true. Furthermore, the getSwizzleGran() method can be used to obtain the operator reordering granularity property value in the shared memory intermediate representation 304.

[0072] In addition, the external object can also use isSwizzle() to determine whether the shared memory intermediate representation 304 has the operator reordering function enabled. The shared memory intermediate representation 304 can also include other functions or features that can be called by external objects, which will not be described one by one in this disclosure.

[0073] Optionally, the shared memory intermediate representation 304 includes the following private parameters that cannot be called / accessed by external objects: tensor_size, swizzleRefShape, swizzleGran, isSwizzle. These parameters can only be modified indirectly through the various public functions described above, thereby protecting the integrity and security of the data.

[0074] refer to Figure 4 The intermediate representation 304 of the shared memory according to the embodiment of the present disclosure can be used to support efficient computing operations in artificial intelligence chips. The intermediate representation 304 of the shared memory according to the embodiment of the present disclosure combines the characteristics of operators and shared memory, and provides a flexible operator rearrangement configuration method. Whether it is automatically inferring the hardware through reference shapes or directly specifying the granularity of operator rearrangement, the analysis module can optimize the data access pattern and thus improve computing performance. This intermediate representation 304 is applicable to a variety of computing tasks and can give full play to the hardware acceleration capabilities.

[0075] Next, refer to Figure 5 To further illustrate some optional details of method 10 according to an embodiment of the present disclosure. Figure 5 A schematic diagram of a method for compiling an operator according to an embodiment of the present disclosure is shown. It is worth noting that method 10 according to an embodiment of the present disclosure can be performed entirely by a device equipped with any assembly language operator generation program, or can be performed in conjunction with a device equipped with other analysis modules and a device equipped with any assembly language operator generation program, and the present disclosure is not limited thereto.

[0076] In operation S110, an intermediate representation of the operator to be compiled is obtained, where the intermediate representation of the operator to be compiled includes at least an intermediate representation of the shared memory and an intermediate representation of the interactive operator, wherein the intermediate representation of the shared memory includes at least a property value indicating the size of the shared memory, and the interactive operator is associated with data interaction between the video memory and the shared memory.

[0077] Previously referenced Figures 2 to 4The intermediate representation of the operator to be compiled, the shared memory intermediate representation, and the interactive operator intermediate representation are described in detail and will not be repeated here. Alternatively, in operation S110, the shared memory intermediate representation 304 can be determined by directly searching for the SharedTensor class in the intermediate representation of the operator to be compiled. By searching for this SharedTensor class, the shared memory intermediate representation can be quickly located.

[0078] However, as mentioned above, the intermediate representation of an interactive operator may involve a variety of scenarios. Therefore, in an optional embodiment, an analysis module (pass) may be used to perform a traversal analysis on the intermediate representation. The analysis module can be a module in the assembly language operator generation program, or another module independent of the assembly language operator generation program, used to perform specific analysis or transformation on the intermediate representation. For example, the analysis module can be used to perform a traversal analysis on the intermediate representation of the operator to be compiled to identify and extract all operators that involve interaction between shared memory and external video memory. These operators will serve as the intermediate representation of the interactive operator. For example, the analysis module can scan each operation (method) or function (function) in the intermediate representation of the operator to be compiled to find all code that reads data from video memory to shared memory, writes data in shared memory back to video memory, or synchronizes data in shared memory with data in video memory, and mark the operators with such code as the intermediate representation of the interactive operator.

[0079] As a specific example, the Tensor Memory Access (TMA) operator is used as an instance of the intermediate representation of the interaction operator. TMA is a general intermediate representation used to represent data read operations, especially data transfer between shared memory and external video memory. During the compilation process of the artificial intelligence chip, TMA can represent a specific data read operation, such as reading tensor data from the video memory and loading it into the shared memory. The design of TMA enables the assembly language operator generation program to more effectively optimize the data access pattern, thereby improving the overall computing performance. In one embodiment, the TMA may contain information such as the source address (location in the video memory), the target address (location in the shared memory), and the data size, which are used to generate the intermediate representation of the interaction operator. Of course, the present disclosure is not limited to this.

[0080] In operation S120 , based on the size of the shared memory and the intermediate representation of the interacting operators, an operator reordering granularity is configured in the intermediate representation of the shared memory to generate a configured intermediate representation of the shared memory.

[0081] Optionally, the operator reordering granularity attribute value may be affected by various factors, such as the hardware structure of the AI ​​chip, the size of shared memory, the execution logic of each operation in the interactive operator, the control logic for concurrent access, and so on. For example, if the shared memory is large, a larger operator reordering granularity can be selected to reduce the number of accesses to global video memory. If the shared memory is small, a smaller operator reordering granularity is required to ensure that data blocks can be fully loaded into the shared memory and avoid frequent memory swaps.

[0082] As another example, if the intermediate representation of an interacting operator indicates that it may need to access shared memory multiple times in a predetermined time period, the operator reordering granularity may be set to be roughly the same as the access range. For example, if an interacting operator may need to access 128 bytes of data continuously, the operator reordering granularity may be set to 128 bytes. Of course, due to the size of the shared memory, the operator reordering granularity may also be set to 64 bytes to strike a balance between satisfying the shared memory size limit and satisfying the execution performance of the intermediate representation of the interacting operator.

[0083] As another example, if multiple interacting operators may need to concurrently access shared memory, the operator reordering granularity may need to take into account the number of concurrent threads to set a relatively balanced operator reordering granularity to evenly distribute shared memory to each interacting operator. Of course, the present disclosure is not limited to this.

[0084] In addition, in an optional embodiment of the present disclosure, in operation S120, the execution granularity of the interacting operator is determined based on the intermediate representation of the interacting operator, and based on the size of the shared memory and the execution granularity of the interacting operator, the attribute value of the operator reordering granularity or the size of the shared memory for single processing is configured in the intermediate representation of the shared memory. Of course, the present disclosure is not limited to this.

[0085] Optionally, during the compilation and execution of an artificial intelligence chip, the execution granularity of an interactive operator refers to the amount of data or the size of a data block processed at one time by the hardware or software when executing the interactive operator. For example, in the operation of transferring data from video memory to shared memory, the execution granularity may represent the number of bytes read or written at a time. The execution granularity of the interactive operator and the operator reordering granularity will jointly affect the efficiency of memory access. Different interactive operators may have different execution granularities. Therefore, in an optional embodiment of the present disclosure, it is also considered to associate the execution granularity of the interactive operator with the operator reordering granularity to determine the attribute value of the operator reordering granularity.

[0086] Optionally, the shared memory reordering granularity can be derived based on the shared memory size and the execution granularity of the interacting operator, and configured in the shared memory's SwizzleGran attribute using the previously described method setSwizzleGran(int user_swizzleGranIn). Specifically, the operator reordering granularity typically needs to be coordinated with the execution granularity to ensure efficient data access. For example, if the intermediate representation of an interacting operator specifies processing 32*8=256 bytes of data at a time (8 represents the number of channels), the shared memory operator reordering granularity can be set to 32 bytes and recorded in the SharedTensor object's SwizzleGran attribute. This allows the assembly language operator generator and hardware to optimize data processing based on the operator reordering granularity.

[0087] Another way is to store the single processing size of the shared memory and pass it to the hardware, which will automatically analyze and determine the appropriate reordering granularity. For example, when the TMA operator is used as an interactive operator, the intermediate representation of the TMA operator may contain information that the single processing size is 64*8 bytes. Through the aforementioned method setSwizzleRefShape (const Shapeuser_swizzleRefShape), the information of a single processing size of 64*8 bytes can be configured in the Shape attribute of the SharedTensor object. In the subsequent compilation process of the interactive operator, the assembly language operator generation program configures this size on the assembly language instruction, and the subsequent hardware uses this information during the execution of the instruction to automatically deduce the operator reordering granularity and execute it. This method reduces the burden on the assembly language operator generation program and allows flexible adjustment of the hardware.

[0088] Optionally, the attribute value of the operator reordering granularity can also be configured in the intermediate representation of the interactive operator to generate the configured intermediate representation of the interactive operator; or based on the configured intermediate representation of the shared memory, the data size of a single process can be configured in the intermediate representation of the interactive operator to generate the configured intermediate representation of the interactive operator. Specifically, the intermediate representations of all interactive operators can contain a SwizzleGran attribute or a SwizzleRefShape attribute to record the operator reordering granularity. For example, in one embodiment, the intermediate representation of the TMA not only specifies a single processing size of 128*64 bytes, but also configures the SwizzleRefShape to be equal to 128*64 bytes through the analysis module. This attribute value is written into the intermediate representation of the TMA during the compilation process for subsequent optimization and hardware execution.

[0089] Optionally, both the attribute value of the operator reordering granularity in the shared memory and the attribute value of the operator reordering granularity in the interactive operator may change dynamically as the operator to be compiled is executed. For example, the attribute value of the operator reordering granularity may match a certain time period or a certain operation and change as the operation changes. This disclosure is not limited to this.

[0090] In operation S130 , the intermediate representation of the operator to be compiled is updated based on the configured intermediate representation of the shared memory.

[0091] Specifically, as the analysis module or the compilation module configures the attribute value of the operator reordering granularity in the intermediate representation of the shared memory, the intermediate representation of the operator to be compiled also needs to be adaptively updated. In the process of generating assembly code using the intermediate representation of the operator to be compiled, if any operator in the intermediate representation of the operator to be compiled involves any shared memory operation, the assembly language operator generation program may configure the shared memory operator reordering granularity attribute to the corresponding instruction, thereby improving data access efficiency.

[0092] Optionally, in one embodiment of the present disclosure, the intermediate representation of the operator to be compiled may be updated based on both the configured intermediate representation of the interactive operator and the configured intermediate representation of the shared memory. Of course, the present disclosure is not limited to this.

[0093] In operation S140 , the intermediate representation of the operator to be compiled is compiled to generate assembly code corresponding to the operator to be compiled.

[0094] In operation S140, the assembly language operator generation program can convert the intermediate representation into an instruction set executable by the hardware (GPU, TPU, NPU), ensuring that the hardware can perform the computing task as expected. More specifically, in the process of generating assembly code using the intermediate representation of the operator to be compiled, since the intermediate representation of the operator involving any shared memory operation includes the operator reordering granularity attribute, the assembly language operator generation program can configure the operator reordering granularity attribute of the shared memory to the corresponding hardware instructions (for example, assembly code), so that the hardware can access the shared memory by operator reordering, thereby improving data access efficiency. Of course, the present disclosure is not limited to this.

[0095] When using IR to generate assembly code, if the IR uses shared memory, the shared memory reordering granularity attribute is configured for the IR so that the hardware accesses the shared memory through operator reordering, improving performance while ensuring correctness.

[0096] Optionally, each operator in the operators to be compiled can be divided into two categories: operators using shared memory and operators not using shared memory. For those intermediate representations of operators using shared memory, operation S140 further includes: configuring the intermediate representation of the operator using shared memory with an attribute value of the granularity of reordering the operators in the shared memory, so that the corresponding assembly instructions for the intermediate representation of the operator using shared memory include the attribute value of the granularity of reordering the operators in the shared memory. Of course, the present disclosure is not limited to this.

[0097] For example, when generating assembly code, if the intermediate representation of an operator using shared memory is configured with a 16-byte operator reordering granularity (with 8 channels) or a single-process shared memory size of 128*64, the assembly language operator generator will embed this operator reordering granularity attribute in the corresponding load instruction. When the hardware executes this instruction, it loads data into shared memory in 128-byte units, reducing memory access latency. Similarly, for store instructions, the assembly language operator generator can optimize the data write mode from shared memory back to video memory by configuring the operator reordering granularity. Similarly, arithmetic (ALU) instructions can also be configured with the operator reordering granularity attribute. In this way, assembly code can fully utilize the hardware characteristics of AI chips, improving computing performance while ensuring correctness.

[0098] In some embodiments of the present disclosure, the intermediate representation of the operator to be compiled may include an intermediate representation for slicing (view) shared memory or an intermediate representation for reusing (make tensor) shared memory. "Slicing" shared memory refers to dividing a shared memory area into multiple smaller sub-areas so that multiple processes or threads can access different parts at the same time. "Reusing" shared memory refers to reusing the same shared memory area between multiple processes or tasks to reduce memory allocation overhead and improve memory usage efficiency. In both cases, the intermediate representation of the existing shared memory may no longer be suitable for the sliced ​​shared memory or the reused shared memory, and a new dedicated intermediate representation needs to be created for these sliced ​​shared memories or reused shared memories.

[0099] In operation S140, for the intermediate representation of the operator used to split the shared memory or the intermediate representation of the operator used to reuse the shared memory, the assembly language operator generation program will create a dedicated intermediate representation of the shared memory for the intermediate representation of the operator used to split the shared memory or the intermediate representation of the operator used to reuse the shared memory based on the existing shared memory intermediate representation SharedTensor. The dedicated intermediate representation of the shared memory is specific to the intermediate representation of the operator used to split the shared memory or the intermediate representation of the operator used to reuse the shared memory. These dedicated intermediate representations can also be named SharedTensor. Of course, the present disclosure is not limited to this.

[0100] These dedicated intermediate representations of shared memory also need to configure the attribute value of the operator reordering granularity. Operation S140 may also include: in response to determining that the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory has been configured, not changing the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory; and in response to determining that the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory has not been configured, configuring the attribute value of the operator reordering granularity in the dedicated intermediate representation of the shared memory to the attribute value of the operator reordering granularity in the intermediate representation of the shared memory. Of course, the present disclosure is not limited to this.

[0101] Specifically, since the dedicated intermediate representation of the shared memory may have been declared in the intermediate representation of the operator to be compiled, the operator reordering attributes of this type of dedicated intermediate representation may have been configured during the previous analysis module traversal process. In this case, the assembly language operator generation program will no longer change the operator reordering granularity attributes of this type of shared memory dedicated intermediate representation. If the operator reordering attributes of this type of dedicated intermediate representation have not been configured during the previous analysis module traversal process, the operator reordering attributes of this type of dedicated intermediate representation will be configured as the attribute value of the operator reordering granularity in the intermediate representation of the shared memory, that is, inherit the attribute value of the operator reordering granularity in the existing dedicated intermediate representation of the shared memory. In this way, the consistency of the memory access pattern can be maintained. Of course, the present disclosure is not limited to this.

[0102] In summary, the embodiments of the present disclosure automatically optimize the shared memory access strategy by automatically configuring the operator reordering granularity in the intermediate representation of the shared memory, thereby improving operator development efficiency and performance, reducing the burden on developers, reducing configuration errors, and improving the adaptability of artificial intelligence chips.

[0103] Figure 6 FIG. 6 is a schematic diagram of an apparatus 60 for compiling operators according to an embodiment of the present disclosure.

[0104] like Figure 6As shown, the apparatus 60 for compiling an operator according to an embodiment of the present disclosure includes a first module 61 , a second module 62 , a third module 63 , and a fourth module 64 .

[0105] The first module 61 can be configured to: obtain an intermediate representation of the operator to be compiled, wherein the intermediate representation of the operator to be compiled includes at least an intermediate representation of the shared memory and an intermediate representation of the interactive operator, wherein the intermediate representation of the shared memory includes at least an attribute value indicating the size of the shared memory, and the interactive operator is associated with the data interaction between the video memory and the shared memory.

[0106] The second module 62 is configured to: configure the operator reordering granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interacting operator, so as to generate the configured intermediate representation of the shared memory.

[0107] The third module 63 is configured to update the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory.

[0108] The fourth module 64 is configured to compile the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled.

[0109] In addition, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory, wherein a computer executable program is stored in the memory, and when the computer executable program is executed by the processor, the above-mentioned method of compiling an operator is executed.

[0110] The processor in the embodiments of the present disclosure can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It can implement or execute the various methods, operations, and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor can be a microprocessor or any conventional processor, including an X86 architecture or an ARM architecture.

[0111] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0112] For example, the method or device according to the embodiment of the present disclosure can also be used by Figure 7 The architecture of the computing device 3000 shown in FIG. Figure 7 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for processing and / or communication of the method provided in the present disclosure, as well as program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 7 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 7 One or more components of a computing device are shown.

[0113] According to yet another aspect of the present disclosure, a computer-readable storage medium is provided. Figure 8 A schematic diagram of a storage medium 4000 according to the present disclosure is shown.

[0114] like Figure 8As shown, the computer-readable storage medium 4020 stores computer-readable instructions 4010. When executed by a processor, the computer-readable instructions 4010 can perform the methods described with reference to the above figures according to the embodiments of the present disclosure. The computer-readable storage medium in the embodiments of the present disclosure can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0115] The present disclosure also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a method according to the present disclosure.

[0116] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0117] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuitry, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the present disclosure are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented, as non-limiting examples, in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or a controller or other computing device, or some combination thereof.

[0118] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art will appreciate that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A method for compiling an operator, characterized in that: The method comprises: Obtaining an intermediate representation of an operator to be compiled, the intermediate representation of the operator to be compiled including at least an intermediate representation of a shared memory and an intermediate representation of an interactive operator, wherein the intermediate representation of the shared memory includes at least a property value indicating a size of the shared memory, and the interactive operator is associated with data interaction between a video memory and the shared memory; Based on the size of the shared memory and the intermediate representation of the interacting operators, configure an operator reordering granularity in the intermediate representation of the shared memory to generate a configured intermediate representation of the shared memory; Based on the configured intermediate representation of the shared memory, updating the intermediate representation of the operator to be compiled, and Compiling the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled, The intermediate representation of the operator is an abstract description form between the source code and the target machine code.

2. The method according to claim 1, wherein in, Configuring the operator reordering granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interacting operators includes: Determining the execution granularity of the interaction operator based on the intermediate representation of the interaction operator, and Based on the size of the shared memory and the execution granularity of the interactive operators, a property value of the operator reordering granularity or the size of the shared memory for single processing is configured in the intermediate representation of the shared memory.

3. The method according to claim 1, wherein The method further comprises: Configuring a property value of the operator reordering granularity in the intermediate representation of the interaction operator to generate a configured intermediate representation of the interaction operator; or Based on the configured intermediate representation of the shared memory, a data size for single processing is configured in the intermediate representation of the interaction operator to generate the configured intermediate representation of the interaction operator.

4. The method according to claim 3, wherein in, The updating of the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory includes: Based on the configured intermediate representation of the interactive operator and the configured intermediate representation of the shared memory, the intermediate representation of the operator to be compiled is updated.

5. The method according to claim 1, wherein in, The intermediate representation of the operator to be compiled includes an intermediate representation of an operator for splitting a shared memory or an intermediate representation of an operator for reusing a shared memory, and compiling the intermediate representation of the operator to be compiled includes: Creating a dedicated intermediate representation of the shared memory according to the intermediate representation of the shared memory, where the dedicated intermediate representation of the shared memory is specific to the intermediate representation of the operator for slicing the shared memory or the intermediate representation of the operator for reusing the shared memory; In response to determining that the property value of the operator reordering granularity in the dedicated intermediate representation in the shared memory has been configured, not changing the property value of the operator reordering granularity in the dedicated intermediate representation in the shared memory; and In response to determining that the property value of the operator reordering granularity in the dedicated intermediate representation in the shared memory is not configured, the property value of the operator reordering granularity in the dedicated intermediate representation in the shared memory is configured as the property value of the operator reordering granularity in the intermediate representation in the shared memory.

6. The method according to claim 1, wherein in, The intermediate representation of the operator to be compiled includes an intermediate representation of an operator for using a shared memory, and compiling the intermediate representation of the operator to be compiled includes: The attribute value of the operator reordering granularity of the shared memory is configured for the operator intermediate representation for using the shared memory, so that the corresponding assembly instruction block for using the operator intermediate representation for using the shared memory includes the attribute value of the operator reordering granularity of the shared memory.

7. The method according to claim 1, wherein in, The obtaining of the intermediate representation of the operator to be compiled includes: The intermediate representation of the operator to be compiled is traversed and analyzed by the analysis module to obtain the intermediate representation of the interaction operator.

8. A device for compiling an operator, characterized in that The device comprises: A first module is configured to: obtain an intermediate representation of an operator to be compiled, wherein the intermediate representation of the operator to be compiled includes at least an intermediate representation of a shared memory and an intermediate representation of an interactive operator, wherein the intermediate representation of the shared memory includes at least a property value indicating a size of the shared memory, and the interactive operator is associated with data interaction between a video memory and the shared memory; A second module is configured to: configure an operator reordering granularity in the intermediate representation of the shared memory based on the size of the shared memory and the intermediate representation of the interacting operator, so as to generate a configured intermediate representation of the shared memory; A third module is configured to: update the intermediate representation of the operator to be compiled based on the configured intermediate representation of the shared memory; and The fourth module is configured to: compile the intermediate representation of the operator to be compiled to generate assembly code corresponding to the operator to be compiled, The intermediate representation of the operator is an abstract description form between the source code and the target machine code.

9. An electronic device, characterized in that: The electronic device comprises: processor; and A memory, wherein a computer executable program is stored in the memory, and when the computer executable program is executed by the processor, the method for compiling an operator according to any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method for compiling an operator according to any one of claims 1 to 7 is implemented.

Images