Code Generation Method, Apparatus, Electronic Device, and Computer-Readable Storage Medium

Abstract

A code generation method, a code generation device, an electronic device, and a computer-readable storage medium. The code generation method includes: obtaining a combined operator for which code is to be generated, the combined operator including a predefined operator and a custom operator; obtaining operator template information for the custom operator; and based on the template information, performing a fusion operation on the predefined operator and the custom operator, and obtaining, from the fused combined operator, fused operator code corresponding to the combined operator. This method can reduce the memory access bandwidth pressure and latency, improve the code parallelization ability, and thus enhance the operation efficiency and performance.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a code generation method, a code generation device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Computational models such as neural network models include a variety of operators, some of which are predefined operators, that is, commonly used operators that have been defined. In addition, in some cases, users need to customize operators. During the process of generating model code, the code of the customized operator needs to be compiled. Summary of the Invention

[0003] At least one embodiment of the present disclosure provides a code generation method, including: obtaining a combined operator for which code is to be generated, where the combined operator includes a predefined operator and a customized operator; obtaining operator template information for the customized operator; and based on the operator template information, performing a fusion operation on the predefined operator and the customized operator, and obtaining fused operator code corresponding to the combined operator from the fused combined operator.

[0004] For example, the code generation method provided by an embodiment of the present disclosure further includes: detecting whether the customized operator meets the fusion condition; where, in response to the customized operator meeting the fusion condition, performing the operations of obtaining the operator template information for the customized operator, and based on the operator template information, performing a fusion operation on the predefined operator and the customized operator, and obtaining fused operator code corresponding to the combined operator from the fused combined operator; where, in response to the customized operator not meeting the fusion condition, separately compiling the predefined operator and the customized operator to obtain non-fused operator code corresponding to the combined operator.

[0005] For example, in the code generation method provided by an embodiment of the present disclosure, the meeting the fusion condition includes meeting at least one of the following conditions: the type of the customized operator is an operator type that supports a fusion operation; the hardware resource allocation meets the fusion requirement; the granularity of the customized operator matches the granularity of the predefined operator.

[0006] For example, in the code generation method provided by an embodiment of the present disclosure, operator template information for the custom operator is obtained, and based on the operator template information, a fusion operation is performed on the predefined operator and the custom operator, and the fused operator code corresponding to the fused joint operator is obtained from the fused joint operator, including: generating assembly code according to the joint operator and obtaining the operator template information, where the assembly code includes the code corresponding to the predefined operator; generating operation code for implementing the calculation function of the custom operator according to the operator template information; and combining the assembly code and the operation code based on the operator template information to obtain the fused operator code corresponding to the joint operator.

[0007] For example, in the code generation method provided by an embodiment of the present disclosure, the operator template information includes the position information of the operation code corresponding to the custom operator in the fused operator code; combining the assembly code and the operation code based on the operator template information includes: obtaining a template operator corresponding to the custom operator with the operation code segment based on the operator template information; and setting the obtained template operator at the corresponding position in the sequence of the assembly code based on the position information.

[0008] For example, in the code generation method provided by an embodiment of the present disclosure, the assembly code further includes code for data loading, data storage, and / or data synchronization for the template operator and corresponding to the custom operator.

[0009] For example, in the code generation method provided by an embodiment of the present disclosure, the operator template information further includes the template type, template parameters, and / or register information for the template operator.

[0010] For example, the code generation method provided by an embodiment of the present disclosure further includes: performing a code optimization operation on the fused operator code.

[0011] For example, in the code generation method provided by an embodiment of the present disclosure, the code optimization operation includes: optimizing instruction scheduling and optimizing register usage.

[0012] At least one embodiment of the present disclosure provides a code generation device, including an acquisition module and a fusion module. The acquisition module is configured to acquire a joint operator for which code is to be generated, where the joint operator includes a predefined operator and a custom operator; the fusion module is configured to acquire operator template information for the custom operator; and perform a fusion operation on the predefined operator and the custom operator based on the operator template information, and obtain the fused operator code corresponding to the joint operator from the fused joint operator.

[0013] For example, the code generation device provided in an embodiment of the present disclosure further includes a detection module configured to detect whether the custom operator meets the fusion condition; wherein, in response to the custom operator meeting the fusion condition, perform the operation of obtaining the operator template information for the custom operator, and based on the operator template information, perform a fusion operation on the predefined operator and the custom operator, and obtain the fused operator code corresponding to the combined operator from the fused combined operator; wherein, in response to the custom operator not meeting the fusion condition, separately compile the predefined operator and the custom operator to obtain the non-fused operator code corresponding to the combined operator.

[0014] For example, in the code generation device provided in an embodiment of the present disclosure, the fusion module includes an operator screening module, a first code generation control module, a second code generation control module, and an operator fusion module. The operator screening module is configured to receive the combined operator; the first code generation control module is configured to generate assembly code according to the combined operator and obtain the operator template information, wherein the assembly code includes the code corresponding to the predefined operator; the second code generation control module is configured to use the operator template information to generate operation code for implementing the calculation function of the custom operator; the operator fusion module is configured to combine the assembly code and the operation code based on the operator template information to obtain the fused operator code corresponding to the combined operator.

[0015] For example, the code generation device provided in an embodiment of the present disclosure further includes an optimization module configured to perform code optimization operations on the fused operator code.

[0016] At least one embodiment of the present disclosure provides an electronic device, including a processor; a memory including one or more computer program modules; wherein, the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the code generation method provided in any embodiment of the present disclosure.

[0017] At least one embodiment of the present disclosure provides a computer-readable storage medium storing non-transitory computer-readable instructions that, when executed by a computer, implement the code generation method provided in any embodiment of the present disclosure. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present disclosure and do not limit the present disclosure.

[0019] Figure 1 The flowchart of a code generation method provided by at least one embodiment of the present disclosure is shown;

[0020] Figure 2 The schematic diagram of a device for implementing the code generation method provided by at least one embodiment of the present disclosure is shown;

[0021] Figure 3 The flowchart of a fusion operation provided by at least one embodiment of the present disclosure is shown;

[0022] Figure 4 The schematic diagram of a code generation device provided by at least one embodiment of the present disclosure is shown;

[0023] Figure 5 The schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure is shown;

[0024] Figure 6 The schematic block diagram of another electronic device provided by at least one embodiment of the present disclosure is shown; and

[0025] Figure 7 The schematic diagram of a computer-readable storage medium provided by at least one embodiment of the present disclosure is shown. Detailed implementation manners

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present disclosure. Obviously, the described embodiments are some, rather than all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the described embodiments of the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0027] Unless otherwise defined, the technical terms or scientific terms used in this disclosure shall have the ordinary meanings as understood by those of ordinary skill in the art to which this disclosure pertains. The terms "first", "second" and similar terms used in this disclosure do not denote any order, quantity or importance, but are only used to distinguish different components. Similarly, the terms such as "a", "an" or "the" do not denote a limitation of quantity, but mean that there is at least one. The terms such as "comprising" or "including" mean that the elements or items appearing before the term cover the elements or items listed after the term and their equivalents, without excluding other elements or items. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0028] Taking a neural network model as an example, in the process of building and deploying most neural network models, users often need to use various complex custom operators such as non-linear activation functions for experiments and tests, and neural network models usually include predefined operators.

[0029] The inventors of this disclosure have noticed that the model code generation can be achieved by independently compiling the custom operator and then simply combining it with the predefined operator (such as a convolution operator or a matrix multiplication operator). The code of the predefined operator generated based on this method and the code of the user-defined operator independently compiled are independent and serial. The data interaction between them often needs to be implemented by using a memory access method. This memory access method includes that when data is transferred between operations in different steps, the previous step (such as the step of performing an operation using a predefined operator) will first write the data (such as the operation result) from the register to the memory, and in the next step (such as the step of performing an operation using a custom operator), these data will be read from the memory into the register to perform the operation of this next step based on the data. And this method results in a strong synchronization method between the two operators, that is, after all the calculations of one operator are completed to obtain all the calculation results, the calculation of the next operator is carried out, which causes the hardware modules to not be fully parallel, restricting the execution efficiency and performance of the code.

[0030] At least one embodiment of this disclosure provides a code generation method, a code generation device, an electronic device and a computer-readable storage medium. The code generation method includes: obtaining a combined operator for which code is to be generated, the combined operator including a predefined operator and a custom operator; obtaining operator template information for the custom operator; and based on the template information, performing a fusion operation on the predefined operator and the custom operator, and obtaining the fused operator code corresponding to the combined operator from the fused combined operator.

[0031] This code generation method can reduce the memory access bandwidth pressure and latency, improve the code parallelization ability, and thus enhance the running efficiency and performance.

[0032] Figure 1 The flowchart of a code generation method provided by at least one embodiment of the present disclosure is shown.

[0033] As Figure 1 shown, the code generation method may include steps S110 to S130.

[0034] Step S110: Obtain the combined operator of the code to be generated, where the combined operator includes predefined operators and custom operators.

[0035] Step S120: Obtain the operator template information for the custom operator.

[0036] Step S130: Based on the operator template information, perform a fusion operation on the predefined operator and the custom operator, and obtain the fused operator code corresponding to the combined operator from the fused combined operator.

[0037] For example, the code generation method of the embodiment of the present disclosure can be applied to the application scenario of code generation in the artificial intelligence chip software stack.

[0038] Figure 2 The schematic diagram of a device for implementing the code generation method provided by at least one embodiment of the present disclosure is shown. As Figure 2 shown, the device includes a central scheduler 210, and the central scheduler 210 includes a detector 201. The detector 201 may include an operator source detection module, and the operator source detection module is used to detect the types of multiple operators included in the combined operator. The operator types include predefined operators or custom operators. The predefined operators may include commonly used (or general) operators in the field. For example, the code of the predefined operator can be written in advance (even compiled) and stored in the operator library for future calls. For example, the predefined operator can be stored in the operator library in the form of a template operator (i.e., a templated operator) that conforms to a predetermined form of programming template. In some embodiments, the predefined operator may further include an operator generated by the operator library according to user requirements. Whether it is the code written in advance or the code instantaneously generated by the operator library, the code generated by the operator library can be understood as a predefined operator.

[0039] Taking the neural network field as an example, the predefined operators may include a convolution operator and a matrix multiplication operator, etc.; the custom operator is a new operator customized by the user.

[0040] For example, a combined operator can be an operator formed by combining two or more operators to perform a specific operation. After obtaining the combined operator, operator source detection can be performed first. There may be three cases for the combined operator. In the first case, all the operators included in the combined operator are predefined operators. In this case, there is no need to fuse custom operators and predefined operators, so there is no need to perform subsequent steps, and the code of the predefined operator can be directly obtained from the operator library using the code generator. In the second case, all the operators included in the combined operator are custom operators. In this case, there is also no need to fuse custom operators and predefined operators, so there is no need to perform subsequent steps, and the compiler can be called to directly compile the custom operators to generate the code of these custom operators. In the third case, the combined operator includes both custom operators and predefined operators. In this case, fusion of the two types of operators is required, so subsequent operations can be performed on this combined operator. For example, by pre-labeling the predefined operator and the custom operator in advance, it is convenient for subsequent operator source detection; or, a list of predefined operators can be maintained in advance for operator source detection. The embodiments of the present disclosure do not limit the methods and means for operator source detection.

[0041] For example, taking the combined operator CBX in a convolutional neural network as an example, where C represents the convolution operator, which can be operated by a proprietary module and predefined by the code generator in the operator library; B represents the Batch Normalization operator, which can be operated by a general-purpose ALU (Arithmetic and Logic Unit) and predefined by the code generator in the operator library; X represents the activation function, which represents a user-defined operator in this embodiment. For example, an operator corresponding to a specific selected activation function. In the step of operator source detection, CBX can be regarded as a combined operator including a custom operator and a predefined operator, and deep fusion can be performed.

[0042] For example, after determining that the combined operator includes a predefined operator and a custom operator, between step S110 and step S120, it may further include: detecting whether the custom operator meets the fusion condition. If the custom operator meets the fusion condition, the custom operator and the predefined operator can be fused using a programming template in order to obtain higher-performance fused code. For example, Figure 2 The shown detector 201 may further include a fusion detection template, and the fusion detection module is used to detect whether the custom operator meets the fusion condition.

[0043] For example, in some embodiments, meeting the fusion condition may include: the type of the custom operator being an operator type that supports fusion operations. For example, an operator type table may be preset, and the operator types involved in the operator type table are operator types supported by the programming template. For example, if the operator type table contains, for example, operator types for per-data-point operations and does not contain operator types that require accumulation in space, then continuing with the above example, if operator X is an operator for per-data-point operations, it belongs to the operator types supported by the programming template, and if operator X is an operator that requires accumulation in space, it does not belong to the operator types supported by the programming template.

[0044] For example, in some embodiments, meeting the fusion condition may include: the allocation of hardware resources meeting the fusion requirements. For example, it may be detected whether the hardware resources required for the execution of the fused code can be satisfied. In some embodiments, it may also include whether the hardware resources required for the fusion process can be satisfied. Hardware resources include, for example, register resources. Since the custom operator and the predefined operator are deeply coupled (i.e., fused) and share register resources, it is necessary to detect whether the user's algorithm and resource allocation meet the resource requirements necessary for template coupling. For example, the programming template expects the user to minimize the control of hardware resources and hand over most of the hardware resource allocation to the software stack to utilize the software stack for flexible allocation and optimization of hardware resources. If the user specifies certain hardware resource restrictions, it may not meet the requirements of the programming template for achieving higher performance, and the custom operator X will be separately taken out to the compiler for compilation to obtain the corresponding code.

[0045] For example, in some embodiments, meeting the fusion condition may include: the granularity of the custom operator matching the granularity of the predefined operator.

[0046] For example, the matching of the operator granularities can be understood as the same operator granularities. The operator granularity can be understood as the data granularity or the computation granularity of the operator. For a matrix with dimensions [N, H, W], if both operators process data of size [1, 1, W] each time they calculate, then the granularities of the two operators are the same. If one operator calculates and processes data of size [1, H, W], while the other operator calculates and processes data of size [1, H, W], then the granularities of the two operators are different, and the granularity is related to the algorithm. N, H, and W can respectively represent, for example, the three dimensions of data such as an image. N represents batch, H represents height, and W represents width.

[0047] For example, in response to the custom operator satisfying the fusion condition, step S120 is executed. In response to the custom operator not satisfying the fusion condition, the code of the predefined operator is obtained respectively, and the custom operator is compiled to obtain the corresponding code. For example, these two types of codes are simply combined (for example, the custom operator code and the predefined operator code are independent and serial) to obtain the non-fused operator code corresponding to the combined operator.

[0048] For example, all or part of the above three fusion conditions can be adopted. In some embodiments, step S120 is executed when all of the above three fusion conditions are satisfied, and the fusion operation is performed using a programming template. Otherwise, the custom operator and the predefined operator are independently compiled and then simply combined. In other embodiments, only one or two of the above three fusion conditions can be detected, and step S120 is executed when the one or two conditions are satisfied. Otherwise, the custom operator and the predefined operator are independently compiled and then simply combined.

[0049] For example, several general operator template information can be defined. The operator template information is set with general information for realizing operator coupling as a framework for parallel programming code. The operator template information includes, for example, template type, parameters, register information, position label information, encapsulation information, etc. For example, non-general information such as the custom algorithm part of the custom operator can be combined with the operator template information to obtain a template operator for the custom operator. The template operator for the custom operator is combined with the template operator for the predefined operator, thereby realizing the fusion (also referred to as coupling) of the custom operator and the predefined operator. For example, the custom operator can be converted into a corresponding template operator based on the operator template information. The template operator can encapsulate the algorithm part of the custom operator and the information for coupling connection with the predefined operator, and then the template operator corresponding to the custom operator is fused into the code including one or more template operators for the predefined operator.

[0050] For example, the fusion operation of a custom operator and a predefined operator can be understood as integrating the algorithmic parts of these two operators into one operator, forming a data interface externally. For the code of the fused operator, during operation, after the data is retrieved from the storage device, the data is processed based on the code of the fused operator, and after all the code of the fused operator is executed, it is output to the storage device, that is, the data is output to the storage device after passing through all the algorithms involved in the fused operator (the overall operator formed by fusing the predefined operator and the custom operator). The operators involved in the fusion (including the custom operator and the predefined operator) can interact with each other through the registers inside the arithmetic unit. The intermediate calculation result of one operator (such as a predefined operator) can be directly output to another operator (such as a custom operator) without the memory access process of outputting the intermediate calculation result and then reading it again. That is to say, the data interaction between the custom operator and the predefined operator no longer needs to be achieved through memory access. Moreover, through this method, the operation of the next operator can start after one operator has obtained partial results, without waiting for the previous operator to obtain all the calculation results, improving the operation efficiency.

[0051] The code generation method according to at least one embodiment of the present disclosure enables the user's custom algorithm and the predefined algorithm to be more deeply integrated compared to simple combination through the defined operator template information. It can reduce the memory access bandwidth pressure and latency, improve the code parallelization ability, and thus enhance the operation efficiency and performance.

[0052] Figure 3 The flowchart of the fusion operation provided by at least one embodiment of the present disclosure is shown. As Figure 3 shown, for example, step S130 may include steps S131 to S133:

[0053] Step S131: Generate assembly code and obtain operator template information according to the combined operator. The assembly code includes the code corresponding to the predefined operator.

[0054] Step S132: Generate operation code for implementing the calculation function of the custom operator according to the operator template information.

[0055] Step S133: Combine the assembly code and the operation code based on the operator template information to obtain the code of the fused operator corresponding to the combined operator.

[0056] As Figure 2As shown, the central scheduler 210 may further include an operator filter 202, and the apparatus for implementing the code generation method may further include a first code generation controller 203, a second code generation controller 204, and an operator fuser 205. For example, after the detector 201 determines that the combined operator CBX meets the fusion requirements, the combined operator CBX may be sent to the operator filter 202. After receiving the combined operator CBX, the operator filter transmits the combined operator CBX to the first code generation controller and labels X as a custom operator. The first code generation controller generates assembly code ASM code according to the combined operator. The assembly code may include all the codes of predefined operators (such as Code Section 0, Code Section 1, …, Code Section N, where N is a positive integer), for example, including the codes of predefined operators C and B. In addition, the assembly code may further include codes for data loading, data storage, and / or data synchronization for the template operator corresponding to the custom operator X. The assembly code does not include the operation code of the custom operator X.

[0057] For example, the first code generation controller 203 may further obtain a template information data structure table for the custom operator X (i.e., the illustrated template information). The template information data structure table may include multiple members, and each member represents an operator template information. For example, the operator template information may include the template type, template parameters, and / or register information, etc. for the template operator.

[0058] For example, the first code generation controller 203 returns the assembly code and the operator template information to the operator filter 202. The operator filter 202 calls the second code generation controller 204 according to the operator template information. The second code generation controller 204 compiles and generates operation codes corresponding to each operator template information (such as Template 0, Template 1, …, Template M, where M is a positive integer) and returns them to the operator filter 202. The operator filter calls the operator fuser 205 according to the obtained codes and the operator template information to obtain the fused operator code corresponding to the combined operator by using the operator fuser 205.

[0059] For example, the operator template information further includes the position information of the operation code corresponding to the custom operator in the fused operator code. Based on the operator template information, the operation code segment can be used to obtain the template operator corresponding to the custom operator, and based on the position information, the obtained template operator can be set at the corresponding position in the sequence of the fused assembly code.

[0060] For example, a label for the custom operator X can be generated in the assembly code. The operator template information table may include this label information. The assembly code with the label set is, for example:

[0061] C-Block0

[0062] B-Block0

[0063] X-0

[0064] C-Block1

[0065] B-Block1

[0066] X-1

[0067] Among them, C-Block0 and C-Block1 are the template operators of operator C in the corresponding code blocks (Block0 or Block1); B-Block0 and B-Block1 are the operator templates of operator B in the corresponding code blocks (Block0 or Block1); X-0 and X-1 are the labels corresponding to the custom operator X, used to indicate the positions of the custom operator X in the assembly code of the corresponding code blocks (Block0 or Block1).

[0068] For example, after the operator filter 202 obtains the assembly code and the operator template information table of CBX, the compiler (the second code generation controller) can be called to obtain the operation code of the custom operator X, and the operation code of the custom operator X can be encapsulated into the template operators X-Block0 and X-Block1 according to the operator template information. The operator fuser 205 inserts the template operators X-Block0 and X-Block1 at the X-0 and X-1 positions in the assembly code of the corresponding code blocks (Block0 or Block1) respectively, to obtain the fused operator code:

[0069] C-Block0

[0070] B-Block0

[0071] X-Block0

[0072] C-Block1

[0073] B-Block1

[0074] X-Block1

[0075] For example, as Figure 2As shown, the apparatus for implementing the code generation method may further include an optimizer 206. After obtaining the fused operator code, the optimizer 206 may be used to perform code optimization operations on the fused operator code. The optimizer 206 may also be referred to as a global optimizer. After obtaining the complete operator code, the global optimizer may perform overall optimization on the fused code, which may include optimizing instruction scheduling and register usage to obtain the optimized final version of high-performance code. For example, after optimization, the order of Block0 and Block1 will be shuffled to implement the CBX high-performance code with less latency.

[0076] The code generation method of at least one embodiment of the present disclosure reduces the difficulty of user code writing and improves the performance of the hardware code for the user to write a complex neural network operation model based on specific hardware.

[0077] The code generation method of at least one embodiment of the present disclosure couples predefined operators and abstract operator templates by defining several abstract operator templates and performing unified resource management and allocation through a central scheduler, thereby maximizing the utilization of hardware computing power resources, reducing the memory access bandwidth pressure, and improving the operating efficiency and performance of the neural network model on specific hardware.

[0078] The code generation method of at least one embodiment of the present disclosure simplifies the workload of the underlying developers when supporting user-defined algorithms by abstracting the algorithm into several general template forms, and reduces the burden on the underlying developers.

[0079] The code generation method of at least one embodiment of the present disclosure simplifies the difficulty of the user in allocating and managing hardware resources when using the hardware and reduces the complexity of the user's hardware programming for user-defined algorithms by defining template programming specifications.

[0080] At least one embodiment of the present disclosure also provides a code generation apparatus. Figure 4 The figure shows a schematic diagram of a code generation apparatus provided by at least one embodiment of the present disclosure. As Figure 4 shown, the code generation apparatus includes an acquisition module 310 and a fusion module 320.

[0081] The acquisition module 310 is configured to acquire a combined operator for which code is to be generated, and the combined operator includes a predefined operator and a user-defined operator. The acquisition module 310 may, for example, execute Figure 1 the steps S110 described.

[0082] The fusion module 320 is configured to acquire operator template information for the user-defined operator; and based on the operator template information, perform a fusion operation on the predefined operator and the user-defined operator, and obtain the fused operator code corresponding to the combined operator from the fused combined operator. The fusion module 320 may, for example, executeFigure 1 The described steps S120 and S130.

[0083] For example, the obtaining module 310 and the fusion module 320 can be hardware, software, firmware, and any feasible combination thereof. For example, the obtaining module 310 and the fusion module 320 can be dedicated or general-purpose circuits, chips, or devices, etc., or can also be a combination of a processor and a memory. Regarding the specific implementation forms of the above-mentioned respective units, the embodiments of the present disclosure do not limit this.

[0084] It should be noted that in the embodiments of the present disclosure, each unit of the code generation device corresponds to each step of the foregoing code generation method. For the specific functions of the code generation device, reference can be made to the relevant descriptions regarding the code generation method, and details are not elaborated herein. Figure 4 The components and structures of the shown code generation device are merely exemplary and not restrictive. According to needs, the code generation device may further include other components and structures.

[0085] For example, the fusion module 320 includes a detection module 321, and the detection module 321 is configured to detect whether a custom operator satisfies the fusion condition. In response to the custom operator satisfying the fusion condition, perform operations of obtaining operator template information for the custom operator, and based on the operator template information, performing a fusion operation on a predefined operator and a custom operator, and obtaining fused operator code corresponding to the fused joint operator from the fused joint operator; in response to the custom operator not satisfying the fusion condition, respectively compile the predefined operator and the custom operator to obtain non-fused operator code corresponding to the joint operator. The detection module 321 can refer to the relevant descriptions regarding the detector above, and details are not elaborated herein.

[0086] For example, the fusion module may further include an operator screening module 322, a first code generation control module 323, a second code generation control module 324, and an operator fusion module 325. The operator screening module 322 is configured to receive a joint operator; the first code generation control module 323 is configured to generate assembly code and obtain operator template information according to the joint operator, where the assembly code includes code corresponding to the predefined operator; the second code generation control module 324 is configured to use the operator template information to generate operation code for implementing the calculation function of the custom operator; the operator fusion module 325 is configured to combine the assembly code and the operation code based on the operator template information to obtain fused operator code corresponding to the joint operator. The operator screening module 322, the first code generation control module 323, the second code generation control module 324, and the operator fusion module 325 can refer to the relevant descriptions regarding the operator screener, the first code generation controller, the second code generation controller, and the operator fuser above, and details are not elaborated herein.

[0087] For example, the fusion module may further include an optimization module 326 configured to perform code optimization operations on the fused operator code. The optimization module 326 may refer to the above-related descriptions about optimization and will not be elaborated herein.

[0088] At least one embodiment of the present disclosure further provides an electronic device, which includes a processor and a memory. The memory includes one or more computer program modules. The one or more computer program modules are stored in the memory and configured to be executed by the processor. The one or more computer program modules include instructions for implementing the above-mentioned code generation method. The electronic device can reduce the memory access bandwidth pressure and latency, improve the code parallelization ability, and thus enhance the operation efficiency and performance.

[0089] Figure 5 A schematic block diagram of an electronic device provided by some embodiments of the present disclosure. As Figure 5 shown, the electronic device 400 includes a processor 410 and a memory 420. The memory 420 is used to store non-transitory computer-readable instructions (such as one or more computer program modules). The processor 410 is used to run the non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are run by the processor 410, one or more steps in the above-mentioned code generation method can be executed. The memory 420 and the processor 410 may be interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0090] For example, the processor 410 may be a central processing unit (CPU), a graphics processing unit (GPU), or other forms of processing units with data processing capabilities and / or program execution capabilities. For example, the central processing unit (CPU) may be of the X86 or ARM architecture, etc. The processor 410 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 400 to perform desired functions.

[0091] For example, the memory 420 may include any combination of one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 410 may run one or more computer program modules to implement various functions of the electronic device 400. Various application programs, various data, and various data used and / or generated by the application programs may also be stored in the computer-readable storage medium.

[0092] It should be noted that in the embodiments of the present disclosure, the specific functions and technical effects of the electronic device 400 may refer to the description of the code generation method in the foregoing text, and will not be elaborated herein.

[0093] Figure 6 FIG. is a schematic block diagram of another electronic device provided in some embodiments of the present disclosure. The electronic device 500 is, for example, suitable for implementing the code generation method provided in the embodiments of the present disclosure. The electronic device 500 may be a terminal device, etc. It should be noted that Figure 6 The illustrated electronic device 500 is merely an example and will not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0094] As Figure 6 shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 510, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 520 or the program loaded from the storage device 580 into the random access memory (RAM) 530. In the RAM 530, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 510, the ROM 520, and the RAM 530 are connected to each other through a bus 540. The input / output (I / O) interface 550 is also connected to the bus 540.

[0095] Typically, the following devices can be connected to the I / O interface 550: input devices 560 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 570 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 580 including, for example, magnetic tapes, hard disks, etc.; and communication devices 590. The communication device 590 can allow the electronic device 500 to communicate with other electronic devices wirelessly or wiredly to exchange data. Although Figure 6 the electronic device 500 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices, and the electronic device 500 can alternatively implement or have more or fewer devices.

[0096] For example, according to an embodiment of the present disclosure, the above code generation method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for executing the above code generation method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 590, or installed from the storage device 580, or installed from the ROM 520. When the computer program is executed by the processing device 510, the functions defined in the code generation method provided by the embodiments of the present disclosure can be realized.

[0097] At least one embodiment of the present disclosure also provides a computer-readable storage medium, which stores non-temporary computer-readable instructions, and when the non-temporary computer-readable instructions are executed by a computer, the above code generation method can be realized. By using this computer-readable storage medium, the memory access bandwidth pressure and latency can be reduced, and the code parallelization ability can be improved, thereby enhancing the operation efficiency and performance..

[0098] Figure 7 Schematic diagram of a storage medium provided for some embodiments of the present disclosure. As Figure 7 shown, the storage medium 600 is used to store non-temporary computer-readable instructions 610. For example, when the non-temporary computer-readable instructions 610 are executed by a computer, one or more steps in the code generation method described above are executed.

[0099] For example, the storage medium 600 can be applied to the above electronic device 400. For example, the storage medium 600 can be the Figure 5 memory 420 in the shown electronic device 400. For example, the relevant description of the storage medium 600 can refer to the corresponding description of the memory 420 in the shown electronic device 400, which will not be elaborated here. Figure 5

[0100] ​The following points need to be noted:

[0101] (1) The accompanying drawings of the embodiments of the present disclosure only relate to the structures involved in the embodiments of the present disclosure, and other structures can refer to the general design.

[0102] (2) Without conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0103] As mentioned above, the above is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A code generation method, comprising: Obtaining a combined operator for which code is to be generated, where the combined operator includes a predefined operator and a custom operator; Obtaining operator template information for the custom operator; and Based on the operator template information, performing a fusion operation on the predefined operator and the custom operator, and obtaining, from the fused combined operator, fused operator code corresponding to the combined operator; Wherein, obtaining the operator template information for the custom operator, and based on the operator template information, performing a fusion operation on the predefined operator and the custom operator, and obtaining, from the fused combined operator, fused operator code corresponding to the combined operator, includes: Generating assembly code according to the combined operator and obtaining the operator template information, where the assembly code includes code corresponding to the predefined operator; Generating operation code for implementing the calculation function of the custom operator according to the operator template information; and Based on the operator template information, combining the assembly code and the operation code to obtain fused operator code corresponding to the combined operator.

2. The code generation method according to claim 1, further comprising: Detecting whether the custom operator meets the fusion condition; Wherein, in response to the custom operator meeting the fusion condition, performing the operation of obtaining the operator template information for the custom operator, and based on the operator template information, performing a fusion operation on the predefined operator and the custom operator, and obtaining, from the fused combined operator, fused operator code corresponding to the combined operator; Wherein, in response to the custom operator not meeting the fusion condition, separately compiling the predefined operator and the custom operator to obtain non-fused operator code corresponding to the combined operator.

3. The code generation method according to claim 2, wherein The meeting the fusion condition includes meeting at least one of the following conditions: The type of the custom operator is an operator type that supports the fusion operation; The allocation of hardware resources meets the fusion requirements; The granularity of the custom operator matches the granularity of the predefined operator.

4. The code generation method according to claim 1, wherein, The operator template information includes position information of the operation code corresponding to the custom operator in the fused operator code; Based on the operator template information, combining the assembly code and the operation code includes: Based on the operator template information, obtaining a template operator corresponding to the custom operator with the operation code segment; Based on the position information, setting the obtained template operator at the corresponding position in the sequence of the assembly code.

5. The code generation method according to claim 4, wherein, The assembly code further includes code for data loading, data storage, and / or data synchronization for the template operator and corresponding to the custom operator.

6. The code generation method according to claim 4, wherein, The operator template information further includes template type, template parameters, and / or register information for the template operator.

7. The code generation method according to any one of claims 1-6, further comprising: Performing a code optimization operation on the fused operator code.

8. The code generation method according to claim 7, wherein, The code optimization operation includes: Optimization of instruction scheduling and optimization of register usage.

9. A code generation device, comprising: An acquisition module configured to acquire combined operators for which code is to be generated, wherein the combined operators include predefined operators and custom operators; A fusion module configured to acquire operator template information for the custom operators; and based on the operator template information, perform a fusion operation on the predefined operators and the custom operators, and obtain fused operator code corresponding to the combined operators from the fused combined operators; Wherein, the fusion module includes: An operator screening module configured to receive the combined operators; A first code generation control module configured to generate assembly code and acquire the operator template information according to the combined operators, wherein the assembly code includes code corresponding to the predefined operators; A second code generation control module configured to generate operation code for implementing the calculation function of the custom operators by using the operator template information; and An operator fusion module configured to combine the assembly code and the operation code based on the operator template information to obtain fused operator code corresponding to the combined operators.

10. The code generation device according to claim 9, further comprising: A detection module configured to detect whether the custom operators meet the fusion conditions; Wherein, in response to the custom operators meeting the fusion conditions, perform the operations of acquiring the operator template information for the custom operators, and based on the operator template information, performing a fusion operation on the predefined operators and the custom operators, and obtaining fused operator code corresponding to the combined operators from the fused combined operators; Wherein, in response to the custom operators not meeting the fusion conditions, separately compile the predefined operators and the custom operators to obtain non-fused operator code corresponding to the combined operators.

11. The code generation device according to claim 9, further comprising: An optimization module configured to perform code optimization operations on the fused operator code.

12. An electronic device, comprising: A processor; A memory including one or more computer program modules / executable codes; Wherein, when the one or more computer program modules / executable codes are executed by the processor, they implement the instructions of the code generation method according to any one of claims 1-8.

13. A computer-readable storage medium storing non-temporary computer-readable instructions, which when executed by a computer, implement the code generation method according to any one of claims 1-8.

Images