Operator library generation method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present disclosure disclose an operator library generation method and device, electronic equipment and a storage medium, relating to the technical field of computers, which comprises: determining a network calculation graph, the network calculation graph comprising scalar nodes and non-scalar nodes, each scalar node having a data dependency relationship with one or more scalar nodes or non-scalar nodes; determining a target node of the network calculation graph based on the data dependency relationship, and fusing the target node to obtain a fused node; and generating an operator library based on the fused node. Embodiments of the present disclosure automatically generate an operator library, reducing human errors such as code vulnerabilities and logical biases introduced by manually generating an operator library, and improving the generation efficiency of the operator library.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to an operator library generation method, apparatus, electronic device, and storage medium. Background Technology

[0002] Operator libraries are software programs that deep learning neural networks rely on to run on specific hardware (CPU, GPU, and accelerators, etc.). For the NPU compiler, each operator library needs to be developed and implemented by the developer. Fusing operators is a common strategy for NPU optimization, and each fusion operator requires the developer to implement the corresponding library. For example, for the convolution + pooling operator, after fusion, the developer needs to implement the convpool library for the NPU to execute it in one go.

[0003] In existing technologies, operator fusion adaptation is usually accomplished by manually writing operator fusion libraries. However, in practical applications, the operator type and number of operators to be fused are dynamically variable. When the deep learning model structure, operator arrangement, or operator combination method changes, the manually built fusion library cannot adapt adaptively. Developers must manually modify and adjust the source code logic and recompile for adaptation.

[0004] However, the manual modification of source code for adaptation is not only cumbersome and has a long development iteration cycle, resulting in low adaptation efficiency, but also highly dependent on human coding experience, which can easily introduce human errors such as code vulnerabilities and logical deviations. It has poor stability and maintainability, making it difficult to meet the actual application needs of rapid model iteration and flexible adaptation to multiple scenarios. Summary of the Invention

[0005] This disclosure provides an operator library generation method, apparatus, electronic device, and storage medium to solve the technical problems in the prior art where manually writing operator fusion libraries is inefficient and prone to errors, and cannot meet the needs of rapid model iteration and flexible adaptation to practical applications in multiple scenarios.

[0006] According to a first aspect of the present disclosure, an operator library generation method is provided, the method being applied to an NPU compiler, the method comprising: A network computation graph is defined, which includes scalar nodes and non-scalar nodes, and each scalar node has a data dependency relationship with one or more scalar nodes or non-scalar nodes; Based on the data dependencies, the target nodes of the network computation graph are determined, and the target nodes are merged to obtain the merged nodes; Based on the fusion node, an operator library is generated.

[0007] As an optional implementation, determining the network computation graph includes: Determine the type of operator and the dependencies between operators, wherein the type of operator is at least one of scalar operator and non-scalar operator; The scalar operator is treated as a scalar node, and the nonscalar operator is treated as a nonscalar node. Based on the dependencies between the operators, the dependencies between each node are determined; based on the dependencies between each node, the network computation graph is determined.

[0008] As an optional implementation, determining the target node of the network computation graph based on the data dependency relationship includes: Determine the root node of the scalar nodes in the network computation graph, and use the root node as the target node; Based on preset rules, the target nodes in each child node of the root node are determined, and the lower-level child nodes of each target node are traversed level by level until all child nodes are traversed to obtain all target nodes.

[0009] As an optional implementation, the preset rules include: If the child node is not a scalar node, then the child node is not the target node. If the child node is a scalar node, determine whether the child node is the target node based on the other parent nodes of the child node.

[0010] As an optional implementation, determining whether a child node is a target node based on its remaining parent nodes includes: If the child node has no other parent node; or if all other parent nodes of the child node are target nodes, then the child node is determined to be a target node. If one or more parent nodes of the child node are not the target node, and the ancestor nodes of the one or more parent nodes include the target node, then the child node is determined not to be the target node. If one or more parent nodes of the child node are not the target node, and the ancestor nodes of the one or more parent nodes do not include the target node, then the child node is determined to be the target node.

[0011] As an optional implementation, generating the operator library based on the fusion node includes: Based on the input and output data of each target node within the fusion node, kernel preprocessing code is generated, which includes the function header definition, parameter declaration, variable definition, and variable initialization operations of the kernel functions; based on the data dependencies between each target node within the fusion node, kernel operation code is generated; based on the operation results of scalar operations on each target node in the fusion node, kernel postprocessing code is generated, which is used to write the operation results of the scalar operations into a specified storage area; The kernel code is determined based on the kernel preprocessing code, the kernel operation code, and the kernel postprocessing code; the kernel code is compiled to obtain an object file, and the operator library is determined based on the object file.

[0012] As an optional implementation, generating kernel computation code based on the data dependencies between target nodes within the fusion node includes: Based on the data dependencies, the target nodes are topologically sorted to obtain the operation order of each target node; The kernel operation code is generated based on the operation order of each target node.

[0013] According to a second aspect of the present disclosure, an operator library generation apparatus is provided, the apparatus comprising: The first processing module is used to determine the network computation graph, which includes scalar nodes and non-scalar nodes, and each scalar node has a data dependency relationship with one or more scalar nodes or non-scalar nodes; The second processing module is used to determine the target nodes of the network computation graph based on the data dependency relationship, and merge the target nodes to obtain the merged nodes; The third processing module is used to generate an operator library based on the fusion node.

[0014] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in any one of the first aspects.

[0015] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method as described in any one of the first aspects.

[0016] The beneficial effects of the technical solutions provided in this disclosure are: This disclosure provides an operator library generation method, apparatus, electronic device, and storage medium. Based on the data dependencies between each scalar node and one or more scalar or non-scalar nodes, this disclosure determines target nodes in the network computation graph and generates an operator library based on these target nodes. This achieves automatic operator library generation, solving the problems of low efficiency and error susceptibility caused by manually writing code to generate operator libraries in existing technologies. It improves the efficiency of operator library generation and meets the practical application needs of rapid model iteration and flexible adaptation to multiple scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0018] Figure 1 A flowchart illustrating an operator library generation method provided in this embodiment of the disclosure; Figure 2 A schematic diagram of a network computation graph provided in an embodiment of this disclosure; Figure 3 This is a schematic diagram of scalar nodes fused in a network computation graph, provided by an embodiment of the present disclosure. Figure 4 This is a schematic diagram of the structure of an operator library generation device provided in an embodiment of the present disclosure; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0019] The embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0020] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, step, operation, element, component, and / or combinations thereof supported by the art. It should be understood that when we say that an element is “connected” or “coupled” to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element are connected through an intermediate element. Furthermore, “connected” or “coupled” as used herein can include wireless connection or wireless coupling. The term “and / or” as used herein indicates at least one of the items defined by the term; for example, “A and / or B” can be implemented as “A,” or as “B,” or as “A and B.”

[0021] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0023] Operator libraries are software programs that deep learning neural networks rely on to run on specific hardware (such as CPUs, GPUs, and accelerators). For the NPU compiler, each operator library needs to be developed and implemented by the developer. Fusing operators is a common strategy for NPU optimization, and each fusion operator requires the developer to implement the corresponding library. For example, for the convolution + pooling operator, after fusion, the developer needs to implement the convpool library for the NPU to execute it in one go.

[0024] In existing technologies, operator fusion adaptation is usually accomplished by manually writing operator fusion libraries. However, in practical applications, the operator type and number of operators to be fused are dynamically variable. When the deep learning model structure, operator arrangement, or operator combination method changes, the manually built fusion library cannot adapt adaptively. Developers must manually modify and adjust the source code logic and recompile for adaptation.

[0025] However, manually modifying source code for adaptation is not only cumbersome and time-consuming, resulting in low efficiency, but also heavily reliant on human coding experience, making it prone to introducing code vulnerabilities, logical deviations, and other human errors. This leads to poor stability and maintainability, making it difficult to meet the practical application requirements of rapid model iteration and flexible adaptation across multiple scenarios. In other words, using traditional manual development methods, manually writing kernel code and customizing operator files for each fusion structure requires significant development costs, resulting in poor adaptability and low versatility, failing to keep pace with the dynamic changes in the computation graph. Furthermore, frequent manual generation of operator libraries leads to resource redundancy, prolonged development cycles, and severely limits model inference efficiency. Therefore, relying on manually customizing fixed operator libraries is unsuitable for dynamic fusion scenarios and lacks engineering practicality.

[0026] To address the dynamic nature of operator libraries, this disclosure proposes an operator library generation method, apparatus, electronic device, and storage medium. Relying on a code transmitter, the dynamically fused target nodes are parsed in real time, automatically completing the compilation process including parameter extraction, variable declaration, topology sorting, computational code generation, and result write-back. This allows for rapid compilation of the dynamically fused subgraph into independent object files without manual intervention, automatically generating an operator library adapted to the current topology.

[0027] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by the technical solutions of this application. It should be noted that the following embodiments can be referenced, learned from, or combined with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be described again.

[0028] It is understood that in the operator library generation method provided in this disclosure, any method step can be executed collaboratively by any one or both of the following: an electronic device equipped with NPU compilation capabilities and a compilation server. Specifically, the operator library generation method can be executed independently by the compilation server, independently by a local electronic device equipped with an NPU compiler, or the entire process of compilation, node fusion, kernel code generation, and operator library construction can be completed collaboratively by the electronic device and the server.

[0029] The compilation server can be a standalone physical compilation server, a server cluster or distributed compilation system composed of multiple physical servers, or a cloud server providing cloud compilation services. It is used to complete core compilation tasks such as computation graph parsing, node optimization, code generation, and operator compilation. Electronic devices adapted to NPU compilation scenarios are mainly terminal devices, embedded devices, or industrial control devices equipped with NPU chips and supporting compilation environments, used to assist in data loading, topology preprocessing, and other operations.

[0030] To clearly illustrate the technical solution of this disclosure, subsequent embodiments will use the NPU compiler as the main execution entity to introduce the operator library generation method. This exemplary description is only used to explain the technical solution and does not constitute a limitation on the scope of protection of this disclosure.

[0031] Figure 1 A flowchart illustrating an operator library generation method provided in this disclosure is shown in the figure. The method is applied to an NPU compiler and includes: S101. Determine the network computation graph, which includes scalar nodes and non-scalar nodes, and each scalar node has a data dependency relationship with one or more scalar nodes or non-scalar nodes.

[0032] In deep learning, artificial intelligence inference, and distributed network computing, the network computing graph, as a visual topological structure representing the logic of the entire computing process, data flow paths, and operational execution relationships, serves as the fundamental carrier for subsequent operator computation, data transmission, model inference, and resource scheduling.

[0033] Specifically, in this embodiment of the disclosure, all nodes in the computation graph are divided into two main types: scalar nodes and non-scalar nodes according to data storage format and data dimension characteristics. Among them, scalar nodes are computation nodes that store a single dimension and a single value. The data inside the node has no row or column dimension and no matrix tensor structure. They are mostly used to store single values ​​such as offsets, thresholds, weight coefficients, and computational constants. Non-scalar nodes are multi-dimensional data nodes that can store composite data such as vectors, matrices, and higher-order tensors, including but not limited to: feature matrices, image tensors, batch sample datasets, convolutional feature maps, and other multi-dimensional computational data.

[0034] Specifically, in this embodiment, no single scalar node in the computation graph possesses independent computational capabilities. It must rely on the output data of other nodes or external data to complete data input. Therefore, each scalar node needs to establish one-to-one or one-to-many data dependency links based on the business computation logic to complete numerical operations such as constant superposition, coefficient multiplication, and threshold comparison. Simultaneously, scalar nodes can also establish data associations with non-scalar nodes, using a single scalar value as a control parameter to perform global correction, scaling, offsetting, and filtering of non-scalar multidimensional data. For example, scalar weight coefficients can be used to correct tensor eigenvalues, and scalar thresholds can be used to filter invalid data in the non-scalar feature matrix.

[0035] It should be noted that data dependencies follow unidirectional flow or bidirectional interaction rules. All dependency links clearly indicate the data input source, data output direction, and operation triggering conditions to ensure that there are no conflicts in data transmission and no errors in the calculation sequence between scalar nodes and related nodes.

[0036] S102. Based on the data dependency relationship, determine the target node of the network computation graph, and merge the target node to obtain the merged node.

[0037] Specifically, in this embodiment of the disclosure, in the network computation graph, a large number of scalar computation operators are usually scattered throughout the topology links. These operators generally have the characteristics of simple computational logic, low data dimensionality, and close computational dependencies. In order to reduce the number of computation graph nodes, simplify the topology structure, and reduce graph scheduling overhead, it is necessary to prioritize the fusion processing of all interconnected scalar computation operators.

[0038] Specifically, in this embodiment of the disclosure, scalar nodes are used as the filtering objects to determine all interconnected scalar nodes, these scalar nodes are taken as target nodes, and the target nodes are merged into a single subgraph node. The above-mentioned fusion of subgraph nodes belongs to the problem of solving the maximum connected subgraph. Therefore, the target node in the network computation graph can be determined by solving the maximum connected subgraph method of the network computation graph.

[0039] It should be noted that network computation graphs are typical directed topological graphs. If the original dependencies are disrupted during the merging process, invalid loop links can easily be generated, leading to problems such as data circular dependencies, repeated operator iterations, and computational deadlocks. Therefore, during the merging of connectivity scalar operators and the construction of large nodes in the subgraph, it is necessary to verify the directed logic of the connectivity paths in real time to ensure that there are no directed loops within the merged composite node or between the composite node and external nodes, thus maintaining the directed acyclic topological characteristics of the computation graph.

[0040] It should also be noted that in the actual operation of the network computation graph, the node topology is not fixed but dynamically variable. It is affected by factors such as the scale of input data, the requirements of inference tasks, and the branching logic of the model. For example, when a new node is added, the number of nodes in the computation graph and the dependencies between nodes will change in real time. Each time a node fusion operation is performed, the structure of the composite subgraph generated based on different connected components is different. Each set of fused connected nodes can be regarded as a brand new custom fusion operator.

[0041] S103. Based on the fusion node, generate an operator library.

[0042] Specifically, in this embodiment of the disclosure, for the fusion node, the kernel code is written based on the code launcher, and a loadable operator library file is generated by combining compilation methods.

[0043] This embodiment of the disclosure determines the target node in the network computation graph based on the data dependency relationship between each scalar node and one or more scalar or non-scalar nodes, and generates an operator library based on the target node. This realizes the automatic generation of the operator library, solves the problems of low efficiency and easy error caused by manually writing code to generate the operator library in the prior art, improves the generation efficiency of the operator library, and meets the practical application needs of rapid model iteration and flexible adaptation to multiple scenarios.

[0044] To facilitate a clearer understanding of the network computation graphs in the embodiments of this application for those skilled in the art, an example is provided below.

[0045] Figure 2 This is a schematic diagram of a network computation graph provided in an embodiment of the present disclosure; as shown in the figure, nodes 1, 2, 3, 4, 5, 6, and 7 are scalar nodes, and the remaining nodes are non-scalar nodes; wherein, the arrow between node 1 and node 2 is used to indicate that the output of node 1 is the input of node 2; the arrows between node 2 and node 3, node 2 and node 4, and node 2 and node 5 are used to indicate that the output of node 2 is the input of node 3, node 4, and node 5; the arrows between node 4 and node 6, and node 4 and node 7 are used to indicate that the output of node 4 is the input of node 6 and node 7.

[0046] Based on the above embodiments, as an optional embodiment, determining the network computation graph includes: Determine the type of operator and the dependencies between operators, wherein the type of operator is at least one of scalar operator and non-scalar operator; The scalar operator is treated as a scalar node, and the nonscalar operator is treated as a nonscalar node. Based on the dependencies between the operators, the dependencies between each node are determined; based on the dependencies between each node, the network computation graph is determined.

[0047] Specifically, in this embodiment of the disclosure, all operational operators in the model are analyzed for type and their dependencies are sorted out. Specifically, according to the difference in the dimension of the operational data, the operators are divided into scalar operators and non-scalar operators; among them, scalar operators are used to perform simple operations on a single numerical value, including arithmetic operations, parameter correction, threshold judgment and other computational logic; non-scalar operators are used to process multi-dimensional tensor data, covering complex tensor calculations such as matrix operations, convolution transformation, feature mapping and so on.

[0048] Specifically, in this embodiment of the disclosure, after clarifying the operator type, a one-to-one mapping rule is adopted to abstract all scalar operators into scalar nodes in the computation graph, and all non-scalar operators into non-scalar nodes. The data dependency relationship between each node is determined according to the data flow between operators. For example, if the output data of the preceding operator is the input data of the following operator, it is determined that there is a dependency relationship between the two operators, which is mapped to the node level as establishing a directed dependency edge between the corresponding nodes.

[0049] The embodiments disclosed herein construct a complete network computation graph based on scalar nodes, non-scalar nodes, and directed dependency edges between nodes. This computation graph can intuitively reflect the operator hierarchy, data flow direction, and topological constraints, providing standardized topological input for subsequent maximum subconnected graph fusion, target node selection, automatic code generation by the code emitter, and operator library compilation and construction.

[0050] Based on the above embodiments, as an optional embodiment, determining the target node of the network computation graph based on the data dependency relationship includes: Determine the root node of the scalar nodes in the network computation graph, and use the root node as the target node; Based on preset rules, the target nodes in each child node of the root node are determined, and the lower-level child nodes of each target node are traversed level by level until all child nodes are traversed to obtain all target nodes.

[0051] Specifically, this embodiment of the disclosure employs a top-down hierarchical traversal filtering method to determine target nodes. Based on data dependency links, the upstream starting node in the network computation graph is selected and defined as the root node of the scalar nodes. This root node serves as the computation starting point for the entire dependency link, undertaking the role of initial data input and basic parameter calculation. After determining the root node, its child nodes are judged according to preset filtering rules, and scalar nodes that meet the fusion conditions are selected and added to the target node set. Further, a step-by-step iterative traversal method is adopted, using the determined target child node as the new traversal starting point, continuously searching its child nodes downwards, repeatedly executing rule judgment and node filtering operations, traversing the entire topology link layer by layer until all associated subordinate child nodes in the computation graph are traversed. Through this top-down, layer-by-layer recursive filtering method, all valid nodes with connectivity relationships and satisfying preset constraints are fully included, ultimately obtaining all target nodes in the computation graph.

[0052] The embodiments disclosed herein can accurately sort out the connected domains of scalar nodes by determining the root node and using hierarchical traversal filtering, avoiding invalid nodes from participating in fusion calculations, and providing accurate and effective support for the subsequent automatic generation of kernel code and operator library by the code launcher.

[0053] Based on the above embodiments, as an optional embodiment, the preset rules include: If the child node is not a scalar node, then the child node is not the target node. If the child node is a scalar node, determine whether the child node is the target node based on the other parent nodes of the child node.

[0054] Specifically, in this embodiment of the disclosure, if a child node is not a scalar node, then the child node does not correspond to a scalar operator, and the operator library in this embodiment of the disclosure is fused with scalar operators; therefore, the child node does not belong to the target node.

[0055] Specifically, in this embodiment of the disclosure, if the child node obtained by the current traversal is a scalar node, it cannot be directly determined to be the target node. It is necessary to further combine all the other parent nodes of the scalar child node except for the current traversal path for judgment.

[0056] In some embodiments, if the other parent nodes of the scalar node are all in the same connected subgraph as the current traversal link, the overall data dependency relationship is unified and there is no cross-branch scattered association, then the scalar node is determined to be fused and is used as the target node; if the scalar node has other parent nodes belonging to different topological branches, independent computing links or isolated scattered nodes, then the scalar node is determined to be fused and is not used as the target node.

[0057] The embodiments of this disclosure can quickly determine the target node in the computation graph through filtering rules, reduce invalid node traversal and avoid the problem of topological loops, and provide an accurate data source for subsequent scalar operator aggregation, topological sorting, automated code generation and dynamic operator library construction, which greatly shortens the time consumed by dynamic fusion and operator compilation.

[0058] Based on the above embodiments, as an optional embodiment, determining whether the child node is a target node based on the remaining parent nodes of the child node includes: If the child node has no other parent node; or if all other parent nodes of the child node are target nodes, then the child node is determined to be a target node. If one or more parent nodes of the child node are not the target node, and the ancestor nodes of the one or more parent nodes include the target node, then the child node is determined not to be the target node. If one or more parent nodes of the child node are not the target node, and the ancestor nodes of the one or more parent nodes do not include the target node, then the child node is determined to be the target node.

[0059] Specifically, in this embodiment, if a child node has no other parent nodes, or if all other parent nodes of the child node are target nodes, it indicates that the scalar node has a simple and clear dependency structure, no additional external branch input, or completely relies on the selected target node to complete data input, and is within the same complete connected topology link as the existing target node, thus meeting the conditions for fusion. Therefore, this node is selected as the target node and participates in the subsequent operator library compilation process.

[0060] Specifically, in this embodiment of the disclosure, if a child node has one or more parent nodes that are not the target node, and the ancestor nodes traced upwards from these parent nodes include the identified target node, it indicates that although such a node has non-same-origin parent nodes, its upstream topology can still be traced back to the predetermined target node, and data flow and computation can be completed by relying on the existing topology links. Therefore, this child node is not considered a target node.

[0061] Specifically, in this embodiment of the disclosure, if a child node has one or more parent nodes that are not target nodes, and none of the ancestor nodes corresponding to these parent nodes contain any target nodes, it indicates that the topological branch in which this type of scalar node is located is independent of the existing target node system and has the conditions for node fusion. Therefore, this child node is taken as the target node.

[0062] This embodiment of the disclosure ensures that scalar nodes in different branches can be fused as needed by judging the other parent nodes of the scalar quantum node, providing accurate and reliable node basis for subsequent topology sorting, automatic kernel code generation and batch construction of dynamic operator library.

[0063] To help those skilled in the art better understand the target node selection method, the following section uses... Figure 2 Let's take an example to illustrate.

[0064] like Figure 2 As shown, node 1 is a scalar node, and node 1 has no parent node that is a scalar node. Therefore, node 1 is the root node, and the merged nodes include node 1.

[0065] Node 1 has two child nodes, where Node 2 is a scalar node and its other parent node is a non-scalar node, which is not in the merge node. Moreover, the ancestor node of this parent node is not in the merge node. Therefore, Node 2 is the target node. At this time, the merge node includes Node 1 and Node 2.

[0066] Node 2 is the target node. Continuing to examine its child nodes, it has three: nodes 3, 4, and 5. For node 3, its other parent node is not within the merge node, and neither is its ancestor node. Therefore, node 3 can be the target node. The merge node now includes nodes 1, 2, and 3. For node 4, its other parent node is node 3, which is already within the merge node. Therefore, node 4 is the target node and can be merged. The merge node now includes nodes 1, 2, 3, and 4. For node 5, its other parent node is a non-scalar node not within the merge node, but its ancestor node is within the merge node (node ​​1). If node 5 is chosen as the target node and merged, a cycle will be created; therefore, node 5 is not chosen as the target node.

[0067] Node 3 has two child nodes; of which, node 4 is already within the merge node, and the other child node is not a scalar node, therefore, it is not considered as a target node.

[0068] Node 4 includes two child nodes, namely Node 6 and Node 7. Since Node 6 and Node 7 each have only one parent node, namely Node 4 itself, all parent nodes are within the merge node. Therefore, Node 6 and Node 7 are selected as target nodes and added to the merge node. At this time, the merge node includes: Node 1, Node 2, Node 3, Node 4, Node 6, and Node 7.

[0069] Since the child nodes of nodes 6 and 7 are not scalar nodes, the process ends here.

[0070] Figure 3 This is a schematic diagram of scalar node fusion in a network computation graph provided by an embodiment of this disclosure; as shown... Figure 3 As shown, the fusion node includes Figure 2 The system contains six scalar operation nodes: node 1, node 2, node 3, node 4, node 6, and node 7. These six nodes meet the selection criteria for scalar node fusion and can be integrated into a single fusion node through the node fusion mechanism to achieve the combined execution of multi-step scalar operations.

[0071] Although node 5 is also a scalar node, it cannot be merged as a target node. Once merged, a data dependency loop structure will be formed inside the merged node. This loop topology structure will cause problems such as data dependency deadlock, inability to determine the order of operations through topological sorting, and computational logic conflicts during the operation process, which will lead to failures such as kernel code generation failure, program compilation errors, and abnormal model inference results.

[0072] Based on the above embodiments, as an optional embodiment, generating the operator library based on the fusion node includes: Based on the input and output data of each target node within the fusion node, kernel preprocessing code is generated, which includes the function header definition, parameter declaration, variable definition, and variable initialization operations of the kernel functions; based on the data dependencies between each target node within the fusion node, kernel operation code is generated; based on the operation results of scalar operations on each target node in the fusion node, kernel postprocessing code is generated, which is used to write the operation results of the scalar operations into a specified storage area; The kernel code is determined based on the kernel preprocessing code, the kernel operation code, and the kernel postprocessing code; the kernel code is compiled to obtain an object file, and the operator library is determined based on the object file.

[0073] Specifically, the code transmitter generates kernel code based on the target node. The kernel code includes three parts: kernel preprocessing code, kernel operation code, and kernel postprocessing code.

[0074] In some embodiments, the kernel preprocessing code is generated as follows: For each scalar operator, its input and output are defined as scalars. If its input is also the input of the entire computation graph, its input definition is added to the kernel parameters; if its output is the output of the entire computation graph, its output is added to the kernel parameters. After obtaining the kernel parameters and completing the variable definitions, commands are executed on the kernel functions to ensure that each kernel function has an independent identifier for easy retrieval and loading after compilation. After completing the kernel function header and parameter definitions, the variable declaration stage begins. Normalized declaration operations are performed according to the actual type and storage format of intermediate variables, temporary storage variables, and operation parameters used in the scalar operation. For non-batch data such as single numerical values ​​and independent operation parameters, ordinary scalar variables are declared; for data with multiple elements that require batch storage and continuous access, arrays are declared. After completing the variable declarations, input data acquisition and variable initialization operations are performed. All external input data for the kernel function has been passed in through function parameters. Therefore, the corresponding values ​​are read directly from the kernel function's input parameters, and the read data is assigned to the corresponding declared variables to complete the variable initialization.

[0075] In some embodiments, before generating kernel operation code, all target nodes need to be sorted according to the data dependencies between nodes in the order of dependency to ensure that the operation order and dependency order are consistent; then, the corresponding operation character code is automatically generated according to the scalar operation type.

[0076] In some embodiments, the result variable of the operation is written to the storage address corresponding to the kernel parameter to generate kernel post-processing code.

[0077] In some embodiments, after writing the preprocessing code, kernel operation code, and kernel postprocessing code for generating kernel code, the kernel code is obtained; the kernel code is then compiled by a compiler to generate corresponding object files, and the object files are packaged and stored to obtain the operator library.

[0078] This disclosed embodiment divides the kernel code into three parts: kernel preprocessing code, kernel operation code, and kernel postprocessing code. This achieves modular and standardized decomposition of the code generation logic, making the responsibilities and boundaries of each stage of data initialization, operation execution, and result writing clear and well-defined. It effectively avoids the problem of mixed and disordered code logic, greatly improves the standardization, readability, and maintainability of the kernel code, and is compatible with various dynamically fused operator libraries.

[0079] Based on the above embodiments, as an optional embodiment, generating kernel operation code according to the data dependency relationship between each target node within the fusion node includes: Based on the data dependencies, the target nodes are topologically sorted to obtain the operation order of each target node; The kernel operation code is generated based on the operation order of each target node.

[0080] Specifically, in this embodiment of the disclosure, since there is a data dependency relationship between the target nodes, the calculation result of one node will be directly used as the input data of another node; therefore, it is necessary to execute according to the calculation order of the nodes, otherwise problems such as undefined variables, unread data, and abnormal calculation results will occur.

[0081] Specifically, in this embodiment of the disclosure, during topological sorting, all scalar type target nodes are traversed based on the directed data dependencies between target nodes. Following the principle of prioritizing nodes with no preceding dependencies and sequentially extending dependencies, all target nodes are linearly sorted to ensure that all parent nodes of any given node complete their operations before that node, forming a strictly ordered, conflict-free, and loop-free execution sequence. After completing the topological sorting, corresponding lines of computation code are generated for each target node according to the obtained computation order, converting the node's computation logic line by line into directly compileable and executable kernel code.

[0082] In this embodiment of the disclosure, the kernel operation code is generated by topological sorting, which can completely match the original data flow logic of the computation graph, ensure that the operation order and dependency relationship are strictly consistent, avoid calculation errors caused by disordered execution order, and completely eliminate abnormal problems such as loops, duplicate calculations, and data dependency conflicts at the code level, providing a stable, correct, and directly runnable operation kernel for subsequent compiler compilation, dynamic operator library generation, and efficient model inference.

[0083] Figure 4 The figure shows a schematic diagram of an operator library generation device provided in an embodiment of the present disclosure. The device may include: a first processing module 4001, a second processing module 4002 and a third processing module 4003.

[0084] The first processing module 4001 is used to determine a network computation graph, which includes scalar nodes and non-scalar nodes, and each scalar node has a data dependency relationship with one or more scalar nodes or non-scalar nodes. The second processing module 4002 is used to determine the target node of the network computation graph based on the data dependency relationship, and merge the target node to obtain the merged node; The third processing module 4003 is used to generate an operator library based on the fusion node.

[0085] The operator library generation device of this disclosure can execute the operator library generation method provided in this disclosure. The implementation principle is similar. The actions performed by each module in the operator library generation device in each embodiment of this application correspond to the steps in the operator library generation method of each embodiment of this application. For detailed functional descriptions of each module of the operator library generation device, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0086] This embodiment of the disclosure determines the target node in the network computation graph based on the data dependency relationship between each scalar node and one or more scalar or non-scalar nodes, and generates an operator library based on the target node. This realizes the automatic generation of the operator library, solves the problems of low efficiency and easy error caused by manually writing code to generate the operator library in the prior art, improves the generation efficiency of the operator library, and meets the practical application needs of rapid model iteration and flexible adaptation to multiple scenarios.

[0087] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure, such as... Figure 5 As shown, the electronic device 5000 includes a processor 5001 and a memory 5003. The processor 5001 and the memory 5003 are connected, for example, via a bus 5002. Optionally, the electronic device 5000 may further include a transceiver 5004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 5004 is not limited to one type, and the structure of the electronic device 5000 does not constitute a limitation on the embodiments of this disclosure.

[0088] Processor 5001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 5001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0089] Bus 5002 may include a pathway for transmitting information between the aforementioned components. Bus 5002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 5002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0090] The memory 5003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0091] The memory 5003 is used to store computer programs that execute embodiments of the present disclosure, and is controlled by the processor 5001 to execute them. The processor 5001 is used to execute the computer programs stored in the memory 5003 to implement the steps shown in the foregoing method embodiments.

[0092] The electronic device package may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0093] This disclosure provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve the following: It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium, a computer-readable medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0094] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments. Compared with the prior art, it can achieve: The terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the illustrations or text descriptions.

[0095] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0096] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of this disclosure.

Claims

1. An operator library generation method characterized by comprising: The method is applied to an NPU compiler, and the method includes: A network computation graph is defined, which includes scalar nodes and non-scalar nodes, and each scalar node has a data dependency relationship with one or more scalar nodes or non-scalar nodes; Based on the data dependencies, the target nodes of the network computation graph are determined, and the target nodes are merged to obtain the merged nodes; Based on the fusion node, an operator library is generated.

2. The operator library generation method according to claim 1, characterized in that, The determination of the network computation graph includes: Determine the type of operator and the dependencies between operators, wherein the type of operator is at least one of scalar operator and non-scalar operator; The scalar operator is treated as a scalar node, and the nonscalar operator is treated as a nonscalar node. Based on the dependencies between the operators, the dependencies between each node are determined; based on the dependencies between each node, the network computation graph is determined.

3. The operator library generation method according to claim 2, characterized in that, The process of determining the target node of the network computation graph based on the data dependency relationship includes: Determine the root node of the scalar nodes in the network computation graph, and use the root node as the target node; Based on preset rules, the target nodes in each child node of the root node are determined, and the lower-level child nodes of each target node are traversed level by level until all child nodes are traversed to obtain all target nodes.

4. The operator library generation method according to claim 3, characterized in that, The preset rules include: If the child node is not a scalar node, then the child node is not the target node. If the child node is a scalar node, determine whether the child node is the target node based on the other parent nodes of the child node.

5. The operator library generation method according to claim 4, characterized in that, Determining whether a child node is a target node based on its remaining parent nodes includes: If the child node has no other parent node; or if all other parent nodes of the child node are target nodes, then the child node is determined to be a target node. If one or more parent nodes of the child node are not the target node, and the ancestor nodes of the one or more parent nodes include the target node, then the child node is determined not to be the target node. If one or more parent nodes of the child node are not the target node, and the ancestor nodes of the one or more parent nodes do not include the target node, then the child node is determined to be the target node.

6. The operator library generation method according to any one of claims 1 to 5, characterized in that, The generation of the operator library based on the fusion node includes: Based on the input and output data of each target node within the fusion node, kernel preprocessing code is generated, which includes the function header definition, parameter declaration, variable definition, and variable initialization operations of the kernel functions; based on the data dependencies between each target node within the fusion node, kernel operation code is generated; based on the operation results of scalar operations on each target node in the fusion node, kernel postprocessing code is generated, which is used to write the operation results of the scalar operations into a specified storage area; The kernel code is determined based on the kernel preprocessing code, the kernel operation code, and the kernel postprocessing code; the kernel code is compiled to obtain an object file, and the operator library is determined based on the object file.

7. The operator library generation method according to claim 6, characterized in that, The step of generating kernel computation code based on the data dependencies between target nodes within the fusion node includes: Based on the data dependencies, the target nodes are topologically sorted to obtain the operation order of each target node; The kernel operation code is generated based on the operation order of each target node.

8. An operator library generation apparatus, characterized in that, The device is used in an NPU compiler, and the device includes: The first processing module is used to determine the network computation graph, which includes scalar nodes and non-scalar nodes, and each scalar node has a data dependency relationship with one or more scalar nodes or non-scalar nodes; The second processing module is used to determine the target nodes of the network computation graph based on the data dependency relationship, and merge the target nodes to obtain the merged nodes; The third processing module is used to generate an operator library based on the fusion node.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images