A processing method and device of a computing model

Abstract

The application discloses a processing method and device of a computing model, the method comprising: judging whether the data dimension of input data and / or output data of a target computing node in the computing model is fixed or not, the computing model comprising a plurality of computing nodes, the target computing node being any one of the plurality of computing nodes, and the target computing node being used for representing a plurality of computing operations in the computing model; determining that the data dimension changes, and compiling a first computing instruction in a plurality of computing instructions corresponding to the target computing node; the plurality of computing instructions corresponding to the plurality of computing operations represented by the target computing node one by one, any one of the computing instructions being used for instructing to execute a corresponding computing operation, and the first computing instruction being used for instructing a computing operation irrelevant to the data dimension of the target computing node. The scheme provided by the application can compile the computing model with dynamic data dimension, improve the compiling speed, reduce the resource cost of compiling, and improve the model compiling efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method and apparatus for processing computational models. Background Technology

[0002] In fields such as artificial intelligence (AI) and machine learning (ML), different computational models can be designed or trained to achieve different functions. Within these models, by designing and establishing computational nodes and corresponding multi-layered computational structures, selecting appropriate inputs and outputs, and then through network learning and optimization, a functional relationship from input to output can be established to represent the real-world correlation between inputs and outputs.

[0003] The computational model includes multiple computation nodes, each representing a computational operation performed on the data within the model. Each computation node has corresponding computational instructions to indicate the computational operation it represents. Different computation nodes are interconnected, representing data transmission relationships between them. At each computation node, corresponding computational operations can be performed on the data input to that node.

[0004] Currently, computational models need to be deployed on hardware devices to achieve practical applications. To improve the adaptability of computational models to various hardware types and enhance model performance, compilers can be used to compile and optimize the models, converting the model definition files of the designed or trained computational models into machine instructions that can run on hardware devices. This allows the computational models to be deployed on hardware devices. After the computational models are deployed, the hardware devices can execute the machine instructions corresponding to the compiled computational models, inputting actual data into the models for computation to obtain the corresponding actual output data.

[0005] Current mainstream model compilers, such as accelerated linear algebra (XLA) compilers, graph lowering (GLOW) compilers, and tensor virtual machine (TVM) compilers, all complete the compilation of computational models and instruction optimization based on a defined data dimension (shape). This means that the actual data dimension of the input data needs to be determined before compilation can be completed.

[0006] However, in real-world business scenarios, the dimensions of the data input to the computational model can be dynamically changing, and the range of these changes can be very large. Furthermore, with the rapid development of artificial intelligence, deep learning algorithms and models are emerging in large numbers, the sources of training data are becoming increasingly diverse, and the specifications of data requiring real-time processing are becoming increasingly varied. This makes the efficient compilation and execution of computational models supporting dynamic dimensions an urgent need. To address this, existing compilation methods such as compilation caching, pre-compiling based on data range, and removing the dynamic range of data dimensions can be used to compile computational models with dynamic dimensions. However, these methods all suffer from long compilation times and high resource consumption, and may also lead to decreased model usability, performance fluctuations, and increased warm-up overhead. Summary of the Invention

[0007] This application provides a method and apparatus for processing computational models, which are used to compile computational models with dynamic data dimensions, thereby improving the compilation speed, reducing the resource overhead of model compilation, and thus improving the compilation efficiency.

[0008] In a first aspect, embodiments of this application provide a method for processing a computational model, the method comprising:

[0009] Determine whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed, wherein the computing model includes multiple computing nodes, the target computing node is any one of the multiple computing nodes, and the target computing node is used to represent multiple computing operations in the computing model; when it is determined that the data dimension changes, compile the first computing instruction among the multiple computing instructions corresponding to the target computing node; wherein the multiple computing instructions correspond one-to-one with the multiple computing operations represented by the target computing node, any one computing instruction is used to instruct the execution of the corresponding computing operation, and the first computing instruction is used to instruct the execution of computing operations unrelated to the data dimension of the target computing node.

[0010] In this method, when compiling the computational model, if the data dimensions of the input and / or output data corresponding to a computational node in the model dynamically change, only the computational instructions corresponding to the computational node that are independent of the data dimensions are compiled. Thus, when running the computational model after compilation, only the computational instructions corresponding to the computational node that are related to the data dimensions need to be compiled, without having to recompile the already compiled computational instructions for that node. Therefore, this method for compiling computational models can adapt to model processing scenarios with dynamically changing data dimensions, improving the versatility and ease of use of the compiled computational model. Simultaneously, it can increase compilation speed, reduce compilation time and resource consumption, thereby improving the compilation efficiency of the model.

[0011] In one possible design, the method further includes: compiling the plurality of calculation instructions when it is determined that the data dimension is fixed.

[0012] In this method, when the data dimension of the computing node remains fixed, the computing instructions corresponding to the computing node can be compiled directly, resulting in a faster compilation speed.

[0013] In one possible design, after compiling the plurality of computation instructions, the method further includes: during the execution of the computation model, performing computation operations corresponding to the plurality of computation instructions according to the compiled plurality of computation instructions.

[0014] In this method, the computation instructions corresponding to the computation nodes in the computation model are first compiled. When the computation model is run, the computation operations corresponding to the compiled computation instructions can be directly executed. Therefore, the execution efficiency of the computation model is high.

[0015] In one possible design, after compiling the first computation instruction among the plurality of computation instructions corresponding to the target computing node, the method further includes: during the execution of the computing model, determining the current data dimension of the target computing node, compiling the second computation instruction among the plurality of computation instructions according to the current data dimension, wherein the second computation instruction is a computation instruction other than the first computation instruction among the plurality of computation instructions; and executing the computation operation corresponding to the plurality of computation instructions according to the compiled plurality of computation instructions.

[0016] In this method, when the data dimension of the computing node changes dynamically, the computing instructions corresponding to the computing node that are independent of the data dimension can be compiled. This avoids the repeated compilation of these computing instructions when running the computing model, thereby improving the overall compilation efficiency and the execution efficiency of the computing model.

[0017] In one possible design, determining whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed includes: determining whether the data dimension is fixed based on the dimension indication information of the computing model; wherein the dimension indication information is used to indicate whether the data dimension is fixed; or, when the data dimension is the data dimension of the input data of the target computing node, determining whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node is fixed; wherein the first computing node is at least one computing node located before the target computing node; or, when the data dimension is the data dimension of the output data of the target computing node, determining whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed.

[0018] In this method, it is possible to determine whether the data dimension of the computing node in the computing model is fixed in a variety of ways, and then select the corresponding compilation strategy based on whether the data dimension of the computing node is fixed.

[0019] In one possible design, determining whether the data dimension is fixed or not based on whether the data dimension of the output data of the first computing node is fixed or not includes: if the data dimension of the output data of the first computing node is fixed or not, then the data dimension is determined to be fixed or not; otherwise, the data dimension is determined to be changing.

[0020] In this method, the input data of a computing node is the output data of the computing nodes preceding it. Therefore, based on the data dimension information of the output data of other computing nodes preceding the current computing node, the data dimension information of the current computing node can be determined easily, quickly and accurately.

[0021] In one possible design, determining whether the data dimension is fixed or unchanging based on whether the data dimension of the input data of the target computing node is fixed or unchanging includes: determining that the data dimension changes when the data dimension of the input data of the target computing node changes; and determining whether the data dimension is fixed or unchanging based on a set dimension analysis function and the data dimension of the input data of the target computing node when the data dimension of the input data of the computing node is fixed or unchanging. The set dimension analysis function is used to determine whether the data dimension of the computing node is fixed or unchanging based on the data dimension of the input data of the computing node.

[0022] In this method, the output data of the computing node is obtained by performing calculations on its input data. Therefore, there is a certain relationship between the input data and the output data of the computing node. By using the data dimension information of the input data of the computing node as a reference to determine the data dimension information of the output data of the computing node, the accuracy of the determined data dimension can be guaranteed.

[0023] In one possible design, executing computational operations corresponding to the compiled plurality of computational instructions includes: executing the computational operations corresponding to the compiled plurality of computational instructions via a first thread; the method further includes: while executing the computational operations corresponding to the plurality of computational instructions via the first thread, executing the following steps via a second thread: determining the current data dimension of a second computational node, wherein the second computational node is at least one computational node located after the target computational node, and the data dimension of the second computational node is variable; compiling a third computational instruction among the plurality of computational instructions corresponding to the second computational node based on the current data dimension of the second computational node, wherein the third computational instruction is used to indicate computational operations related to the data dimension of the second computational node.

[0024] In this method, for computation nodes with dynamically changing data dimensions, the data dimension information of the current computation node and the compilation of data dimension-related computation instructions can be determined simultaneously while executing the computation instructions of other computation nodes preceding it. This hides the preparation overhead of the current computation node within the execution process of other computation nodes. Thus, after other nodes have finished running, the current node can start running, ensuring the continuity of computation node operation and improving the execution efficiency of the computation model. Furthermore, performing the compilation and execution of computation instructions on different threads ensures the continuity of the computational operation execution flow within the computation model, avoiding the generation of additional overhead that cannot be hidden.

[0025] In one possible design, the method further includes: executing multiple computation instructions corresponding to the compiled second computing node through the first thread.

[0026] In this method, the thread that executes the computation instructions corresponding to the computing node is the same thread that executed the computation instructions corresponding to the computing node before this computing node, which can ensure the continuity of the execution flow of the computation operation corresponding to the computing node.

[0027] Secondly, this application provides a computing device, comprising:

[0028] A judgment unit is used to determine whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed, wherein the computing model includes multiple computing nodes, the target computing node is any one of the multiple computing nodes, and the target computing node is used to represent multiple computing operations in the computing model; a processing unit is used to compile a first computing instruction among multiple computing instructions corresponding to the target computing node when it is determined that the data dimension has changed; wherein the multiple computing instructions correspond one-to-one with the multiple computing operations represented by the target computing node, any one computing instruction is used to instruct the execution of the corresponding computing operation, and the first computing instruction is used to instruct the execution of a computing operation that is unrelated to the data dimension of the target computing node.

[0029] In one possible design, the processing unit is further configured to: compile the plurality of calculation instructions when it is determined that the data dimension is fixed.

[0030] In one possible design, after compiling the first computation instruction among the plurality of computation instructions corresponding to the target computing node, the processing unit is further configured to: determine the current data dimension of the target computing node during the execution of the computing model; compile the second computation instruction among the plurality of computation instructions according to the current data dimension, wherein the second computation instruction is a computation instruction other than the first computation instruction among the plurality of computation instructions; and execute the computation operation corresponding to the plurality of computation instructions according to the compiled plurality of computation instructions.

[0031] In one possible design, when the judgment unit determines whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed, it is specifically used to: determine whether the data dimension is fixed based on the dimension indication information of the computing model; wherein the dimension indication information is used to indicate whether the data dimension is fixed; or, when the data dimension is the data dimension of the input data of the target computing node, determine whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node is fixed; wherein the first computing node is at least one computing node located before the target computing node; or, when the data dimension is the data dimension of the output data of the target computing node, determine whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed.

[0032] In one possible design, when the determination unit determines whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node is fixed, it is specifically used to: if the data dimension of the output data of the first computing node is fixed, then determine that the data dimension is fixed; otherwise, determine that the data dimension is changing.

[0033] In one possible design, when the determination unit determines whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed, it is specifically used for: determining that the data dimension changes when the data dimension of the input data of the target computing node changes; and determining whether the data dimension is fixed when the data dimension of the input data of the target computing node is fixed based on a set dimension analysis function and the data dimension of the input data of the target computing node, wherein the set dimension analysis function is used to determine whether the data dimension of the computing node is fixed based on the data dimension of the input data of the computing node.

[0034] In one possible design, when the processing unit executes the computational operations corresponding to the compiled plurality of computational instructions, it is specifically configured to: execute the computational operations corresponding to the compiled plurality of computational instructions through a first thread; the processing unit is further configured to: when executing the computational operations corresponding to the plurality of computational instructions through the first thread, execute the following steps through a second thread: determine the current data dimension of the second computing node, wherein the second computing node is at least one computing node located after the target computing node, and the data dimension of the second computing node is variable; compile a third computational instruction among the plurality of computational instructions corresponding to the second computing node according to the current data dimension of the second computing node, wherein the third computational instruction is used to indicate a computational operation related to the data dimension of the second computing node.

[0035] In one possible design, the processing unit is further configured to: execute multiple computation instructions corresponding to the compiled second computing node via the first thread.

[0036] Thirdly, this application provides a computing device including a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computing program stored in the memory to implement the method described in the first aspect or any possible design of the first aspect.

[0037] Fourthly, this application provides a computing device including at least one processor and an interface; the interface is used to provide program instructions or data to the at least one processor; the at least one processor is used to execute the program instructions to implement the method described in the first aspect or any possible design of the first aspect.

[0038] Fifthly, this application provides a computer-readable storage medium storing a computer program that, when run on a computing device, causes the computing device to perform the method described in the first aspect or any possible design of the first aspect.

[0039] In a sixth aspect, this application provides a computer program product comprising a computer program or instructions that, when executed by a computing device, implement the method described in the first aspect or any possible design of the first aspect.

[0040] In a seventh aspect, this application provides a chip system including at least one processor and an interface, the interface being used to provide program instructions or data to the at least one processor, the at least one processor being used to execute the program instructions to implement the method described in the first aspect or any possible design of the first aspect.

[0041] In one possible design, the chip system further includes a memory for storing program instructions and data.

[0042] In one possible design, the chip system consists of chips, or includes chips and other discrete devices.

[0043] For the beneficial effects described in the third to seventh aspects above, please refer to the description of the beneficial effects in the first or second aspects above, which will not be repeated here. Attached Figure Description

[0044] Figure 1 A schematic diagram of the system architecture for a possible application of the processing method of the computational model provided in the embodiments of this application;

[0045] Figure 2 A schematic diagram illustrating the processing flow of a compiler and executor provided in an embodiment of this application;

[0046] Figure 3 A schematic diagram illustrating a processing method for a computational model provided in an embodiment of this application;

[0047] Figure 4 A schematic diagram of a processing method for another computational model provided in an embodiment of this application;

[0048] Figure 5 A schematic diagram illustrating the processing flow of a compiler and executor provided in an embodiment of this application;

[0049] Figure 6 A schematic diagram of a computation instruction corresponding to an execution node provided in an embodiment of this application;

[0050] Figure 7 A schematic diagram illustrating a method for refreshing calculation instructions based on an object address mirror table, provided in an embodiment of this application;

[0051] Figure 8 A schematic diagram of a computing device provided in an embodiment of this application;

[0052] Figure 9 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. In the description of the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0054] For ease of understanding, exemplary descriptions of concepts related to this application are provided for reference.

[0055] 1) Compilation: Compilation is the process of converting a program written in one programming language (source language) into a program written in another language (target language). The source language can be the language used by the user when writing the target program, and the target language can be the language used by the device the user wants to run the target program on. For example, compilation can convert the high-level language used when writing the source program into binary language that the machine (such as a computer, actuator, etc.) can understand, so that the machine can recognize and execute it.

[0056] 2) Computational graphs: By defining the computational model and solving its parameters (which can be called model training), a unique computational logic can be determined. This computational logic, after transformation, can be applied to inference computation (which can also be called model inference or application). This computational logic can be represented by a graph, which is called a computational graph.

[0057] A computation graph, represented as a directed graph, defines how data flows, how it is computed, and the dependencies between various computations. The computation graph of a computational model consists of nodes and edges. Nodes represent applied mathematical operations, the start point of data input / output, or the end point of reading / writing persistent variables. Edges represent the input / output relationships between nodes. Edges can transmit dynamically scalable multidimensional data arrays, also known as tensors. Tensors are data structures used to represent data in a model; a tensor can correspond to an n-dimensional array or list, where n is an integer greater than or equal to zero. Tensors have two properties: shape and rank. Furthermore, tensors can flow between nodes in the computation graph.

[0058] Computation graphs can be divided into static computation graphs and dynamic computation graphs. In a static computation graph, the data dimensions of the input or output data of the computation nodes are fixed, while in a dynamic computation graph, the data dimensions of the input or output data of some or all of the computation nodes are dynamically changing.

[0059] It should be noted that, in the embodiments of this application, a node may also be referred to as a computing node, task node, computing task, operator, operation (OP), computation layer, etc., and a data dimension may also be referred to as dimension, shape, etc. Furthermore, the computing model described in the embodiments of this application may be a deep learning model, a neural network model, etc.

[0060] It should be understood that in the embodiments of this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple.

[0061] Currently, in fields such as artificial intelligence and machine learning, the following two methods are mainly used to compile computational models:

[0062] Method 1: Runtime compilation (just-in-time compiler, JIT)

[0063] In this approach, the model is compiled before execution each time it is run, based on the dimensions of the actual input data. Therefore, this method requires recompilation every time the model is run.

[0064] The compilation overhead of a typical computational model is far greater than its execution overhead, sometimes by hundreds of times. Therefore, when the data dimensions in a computational model vary widely and frequently, the time and resource overhead of recompilation far exceeds the execution overhead, resulting in very low compilation and execution efficiency. While techniques such as compiler caching can reduce the number of recompilations to some extent, they also introduce additional cache resources. When the data dimensions vary too much or even approach infinity, the hardware devices used to compile and execute the computational model may become unusable. Furthermore, in some edge computing scenarios, computational models already deployed on hardware devices need to support multiple specifications of raw input data (e.g., image resolution, sentence length), requiring redeployment after recompiling the model.

[0065] Method 2: Compile ahead of time

[0066] In this approach, the computational model can be pre-compiled for all possible data dimensions based on the range of possible data dimensions of the input data. In scenarios such as inference where precision is not particularly sensitive, some dedicated inference engines can divide a large range of data dimensions into multiple tiers. When executing the computational model, the actual input data is padded to the closest tier, and then the compiled computational model corresponding to the data dimensions of that tier is used for calculation.

[0067] This method incurs no additional compilation overhead, and since the data dimension information in the computational model is fixed after allocating specific data dimension levels, the fusion of dynamic computation graphs can achieve the same benefits as static computation graphs. However, this method requires the business to prepare all data dimension ranges in advance. For some businesses, such as the sparse models commonly used in recommendation services or detection models in detection-type businesses, the data dimension range can reach hundreds or even tens of thousands. Therefore, the compilation time and resource consumption in advance are non-negligible additional overheads. This method still suffers from the problem of high compilation time and resource consumption, leading to low compilation and execution efficiency of the model.

[0068] In view of this, embodiments of this application provide a method for processing a computational model. This method can be used to compile computational models with dynamic data dimensions, while improving the model compilation speed, reducing the resource overhead of model compilation, and thus improving the model compilation efficiency.

[0069] The processing method of the computational model provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0070] Figure 1 This is a schematic diagram of the system architecture for one possible application of the processing method for the computational model provided in the embodiments of this application. The system can be used to compile and run the computational model. For example, the system may include a compiler 101 and an executor (or execution engine) 102.

[0071] The computational model in this application embodiment includes multiple computational nodes, and any one computational node can be used to represent one or more computational operations in the computational model.

[0072] For example, a computational model can be used Figure 1 The computation graph shown represents the computational model, where computational nodes A, B, C, D, E, F, and G represent multiple computational operations. The connections between the computational nodes indicate the data flow and data dependencies between them. For example, if computational node A is connected to computational nodes B, C, and D, then the output data of node A can be input to nodes B, C, and D. Nodes B, C, and D can be considered data-dependent nodes, all depending on node A.

[0073] When implementing the computational model, the model file of the designed computational model can be loaded into the compiler 101. The compiler 101 compiles the computational model, thereby converting the model file into a binary file that the executor 102 can recognize. The binary file is then transmitted to the executor 102, which can then run the computational model according to the binary file to perform computational processing on the data input to the computational model.

[0074] Figure 2 This is a schematic diagram of the processing flow of a compiler and executor provided in an embodiment of this application.

[0075] like Figure 2As shown in diagram (a), when compiling the computational model, the compiler splits the computational nodes in the model into two parts: a known part and an unknown part. The data dimensions of the known part are fixed, while the data dimensions of the unknown part are dynamically changing, including the data dimensions of the input and / or output data. In other words, the data dimensions of the input and / or output data of the known part of the computational model are static and can be uniquely determined, while the data dimensions of the input and / or output data of the unknown part are dynamically changing and cannot be uniquely determined.

[0076] For the known computation nodes in the computation model, the compiler can perform full compilation, that is, compile all computation instructions corresponding to the computation node. The specific processing flow may include compilation preparation, optimization / fusion, node compilation, memory allocation, data stream allocation, and task generation.

[0077] For computation nodes in the computational model that are unknown, the compiler can perform partial compilation. Specifically, it can compile computation instructions corresponding to the computation nodes that are independent of the data dimensions of the input and output data. Here, computation instructions independent of the data dimensions of the input and output data can be understood as computation operations indicated by the instructions that do not contain any information related to the data dimensions of the input and / or output data. The specific processing flow may include compilation preparation, optimization / fusion, node compilation, and task generation.

[0078] It should be understood that the compiler does not strictly follow the operations described in the above process during actual compilation. The operations described above are only examples, and the compiler can reasonably adjust the process during actual execution, such as adding or removing some operations, or adjusting the execution order of some operations. For the specific implementation of the above operations, please refer to the compilation methods of model compilers in the field of machine learning, which will not be detailed here.

[0079] like Figure 2As shown in diagram (b), during the execution of the computational model compiled by the compiler, for computational nodes in the known part of the computational model, the execution worker of the executor can directly execute the computational operations corresponding to the compiled computational instructions. For computational nodes in the unknown part of the computational model, the update worker of the executor can first compile the computational instructions (i.e., computational instructions not compiled by the compiler in advance) related to the data dimensions of the input data and / or output data of the computational node according to the data dimensions of the current actual input data and / or output data of the computational node, and then execute the compiled computational instructions corresponding to the computational node.

[0080] For example, the above Figure 1 In the computational model shown, it is assumed that the data dimensions of the input and output data for computational nodes A, B, C, and F are fixed, while the data dimensions of the input and output data for computational nodes D, E, and G are variable. During compilation, the compiler performs full compilation of the computational instructions corresponding to computational nodes A, B, C, and F, and compiles only the computational instructions for computational nodes D, E, and G that are independent of data dimensions. When the executor runs the compiled computational model, it can directly execute the compiled computational instructions for computational nodes A, B, C, and F. For computational nodes D, E, and G, it first updates and compiles the uncompiled content based on the data dimensions of their input and output data before executing the compiled computational instructions for those nodes.

[0081] The above method combines pre-compilation and runtime update. On the one hand, during the compilation of the computation model, the compiler only needs to compile the computation model once, so it does not need to recompile the computation model every time the data dimension changes. On the other hand, during the operation of the computation model, the executor can drive the compilation of dynamic data dimension-related content according to the actual input data of the computing node, ensuring the normal operation of the computing node.

[0082] It should be noted that the above Figure 1 or Figure 2 The compiler and executor shown can be deployed in different devices or equipment, or they can be integrated in the same device or equipment. No specific limitation is made in the embodiments of this application.

[0083] For example, compilers and executors can be integrated onto the same computing device. In this embodiment, the computing device can also be referred to as a computer system. From a logical layering perspective, the computing device may include a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer may include hardware such as processors, memory, and memory resource managers. The operating system can be any one or more computer operating systems that implement business processing through processes, for example... operating system, operating system, operating system, operating system, Operating system or Operating systems, etc. The application layer can contain applications.

[0084] In the embodiments of this application, the computing device may be a terminal device such as a smartphone or a personal computer. This application does not impose any particular limitation, as long as it can read the program code of the method of the embodiment of this application and run the program code to process the computing model according to the processing method of the computing model of the embodiment of this application.

[0085] The processing method of the computational model provided in the embodiments of this application or Figure 1 The system shown can also be applied to the operating environment of data centers and other related equipment. Through software modification, it can achieve generalized support for accelerated execution of computing models with dynamic data dimension requirements.

[0086] For ease of explanation, the following describes the processing method of the computing model provided in the embodiments of this application using the example of a computing device executing a computing model. The computing device may include a compiler and a processor. The compiler is used to execute a method for compiling the computing model, and the executor is used to execute a method for running the computing model.

[0087] Figure 3 A schematic diagram of a processing method for a computational model provided in an embodiment of this application is shown below. Figure 3 As shown, the method includes:

[0088] S301: The compiler determines whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed. The computing model contains multiple computing nodes, and the target computing node is any one of the multiple computing nodes. The target computing node is used to represent multiple computing operations in the computing model.

[0089] S302: When the data dimension change is determined, the compiler compiles the first calculation instruction among the multiple calculation instructions corresponding to the target computing node; wherein, the multiple calculation instructions correspond one-to-one with the multiple calculation operations represented by the target computing node, any one of the calculation instructions is used to indicate the execution of the corresponding calculation operation, and the first calculation instruction is used to indicate a calculation operation that is not related to the data dimension of the target computing node.

[0090] Optionally, the method may further include the following step S303:

[0091] S303: When it is determined that the data dimension is fixed, the compiler compiles the plurality of calculation instructions.

[0092] In step S301 above, when the compiler determines whether the data dimensions of the input data and / or output data of the target computation node in the computation model are fixed, it can use at least one of the following methods:

[0093] Method 1: Determine whether the data dimension of the target computing node is fixed based on the dimension indication information of the computing model, wherein the dimension indication information is used to indicate whether the data dimension of the input data and / or output data of the target computing node is fixed.

[0094] In this method, the data dimension indication information can be preset or input by the user.

[0095] As an optional implementation, the data dimension indication information can directly indicate that the data dimension of the target computing node is fixed or directly indicate that the data dimension of the target computing node is changing.

[0096] As another optional implementation, when the data dimension of the target computing node remains fixed, the dimension indication information can indicate the data dimension of the target computing node. The compiler can determine that the data dimension of the target computing node is fixed based on this dimension indication information, and can also determine the specific data dimensions of the input data and / or output data of the target computing node. When the data dimension of the target computing node changes, the dimension indication information can indicate that the data dimension of the input data of the target computing node changes.

[0097] In existing methods for compiling computational models, the data dimension information of the input data for the first computational node is generally known and is input into the compiler along with the computational model to be compiled. Therefore, in some embodiments of this application, this method can be used to determine the data dimension of the first computational node in the computational model. Of course, the data dimension information of other computational nodes in the computational model can also be determined in this way, thereby speeding up the compiler's judgment.

[0098] Method 2: When the data dimension is the data dimension of the input data of the target computing node, determine whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node is fixed; wherein, the first computing node is at least one computing node located before the target computing node, and the input data of the target computing node is the output data of the first computing node.

[0099] This method can be used to determine the data dimensions of the input data from the second to the last computation node in a computation model.

[0100] In this method, if the compiler determines that the data dimension of the output data of the first computing node is fixed, then it determines that the data dimension of the input data of the target computing node is fixed; otherwise, it determines that the data dimension of the input data of the target computing node is not fixed.

[0101] For example, in such Figure 1 In the computational model shown, it is assumed that the data dimensions of both the input and output data of computation node A are fixed. Therefore, computation node A belongs to the category of computation nodes with determinable data dimensions. When the target computation node is computation node B, the first computation node is computation node A. Since the output data of computation node A is input to computation node B and serves as its input data, the data dimension of the output data of computation node A is the same as the data dimension of the input data of computation node B. Therefore, it can be determined that the data dimension of the input data of computation node B is also fixed, and computation node B also belongs to the category of computation nodes with determinable data dimensions.

[0102] In some embodiments of this application, when the target computing node is connected to multiple first computing nodes, the data dimension of the input data of the target computing node is determined to be fixed only if the data dimensions of the input data and output data of the multiple first computing nodes are both fixed. If the data dimension of the input data or output data of at least one of the multiple first computing nodes is dynamically changing, then the data dimension of the input data of the target computing node is determined to be dynamically changing.

[0103] Method 3: When the data dimension is the data dimension of the output data of the target computing node, determine whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed.

[0104] This method can be used to determine the data dimension of the output data of any computing node in the computing model.

[0105] In this method, if the compiler determines that the data dimension of the input data of the target computing node is fixed, it then determines whether the data dimension of the output data of the target computing node is fixed based on the set data dimension analysis function and the data dimension of the input data. The set data dimension analysis function is used to determine whether the data dimension of the output data of the computing node is fixed based on the data dimension of the input data. If it is determined that the data dimension of the input data of the target computing node is not fixed, then it is determined that the data dimension of the output data of the target computing node is not fixed.

[0106] Specifically, after determining the data dimension of the input data of the target computation node, the compiler can use a predefined data dimension analysis function to deduce the data dimension of the output data based on the data dimension of the input data. If a definite data dimension of the output data can be derived, then the data dimension of the output data is determined to be fixed; otherwise, the data dimension of the output data is determined to be non-fixed. The predefined data dimension analysis function can be the data dimension derivation function `infer shape()`.

[0107] In some embodiments of this application, when the compiler determines any output data of the target computing node or a change in the data dimension of the output data, it determines that the target computing node does not belong to a computing node with a determinable data dimension and can mark the target computing node as unknown; otherwise, it determines that the target computing node belongs to a computing node with a determinable data dimension and can mark the target computing node as known.

[0108] In some embodiments of this application, when the compiler determines that the data dimension of the output data of the target computing node is fixed, it passes the information to the next computing node, where the input data of the next computing node is the output data of the target computing node.

[0109] For example, in such Figure 1 In the computational model shown, if the data dimension of the input data of computation node A is fixed, the data dimension of the output data of computation node A can be derived using the infer shape() function. If a definite data dimension of the output data can be derived, then it can be determined that the data dimension of the output data of computation node A is fixed. Therefore, computation node A belongs to computation nodes with a determinable data dimension and can be marked as known.

[0110] For example, if the data dimension of the output data of computation node A remains fixed, it can be determined that the data dimension of the input data of computation node D is also fixed, and its data dimension is the same as the data dimension of the output data of computation node A. Then, the data dimension of the output data of computation node D can be derived using the `infer shape()` function. If a definite data dimension of the output data cannot be derived, it can be determined that the data dimension of the output data of computation node D is dynamically changing. Therefore, computation node D does not belong to the computation nodes with a determinable data dimension and can be marked as "unknown".

[0111] For example, when it's determined that the data dimension of the output data of computing node D has changed, computing node D outputs -1. -1 indicates that the data dimension of the output data is not fixed. If the next computing node E receives -1, it can be determined that the data dimension of the input data of computing node E has also changed. Therefore, computing node E does not belong to the category of computing nodes whose data dimension can be determined and can be marked as unknown. Furthermore, computing node E belongs to the category of computing nodes whose data dimension cannot be determined due to the blocking of the preceding computing node.

[0112] In some embodiments of this application, if there are other data transmission paths between two computing nodes that do not belong to a determinable data dimension, then the data dimensions of all computing nodes on that transmission path are not fixed.

[0113] For example, in such Figure 1 In the computational model shown, assuming that computational nodes A, D, E, and G on the determined transmission path ADEG are all marked as unknown, since there is another transmission path ACFG between computational nodes A and G, it can be determined that computational nodes C and F located between computational nodes A and G on this transmission path are computational nodes with undetermined data dimensions, and computational nodes C and F can be marked as unknown.

[0114] In steps S302 and S303 above, for a computation node with a known data dimension, the compiler compiles all computation instructions corresponding to the computation node to generate a complete execution task, i.e., the compiled computation instructions. For a computation node with an unknown data dimension, the compiler performs pre-compilation to generate an execution task, but the data dimension content that depends on the input and output data of the computation node is reserved and not generated. That is, among the multiple computation instructions corresponding to the computation node, the first computation instruction used to indicate computation operations that are not related to the data dimension of the target computation node is selected, the selected first computation instruction is compiled, and the remaining uncompiled second computation instructions are retained.

[0115] In the above embodiments, when compiling the computational model, the compiler can directly compile the computation instructions for computation nodes with fixed data dimensions. For computation instructions for computation nodes with dynamically changing data dimensions, only the computation instructions unrelated to the data dimension information are compiled. Based on this compilation method, when running the computational model after compilation, only the uncompiled computation instructions corresponding to the computation nodes need to be compiled, without having to recompile the already compiled computation instructions corresponding to the computation nodes. This method can adapt to scenarios with dynamically changing data dimensions, and the range of dynamic changes in data dimensions is not limited. Therefore, the compiled model has high versatility and ease of use. At the same time, only the computation instructions that do not depend on the data dimension need to be compiled once during the pre-run compilation process, and these computation instructions do not need to be recompiled during runtime. Therefore, it can significantly reduce compilation time and resource consumption, and improve the compilation efficiency of the model.

[0116] The above embodiments describe a method for a computing device's compiler to compile a computing model. The following describes a method for a computing device's executor to run the compiled computing model.

[0117] Figure 4 This is a schematic diagram illustrating a processing method for another computational model provided in an embodiment of this application. For example... Figure 4 As shown, in performing the above embodiments Figure 3 Following the method shown, the processing method for the computational model provided in this application embodiment may further include the following steps:

[0118] S401: If the executor determines that the multiple computation instructions corresponding to the target computing node are in a fully compiled state, then during the running of the computing model, the executor executes the computation operations corresponding to the multiple computation instructions according to the compiled multiple computation instructions.

[0119] In this case, if the executor determines that the data dimensions of the input and output data of the target computing node are fixed, or if it determines that all computing instructions contained in the target computing node have been compiled, and the computing instructions corresponding to the target computing node are in a fully compiled state, then the executor can directly execute the corresponding computing operation according to all computing instructions corresponding to the target computing node.

[0120] S402: If the executor determines that the multiple computation instructions corresponding to the target computing node are in a partially compiled state, then during the execution of the computing model, it determines the current data dimension of the target computing node, compiles the second computation instruction among the multiple computation instructions according to the current data dimension, and executes the computation operation corresponding to the multiple computation instructions according to the compiled multiple computation instructions; wherein, the second computation instruction is the computation instruction other than the first computation instruction among the multiple computation instructions.

[0121] When the executor determines that the data dimension of the input or output data of the target computing node has changed, or when it determines that the target computing node contains uncompiled computing instructions, it determines that the computing instructions corresponding to the target computing node are not in a fully compiled state, but in a partially compiled state.

[0122] When the executor compiles the second calculation instruction among the plurality of calculation instructions, it can determine the data dimensions of the input and output data of the target computing node based on the input data of the target computing node, and compile the second calculation instruction among the plurality of calculation instructions based on the data dimensions of the input and output data of the target computing node.

[0123] Furthermore, during the execution of the computational operations corresponding to the compiled computational instructions, the executor can simultaneously determine the data dimensions of the input and output data of the next computational node of the target computational node, and compile the computational instructions related to the data dimensions corresponding to the next computational node. Specifically, the executor can execute the computational operations corresponding to the multiple computational instructions through a first thread, while simultaneously determining the current data dimension of the second computational node through a second thread, and compiling the third computational instruction among the multiple computational instructions corresponding to the second computational node based on the current data dimension of the second computational node. Here, the second computational node is at least one computational node located after the target computational node, the input data of the second computational node is the output data of the target computational node, and the data dimension of the second computational node is variable; the third computational instruction is used to indicate the computational operations related to the data dimensions of the second computational node. After the computational operations corresponding to the multiple computational instructions are executed through the first thread, the compiled multiple computational instructions corresponding to the second computational node are executed again through the first thread.

[0124] The above method will be introduced below with specific examples.

[0125] Figure 5 This is a schematic diagram illustrating the processing flow of a compiler and executor provided in an embodiment of this application. Figure 5As shown, during the compilation process of the compiler on the input computational model, the process for handling the known computational nodes may include operations such as optimization, fusion, task generation, and resource allocation. The process for handling the unknown computational nodes may include operations such as optimization, fusion, and partial task generation. After compiling the input computational model, the compiler outputs it to the executor. After compilation, each computational node can correspond to a separate or optimized / fused computational node (operator) from before compilation.

[0126] like Figure 5 As shown, when the executor receives the compiled computational model, it sorts the computational nodes (computational tasks) in the compiled model and determines the compilation status of the corresponding computational nodes in turn according to the sorting. If it is determined that the computational node is in a fully compiled state, the computational tasks of that node are pushed into the execut queue. If it is determined that the computational node is in an incompletely compiled state, the computational tasks of that node are pushed into the update queue. The executor can sort the computational nodes in the compiled computational model according to the dependencies between the computational nodes, following either a depth-first or breadth-first search principle.

[0127] For computation nodes in the queue, the executor's execution thread can pop the computation tasks of the computation nodes from the queue and directly issue them for execution. Specifically, the execution thread sequentially performs operations such as loading computation instructions, executing computation instructions, and executing callbacks to complete the execution of the compiled computation instructions corresponding to the computation node.

[0128] The compute nodes in the update queue are in an incompletely compiled state. Therefore, the uncompiled compute instructions must first be compiled to obtain the complete compiled compute instructions corresponding to the compute node, and then the compiled complete compute instructions are executed. For compute nodes in the update queue, the executor's update thread can first obtain the output data of the previous compute node, use this output data as the input data of the target compute node, and deduce the data dimension of the target compute node's output data based on the data dimension of the input data. Then, based on the determined data dimensions of the target compute node's input and output data, the uncompiled compute instructions corresponding to the target compute node are compiled. Finally, based on the input data of the target compute node, all the compiled compute instructions corresponding to the target compute node are executed.

[0129] If the next computing node connected to the target computing node in the queue to be executed is also a computing node in the queue to be executed, then after the executor finishes executing the computing instructions corresponding to the target computing node, it will directly execute the computing instructions corresponding to the next computing node connected to the target computing node.

[0130] If the next computing node connected to the target computing node in the queue to be executed is a computing node in the queue to be updated, then after the executor finishes executing the computing instructions corresponding to the target computing node, it determines the data dimensions of the input and output data of the target computing node based on the actual input data of the target computing node, compiles the uncompiled computing instructions corresponding to the target computing node accordingly, and finally executes all the compiled computing instructions corresponding to the target computing node on the input data of the target computing node.

[0131] Figure 6 This is a schematic diagram illustrating the computation instructions corresponding to an execution node, as provided in an embodiment of this application. For example... Figure 6 As shown in the diagram, assume that the eight computational nodes in the pre-compilation computational model are ordered as computational node A, computational node B, computational node C, computational node D, computational node E, computational node F, computational node G, and computational node H. The data dimensions of the input and output data for computational nodes A, B, G, and H are fixed and can be fully compiled. The data dimensions of the input and output data for computational nodes C, D, E, and F are not fixed and require partial compilation.

[0132] The compute nodes in the compiled compute model include compute node K1, compute node C, compute node D, compute node E, compute node F, and compute node K2. During compilation, compute node A and compute node B can be merged into a single compute node K1, and compute node G and compute node H can be merged into a single compute node K2. This node fusion method avoids frequent back-and-forth reading and writing of intermediate data in registers and memory, thereby improving overall compute performance.

[0133] When the execution thread executes the compiled computation instructions corresponding to the computation node, if the target computation node is a fully compiled computation node, the computation instructions corresponding to the target computation node are executed directly; if the target computation node is a data-dependent computation node, when the executor starts executing the compiled computation instructions corresponding to the previous computation node of the target computation node, it simultaneously starts the update preparations such as data dimension inference, resource preparation, and strategy refresh, as well as some computation instruction compilation actions corresponding to the target computation node.

[0134] Ideally, after the compiled computation instructions for the preceding computation node of the target computation node have been executed, the update preparation and compilation of some computation instructions for the target computation node have also been completed. Then, the target computation node begins executing its compiled computation instructions, and simultaneously initiates update preparations such as data dimension derivation, resource preparation, and policy refresh for the next computation node, as well as the compilation of some computation instructions. This completely hides the overhead of updating computation nodes within the computational model, maximizing compiler utilization to approximately 100%.

[0135] For example, such as Figure 6 As shown, during the execution of the computation model, computation nodes C, D, E, and F are dependent computation nodes. If the executor only executes the computation instructions corresponding to each computation node sequentially on the execution thread, the compilation process of computation instructions related to data dimensions must be performed before executing the computation instructions corresponding to computation node C. This will generate additional overhead and cause the execution of computation operations between computation nodes to be discontinuous.

[0136] Therefore, in the method provided in this application embodiment, the operation instructions corresponding to the computing node can be executed by only one execution thread, while the compilation process of the computing instructions related to the data dimension can be executed by other threads. Specifically, as shown... Figure 6As shown, while the executor executes the computation instructions corresponding to computation node K1, it also performs preparatory processes such as data dimension derivation for node C and compilation of computation instructions related to the data dimensions through thread 1 corresponding to node C. After the execution thread finishes executing the computation instructions corresponding to computation node K1, the preparatory process for computation node C on thread 1 is also completed. At this point, all computation instructions corresponding to computation node C have been compiled, and the execution thread can directly execute the compiled computation instructions for computation node C based on the processing results of thread 1. This process continues, ensuring that the execution thread only executes the computation instructions corresponding to each computation node, while threads 1 to 5 execute the preparatory processes for each computation node. Therefore, it can hide the additional preparation overhead during the operation of the computation model and ensure the continuity of the execution of computation instructions corresponding to computation nodes. In this system, the output data of computing node C serves as the input data for computing node D. When the execution thread executes the computation instructions corresponding to computing node C, the data dimension of the output data of computing node C is already determined. Therefore, the data dimension of the input data of computing node D can be obtained. Thread 2, corresponding to computing node D, can simultaneously initiate the preparation process for computing node D, deriving the data dimension of the output data of computing node D based on the data dimension of its input data, and compiling the previously uncompiled computation instructions based on the determined data dimension. After the computation instructions of computing node C are executed, the executor has also completed the compilation of the previously uncompiled computation instructions for computing node D, and can then begin executing the computation instructions corresponding to computing node D.

[0137] In some embodiments of this application, when the executor runs the computation model, it can also send the computation instructions corresponding to the computation nodes in the computation model to other devices, and the other devices can perform the corresponding computation operations to obtain the computation results, thereby reducing the processing load of the executor and improving the processing efficiency.

[0138] Specifically, the computing device may further include at least one hardware device, which can be used to execute specific computing instructions. Based on this, the executor may include a host central processing unit (host CPU), which runs the compiled computing model. The host CPU can be connected to at least one hardware device, which may be a graphics processing unit (GPU), an embedded neural network processing unit (NPU), a coprocessor, etc. The host CPU and the at least one hardware device can be connected via an interface. The host CPU can copy the computing instructions corresponding to the computing nodes in the compiled computing model to the hardware device and notify the hardware device's task schedule to trigger the hardware device to execute the computing instructions.

[0139] Since the data dimensions of the input and output data of a fully compiled compute node do not change during the execution of the entire compute model, the resources occupied by a fully compiled compute node are persistent. When the allocator allocates memory resources for the compute node, it only needs to refresh the storage address of the compute node's input data and the storage address of its output data.

[0140] The allocator, also known as a memory resource manager, is used to manage data storage addresses in the hardware device. It should be noted that the allocator can be deployed on the hardware device or the executor, or it can be deployed independently; this embodiment does not impose specific limitations.

[0141] In some embodiments of this application, the computation instructions corresponding to the computing node include head control information and computation operation information. The head control information is used to indicate the data storage address of the input data and output data of the computing node, and the computation operation information is used to indicate the computation operation corresponding to the computing node.

[0142] In some embodiments of this application, the host CPU can add a Level 2 base address to the head control information of the computation instructions corresponding to the computing node. This Level 2 base address is used to determine the data storage address of the input data of the computing node. Simultaneously, the host CPU generates an argument (args) address table to represent the correspondence between the data storage addresses of the input and output data of the computing node and the computing node itself.

[0143] Figure 7 This is a schematic diagram illustrating a method for refreshing computation instructions based on an object address mirror table (args table mirror) provided in an embodiment of this application.

[0144] In some embodiments of this application, when the executor determines that the computing node is a fully compiled computing node during the execution of the computing model, it executes the computing instructions corresponding to the computing node and stores the output data to the data memory address of the output data after the execution is completed.

[0145] When the executor executes the computation instructions corresponding to the computing node, it writes the computation instructions into the hardware device through kernel startup (kernellaunch) and triggers the hardware device to directly execute the computation instructions to obtain the corresponding output data.

[0146] For example, such as Figure 7 As shown in diagram (b), if the current computing node is computing node 1 (op 1) as shown in diagram (b), then after the executor and hardware device execute the computing instructions corresponding to computing node 1, they return data to the data memory address (output ptr) of the output data.

[0147] When the executor determines that a computation node is not fully compiled during the execution of the computation model, it performs the following steps:

[0148] Step 1: The executor derives the data dimension of the output data of the current input computing node based on the data dimension of the current input computing node, and compiles the computation instructions that the compiler has not yet compiled based on the obtained data dimension.

[0149] like Figure 7 As shown in diagram (a), the executor can use the dimension derivation function (infershape funclib) to derive the data dimension of the output data of the computing node, and use the strategy optimization function (tiling funclib) to perform various preparatory operations before executing instructions, such as compiling the computation instructions that have not been compiled by the compiler based on the data dimension of the input and output data of the computing node.

[0150] Step 2: The executor sends the compiled calculation instructions to the hardware device, which then performs the corresponding calculation operations.

[0151] Step 3: When the data dimension of the input data of the compute node changes, the host CPU refreshes the address information in the object address mirror table according to the data memory address of the new input data.

[0152] When the executor executes the computing node for the first time, the host CPU requests a data memory address for the input data of the computing node, writes the correspondence between the obtained data memory address and the computing node into the object address mirror table, and then writes the computing instructions corresponding to the computing node into the device, which then executes the computing instructions.

[0153] Subsequently, when the data dimension of the input data of the compute node changes, the host CPU re-determines the size of the data memory required for the changed input data of the compute node, and then requests a new data memory address from the allocator. After obtaining the new data memory address, the executor looks up the address of the compute node's Level 2 base address in the object address mirror table, and refreshes the Level 2 base address using the obtained new data memory address, without needing to rewrite the computation instructions corresponding to the compute node into the device. After the address in the object address mirror table is refreshed, the device can determine the corresponding data memory address based on the refreshed Level 2 base address, retrieve the changed input data of the compute node from it, and execute the computation instructions on the input data to obtain the corresponding output data. Therefore, the above method can avoid the resource consumption of frequent data copying, computation instruction instruction delivery, and memory address passing between the host CPU and the hardware device.

[0154] For example, if the target computing node is Figure 7The computation node 2 shown in diagram (b) receives input data including the output data obtained by the previous computation node 1 based on input data 1, and additional input data 2. If the executor executes computation node 2 for the first time, it requests the allocation of a memory address corresponding to computation node 2. The allocator uses a two-level base addressing method to allocate data memory addresses for computation node 2. The data memory address allocated by the allocator is represented as a combination of an offset and a two-level base address. Combining the two-level base address and the corresponding offset determines the corresponding data memory address. For example, the data memory address of computation node 2 may include data memory address 1, data memory address 2, and data memory address 3. Data memory address 1 can be used to store the data input from computation node 1 to computation node 2. Data memory address 1 can be obtained based on the input 0 offset corresponding to the data input from computation node 1 to computation node 2 and base address 1. Data memory address 2 can contain different memory spaces, used to store input data 2, output data of computing node 2, and data storage for intermediate computation operations within computing node 2. Data memory address 2 can be obtained based on the input 1 offset and base address 2 corresponding to input data 2, the output 0 offset and base address 2 corresponding to the output data of computing node 2, and the workspace offset and base address 2 corresponding to the data from intermediate computation operations. Data memory address 3 can be used to store data related to various preparation operations, such as data generated during the compilation process of computation instructions related to data dimensions corresponding to computing node 2. Data memory address 3 can be obtained based on the tiling data offset and base address 3 corresponding to the preparation operation.

[0155] It should be noted that in the allocation and representation of data memory addresses, the same Level 2 base address can correspond to different offsets, and different combinations of Level 2 base addresses and offsets can correspond to different data memory addresses, or to different memory spaces within the same data memory address.

[0156] After obtaining the data memory address corresponding to compute node 2, the executor writes it into the object address mirror table and then sends the computation instructions corresponding to compute node 2 to the hardware device through the kernel launch process. The obtained data memory address of compute node 2 can be written to the hardware device via a host-to-device (H2D) process. If the executor is not executing compute node 2 for the first time, when it determines that the data dimension of the compute node has changed, the executor only needs to redetermine the offset corresponding to the data memory address and notify the hardware device of the change in the data memory address; it does not need to send the computation instructions corresponding to compute node 2 to the hardware device again through the kernel launch process.

[0157] In this embodiment of the application, the object address mirror table is used to represent the correspondence between the memory addresses of the input data and / or output data of the computing node and the computing node.

[0158] For example, such as Figure 7 In the object address mirror table shown in diagram (a), Conv_1, Bn_1, and Relu_1 represent three consecutive computing nodes, and Args_1ptr, Args_2ptr, and Args_3ptr are three consecutive data memory addresses, representing the three memory addresses allocated to the three computing nodes Conv_1, Bn_1, and Relu_1, respectively. When the data dimension of the input data of a computing node changes, the memory address allocated to that computing node (such as the size of the memory address space) also needs to be adjusted accordingly. Therefore, the memory addresses of subsequent computing nodes also need to be adjusted accordingly. Thus, when the data dimension of the input data of a computing node changes, the executor needs to request a new memory address for that computing node based on the changed data dimension of the input data, and refresh the memory addresses in the object address mirror table according to the new memory addresses and tiling results, thereby updating the memory addresses corresponding to each computing node in the computing model.

[0159] Step 4: After the data dimension of the input data of the computing node changes, the executor triggers the hardware device to execute the computing operation corresponding to the latest computing instruction.

[0160] After step 3 above is completed, the host CPU refreshes the address information in the object address mirror table and notifies the hardware device of the address change in the object address mirror table. The hardware device can then obtain the input data of the changed computing node based on the changed data memory address and execute the calculation instructions on the input data.

[0161] In the above embodiments, when compiling the computational model, the compiler of the computing device directly compiles all computation instructions corresponding to the computational nodes with fixed data dimensions, and only compiles the computation instructions corresponding to the computational nodes that are independent of the data dimensions for computational nodes with dynamically changing data dimensions. After compilation, when running the computational model, the uncompiled computation instructions corresponding to the computational nodes are compiled based on the actual input data of the computational nodes, and the computation instructions corresponding to the computational nodes can be executed. This can adapt to computational model processing scenarios where the data dimensions of the computational nodes change dynamically, and the range of dynamic changes in the data dimensions of the computational nodes is not limited. Therefore, the compiled model has high versatility and ease of use. At the same time, it avoids the repeated compilation of some computation instructions, which can improve the model compilation speed, greatly reduce compilation time and resource consumption, and thus improve the compilation efficiency of the computational model.

[0162] Based on the above embodiments and the same concept, this application also provides a computing device, such as... Figure 8 As shown, the computing device 800 may include:

[0163] The judgment unit 801 is used to determine whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed, wherein the computing model includes multiple computing nodes, the target computing node is any one of the multiple computing nodes, and the target computing node is used to represent multiple computing operations in the computing model; the processing unit 802 is used to compile a first computing instruction among the multiple computing instructions corresponding to the target computing node when it is determined that the data dimension has changed; wherein the multiple computing instructions correspond one-to-one with the multiple computing operations represented by the target computing node, any one computing instruction is used to instruct the execution of the corresponding computing operation, and the first computing instruction is used to instruct the execution of a computing operation that is unrelated to the data dimension of the target computing node.

[0164] In one possible design, the processing unit 802 is further configured to: compile the plurality of calculation instructions when it is determined that the data dimension is fixed.

[0165] In one possible design, after compiling the first computation instruction among the plurality of computation instructions corresponding to the target computing node, the processing unit 802 is further configured to: determine the current data dimension of the target computing node during the execution of the computing model; compile the second computation instruction among the plurality of computation instructions according to the current data dimension, wherein the second computation instruction is a computation instruction other than the first computation instruction among the plurality of computation instructions; and execute the computation operation corresponding to the plurality of computation instructions according to the compiled plurality of computation instructions.

[0166] In one possible design, when the judgment unit 801 judges whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed, it is specifically used to: determine whether the data dimension is fixed based on the dimension indication information of the computing model; wherein the dimension indication information is used to indicate whether the data dimension is fixed; or, when the data dimension is the data dimension of the input data of the target computing node, determine whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node is fixed; wherein the first computing node is at least one computing node located before the target computing node; or, when the data dimension is the data dimension of the output data of the target computing node, determine whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed.

[0167] In one possible design, when the determination unit 801 determines whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node is fixed, it is specifically used to: if the data dimension of the output data of the first computing node is fixed, then determine that the data dimension is fixed; otherwise, determine that the data dimension is changing.

[0168] In one possible design, when the judgment unit 801 determines whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed, it is specifically used for: determining that the data dimension changes when the data dimension of the input data of the target computing node changes; and determining whether the data dimension is fixed when the data dimension of the input data of the target computing node is fixed based on a set dimension analysis function and the data dimension of the input data of the target computing node, wherein the set dimension analysis function is used to determine whether the data dimension of the computing node is fixed based on the data dimension of the input data of the computing node.

[0169] In one possible design, when the processing unit 802 executes the computational operations corresponding to the compiled plurality of computational instructions, it is specifically configured to: execute the computational operations corresponding to the compiled plurality of computational instructions through a first thread; the processing unit 802 is further configured to: when executing the computational operations corresponding to the plurality of computational instructions through the first thread, execute the following steps through a second thread: determine the current data dimension of the second computing node, wherein the second computing node is at least one computing node located after the target computing node, and the data dimension of the second computing node is variable; compile a third computational instruction among the plurality of computational instructions corresponding to the second computing node according to the current data dimension of the second computing node, wherein the third computational instruction is used to indicate computational operations related to the data dimension of the second computing node.

[0170] In one possible design, the processing unit 802 is further configured to: execute multiple computation instructions corresponding to the compiled second computing node through the first thread.

[0171] The unit division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into a single processor, exist as separate physical units, or be integrated into a single unit. The integrated units described above can be implemented in hardware or as software functional units.

[0172] Figure 8 One or more of the various units within can be implemented using software, hardware, firmware, or a combination thereof. The software or firmware includes, but is not limited to, computer program instructions or code, and can be executed by a hardware processor. The hardware includes, but is not limited to, various integrated circuits, such as a central processing unit (CPU), a digital signal processor (DSP), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

[0173] Based on the above embodiments and the same concept, this application also provides a computing device for implementing the processing method of the computing model provided in this application. For example... Figure 9As shown, the computing device 900 may include one or more processors 901, a memory 902, and one or more computer programs (not shown in the figure). In one implementation, the aforementioned devices may be coupled via one or more communication lines 903. The memory 902 stores one or more computer programs, which include instructions; the processor 901 invokes the instructions stored in the memory 902, causing the computing device 900 to execute the software program processing method provided in this application embodiment.

[0174] In the embodiments of this application, the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0175] In the embodiments of this application, the memory can be volatile memory or non-volatile memory, or it can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. The memory in the embodiments of this application may also be circuitry or any other means capable of implementing storage functions.

[0176] In one implementation, the computing device 900 may further include a communication interface 904 for communicating with other devices via a transmission medium. For example, the computing device 900 can interact with a first server, a second server, or a database through the communication interface 904. In this embodiment, the communication interface may be a transceiver, a circuit, a bus, a module, or other types of communication interface. In this embodiment, when the communication interface is a transceiver, the transceiver may include an independent receiver, an independent transmitter, or a transceiver with integrated transceiver functions, or an interface circuit.

[0177] In some embodiments of this application, the processor 901, memory 902, and communication interface 904 can be interconnected via a communication line 903. The communication line 903 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication line 903 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0178] The methods provided in this application can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented in software, they can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., digital video disc (DVD), or semiconductor media (e.g., SSD), etc.

[0179] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A processing method of a computational model, characterized by, The method includes: Determine whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed and unchanging, wherein the computing model contains multiple computing nodes, the target computing node is any one of the multiple computing nodes, and the target computing node is used to represent multiple computing operations in the computing model; When the change in the data dimension is determined, the first calculation instruction among the multiple calculation instructions corresponding to the target computing node is compiled; wherein, the multiple calculation instructions correspond one-to-one with the multiple calculation operations represented by the target computing node, any one of the calculation instructions is used to instruct the execution of the corresponding calculation operation, and the first calculation instruction is used to instruct the calculation operation that is not related to the data dimension of the target computing node.

2. The method according to claim 1, characterized in that, The method further includes: When the data dimension is determined to be fixed, the multiple calculation instructions are compiled.

3. The method according to claim 1 or 2, characterized in that, After compiling the first computation instruction among multiple computation instructions corresponding to the target computing node, the method further includes: During the execution of the computing model, the current data dimension of the target computing node is determined, and the second computing instruction among the plurality of computing instructions is compiled according to the current data dimension, wherein the second computing instruction is a computing instruction other than the first computing instruction among the plurality of computing instructions; Based on the compiled multiple computation instructions, execute the computation operations corresponding to the multiple computation instructions.

4. The method according to claim 1 or 2, characterized in that, Determining whether the data dimensions of the input and / or output data of the target computation node in the computation model remain fixed includes: Based on the dimension indication information of the computational model, it is determined whether the data dimension is fixed; wherein, the dimension indication information is used to indicate whether the data dimension is fixed; or When the data dimension is the same as the data dimension of the input data of the target computing node, it is determined whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node remains constant; wherein, the first computing node is at least one computing node located before the target computing node; or When the data dimension is the same as the data dimension of the output data of the target computing node, it is determined whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed.

5. The method according to claim 4, characterized in that, Determining whether the data dimension is fixed based on whether the output data of the first computing node remains constant includes: If the data dimension of the output data of the first computing node remains fixed, then the data dimension is determined to be fixed; otherwise, the data dimension is determined to be variable.

6. The method according to claim 4, characterized in that, Determining whether the data dimension is fixed based on whether the input data dimension of the target computing node remains constant includes: When the data dimension of the input data of the target computing node changes, it is determined that the data dimension has changed; When the data dimension of the input data of the target computing node remains fixed, the data dimension is determined to be fixed based on the set dimension analysis function and the data dimension of the input data of the target computing node. The set dimension analysis function is used to determine whether the data dimension of the computing node is fixed based on the data dimension of the input data of the computing node.

7. The method according to claim 3, characterized in that, Based on the compiled plurality of computation instructions, execute the computation operations corresponding to the plurality of computation instructions, including: Based on the compiled multiple computation instructions, the computation operations corresponding to the multiple computation instructions are executed by the first thread; The method further includes: When the computational operations corresponding to the multiple computational instructions are executed through the first thread, the following steps are executed through the second thread: Determine the current data dimension of the second computing node, wherein the second computing node is at least one computing node located after the target computing node, and the data dimension of the second computing node is variable; Based on the current data dimension of the second computing node, the third computing instruction among the multiple computing instructions corresponding to the second computing node is compiled, wherein the third computing instruction is used to indicate the computing operation related to the data dimension of the second computing node.

8. The method according to claim 7, characterized in that, The method further includes: The first thread executes multiple computation instructions corresponding to the compiled second computing node.

9. A computing device, characterized in that, include: The judgment unit is used to determine whether the data dimension of the input data and / or output data of the target computing node in the computing model is fixed. The computing model includes multiple computing nodes, and the target computing node is any one of the multiple computing nodes. The target computing node is used to represent multiple computing operations in the computing model. The processing unit is configured to compile a first calculation instruction among a plurality of calculation instructions corresponding to the target computing node when the data dimension is determined to have changed; wherein the plurality of calculation instructions correspond one-to-one with a plurality of calculation operations represented by the target computing node, any one of the calculation instructions is used to instruct the execution of the corresponding calculation operation, and the first calculation instruction is used to instruct a calculation operation that is not related to the data dimension of the target computing node.

10. The apparatus according to claim 9, characterized in that, The processing unit is also used for: When the data dimension is determined to be fixed, the multiple calculation instructions are compiled.

11. The apparatus according to claim 9 or 10, characterized in that, After compiling the first computation instruction among multiple computation instructions corresponding to the target computing node, the processing unit is further configured to: During the execution of the computing model, the current data dimension of the target computing node is determined, and the second computing instruction among the plurality of computing instructions is compiled according to the current data dimension, wherein the second computing instruction is a computing instruction other than the first computing instruction among the plurality of computing instructions; Based on the compiled multiple computation instructions, execute the computation operations corresponding to the multiple computation instructions.

12. The apparatus according to claim 9 or 10, characterized in that, When the judgment unit determines whether the data dimensions of the input data and / or output data of the target computing node in the computing model are fixed, it is specifically used for: Based on the dimension indication information of the computational model, it is determined whether the data dimension is fixed; wherein, the dimension indication information is used to indicate whether the data dimension is fixed; or When the data dimension is the same as the data dimension of the input data of the target computing node, it is determined whether the data dimension is fixed based on whether the data dimension of the output data of the first computing node remains constant; wherein, the first computing node is at least one computing node located before the target computing node; or When the data dimension is the same as the data dimension of the output data of the target computing node, it is determined whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node is fixed.

13. The apparatus according to claim 12, characterized in that, When the judgment unit determines whether the data dimension of the output data of the first computing node is fixed, it is specifically used for: If the data dimension of the output data of the first computing node remains fixed, then the data dimension is determined to be fixed; otherwise, the data dimension is determined to be variable.

14. The apparatus according to claim 12, characterized in that, The judgment unit determines whether the data dimension is fixed based on whether the data dimension of the input data of the target computing node remains constant. Specifically, this is done by: When the data dimension of the input data of the target computing node changes, it is determined that the data dimension has changed; When the data dimension of the input data of the target computing node remains fixed, the data dimension is determined to be fixed based on the set dimension analysis function and the data dimension of the input data of the target computing node. The set dimension analysis function is used to determine whether the data dimension of the computing node is fixed based on the data dimension of the input data of the computing node.

15. The apparatus according to claim 11, characterized in that, When the processing unit executes the computation operations corresponding to the compiled plurality of computation instructions, it is specifically used for: Based on the compiled multiple computation instructions, the computation operations corresponding to the multiple computation instructions are executed by the first thread; The processing unit is also used for: When the computational operations corresponding to the multiple computational instructions are executed through the first thread, the following steps are executed through the second thread: Determine the current data dimension of the second computing node, wherein the second computing node is at least one computing node located after the target computing node, and the data dimension of the second computing node is variable; Based on the current data dimension of the second computing node, the third computing instruction among the multiple computing instructions corresponding to the second computing node is compiled, wherein the third computing instruction is used to indicate the computing operation related to the data dimension of the second computing node.

16. The apparatus according to claim 15, characterized in that, The processing unit is also used for: The first thread executes multiple computation instructions corresponding to the compiled second computing node.

17. A computing device, characterized in that, Includes memory and at least one processor; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory to implement the method as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when run on a computing device, causes the computing device to perform the method as described in any one of claims 1 to 8.

Images