Model compilation method and device, compiler and model running system

Abstract

The present disclosure provides a model compiling method and device, a compiler and a model running system, and relates to the technical field of artificial intelligence. The method comprises: in response to a running instruction, obtaining a model file corresponding to a machine learning model comprising a plurality of computing layers; parsing the model file to obtain a first computing graph executable by hardware, the first computing graph comprising a plurality of first operators, each first operator corresponding to at least one computing layer; determining running information of a first group of first operators according to input from a user and the first computing graph, the first group of first operators comprising at least some of the plurality of first operators; determining hardware configuration information corresponding to each first operator in the first group of first operators according to the running information of the first operator; and sending the hardware configuration information corresponding to each first operator in the plurality of first operators to a data flow-based executable device, so that the executable device performs the calculation corresponding to each first operator.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a model compilation method, apparatus, compiler, and model running system. Background Technology

[0002] Machine learning models have wide applications in artificial intelligence fields such as speech recognition, natural language processing, and image recognition and processing. The inference process of machine learning models typically involves a large number of computational tasks, requiring high computing power and often needing to run on executable devices capable of handling large amounts of computation.

[0003] In related technologies, machine learning models are typically compiled into executable program instructions, and then an instruction set-based executable device is used to execute the instructions of the executable program to run the machine learning model, thereby realizing the inference process of the machine learning model. Summary of the Invention

[0004] The inventors noted that the methods in the related technologies have certain limitations, namely that data stream-based executable devices cannot read the instructions of executable programs, which makes it impossible for data stream-based executable devices to run machine learning models.

[0005] To address the aforementioned problems, the present disclosure proposes the following solutions.

[0006] According to one aspect of the present disclosure, a model compilation method is provided, comprising: in response to a run instruction, acquiring a model file corresponding to a machine learning model including multiple computational layers; parsing the model file to obtain a hardware-executable first computational graph, the first computational graph including multiple first operators, each first operator corresponding to at least one computational layer; determining run information of a first group of first operators based on input from a user and the first computational graph, the first group of first operators including at least some of the multiple first operators; determining hardware configuration information corresponding to each first operator based on the run information of each first operator in the first group of first operators; and sending the hardware configuration information corresponding to each first operator in the multiple first operators to a data stream-based executable device so that the executable device performs computations corresponding to each first operator.

[0007] In some embodiments, the parsing further obtains static data required to perform the computation corresponding to each of the plurality of first operators; determining the running information of the first group of first operators based on the user input and the first computation graph includes: determining the output information of the first operator based on the input, the at least one computation layer corresponding to each of the first operators in the first group of first operators, and the static data of the first operator, wherein the running information of the first operator includes the output information of the first operator.

[0008] In some embodiments, the plurality of first operators include a first group of first operators and a second group of first operators other than the first group of first operators, and the method further includes: obtaining hardware configuration information corresponding to each first operator in the second group of operators.

[0009] In some embodiments, the method further includes: receiving structural information of the machine learning model input by the user, the structural information including the type, number, and connection method of the plurality of computational layers; generating a second computational graph corresponding to the machine learning model based on the structural information, the second computational graph including a plurality of second operators corresponding one-to-one with the plurality of computational layers; converting the second computational graph into a first computational graph, each of the plurality of first operators corresponding to at least one second operator; and obtaining the model file based on the first computational graph.

[0010] In some embodiments, the structural information further includes layer parameters for each of the plurality of computational layers, and the method further includes: determining static data of a second operator corresponding to the computational layer based on the layer parameters of each computational layer; obtaining static data required to perform the computation corresponding to the first operator based on the static data of at least one second operator corresponding to each of the plurality of first operators; obtaining the model file based on the first computational graph includes: serializing the first computational graph and the static data required to perform the computation corresponding to each of the plurality of first operators to obtain the model file.

[0011] In some embodiments, at least one of the plurality of first operators corresponds to a plurality of second operators.

[0012] In some embodiments, the second computation graph includes a computation subgraph, which is pre-generated based on the type, number, and connection method of at least two computation layers.

[0013] In some embodiments, the running instruction is received via a first application programming interface (API), and the structure information is received via a second API, wherein the first API and the second API are located in the same user interface.

[0014] In some embodiments, the executable device includes an artificial intelligence accelerator.

[0015] In some embodiments, the machine learning model is a neural network model.

[0016] According to another aspect of the present disclosure, a model compilation apparatus is provided, comprising: an acquisition module configured to acquire a model file corresponding to a machine learning model including multiple computational layers in response to a run instruction; a parsing module configured to parse the model file to obtain a hardware-executable first computational graph, the first computational graph including multiple first operators, each first operator corresponding to at least one computational layer; a first determining module configured to determine run information of a first group of first operators based on input from a user and the first computational graph, the first group of first operators including at least some of the multiple first operators; a second determining module configured to determine hardware configuration information corresponding to each first operator in the first group of first operators based on the run information of each first operator; and a sending module configured to send the hardware configuration information corresponding to each first operator in the multiple first operators to a data stream-based executable device, so that the executable device performs the computation corresponding to each first operator.

[0017] According to another aspect of the present disclosure, a model compilation apparatus is provided, comprising: a memory; and a processor coupled to the memory, the processor being configured to execute the method described in any of the above embodiments based on instructions stored in the memory.

[0018] According to another aspect of the present disclosure, a compiler is provided, including the model compilation apparatus described in any of the foregoing embodiments.

[0019] According to another aspect of the present disclosure, a model running system is provided, including: a compiler as described in any of the above embodiments; and an executable device, the executable device being configured to perform calculations corresponding to each of a plurality of first operators sent by the compiler as described in any of the above embodiments.

[0020] According to another aspect of the present disclosure, a computer-readable storage medium is provided, including computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method described in any of the above embodiments.

[0021] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements the method described in any of the above embodiments.

[0022] In this embodiment, based on user input and a first computational graph obtained by parsing the model file corresponding to the machine learning model, hardware configuration information of at least some first operators can be determined. Therefore, a dataflow-based executable device can execute computations corresponding to each first operator based on the hardware configuration information corresponding to each first operator. In this approach, compiling the machine learning model does not yield executable program instructions, but rather hardware configuration information corresponding to each first operator in the hardware-executable first computational graph. The dataflow-based executable device can run the machine learning model by reading the hardware configuration information corresponding to each first operator, thereby overcoming the technical problem that dataflow-based executable devices cannot run machine learning models.

[0023] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic flowchart of a model compilation method according to some embodiments of this disclosure;

[0026] Figure 2 This is a flowchart illustrating a model compilation method according to other embodiments of this disclosure;

[0027] Figure 3 This is a flowchart illustrating a model compilation method according to some embodiments of the present disclosure;

[0028] Figure 4 This is a schematic diagram of the structure of a model compilation apparatus according to some embodiments of the present disclosure;

[0029] Figure 5 This is a schematic diagram of the structure of a model compilation apparatus according to other embodiments of the present disclosure;

[0030] Figure 6 This is a schematic diagram of the structure of a model compilation apparatus according to some embodiments of the present disclosure;

[0031] Figure 7 This is a schematic diagram of the structure of a model operating system according to some embodiments of the present disclosure. Detailed Implementation

[0032] The technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0033] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0034] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0035] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0036] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0037] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0038] Figure 1 This is a schematic flowchart of a model compilation method according to some embodiments of the present disclosure.

[0039] In step 102, in response to the run instruction, the model file corresponding to the machine learning model, which includes multiple computing layers, is obtained.

[0040] In some embodiments, user input of execution instructions can be received through a first application programming interface (API). For example, the first API may be a C++ interface.

[0041] In some embodiments, the model file may be a binary file with the .bin extension.

[0042] In some embodiments, the machine learning model may be a neural network model, and the multiple computational layers may include convolutional layers, pooling layers, fully connected layers, etc. in the neural network model.

[0043] In step 104, the model file is parsed to obtain a hardware-executable first computational graph.

[0044] Here, the first computation graph includes multiple first operators, each of which corresponds to at least one computation layer.

[0045] In some embodiments, the type, number, and connection method of a plurality of first operators that can be executed by hardware can be obtained according to the first computation graph, wherein the type of the first operator can be determined according to the computation layer corresponding to each first operator, and the computation order between the multiple first operators can be determined according to the connection method between the first operators.

[0046] It should be understood that the computations required when a first operator is executed can be determined based on at least one computation layer corresponding to each first operator.

[0047] In some embodiments, a first operator may correspond to a computation layer. In this case, performing the computation corresponding to the first operator is equivalent to performing the computation corresponding to the computation layer. For example, if a first operator corresponds to a convolutional layer, then performing the computation corresponding to the first operator is equivalent to performing the computation corresponding to the convolutional layer.

[0048] In other embodiments, a first operator may correspond to multiple computation layers. In this case, performing the computation corresponding to the first operator is equivalent to performing multiple computations corresponding to the multiple computation layers. For example, if a first operator corresponds to a convolutional layer and a fully connected layer, then performing the computation corresponding to the first operator is equivalent to performing the computation corresponding to the convolutional layer and the computation corresponding to the fully connected layer.

[0049] In step 106, the running information of the first group of first operators is determined based on the input from the user and the first computation graph.

[0050] Here, the first group of first operators includes at least some of the first operators among a plurality of first operators.

[0051] In some embodiments, the runtime information may include input information for each first operator in the first group of first operators. For example, the input information for each first operator may include the size of the input to that first operator. Taking the input information including the size of the input as an example, the size of the input to the first first operator in the first computation graph can be determined based on the size of the input from the user, and the size of the input to other first operators can be determined based on the size of the output of the previous first operator. In some cases, the size of the output of the previous first operator depends only on the type of the previous first operator; in other cases, the size of the output of the previous first operator depends on the type of the previous first operator and the static data required to perform the computation corresponding to the previous first operator.

[0052] For example, the first computation graph may include two first operators, A and B, where the output of A serves as the input of B. Assuming the user input is an image with a pixel size of 100×100, then the input size of A is 100×100; assuming the output size of A, determined according to the first computation graph, is 50×50, then the input size of B is 50×50.

[0053] In some embodiments, the runtime information may include only the input information of each of the first operators in the first group of first operators.

[0054] In other embodiments, the operation information of the first group of first operators may include the input information and output information of each first operator in the first group of first operators. The following will describe, in conjunction with different embodiments, the case where the operation information also includes output information.

[0055] In step 108, the hardware configuration information corresponding to the first operator is determined based on the running information of each first operator in the first group of first operators.

[0056] In some embodiments, the hardware configuration information corresponding to the first operator may include the values ​​of configuration parameters required to configure the hardware to perform the calculation corresponding to the first operator. For example, if the operating information of a first operator includes the input size of 56×56×256, then the hardware configuration information corresponding to the first operator may include the values ​​of the configuration parameters ih, iw, and ic corresponding to the input size, i.e., ih = 56, iw = 56, and ic = 256. In the input size of 56×56×256, the first 56 represents the height of the input, the second 56 represents the width of the input, and 256 represents the number of input channels.

[0057] In step 110, the hardware configuration information corresponding to each of the multiple first operators is sent to the executable device so that the executable device can perform the calculation corresponding to each first operator.

[0058] In some embodiments, multiple first operators are identical to a first group of first operators. In this case, the hardware configuration information corresponding to each first operator needs to be dynamically determined based on user input.

[0059] In other embodiments, the plurality of first operators may include a first group of first operators and a second group of first operators other than the first group of first operators. In this case, the hardware configuration information of each first operator in the first group of first operators can be determined according to the specific implementation of steps 106 and 108, and the hardware configuration information corresponding to each first operator in the second group of first operators can be directly obtained.

[0060] In some embodiments, the executable device may include an Artificial Intelligence (AI) accelerator. For example, the AI ​​accelerator may be an AI scalar accelerator or an AI vector accelerator. In some cases, the computation corresponding to each first operator may be performed by the central processing unit (CPU) of the AI ​​accelerator.

[0061] In the above embodiments, based on the user input and the first computation graph obtained by parsing the model file corresponding to the machine learning model, the hardware configuration information of at least some of the first operators can be determined. Therefore, the dataflow-based executable device can execute the computation corresponding to each first operator according to the hardware configuration information corresponding to each first operator. In this way, the result of compiling the machine learning model is not the instructions for an executable program, but rather the hardware configuration information corresponding to each first operator in the hardware-executable first computation graph. The dataflow-based executable device can run the machine learning model by reading the hardware configuration information corresponding to each first operator, thereby overcoming the technical problem that dataflow-based executable devices cannot run machine learning models.

[0062] Furthermore, regardless of the size of the user's input, the operating information of at least some of the first operators in the first computation graph can be determined directly from the user's input and the first computation graph, without the need for additional processing of the user's input, thereby improving the operating efficiency of the machine learning model.

[0063] In some embodiments, parsing the model file corresponding to the machine learning model in step 104 can also obtain the static data required to perform the computation corresponding to each of the multiple first operators. It should be understood that the static data remains unchanged for a given first operator. For example, performing the computation corresponding to a first operator can be a convolution computation, and the static data for this first operator can be parameters such as the kernel size, stride size, and weights required to perform the convolution computation.

[0064] In some embodiments, the output information of a first operator can be determined based on user input, at least one computational layer corresponding to each first operator in the first group of first operators, and the static data of the first operator. Here, the operational information of the first operator may include the output information of the first operator. For example, the output information of each first operator may include the size of the output of the first operator. Taking the output information including the size of the output as an example, a certain first operator B1 in the first group of first operators may correspond to a convolutional layer. Assuming that the user input is a 256×256 image, based on the computation corresponding to B1 and the static data required to perform the computation corresponding to B1 (e.g., the size of the convolutional kernel, the size of the padding value, and the stride size, etc.), the size of the output of B1 can be determined to be 63×63.

[0065] Figure 2 This is a schematic flowchart of a model compilation method according to other embodiments of this disclosure.

[0066] In some embodiments, the model compilation method includes, in addition to Figure 1 In addition to the steps shown, it also includes Figure 2 Steps 202 to 208 are shown.

[0067] In step 202, the structural information of the machine learning model input by the user is received.

[0068] Here, the structural information includes the type, number, and connection method of multiple computing layers.

[0069] In some embodiments, structural information of the machine learning model input by the user can be received through a second application programming interface.

[0070] In some embodiments, the first application programming interface (API) and the second API can reside in the same user interface. This allows users to send structural information and operational instructions through the same user interface, reducing the complexity of user operations.

[0071] In some embodiments, the machine learning model input by the user can be a neural network model. In this case, the types of computational layers in the machine learning model can include convolution, pooling, and fully connected layers, etc. The computational order between multiple computational layers can be determined according to the connection method between them. For example, based on the connection method between convolutional layers, pooling layers, and fully connected layers in a neural network model, the computational order can be determined as follows: first, the convolutional layer is computed; then, the pooling layer is computed; and finally, the fully connected layer is computed.

[0072] In step 204, a second computational graph corresponding to the machine learning model is generated based on the structural information.

[0073] Here, the second computation graph includes multiple second operators that correspond one-to-one with multiple computation layers.

[0074] In some embodiments, the type, number, and connection method of multiple computation layers in a machine learning model can be obtained from the second computation graph.

[0075] In step 206, the second computation graph is converted into the first computation graph.

[0076] Here, the first computation graph includes multiple first operators, and each first operator corresponds to at least one second operator.

[0077] In some cases, a first operator can correspond to a second operator. In other cases, a first operator can correspond to multiple second operators.

[0078] In step 208, the model file is obtained based on the first calculation graph.

[0079] In some embodiments, the first computation graph can be serialized to obtain a model file corresponding to the machine learning model.

[0080] It should be understood that the structural information of multiple machine learning models can be pre-input to obtain the model file corresponding to each machine learning model, so that during operation, the model file corresponding to a certain machine learning model can be obtained in response to the running instruction of running a certain machine learning model.

[0081] In the above embodiments, during the compilation of the machine learning model into a model file, the user does not need to input the source code corresponding to the machine learning model, but can directly input the structural information of the machine learning model, further reducing the complexity of the user operation. Furthermore, after obtaining the model file, errors in the compilation process can be identified directly by viewing the model file. Compared to related technologies that require hardware execution of the executable program's instructions to find compilation errors, this improves the efficiency of error finding.

[0082] In some embodiments, at least one of the multiple first operators can correspond to multiple second operators. For example, during the process of converting a second computation graph into a hardware-executable first computation graph, the second operators corresponding to the convolutional layer and the second operators corresponding to the pooling layer can be fused into a single hardware-executable first operator according to the hardware execution requirements. Thus, during the execution of a machine learning model, the hardware executing the computation corresponding to one first operator is equivalent to executing the computation corresponding to multiple second operators, thereby further improving the running efficiency of the machine learning model.

[0083] In some embodiments, the second computation graph may include a computation subgraph, which may be pre-generated based on the type, number, and connection method of at least two computation layers. For example, for some frequently used computation layers, computation subgraphs may be pre-generated according to the type, number, and connection method of these computation layers, so that if the user inputs structural information containing the structural information corresponding to these computation layers, the corresponding computation subgraph can be directly invoked.

[0084] In the above embodiments, by pre-generating computational subgraphs, the generated computational subgraphs can be directly called during the generation of the second computational graph, thereby improving the compilation efficiency of the machine learning model.

[0085] Figure 3 This is a schematic flowchart of a model compilation method according to some embodiments of the present disclosure.

[0086] In some embodiments, the structural information of the machine learning model received as user input in step 202 may further include layer parameters for each of the multiple computational layers. In this case, the model compilation method, in addition to including... Figure 1 and Figure 2 In addition to the steps shown, it also includes Figure 3 Steps 302 to 306 are shown.

[0087] In step 302, the static data of the second operator corresponding to each computational layer is determined based on the layer parameters of each computational layer.

[0088] In some embodiments, the layer parameters of each computational layer can be rearranged according to the execution requirements of the hardware to determine the static data of the second operator corresponding to that computational layer. For example, the layer parameters of the convolutional layer include a kernel size of 50×50×32, and the hardware is an AI accelerator based on a dataflow architecture. Since executing the computation corresponding to the operator in the AI ​​accelerator based on the dataflow architecture requires pre-determining the arrangement order of the data in the computation module according to the execution requirements of the AI ​​accelerator, the pre-determined arrangement order of the data can be to arrange parameter 32 first, then parameter 50×50. Therefore, the static data of the second operator can be determined to include 32×50×50.

[0089] In step 304, static data required to perform the calculation corresponding to each of the multiple first operators is obtained based on the static data of at least one second operator corresponding to each of the multiple first operators.

[0090] In some embodiments, one first operator may correspond to one second operator. In this case, the static data required to perform the calculation corresponding to the first operator may be the static data of the second operator. In other embodiments, one first operator may correspond to multiple second operators. In this case, the static data required to perform the calculation corresponding to the first operator may include the static data of the multiple second operators.

[0091] In step 306, the first computation graph and the static data required to perform the computation corresponding to each first operator are serialized to obtain the model file.

[0092] In some embodiments, the first computation graph and the static data required to perform the computation corresponding to each first operator in the first computation graph can be serialized into a single model file. This avoids the time consumption caused by separately calling the first computation graph and the corresponding static data during runtime, as well as the runtime errors caused by the mismatch between the separately called first computation graph and the static data, thus improving the running efficiency and accuracy of the machine learning model.

[0093] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus embodiments, since they largely correspond to the method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0094] Figure 4 This is a schematic diagram of the structure of a model compilation apparatus according to some embodiments of the present disclosure.

[0095] like Figure 4 As shown, the model compilation device 400 includes: an acquisition module 401, a parsing model 402, a first model determination module 403, a second model determination module 404, and a sending module 405.

[0096] The acquisition module 401 is configured to acquire the model file corresponding to the machine learning model, which includes multiple computational layers, in response to a run instruction.

[0097] The parsing module 402 is configured to parse the model file to obtain a hardware-executable first computation graph, which includes a plurality of first operators, each corresponding to at least one computation layer.

[0098] The first determining module 403 is configured to determine the operation information of a first group of first operators based on input from the user and a first computation graph. The first group of first operators includes at least some of the first operators among a plurality of first operators.

[0099] The second determining module 404 is configured to determine the hardware configuration information corresponding to the first operator based on the running information of each first operator in the first group of first operators.

[0100] The sending module 405 is configured to send hardware configuration information corresponding to each of the plurality of first operators to a data stream-based executable device, so that the executable device can perform the calculation corresponding to each first operator.

[0101] In some embodiments, the parsing module 402 may also obtain static data required to perform calculations corresponding to each of the multiple first operators when parsing the model file. In this case, the parsing module 402 may also be configured to determine the output information of the first operator based on the user's input, at least one computational layer corresponding to each first operator in the first group of first operators, and the static data of the first operator, wherein the running information of the first operator includes the output information of the first operator.

[0102] In some embodiments, the plurality of first operators includes a first group of first operators and a second group of first operators excluding the first group of first operators. In these embodiments, the acquisition module 401 may also be configured to acquire hardware configuration information corresponding to each first operator in the second group of operators.

[0103] Figure 5 This is a schematic diagram of the structure of a model compilation apparatus according to other embodiments of the present disclosure.

[0104] like Figure 5 As shown, in some embodiments, the model compilation device 500 includes, in addition to, Figure 4 In addition to the modules in the model compilation device 400 shown, it also includes a receiving module 501, a generating module 502, a conversion module 503, and an obtaining module 504.

[0105] The receiving module 501 is configured to receive structural information of the machine learning model input by the user, including the type, number, and connection method of multiple computation layers.

[0106] The generation module 502 is configured to generate a second computation graph corresponding to the machine learning model based on the structural information input by the user. The second computation graph includes multiple second operators that correspond one-to-one with multiple computation layers.

[0107] In some embodiments, the generation module 502 can also be configured to pre-generate a computational subgraph based on the type, number, and connection method of at least two computational layers. In this case, the second computational graph corresponding to the machine learning model generated based on the structural information input by the user may include a computational subgraph.

[0108] The conversion module 503 is configured to convert the second computation graph into a first computation graph. In some cases, each of the plurality of first operators in the first computation graph corresponds to at least one second operator. In other cases, at least one of the plurality of first operators in the first computation graph corresponds to a plurality of second operators.

[0109] Module 504 is configured to obtain a model file based on the first computation graph.

[0110] In some embodiments, the structural information received by the receiving module 501 from user input may further include layer parameters for each of the plurality of computational layers. In this case, the model compilation apparatus 500 may further include a third determining module 505. The third determining module 505 is configured to determine the static data of the second operator corresponding to each computational layer based on the layer parameters of each computational layer. The obtaining module 504 may further be configured to obtain the static data required to perform the computation corresponding to the first operator based on the static data of at least one second operator corresponding to each of the plurality of first operators; and to serialize the first computation graph and the static data required to perform the computation corresponding to each of the plurality of first operators to obtain a model file.

[0111] Figure 6 This is a schematic diagram of the structure of a model compilation apparatus according to some embodiments of the present disclosure.

[0112] like Figure 6 As shown, the model compilation apparatus 600 includes a memory 601 and a processor 602 coupled to the memory 601. The processor 602 is configured to execute the method of any of the above embodiments based on instructions stored in the memory 601.

[0113] The memory 601 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory may store, for example, an operating system, application programs, a boot loader, and other programs.

[0114] The model compilation device 600 may also include an input / output interface 603, a network interface 604, and a storage interface 605. These interfaces 603, 604, and 605, as well as the memory 601 and processor 602, can be connected, for example, via a bus 606. The input / output interface 603 provides a connection interface for input / output devices such as displays, mice, keyboards, and touchscreens. The network interface 604 provides a connection interface for various networked devices. The storage interface 605 provides a connection interface for external storage devices such as SD cards and USB flash drives.

[0115] This disclosure also provides a compiler, including the model compilation apparatus of any of the above embodiments.

[0116] Figure 7 This is a schematic diagram of the structure of a model operating system according to some embodiments of the present disclosure.

[0117] like Figure 7As shown, the model execution system 700 includes a compiler 701 and an executable device 702. The compiler 701 includes the model compilation apparatus of any of the above embodiments, such as model compilation apparatus 400 / 500 / 600. The executable device 702 is configured to perform calculations corresponding to each of the plurality of first operators sent by the compiler 701, based on hardware configuration information corresponding to each first operator.

[0118] This disclosure also provides a computer-readable storage medium including computer program instructions, wherein the computer program instructions, when executed by a processor, implement the method of any of the above embodiments.

[0119] This disclosure also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, it implements the method of any of the above embodiments.

[0120] The embodiments of this disclosure have now been described in detail. To avoid obscuring the concept of this disclosure, some details known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein based on the above description.

[0121] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0122] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that the functions specified in one or more flowchart illustrations and / or one or more blocks in a block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate functions for implementing the functions in the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] While specific embodiments of this disclosure have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments or equivalent substitutions can be made to some technical features without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A model compilation method, comprising: In response to the run instruction, retrieve the model file corresponding to the machine learning model, which includes multiple computational layers; The model file is parsed to obtain a hardware-executable first computation graph, which includes a plurality of first operators, each of which corresponds to at least one computation layer; Based on user input and the first computation graph, determine the operation information of the first group of first operators, wherein the first group of first operators includes at least some of the plurality of first operators; Based on the running information of each first operator in the first group of first operators, the hardware configuration information corresponding to the first operator is determined. The running information of each first operator in the first group of first operators includes the input information of the first operator, and the hardware configuration information corresponding to each first operator includes the values ​​of the configuration parameters required to configure the hardware to perform the calculation corresponding to the first operator. The hardware configuration information corresponding to each of the plurality of first operators is sent to the data stream-based executable device so that the executable device can perform the calculation corresponding to each first operator.

2. The method according to claim 1, wherein, Parsing the model file also yields the static data required to perform the calculations corresponding to each of the plurality of first operators; Based on the user input and the first computation graph, the runtime information of the first group of first operators is determined as follows: Based on the input from the user, the at least one computational layer corresponding to each first operator in the first group of first operators, and the static data of the first operator, the output information of the first operator is determined, and the running information of the first operator also includes the output information of the first operator.

3. The method according to claim 1, wherein, The plurality of first operators includes a first group of first operators and a second group of first operators excluding the first group of first operators. The method further includes: Obtain the hardware configuration information corresponding to each first operator in the second group of first operators.

4. The method according to claim 1, further comprising: The system receives structural information about the machine learning model input by the user, the structural information including the type, number, and connection method of the multiple computational layers; A second computation graph corresponding to the machine learning model is generated based on the structural information. The second computation graph includes multiple second operators that correspond one-to-one with the multiple computation layers. The second computation graph is converted into the first computation graph, and each of the plurality of first operators corresponds to at least one second operator; The model file is obtained based on the first computational graph.

5. The method according to claim 4, wherein, The structural information also includes layer parameters for each of the plurality of computation layers, and the method further includes: The static data of the second operator corresponding to each computational layer are determined based on the layer parameters of each computational layer. Based on the static data of at least one second operator corresponding to each of the plurality of first operators, the static data required to perform the calculation corresponding to the first operator is obtained; The model file obtained based on the first computational graph includes: The first computation graph and the static data required to perform the computation corresponding to each of the plurality of first operators are serialized to obtain the model file.

6. The method according to claim 4, wherein, At least one of the plurality of first operators corresponds to a plurality of second operators.

7. The method according to claim 4, wherein, The second computation graph includes a computation subgraph, which is pre-generated based on the type, number, and connection method of at least two computation layers.

8. The method according to claim 4, wherein, The system receives the operation instruction via a first application programming interface (API) and the structure information via a second API, wherein the first API and the second API are located in the same user interface.

9. The method according to any one of claims 1-8, wherein, The executable device includes an artificial intelligence accelerator.

10. The method according to any one of claims 1-8, wherein, The machine learning model is a neural network model.

11. A model compilation apparatus, comprising: The acquisition module is configured to retrieve model files corresponding to machine learning models that include multiple computational layers in response to a run instruction; The parsing module is configured to parse the model file to obtain a hardware-executable first computation graph, the first computation graph including a plurality of first operators, each first operator corresponding to at least one computation layer; The first determining module is configured to determine the operation information of a first group of first operators based on input from the user and the first computation graph, wherein the first group of first operators includes at least a portion of the plurality of first operators; The second determining module is configured to determine the hardware configuration information corresponding to the first operator based on the running information of each first operator in the first group of first operators. The running information of each first operator in the first group of first operators includes the input information of the first operator, and the hardware configuration information corresponding to each first operator includes the values ​​of the configuration parameters required to configure the hardware to perform the calculation corresponding to the first operator. The sending module is configured to send hardware configuration information corresponding to each of the plurality of first operators to a data stream-based executable device, so that the executable device performs the calculation corresponding to each first operator.

12. A model compilation apparatus, comprising: Memory; as well as A processor coupled to the memory is configured to execute the method of any one of claims 1-10 based on instructions stored in the memory.

13. A compiler comprising the model compilation apparatus of claim 11 or 12.

14. A model running system, comprising: The compiler as described in claim 13; as well as An executable device configured to perform computations corresponding to each of a plurality of first operators sent by the compiler, based on hardware configuration information corresponding to each first operator.

15. A computer-readable storage medium comprising computer program instructions, wherein, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1-10.

16. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the method described in any one of claims 1-10.

Images