Probabilistic model ai domain compiler system and method based on multi-level intermediate representation

Abstract

This invention provides a probabilistic model compilation system and method for the AI ​​field based on multi-level intermediate representations, comprising: a front-end conversion layer: taking the model described in Python as input, traversing the abstract syntax tree to map the operation nodes to operation nodes in the MLIR dialect and outputting them; a compilation layer: taking the IR represented in the MLIR dialect as input, decomposing, downgrading, and optimizing the operations; and a back-end layer: integrating two back-ends, LLVM JIT and IREE, receiving the final form of the IR as input, generating corresponding code according to the target platform, and performing calculations. This invention's probabilistic model compilation method based on multi-level intermediate representations uses static univariate assignment as the intermediate representation result for compiling and running probabilistic programming code, converting it into corresponding back-end code according to front-end configuration requirements, thus increasing the portability of the probabilistic model.

Description

Technical Field

[0001] This invention relates to the field of computer compilers, and more specifically, to a compilation system and method for probabilistic models in the field of AI based on multi-level intermediate representations. Background Technology

[0002] Currently, probabilistic programming systems can be divided into two main categories: one is specifically designed for particular inference algorithms, such as BUGS, Stan, and Infer.NET; the other focuses on general-purpose Turing-complete languages, known as general-purpose probabilistic programming languages, such as WebPPL, Church, Edward, Pyro, and BLOG. BUGS, an early probabilistic programming system launched in 2014, proposed a modeling language that can describe a directed graphical model. It primarily applies Gibbs sampling to hierarchical Bayesian models and uses Markov Monte Carlo (MCMC) methods for inference. Stan, similar to BUGS, proposed its own modeling language, focusing on Hamiltonian Monte Carlo (HMC) algorithms. Therefore, the two differ significantly in their applicable scope. BUGS is mainly suitable for various types of hierarchical Bayesian models, while Stan is mainly suitable for models with jointly differentiable probabilities, supports a very limited set of variational inference algorithms, and uses Gaussian variational distributions and reparameterized gradient estimators for continuous latent variables. Infer.NET, based on the .NET language, is primarily designed for message-propagation-based inference algorithms in probabilistic models. These algorithms often employ strong assumptions, including Variational Message Passing (VMP) and Expectation Propagation (EP). These specialized probabilistic programming systems are designed for specific inference algorithms and are applied in specific fields such as mathematics and statistics.

[0003] Probabilistic programming systems offer more comprehensive model language description capabilities, supporting high-level programming languages ​​and enabling the expression of more complex and diverse models. Church, built on a subset of the Scheme language, employs relatively general sampling algorithms in its original implementation, such as rejection sampling. However, due to the lack of guaranteed efficiency, it is difficult to apply to real-world machine learning problems. Venture, similar to Church, specifies probabilistic models through a Turing-complete high-level programming language, maintaining the language's versatility while supporting flexible algorithm combinations, thus enhancing the flexibility of the inference system. Venture also supports variational inference algorithms based on the REINFORCE gradient estimator. Anglican, built on the Clojure language, differs in that it introduces particle-based inference algorithms, including sequence Monte Carlo and particle Markov chain Monte Carlo. WebPPL is a lightweight probabilistic programming language built on JavaScript, supporting the construction of probabilistic models and the execution of statistical inference. Its unique feature is that it is designed specifically for browser use, allowing web developers to easily integrate WebPPL into existing applications, providing an interactive, real-time probabilistic inference platform. PyMC3 is a probabilistic programming Python library based on Theano, utilizing Bayesian inference. It provides high-level interfaces for defining models and performing inference, supporting Markov Monte Carlo and variational inference. Gen is a probabilistic programming language based on Julia, supporting various inference methods, including Sequence Monte Carlo (SMC), Hamiltonian Monte Carlo (HMC), and variational inference. Edward is a Python library developed by Columbia University in collaboration with Adobe Research, Google Research, and others. Built on top of TensorFlow, it is used for probabilistic modeling, inference, and evaluation, supporting everything from classic hierarchical models on small datasets to complex models on large datasets. Edward inherits the advantages of TensorFlow, supporting graph computation, automatic differentiation, and cross-platform capabilities. Pyro is an open-source probabilistic programming language from Uber AI Labs, based on PyTorch. It combines the advantages of deep learning and Bayesian modeling, focusing on variational inference, supporting composable inference algorithms, and flexibly expressing complex probabilistic models. Pyro and Edward are built on PyTorch and TensorFlow respectively, making them suitable for applications in the field of deep learning.

[0004] With the increasing importance of probabilistic models, various probabilistic programming languages ​​and frameworks have emerged, providing interfaces and computational libraries for expressing probabilistic models. Different probabilistic programming languages ​​and frameworks have their own focuses, as well as some drawbacks and limitations: (1) High computational cost: When dealing with large datasets or complex models, a large amount of computational resources are required and time is consumed due to the need to infer from many variables. (2) Difficult debugging and performance optimization: Existing systems usually implement specific functions by calling underlying C / C++ libraries, such as generating random numbers. The opaque calling process makes it difficult to debug and perform targeted optimization. (3) Lack of standardization: It is difficult to integrate with existing systems and has poor portability. Probabilistic programming systems can be divided into two main categories: general-purpose and special-purpose. Special-purpose probabilistic programming languages ​​are designed for specific inference algorithms and have unique syntax and usage methods; general-purpose probabilistic programming languages ​​are usually built on Turing-complete language systems. The large variety and differences between languages ​​make it difficult for users to compare and reproduce models between different frameworks, and also make it difficult to deploy and run on different platforms. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a probabilistic model compilation system and method for the AI ​​field based on multi-level intermediate representations.

[0006] According to the present invention, a probabilistic model AI domain compilation system based on multi-level intermediate representation includes:

[0007] Front-end transformation layer: Taking the model described in Python as input, it traverses the abstract syntax tree to map the operation nodes to operation nodes in the MLIR dialect and outputs them;

[0008] Compilation layer: Takes IR represented in MLIR dialect as input, and decomposes, downgrades and optimizes the operations;

[0009] Backend layer: Integrates two backends, LLVM JIT and IREE, receives the final form of IR as input, generates corresponding code and performs calculations according to the target platform, and performs feature extraction and image reconstruction.

[0010] Preferably, in the front-end conversion layer:

[0011] Abstract syntax trees (ASTs) are used to convert Python programs into intermediate representations. An AST is an intermediate product from Python source code to bytecode. The nodes of the tree contain node type, operation type, and parameters, and can be traversed and modified. It maps Python operations one by one to operations in a dialect and outputs them as intermediate representations in text form.

[0012] Obtain the abstract syntax tree structure, traverse the AST nodes in a top-down order of module, function, task, and return, perform type checking and construct a symbol table during the process, insert the constructed intermediate representation dialect operation information as attributes into the corresponding nodes, and output the transformation result of the intermediate representation when the traversal is completed.

[0013] Preferably, the conversion process from Python source code to intermediate representation is completed before the original program is executed, avoiding modifications to the code that exceed the preset standard. Python function decorators are used to add functionalities without modifying the original function. A specified decorator function is added before the original function; the instructions in the decorator are executed first, returning a new function object to replace the original function. The front-end conversion layer inserts a conversion function `@backend` before the source code function as a decorator. This function traverses and rewrites the abstract syntax tree, constructs dialect operation nodes, generates the Interpreter Representation (IR), and calls the compilation layer interface within the conversion function to downgrade the preset high-level dialect to an external dialect or a lower-level dialect. The back-end layer then compiles this into a callable function. Corresponding interfaces are called for different back-end platforms, using parameterized function decorators to pass the specified back-end platform to the conversion process. The conversion to IR is executed once at function definition, compiled once, and used multiple times.

[0014] Static type checking is performed during compilation, and type information is written into intermediate representation code and passed along with the intermediate representation conversion until assembly code or machine instructions are generated.

[0015] Preferably, in the compilation layer:

[0016] The compilation layer performs a downgrade transformation on the intermediate representation output by the front-end layer, reducing it to a form acceptable to the back-end runtime layer. During the downgrade transformation, optimization operations are inserted, including common subexpression elimination and dead code elimination. The compilation layer is implemented using the C++ language and the TableGen mechanism, providing Python with callable APIs.

[0017] The compiler layer processes input parameters as follows:

[0018] Register all dialects that can be used during the conversion process with the DialectRegistry. The list of dialects that can be used is loaded into the context when the actual conversion operation occurs.

[0019] Register all supported conversion processes with PassRegistry, including built-in dialect conversion processes, inter-dialect conversion processes, and general conversion optimization processes;

[0020] Parse the input command and populate the command configuration options, including the input IR file, output file, and transformation target;

[0021] Obtain the PassManager object, add the required Passes to it, and build the compilation pipeline;

[0022] The intermediate representation code of the input is parsed, and the compilation pipeline is started through PassManager. The Passes in the pipeline are executed in sequence to transform the input IR and write the result to the target file.

[0023] Preferably, the compilation layer performs multi-level transformation and rewriting on the intermediate code represented by the custom dialect, downgrading it to the LLVM dialect or the TOSA dialect, which can be read and executed by the LLVM JIT and IREE backends respectively. One transformation process is one pass.

[0024] Pass is defined through the TableGen mechanism and implemented in C++; the underlying implementation of probability distribution operations is implemented at the compilation layer, which degrades the sampling operations from the probability distribution defined by the pre-defined high-level abstract dialect into control flow and basic operation operations; the implementation of the uniform distribution sampling algorithm generates a random sequence that follows a standard uniform distribution through a linear congruential algorithm, and then transforms the standard uniform distribution result according to the distribution parameters;

[0025] For the normal distribution, the Box-Muller algorithm is used to sample the standard normal distribution based on the uniform distribution. Then, the sampling results of the standard normal distribution are linearly transformed according to the mean and variance of the target distribution: two sets of independent random numbers U1 and U2 are generated based on the standard uniform distribution. The following transformation is applied to generate independent random variables Z0 and Z1 that follow a normal distribution.

[0026]

[0027]

[0028] For the exponential distribution, the probability density function f(x; λ) = e^(-λ / λ) is obtained using the inverse transform method. {-λx} For exponential distribution sampling; for multinomial distribution, the idea of ​​rejection sampling is learned, and the Alias ​​Method is used to implement sampling by constructing probability tables and alias tables;

[0029] At the compilation layer, sampling operations are converted into intermediate representation code using the Standard and Linalg dialects built into MLIR. Then, the runtime layer generates the corresponding backend code based on the parameter configuration and performs the computation. When facing an LLVM backend, the compilation layer converts the IR into an LLVM IR, which is then loaded and run by the LLVM JIT. When facing an IREE backend, the custom high-level IR is converted into a TOSA or MHLO IR, and the code is executed by dylib or vulkan as the backend driver.

[0030] The present invention provides a method for compiling probabilistic models in the AI ​​domain based on multi-level intermediate representations, comprising:

[0031] Step S1: The front-end transformation layer takes the model described in Python as input, traverses the abstract syntax tree to map the operation nodes to operation nodes in the MLIR dialect and outputs them;

[0032] Step S2: The compilation layer takes the IR represented by the MLIR dialect as input and decomposes, downgrades, and optimizes the operation;

[0033] Step S3: The backend layer integrates two backends, LLVM JIT and IREE, receives the final form of IR as input, generates corresponding code according to the target platform and performs calculations to extract features and reconstruct the image.

[0034] Preferably, in step S1:

[0035] Abstract syntax trees (ASTs) are used to convert Python programs into intermediate representations. An AST is an intermediate product from Python source code to bytecode. The nodes of the tree contain node type, operation type, and parameters, and can be traversed and modified. It maps Python operations one by one to operations in a dialect and outputs them as intermediate representations in text form.

[0036] Obtain the abstract syntax tree structure, traverse the AST nodes in a top-down order of module, function, task, and return, perform type checking and construct a symbol table during the process, insert the constructed intermediate representation dialect operation information as attributes into the corresponding nodes, and output the transformation result of the intermediate representation when the traversal is completed.

[0037] Preferably, the conversion process from Python source code to intermediate representation is completed before the original program is executed, avoiding modifications to the code that exceed the preset standard. Python function decorators are used to add functionalities without modifying the original function. A specified decorator function is added before the original function; the instructions in the decorator are executed first, returning a new function object to replace the original function. The front-end conversion layer inserts a conversion function `@backend` before the source code function as a decorator. This function traverses and rewrites the abstract syntax tree, constructs dialect operation nodes, generates the Interpreter Representation (IR), and calls the compilation layer interface within the conversion function to downgrade the preset high-level dialect to an external dialect or a lower-level dialect. The back-end layer then compiles this into a callable function. Corresponding interfaces are called for different back-end platforms, using parameterized function decorators to pass the specified back-end platform to the conversion process. The conversion to IR is executed once at function definition, compiled once, and used multiple times.

[0038] Static type checking is performed during compilation, and type information is written into intermediate representation code and passed along with the intermediate representation conversion until assembly code or machine instructions are generated.

[0039] Preferably, in step S2:

[0040] The compilation layer performs a downgrade transformation on the intermediate representation output by the front-end layer, reducing it to a form acceptable to the back-end runtime layer. During the downgrade transformation, optimization operations are inserted, including common subexpression elimination and dead code elimination. The compilation layer is implemented using the C++ language and the TableGen mechanism, providing Python with callable APIs.

[0041] The compiler layer processes input parameters as follows:

[0042] Register all dialects that can be used during the conversion process with the DialectRegistry. The list of dialects that can be used is loaded into the context when the actual conversion operation occurs.

[0043] Register all supported conversion processes with PassRegistry, including built-in dialect conversion processes, inter-dialect conversion processes, and general conversion optimization processes;

[0044] Parse the input command and populate the command configuration options, including the input IR file, output file, and transformation target;

[0045] Obtain the PassManager object, add the required Passes to it, and build the compilation pipeline;

[0046] The intermediate representation code of the input is parsed, and the compilation pipeline is started through PassManager. The Passes in the pipeline are executed in sequence to transform the input IR and write the result to the target file.

[0047] Preferably, the compilation layer performs multi-level transformation and rewriting on the intermediate code represented by the custom dialect, downgrading it to the LLVM dialect or the TOSA dialect, which can be read and executed by the LLVM JIT and IREE backends respectively. One transformation process is one pass.

[0048] Pass is defined through the TableGen mechanism and implemented in C++; the underlying implementation of probability distribution operations is implemented at the compilation layer, which degrades the sampling operations from the probability distribution defined by the pre-defined high-level abstract dialect into control flow and basic operation operations; the implementation of the uniform distribution sampling algorithm generates a random sequence that follows a standard uniform distribution through a linear congruential algorithm, and then transforms the standard uniform distribution result according to the distribution parameters;

[0049] For the normal distribution, the Box-Muller algorithm is used to sample the standard normal distribution based on the uniform distribution. Then, the sampling results of the standard normal distribution are linearly transformed according to the mean and variance of the target distribution: two sets of independent random numbers U1 and U2 are generated based on the standard uniform distribution. The following transformation is applied to generate independent random variables Z0 and Z1 that follow a normal distribution.

[0050]

[0051]

[0052] For the exponential distribution, the probability density function f(x; λ) = e^(-λ / λ) is obtained using the inverse transform method. {-λx} For exponential distribution sampling; for multinomial distribution, the idea of ​​rejection sampling is learned, and the Alias ​​Method is used to implement sampling by constructing probability tables and alias tables;

[0053] At the compilation layer, sampling operations are converted into intermediate representation code using the Standard and Linalg dialects built into MLIR. Then, the runtime layer generates the corresponding backend code based on the parameter configuration and performs the computation. When facing an LLVM backend, the compilation layer converts the IR into an LLVM IR, which is then loaded and run by the LLVM JIT. When facing an IREE backend, the custom high-level IR is converted into a TOSA or MHLO IR, and the code is executed by dylib or vulkan as the backend driver.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The probability model compilation method based on multi-level intermediate representation of the present invention uses static single variable assignment as the intermediate representation result for compiling and running probability programming program code, and converts it into corresponding back-end code according to front-end configuration requirements, thereby increasing the portability of the probability model;

[0056] 2. The probability model compilation method based on multi-level intermediate representation of this invention converts the process of generating random numbers conforming to a specific probability distribution using LCG, Box-Muller, and inverse transformation methods into the intermediate dialect of MLIR, increasing the possibility of multi-terminal conversion. Attached Figure Description

[0057] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0058] Figure 1 This is a diagram illustrating the overall architecture of a probabilistic model compilation method based on multi-level intermediate representations.

[0059] Figure 2This is a schematic diagram of the source code to IR conversion process;

[0060] Figure 3 This is a diagram illustrating static type checking.

[0061] Figure 4 This is a schematic diagram of a uniform distribution sampling algorithm;

[0062] Figure 5 This is a schematic diagram of uniformly distributed SSA IR results. Detailed Implementation

[0063] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0064] Example 1:

[0065] This invention relates to the fields of computer compilers, high-level programming languages, and probabilistic programming, and particularly to the design and implementation of a compiler for probabilistic programming models constructed through a multi-level intermediate representation.

[0066] According to the present invention, a probabilistic model AI domain compilation system based on multi-level intermediate representation is provided, such as... Figures 1-5 As shown, it includes:

[0067] Front-end transformation layer: Taking the model described in Python as input, it traverses the abstract syntax tree to map the operation nodes to operation nodes in the MLIR dialect and outputs them;

[0068] Specifically, in the front-end conversion layer:

[0069] Abstract syntax trees (ASTs) are used to convert Python programs into intermediate representations. An AST is an intermediate product from Python source code to bytecode. The nodes of the tree contain node type, operation type, and parameters, and can be traversed and modified. It maps Python operations one by one to operations in a dialect and outputs them as intermediate representations in text form.

[0070] Obtain the abstract syntax tree structure, traverse the AST nodes in a top-down order of module, function, task, and return, perform type checking and construct a symbol table during the process, insert the constructed intermediate representation dialect operation information as attributes into the corresponding nodes, and output the transformation result of the intermediate representation when the traversal is completed.

[0071] Specifically, the conversion process from Python source code to intermediate representation is completed before the original program is executed, avoiding modifications to the code that exceed the preset standards. Python function decorators are used to add features without modifying the original functions. A specified decorator function is added before the original function; the instructions in the decorator are executed first, returning a new function object to replace the original function. The front-end conversion layer inserts a conversion function `@backend` before the source code functions as a decorator. This function traverses and rewrites the abstract syntax tree, constructs dialect operation nodes, generates the Interpreter Representation (IR), and calls the compilation layer interface within the conversion function to downgrade the preset high-level dialect to an external dialect or a lower-level dialect. The back-end layer then compiles this into a callable function. Different interfaces are called for different back-end platforms, using parameterized function decorators to pass the specified back-end platform to the conversion process. The conversion to IR is executed once at function definition, compiled once, and used multiple times.

[0072] Static type checking is performed during compilation, and type information is written into intermediate representation code and passed along with the intermediate representation conversion until assembly code or machine instructions are generated.

[0073] Compilation layer: Takes IR represented in MLIR dialect as input, and decomposes, downgrades and optimizes the operations;

[0074] Specifically, in the compilation layer:

[0075] The compilation layer performs a downgrade transformation on the intermediate representation output by the front-end layer, reducing it to a form acceptable to the back-end runtime layer. During the downgrade transformation, optimization operations are inserted, including common subexpression elimination and dead code elimination. The compilation layer is implemented using the C++ language and the TableGen mechanism, providing Python with callable APIs.

[0076] The compiler layer processes input parameters as follows:

[0077] Register all dialects that can be used during the conversion process with the DialectRegistry. The list of dialects that can be used is loaded into the context when the actual conversion operation occurs.

[0078] Register all supported conversion processes with PassRegistry, including built-in dialect conversion processes, inter-dialect conversion processes, and general conversion optimization processes;

[0079] Parse the input command and populate the command configuration options, including the input IR file, output file, and transformation target;

[0080] Obtain the PassManager object, add the required Passes to it, and build the compilation pipeline;

[0081] The intermediate representation code of the input is parsed, and the compilation pipeline is started through PassManager. The Passes in the pipeline are executed in sequence to transform the input IR and write the result to the target file.

[0082] Specifically, the compilation layer performs multi-level transformations and rewrites on the intermediate code represented by the custom dialect, downgrading it to the LLVM dialect or the TOSA dialect, which can be read and executed by the LLVM JIT and IREE backends respectively. One transformation process is one pass.

[0083] Pass is defined through the TableGen mechanism and implemented in C++; the underlying implementation of probability distribution operations is implemented at the compilation layer, which degrades the sampling operations from the probability distribution defined by the pre-defined high-level abstract dialect into control flow and basic operation operations; the implementation of the uniform distribution sampling algorithm generates a random sequence that follows a standard uniform distribution through a linear congruential algorithm, and then transforms the standard uniform distribution result according to the distribution parameters;

[0084] For the normal distribution, the Box-Muller algorithm is used to sample the standard normal distribution based on the uniform distribution. Then, the sampling results of the standard normal distribution are linearly transformed according to the mean and variance of the target distribution: two sets of independent random numbers U1 and U2 are generated based on the standard uniform distribution. The following transformation is applied to generate independent random variables Z0 and Z1 that follow a normal distribution.

[0085]

[0086]

[0087] For the exponential distribution, the probability density function f(x; λ) = e^(-λ / λ) is obtained using the inverse transform method. {-λx} For exponential distribution sampling; for multinomial distribution, the idea of ​​rejection sampling is learned, and the Alias ​​Method is used to implement sampling by constructing probability tables and alias tables;

[0088] At the compilation layer, sampling operations are converted into intermediate representation code using the Standard and Linalg dialects built into MLIR. Then, the runtime layer generates the corresponding backend code based on the parameter configuration and performs the computation. When facing an LLVM backend, the compilation layer converts the IR into an LLVM IR, which is then loaded and run by the LLVM JIT. When facing an IREE backend, the custom high-level IR is converted into a TOSA or MHLO IR, and the code is executed by dylib or vulkan as the backend driver.

[0089] Backend layer: Integrates two backends, LLVM JIT and IREE, receives the final form of IR as input, generates corresponding code and performs calculations according to the target platform, and performs feature extraction and image reconstruction.

[0090] Example 2:

[0091] Example 2 is a preferred embodiment of Example 1, and is used to illustrate the present invention in more detail.

[0092] The present invention also provides a probabilistic model AI domain compilation system based on multi-level intermediate representation. The probabilistic model AI domain compilation system based on multi-level intermediate representation can be implemented by executing the process steps of the probabilistic model AI domain compilation method based on multi-level intermediate representation. That is, those skilled in the art can understand the probabilistic model AI domain compilation method based on multi-level intermediate representation as a preferred embodiment of the probabilistic model AI domain compilation system based on multi-level intermediate representation.

[0093] The present invention provides a method for compiling probabilistic models in the AI ​​domain based on multi-level intermediate representations, comprising:

[0094] Step S1: The front-end transformation layer takes the model described in Python as input, traverses the abstract syntax tree to map the operation nodes to operation nodes in the MLIR dialect and outputs them;

[0095] Specifically, in step S1:

[0096] Abstract syntax trees (ASTs) are used to convert Python programs into intermediate representations. An AST is an intermediate product from Python source code to bytecode. The nodes of the tree contain node type, operation type, and parameters, and can be traversed and modified. It maps Python operations one by one to operations in a dialect and outputs them as intermediate representations in text form.

[0097] Obtain the abstract syntax tree structure, traverse the AST nodes in a top-down order of module, function, task, and return, perform type checking and construct a symbol table during the process, insert the constructed intermediate representation dialect operation information as attributes into the corresponding nodes, and output the transformation result of the intermediate representation when the traversal is completed.

[0098] Specifically, the conversion process from Python source code to intermediate representation is completed before the original program is executed, avoiding modifications to the code that exceed the preset standards. Python function decorators are used to add features without modifying the original functions. A specified decorator function is added before the original function; the instructions in the decorator are executed first, returning a new function object to replace the original function. The front-end conversion layer inserts a conversion function `@backend` before the source code functions as a decorator. This function traverses and rewrites the abstract syntax tree, constructs dialect operation nodes, generates the Interpreter Representation (IR), and calls the compilation layer interface within the conversion function to downgrade the preset high-level dialect to an external dialect or a lower-level dialect. The back-end layer then compiles this into a callable function. Different interfaces are called for different back-end platforms, using parameterized function decorators to pass the specified back-end platform to the conversion process. The conversion to IR is executed once at function definition, compiled once, and used multiple times.

[0099] Static type checking is performed during compilation, and type information is written into intermediate representation code and passed along with the intermediate representation conversion until assembly code or machine instructions are generated.

[0100] Step S2: The compilation layer takes the IR represented by the MLIR dialect as input and decomposes, downgrades, and optimizes the operation;

[0101] Specifically, in step S2:

[0102] The compilation layer performs a downgrade transformation on the intermediate representation output by the front-end layer, reducing it to a form acceptable to the back-end runtime layer. During the downgrade transformation, optimization operations are inserted, including common subexpression elimination and dead code elimination. The compilation layer is implemented using the C++ language and the TableGen mechanism, providing Python with callable APIs.

[0103] The compiler layer processes input parameters as follows:

[0104] Register all dialects that can be used during the conversion process with the DialectRegistry. The list of dialects that can be used is loaded into the context when the actual conversion operation occurs.

[0105] Register all supported conversion processes with PassRegistry, including built-in dialect conversion processes, inter-dialect conversion processes, and general conversion optimization processes;

[0106] Parse the input command and populate the command configuration options, including the input IR file, output file, and transformation target;

[0107] Obtain the PassManager object, add the required Passes to it, and build the compilation pipeline;

[0108] The intermediate representation code of the input is parsed, and the compilation pipeline is started through PassManager. The Passes in the pipeline are executed in sequence to transform the input IR and write the result to the target file.

[0109] Specifically, the compilation layer performs multi-level transformations and rewrites on the intermediate code represented by the custom dialect, downgrading it to the LLVM dialect or the TOSA dialect, which can be read and executed by the LLVM JIT and IREE backends respectively. One transformation process is one pass.

[0110] Pass is defined through the TableGen mechanism and implemented in C++; the underlying implementation of probability distribution operations is implemented at the compilation layer, which degrades the sampling operations from the probability distribution defined by the pre-defined high-level abstract dialect into control flow and basic operation operations; the implementation of the uniform distribution sampling algorithm generates a random sequence that follows a standard uniform distribution through a linear congruential algorithm, and then transforms the standard uniform distribution result according to the distribution parameters;

[0111] For the normal distribution, the Box-Muller algorithm is used to sample the standard normal distribution based on the uniform distribution. Then, the sampling results of the standard normal distribution are linearly transformed according to the mean and variance of the target distribution: two sets of independent random numbers U1 and U2 are generated based on the standard uniform distribution. The following transformation is applied to generate independent random variables Z0 and Z1 that follow a normal distribution.

[0112]

[0113]

[0114] For the exponential distribution, the probability density function f(x; λ) = e^(-λ / λ) is obtained using the inverse transform method. {-λx} For exponential distribution sampling; for multinomial distribution, the idea of ​​rejection sampling is learned, and the Alias ​​Method is used to implement sampling by constructing probability tables and alias tables;

[0115] At the compilation layer, sampling operations are converted into intermediate representation code using the Standard and Linalg dialects built into MLIR. Then, the runtime layer generates the corresponding backend code based on the parameter configuration and performs the computation. When facing an LLVM backend, the compilation layer converts the IR into an LLVM IR, which is then loaded and run by the LLVM JIT. When facing an IREE backend, the custom high-level IR is converted into a TOSA or MHLO IR, and the code is executed by dylib or vulkan as the backend driver.

[0116] Step S3: The backend layer integrates two backends, LLVM JIT and IREE, receives the final form of IR as input, generates corresponding code according to the target platform and performs calculations to extract features and reconstruct the image.

[0117] Example 3:

[0118] Example 3 is a preferred example of Example 1, and is used to illustrate the present invention in more detail.

[0119] The invention will be described in detail with reference to the accompanying drawings.

[0120] Figure 1 This paper presents the overall architecture diagram of a probabilistic model compilation framework built based on a multi-level intermediate representation. Functionally, the work can be divided into a front-end conversion layer, a compilation layer, and a back-end runtime layer. Data transmission between layers is achieved through text-based intermediate representations and simple APIs (Application Programming Interfaces). This layered design effectively reduces the coupling of the framework.

[0121] The front-end transformation layer takes the model described in Python as input, traverses the abstract syntax tree to map the operation nodes to operation nodes in the MLIR (Multi-Level Intermediate Representation) dialect and outputs them;

[0122] The compilation layer takes the Intermediate Representation (IR) of the high-level dialect as input, and decomposes, downgrades and optimizes the complex operations in it. For example, it converts the calculation of probability density function into a combination of basic operations such as multiplication and addition according to mathematical formulas or algorithms, and outputs the rewritten IR.

[0123] The backend layer receives the final form of the IR as input, generates the corresponding code based on the target platform, and performs the actual computation. This invention integrates two backends: LLVM JIT (LLVM Just-In-Time) and IREE (Intermediate Representation Execution Environment). LLVM targets the CPU, while IREE supports both CPU and GPU platforms.

[0124] The front-end conversion layer utilizes an Abstract Syntax Tree (AST) to convert Python programs into intermediate representations. An AST is an intermediate product in the conversion from Python source code to bytecode. The nodes of the tree contain important information such as node type, operation type, and parameters, and can be traversed and modified. Constructing the intermediate representation requires mapping each operation in Python to an operation in a dialect and outputting the intermediate representation as text. This invention obtains the AST structure, traverses the AST nodes in a top-down order of Module-Function-Assign-Return, performs type checking and constructs a symbol table during the process, inserts the dialect operation information of the constructed intermediate representation as attributes into the corresponding nodes, and outputs the conversion result of the intermediate representation upon completion of the traversal. The conversion process from Python source code to intermediate representation should be completed before the original program is executed, while avoiding intrusive modifications to the code. Python function decorators can meet these requirements. A Python decorator is a special function that takes another function as a parameter, adding functionality to the original function without modifying it; it is usually a wrapper around the original function. Adding a decorator function specified by the "@" symbol before the original function will first execute the instructions in the decorator, returning a new function object to replace the original function. The front-end transformation layer inserts a transformation function `@backend` as a decorator before the source code function, traversing and rewriting the abstract syntax tree, constructing dialect operation nodes, and generating the IR. However, at this point, the IR only consists of high-level dialects and cannot be used as input for back-end computation. Therefore, the transformation function calls the compiler layer interface to downgrade the high-level dialect to external dialects such as TOSA (Tensor Operator Set Architecture), LLVM, or lower-level dialects, which are then compiled into callable functions by the back-end layer. Different interfaces are called for different back-end platforms; therefore, parameterized function decorators are used to pass the specified back-end platform to the transformation process. The above IR transformation process is executed only once at function definition, i.e., "compile once, use many times." A diagram of the transformation process is shown below. Figure 2 As shown.

[0125] To reduce the overhead of runtime type checking, this invention performs static type checking at compile time and writes type information into intermediate representation code, which is then passed along with the intermediate representation's conversions until assembly code or machine instructions are generated. Python provides a simple and clean type system; all its types derive from the Object base class, and variable definitions and declarations are not subject to strong type constraints. This flexibility allows Python to be quickly applied in many fields. As an interpreted language, Python only checks type consistency when necessary, checking the existence of methods required by a given program—a method known as duck typing. While flexible typing facilitates rapid program development, such as in web applications, it has not been effective in many areas, instead leading to performance penalties. For example, in deep learning, computations are often performed on basic types such as integers and floating-point numbers; even though CPython implements these basic types in C code, there is still significant overhead. Figure 3 The right side uses floating-point types as an example to demonstrate the static type inference process, which converts the floating-point variable x into a tensor. <f32>Encoded into the IR of the dialect representation, it carries type information during the conversion from high-level dialect to low-level dialect, and is converted to llvm.ptr when descending to LLVM dialect. <f32>The type represents floating-point numbers, and the final generated machine code. This is suitable for scenarios where the data type of the dataset is typically known and fixed, taking into account the characteristics of probabilistic models.

[0126] The compilation layer performs a downgrade transformation on the intermediate representation output by the front-end layer, reducing it to a form acceptable to the back-end runtime layer, such as LLVM IR. During the downgrade transformation, optimization operations such as common subexpression elimination and dead code elimination are inserted. The compilation layer is implemented using C++ and the TableGen mechanism, providing a callable API to Python. The compilation layer's processing of input parameters is as follows:

[0127] Register all available dialects during the conversion process with the DialectRegistry. The list of available dialects will be loaded into the context when the actual conversion operation occurs.

[0128] Register all supported conversion processes with PassRegistry, including built-in dialect conversion processes, inter-dialect conversion processes, and general conversion optimization processes;

[0129] Parse the input commands and populate the command configuration options, including parameters such as the input IR file, output file, and transformation target.

[0130] Obtain the PassManager object, add the required Passes to it, and build the compilation pipeline;

[0131] The intermediate representation code of the input is parsed, and the compilation pipeline is started through PassManager. The Passes in the pipeline are executed in sequence to transform the input IR and write the result to the target file.

[0132] The compiler layer performs multi-level transformations and rewrites on the intermediate code represented by the custom dialect, downgrading it to the LLVM dialect or the TOSA dialect, which can be read and executed by the LLVM JIT and IREE backends, respectively. One transformation process is called a Pass. Passes are defined through the TableGen mechanism and implemented in C++. This invention implements the low-level implementation of probability distribution operations at the compiler layer, degrading the sampling operations from probability distributions defined in the high-level abstract dialect into control flow and basic computational operations. The implementation of the uniform distribution sampling algorithm is as follows: Figure 4 As shown, a random sequence following a standard uniform distribution is generated using a linear congruential algorithm, and then the result of the standard uniform distribution is transformed according to the parameters of the distribution.

[0133] For the normal distribution, the Box-Muller algorithm is used to sample the standard normal distribution based on the uniform distribution. Then, the sampling results of the standard normal distribution are linearly transformed according to the mean and variance of the target distribution: First, two sets of independent random numbers U1 and U2 are generated based on the standard uniform distribution. The following transformation is applied to generate independent random variables Z0 and Z1 that follow a normal distribution.

[0134]

[0135]

[0136] For the exponential distribution, the probability density function f(x; λ) = e^(-λ / λ) is obtained using the inverse transform method. {-λx} For exponential distribution sampling, and for multinomial distributions, the idea of ​​rejection sampling is learned, and sampling is achieved by constructing probability tables and alias tables using the Alias ​​Method.

[0137] This invention converts the sampling operation into intermediate representation code in the MLIR's built-in Standard and Linalg dialects at the compilation layer. Figure 5 This demonstrates the intermediate representation results of the uniformly distributed sampling operation transformation. The runtime layer then generates the corresponding backend code based on the parameter configuration and performs the computation. For LLVM backends, the compilation layer converts the IR to LLVM IR, which is then loaded and run by the LLVM JIT. For IREE backends, the custom high-level IR is converted to TOSA or MHLO (Meta High Level Operations) IR, and the code is executed by dylib or vulkan as the backend driver.

[0138] This invention facilitates the rapid migration of deep learning models to different backends by specifying the runtime platform through decorators inserted into designated functions. It has been successfully implemented on Variational Auto Encoder (VAE) models. Using the VAE model as an example, a PyTorch-described frontend model is converted into an intermediate representation describing a high-level dialect. This is followed by a step-by-step downgrade transformation through the compilation layer, and finally, training and validation are performed on IREE's Vulkan and dylib backends. Feature extraction and image reconstruction are then performed on face images from the celebA dataset, commonly used for image compression and image generation.

[0139] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0140] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A probabilistic model AI domain compilation system based on multi-level intermediate representation, characterized in that, include: Front-end transformation layer: Taking the model described in Python as input, it traverses the abstract syntax tree to map the operation nodes to operation nodes in the MLIR dialect and outputs them; Compilation layer: Takes IR represented in MLIR dialect as input, and decomposes, downgrades and optimizes the operations; Backend layer: Integrates two backends, LLVM JIT and IREE, receives the final form of IR as input, generates corresponding code and performs calculations according to the target platform, and performs feature extraction and image reconstruction on the image; In the compilation layer: The compilation layer performs a downgrade transformation on the intermediate representation output by the front-end layer, reducing it to a form acceptable to the back-end runtime layer. During the downgrade transformation, optimization operations are inserted, including common subexpression elimination and dead code elimination. The compilation layer is implemented using the C++ language and the TableGen mechanism, providing Python with APIs that can be called. The compiler layer processes input parameters as follows: Register all dialects that can be used during the conversion process with the DialectRegistry. The list of dialects that can be used is loaded into the context when the actual conversion operation occurs. Register all supported conversion processes with PassRegistry, including built-in dialect conversion processes, inter-dialect conversion processes, and general conversion optimization processes; Parse the input command and populate the command configuration options, including the input IR file, output file, and transformation target; Obtain the PassManager object, add the required Passes to it, and build the compilation pipeline; The intermediate representation code of the input is parsed, the compilation pipeline is started through PassManager, the Passes in the pipeline are executed in sequence, the input IR is transformed, and the result is written to the target file; The compilation layer performs multi-level transformations and rewrites on the intermediate code represented by the custom dialect, downgrading it to the LLVM dialect or the TOSA dialect. These dialects are then read and executed by the LLVM JIT and IREE backends, respectively. One transformation process is called a Pass. Pass is defined through the TableGen mechanism and implemented in C++; the underlying implementation of probability distribution operations is implemented at the compilation layer, which degrades the sampling operations from the probability distribution defined by the pre-defined high-level abstract dialect into control flow and basic operation operations; the implementation of the uniform distribution sampling algorithm generates a random sequence that follows a standard uniform distribution through a linear congruential algorithm, and then transforms the standard uniform distribution result according to the distribution parameters; For the normal distribution, the Box-Muller algorithm is used to sample the standard normal distribution based on the uniform distribution. Then, the sampling results of the standard normal distribution are linearly transformed according to the mean and variance of the target distribution: two sets of independent random numbers U1 and U2 are generated based on the standard uniform distribution. The following transformation is applied to generate independent random variables Z0 and Z1 that follow a normal distribution. For the exponential distribution, the inverse transformation method is used to implement the exponential distribution sampling of the probability density function; for the multinomial distribution, the idea of ​​rejection sampling is learned, and the Alias ​​Method is used to implement sampling by constructing probability tables and alias tables. At the compilation layer, sampling operations are converted into intermediate representation code using the Standard and Linalg dialects built into MLIR. Then, the runtime layer generates the corresponding backend code based on the parameter configuration and performs the computation. When facing an LLVM backend, the compilation layer converts the IR into an LLVM IR, which is then loaded and run by the LLVM JIT. When facing an IREE backend, the custom high-level IR is converted into TOSA or MHLOIR, and the code is executed by dylib or vulkan as the backend driver.

2. The AI ​​domain compilation system for probabilistic models based on multi-level intermediate representations according to claim 1, characterized in that, In the front-end conversion layer: The abstract syntax tree is used to perform the conversion from Python programs to intermediate representations; Abstract Syntax Tree (AST) is an intermediate product between Python source code and bytecode. The nodes of the tree contain node type, operation type, and parameters, and can be traversed and modified. It maps Python operations one by one to operations in dialects and outputs them as intermediate representations in text form. Obtain the abstract syntax tree structure, traverse the AST nodes in a top-down order of module, function, task, and return, perform type checking and construct a symbol table during the process, insert the constructed intermediate representation dialect operation information as attributes into the corresponding nodes, and output the transformation result of the intermediate representation when the traversal is completed.

3. The AI ​​domain compilation system for probabilistic models based on multi-level intermediate representations according to claim 2, characterized in that: The conversion process from Python source code to intermediate representation is completed before the original program is executed, avoiding modifications to the code that exceed the preset standards. Python function decorators are used to add features without modifying the original functions. A specified decorator function is added before the original function; the instructions in the decorator are executed first, returning a new function object to replace the original function. The front-end conversion layer inserts a conversion function `@backend` before the source code functions as a decorator. This function traverses and rewrites the abstract syntax tree, constructs dialect operation nodes, generates the Interpreter Representation (IR), and calls the compiler layer interface within the conversion function to downgrade the preset high-level dialect to an external dialect or a lower-level dialect. The back-end layer then compiles this into a callable function. Different interfaces are called for different back-end platforms, using parameterized function decorators to pass the specified back-end platform to the conversion process. The conversion to IR is executed once at function definition, compiled once, and used multiple times. Static type checking is performed during compilation, and type information is written into intermediate representation code and passed along with the intermediate representation conversion until assembly code or machine instructions are generated.

4. A compilation method for probabilistic models in the AI ​​domain based on multi-level intermediate representations, characterized in that, include: Step S1: The front-end transformation layer takes the model described in Python as input, traverses the abstract syntax tree to map the operation nodes to operation nodes in the MLIR dialect and outputs them; Step S2: The compilation layer takes the IR represented by the MLIR dialect as input and decomposes, downgrades and optimizes the operation; Step S3: The backend layer integrates two backends, LLVM JIT and IREE, receives the final form of IR as input, generates corresponding code according to the target platform and performs calculations to extract features and reconstruct the image. In step S2: The compilation layer performs a downgrade transformation on the intermediate representation output by the front-end layer, reducing it to a form acceptable to the back-end runtime layer. During the downgrade transformation, optimization operations are inserted, including common subexpression elimination and dead code elimination. The compilation layer is implemented using the C++ language and the TableGen mechanism, providing Python with APIs that can be called. The compiler layer processes input parameters as follows: Register all dialects that can be used during the conversion process with the DialectRegistry. The list of dialects that can be used is loaded into the context when the actual conversion operation occurs. Register all supported conversion processes with PassRegistry, including built-in dialect conversion processes, inter-dialect conversion processes, and general conversion optimization processes; Parse the input command and populate the command configuration options, including the input IR file, output file, and transformation target; Obtain the PassManager object, add the required Passes to it, and build the compilation pipeline; The intermediate representation code of the input is parsed, the compilation pipeline is started through PassManager, the Passes in the pipeline are executed in sequence, the input IR is transformed, and the result is written to the target file; The compilation layer performs multi-level transformations and rewrites on the intermediate code represented by the custom dialect, downgrading it to the LLVM dialect or the TOSA dialect. These dialects are then read and executed by the LLVM JIT and IREE backends, respectively. One transformation process is called a Pass. Pass is defined through the TableGen mechanism and implemented in C++; the underlying implementation of probability distribution operations is implemented at the compilation layer, which degrades the sampling operations from the probability distribution defined by the pre-defined high-level abstract dialect into control flow and basic operation operations; the implementation of the uniform distribution sampling algorithm generates a random sequence that follows a standard uniform distribution through a linear congruential algorithm, and then transforms the standard uniform distribution result according to the distribution parameters; For the normal distribution, the Box-Muller algorithm is used to sample the standard normal distribution based on the uniform distribution. Then, the sampling results of the standard normal distribution are linearly transformed according to the mean and variance of the target distribution: two sets of independent random numbers U1 and U2 are generated based on the standard uniform distribution. The following transformation is applied to generate independent random variables Z0 and Z1 that follow a normal distribution. For the exponential distribution, the inverse transformation method is used to implement the exponential distribution sampling of the probability density function; for the multinomial distribution, the idea of ​​rejection sampling is learned, and the Alias ​​Method is used to implement sampling by constructing probability tables and alias tables. At the compilation layer, sampling operations are converted into intermediate representation code using the Standard and Linalg dialects built into MLIR. Then, the runtime layer generates the corresponding backend code based on the parameter configuration and performs the computation. When facing an LLVM backend, the compilation layer converts the IR into an LLVM IR, which is then loaded and run by the LLVM JIT. When facing an IREE backend, the custom high-level IR is converted into TOSA or MHLOIR, and the code is executed by dylib or vulkan as the backend driver.

5. The AI ​​domain compilation method for probabilistic models based on multi-level intermediate representations according to claim 4, characterized in that, In step S1: The abstract syntax tree is used to perform the conversion from Python programs to intermediate representations; Abstract Syntax Tree (AST) is an intermediate product between Python source code and bytecode. The nodes of the tree contain node type, operation type, and parameters, and can be traversed and modified. It maps Python operations one by one to operations in dialects and outputs them as intermediate representations in text form. Obtain the abstract syntax tree structure, traverse the AST nodes in a top-down order of module, function, task, and return, perform type checking and construct a symbol table during the process, insert the constructed intermediate representation dialect operation information as attributes into the corresponding nodes, and output the transformation result of the intermediate representation when the traversal is completed.

6. The AI ​​domain compilation method for probabilistic models based on multi-level intermediate representations according to claim 5, characterized in that: The conversion process from Python source code to intermediate representation is completed before the original program is executed, avoiding modifications to the code that exceed the preset standards. Python function decorators are used to add features without modifying the original functions. A specified decorator function is added before the original function; the instructions in the decorator are executed first, returning a new function object to replace the original function. The front-end conversion layer inserts a conversion function `@backend` before the source code functions as a decorator. This function traverses and rewrites the abstract syntax tree, constructs dialect operation nodes, generates the Interpreter Representation (IR), and calls the compiler layer interface within the conversion function to downgrade the preset high-level dialect to an external dialect or a lower-level dialect. The back-end layer then compiles this into a callable function. Different interfaces are called for different back-end platforms, using parameterized function decorators to pass the specified back-end platform to the conversion process. The conversion to IR is executed once at function definition, compiled once, and used multiple times. Static type checking is performed during compilation, and type information is written into intermediate representation code and passed along with the intermediate representation conversion until assembly code or machine instructions are generated.

Images