Code compilation method and apparatus, and computing device cluster

Abstract

A code compilation method, comprising: acquiring from a runtime first bytecodes corresponding to a target code, wherein the target code is used for processing a machine learning model; compiling bytecodes among the first bytecodes that are executed before a graph break point bytecode into a first computation graph, wherein the graph break point bytecode is the first bytecode among the first bytecodes that requires to be dynamically executed, the graph break point bytecode is called among the first bytecodes, and the graph break point bytecode does not call other bytecodes among the first bytecodes; and executing the first computation graph and the graph break point bytecode. In this way, during code compilation, when a bytecode that requires to be dynamically executed is identified, bytecodes that are executed before the bytecode are compiled into a computation graph, such that call overheads between a plurality of computation graphs can be avoided. In addition, by forming a large computation graph, a compiler may have a larger global vision, such that the compiler has more opportunities to perform compilation optimization, thereby further improving the performance.

Description

A code compilation method, apparatus, and computing device cluster

[0001] This application claims priority to Chinese Patent Application No. 202411693131.2, filed on November 22, 2024, entitled “A code compilation method, apparatus and computing device cluster”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI) technology, and in particular to a code compilation method, apparatus and computing device cluster. Background Technology

[0003] Current deep learning frameworks (such as MindSpore, PyTorch, and TensorFlow) have two operating modes: dynamic graph mode and static graph mode. In dynamic graph mode, the deep learning framework works in conjunction with the Python interpreter. Each time the interpreter executes code line by line, the deep learning framework captures the corresponding operations and performs the appropriate computations. Because this mode follows the Python interpreter's execution method—executing code line by line—it lacks a global perspective for optimization, resulting in a lower performance ceiling, but better usability. In static graph mode, the pre-built computation graph is converted into a complete intermediate representation (IR) graph by compiling user code. This process allows the compiler to apply optimization strategies more broadly, significantly improving model execution speed and resource utilization. However, because static graph mode requires compilation before execution, it imposes stricter requirements on the code written by the user. If the user uses syntax not supported by the static graph compiler, the static graph compilation will directly report an error. Therefore, static graph mode offers better performance but is less user-friendly.

[0004] In some complex application scenarios, user-written code may contain functions that can be converted into static graphs as well as functions that can only be converted into dynamic graphs, and there may also be arbitrary call relationships between functions. In this case, whether the code is processed using a dynamic graph pattern or a static graph pattern, the compilation and execution of the code will become more difficult. Summary of the Invention

[0005] This application provides a code compilation method, apparatus, computing device cluster, computer storage medium, and computer product that can improve code compilation and execution efficiency in subgraph-split graph scenarios.

[0006] In a first aspect, this application provides a code compilation method, comprising: obtaining a first bytecode corresponding to a target code from runtime, the target code being used to process a machine learning model; compiling the bytecode in the first bytecode that is executed before the split graph point bytecode into a first computation graph, wherein the split graph point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed, the split graph point bytecode is called in the first bytecode, and the split graph point bytecode does not call other bytecode in the first bytecode; and executing the first computation graph and the split graph point bytecode.

[0007] In this way, during the code compilation process, when bytecode that needs to be dynamically executed is identified, the overhead of calling multiple static graphs can be avoided by compiling the bytecode that executes before that bytecode into a computation graph. Furthermore, forming a large computation graph gives the compiler a larger global view, which in turn gives the compiler more opportunities to perform compilation optimizations and further improve performance.

[0008] In one possible implementation, the output of the first computation graph is the active variables of a plurality of second computation graphs; wherein, a second computation graph is obtained by compiling bytecode that is executed in the first bytecode before the split graph point bytecode and is in the same call level.

[0009] One possible implementation also includes: combining the bytecode in the first bytecode that executes later than the bytecode at the split point in the graph into a second bytecode, and then compiling the second bytecode. In this way, during code compilation, when the first bytecode that needs to be dynamically executed and only acts as a callee is identified, combining the bytecode that executes later than that bytecode into a single bytecode provides an opportunity for subsequent unanalyzed code to be incorporated into a computation graph, thereby increasing the upper limit of compilation optimization for subsequent code.

[0010] In one possible implementation, bytecode that executes later than the split-point bytecode in the first bytecode is combined into a second bytecode. This includes: adding variable handling bytecode before the bytecode that is at the same call level as the split-point bytecode and executes later than it; the variable handling bytecode is used to perform stack restoration and local variable operations; in the first bytecode, within each call level that has a call relationship with the split-point bytecode, adding variable handling bytecode and call bytecode before the bytecode that executes later than it, wherein the call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level; and combining the bytecode with added variable handling bytecode and call bytecode, as well as the bytecode with added variable handling bytecode, according to the call level related to the split-point bytecode in the first bytecode, to obtain the second bytecode. In this way, by modifying the bytecode following the bytecode that needs to be dynamically executed, they can be integrated together to form a single bytecode. In addition, during the integration process, some bytecode for data processing (such as parameter import and calling) is added to the original bytecode, without modifying other parts of the original bytecode. Therefore, this method will not have problems such as variable name conflicts or ambiguous namespaces.

[0011] In one possible implementation, the bytecode in the first bytecode that executes later than the split-point bytecode is compiled into second bytecode. This includes: constructing a second sub-bytecode, which comprises multiple calling bytecodes. One calling bytecode calls a bytecode within the call hierarchy that has a call relationship with the first bytecode and executes later than the split-point bytecode, or it calls a bytecode in the first bytecode that is at the same call hierarchy as the split-point bytecode and executes later than the split-point bytecode. The execution order of the calling bytecodes is the reverse of the call hierarchy of the bytecodes they call. The second sub-bytecode is then combined with the bytecode in the first bytecode that executes later than the split-point bytecode to obtain the second bytecode. In this way, by modifying the bytecode following the dynamically executed bytecode, they can be integrated together to form a single bytecode. In addition, during the integration process, some bytecode for data processing (such as parameter import and calling) is added to the original bytecode, without modifying other parts of the original bytecode. Therefore, this method will not have problems such as variable name conflicts or ambiguous namespaces.

[0012] In one possible implementation, after obtaining the first computation graph and the second bytecode, the process further includes: constructing and executing scheduling bytecode to execute the first computation graph and the split graph point bytecode, and compiling the second bytecode; wherein the scheduling bytecode includes a first call bytecode, a second call bytecode, and a third call bytecode arranged sequentially; the first call bytecode is used to call the first computation graph, the second call bytecode is used to call the split graph point bytecode, and the third call bytecode is used to call the second bytecode. In this way, the execution of the first computation graph, the split graph point bytecode, and the second bytecode can be achieved through the scheduling bytecode, and the second bytecode can be re-entered into the graph capture process.

[0013] Secondly, this application provides a code compilation method, comprising: obtaining a first bytecode corresponding to a target code from runtime, the target code being used to process a machine learning model; combining bytecode in the first bytecode that is executed later than the split-map point bytecode into a second bytecode, wherein the split-map point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed, the split-map point bytecode is called in the first bytecode, and the split-map point bytecode does not call other bytecode in the first bytecode; and compiling the second bytecode.

[0014] In one possible implementation, the bytecode in the first bytecode that executes later than the split-map point bytecode is combined into the second bytecode. This includes: adding variable handling bytecode before the bytecode that is at the same call level as the split-map point bytecode and executes later than the split-map point bytecode. The variable handling bytecode is used to perform stack restoration and local variable operations. In the first bytecode, within each call level that has a call relationship with the split-map point bytecode, variable handling bytecode and call bytecode are added before the bytecode that executes later than the split-map point bytecode. The call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. The bytecode with added variable handling bytecode and call bytecode, as well as the bytecode with added variable handling bytecode, are combined according to the call level related to the split-map point bytecode in the first bytecode to obtain the second bytecode.

[0015] In one possible implementation, combining bytecode in the first bytecode that executes later than the split-map point bytecode into a second bytecode includes: constructing a second sub-bytecode, which includes: multiple calling bytecodes, one of which calls a bytecode that has a calling relationship with the first bytecode in the call hierarchy and executes later than the split-map point bytecode, or calls a bytecode in the first bytecode that is at the same call hierarchy as the split-map point bytecode and executes later than the split-map point bytecode; the execution order of the calling bytecodes is the reverse of the call hierarchy of the bytecodes called by the calling bytecode; and combining the second sub-bytecode with the bytecode in the first bytecode that executes later than the split-map point bytecode to obtain the second bytecode.

[0016] Thirdly, this application provides a code compilation apparatus, comprising: an acquisition module and a processing module. The acquisition module is used to acquire first bytecode from runtime. The processing module is used to compile the bytecode in the first bytecode that executes before the split-map point bytecode into a first computation graph, wherein the split-map point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed, the split-map point bytecode is called within the first bytecode, and the split-map point bytecode does not call other bytecode within the first bytecode. The processing module is also used to execute the first computation graph and the split-map point bytecode.

[0017] In one possible implementation, the output of the first computation graph is the active variables of a plurality of second computation graphs; wherein, a second computation graph is obtained by compiling bytecode that is executed in the first bytecode before the split graph point bytecode and is in the same call level.

[0018] In one possible implementation, the processing module is further configured to: combine bytecode in the first bytecode that is executed later than the split point bytecode into a second bytecode; and compile the second bytecode.

[0019] In one possible implementation, when the processing module combines the bytecode executed later than the split point bytecode in the first bytecode into the second bytecode, it specifically does the following: It adds variable processing bytecode before the bytecode executed later than the split point bytecode, which is at the same call level as the split point bytecode. This variable processing bytecode is used to perform stack restoration and local variable operations. In the first bytecode, within each call level that has a call relationship with the split point bytecode, it adds variable processing bytecode and call bytecode before the bytecode executed later than the split point bytecode. The call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. According to the call level related to the split point bytecode in the first bytecode, it combines the bytecode with added variable processing bytecode and call bytecode, as well as the bytecode with added variable processing bytecode, to obtain the second bytecode.

[0020] In one possible implementation, when the processing module combines the bytecode in the first bytecode that is executed later than the split-map point bytecode into the second bytecode, it specifically performs the following: constructing a second sub-bytecode, which includes: multiple calling bytecodes, one of which calls a bytecode that is executed later than the split-map point bytecode within the calling hierarchy of the first bytecode, or calls a bytecode that is executed later than the split-map point bytecode within the same calling hierarchy of the first bytecode; the execution order of the calling bytecodes is the opposite of the calling hierarchy of the bytecodes they call; and combining the second sub-bytecode with the bytecode in the first bytecode that is executed later than the split-map point bytecode to obtain the second bytecode.

[0021] In one possible implementation, after obtaining the first computation graph and the second bytecode, the processing module is further configured to: construct scheduling bytecode and execute the scheduling bytecode to execute the first computation graph and the split graph point bytecode, and compile the second bytecode; wherein the scheduling bytecode includes a first call bytecode, a second call bytecode, and a third call bytecode arranged in sequence; the first call bytecode is used to call the first computation graph, the second call bytecode is used to call the split graph point bytecode, and the third call bytecode is used to call the second bytecode.

[0022] Fourthly, this application provides a code compilation apparatus, comprising: an acquisition module and a processing module. The acquisition module is used to acquire first bytecode from runtime. The processing module is used to combine bytecode in the first bytecode that is executed later than the split-map point bytecode into second bytecode, wherein the split-map point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed, the split-map point bytecode is called in the first bytecode, and the split-map point bytecode does not call other bytecode in the first bytecode. The processing module is also used to compile the second bytecode.

[0023] In one possible implementation, when the processing module combines the bytecode executed later than the split point bytecode in the first bytecode into the second bytecode, it specifically does the following: It adds variable processing bytecode before the bytecode executed later than the split point bytecode, which is at the same call level as the split point bytecode. This variable processing bytecode is used to perform stack restoration and local variable operations. In the first bytecode, within each call level that has a call relationship with the split point bytecode, it adds variable processing bytecode and call bytecode before the bytecode executed later than the split point bytecode. The call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. According to the call level related to the split point bytecode in the first bytecode, it combines the bytecode with added variable processing bytecode and call bytecode, as well as the bytecode with added variable processing bytecode, to obtain the second bytecode.

[0024] In one possible implementation, when the processing module combines the bytecode in the first bytecode that is executed later than the split-map point bytecode into the second bytecode, it specifically performs the following: constructing a second sub-bytecode, which includes: multiple calling bytecodes, one of which calls a bytecode that is executed later than the split-map point bytecode within the calling hierarchy of the first bytecode, or calls a bytecode that is executed later than the split-map point bytecode within the same calling hierarchy of the first bytecode; the execution order of the calling bytecodes is the opposite of the calling hierarchy of the bytecodes they call; and combining the second sub-bytecode with the bytecode in the first bytecode that is executed later than the split-map point bytecode to obtain the second bytecode.

[0025] Fifthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect, or performs the method described in the second aspect or any possible implementation of the second aspect.

[0026] Sixthly, this application provides a computer-readable storage medium including computer program instructions. When executed by a computing device, the computing device performs the method described in the first aspect or any possible implementation thereof; or, when executed by a cluster of computing devices, the cluster of computing devices performs the method described in the first aspect or any possible implementation thereof, or performs the method described in the second aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0027] In a seventh aspect, this application provides a computer program product comprising instructions that, when executed by a computing device, cause the computing device to perform the method described in the first aspect or any possible implementation thereof; or, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof; or, perform the method described in the second aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0028] Eighthly, embodiments of this application provide a code compilation apparatus, including at least one processor and an interface; the at least one processor obtains program instructions or data through the interface; the at least one processor is configured to execute program line instructions to implement the method described in the first aspect or any possible implementation of the first aspect, or to implement the method described in the second aspect or any possible implementation of the second aspect. Exemplarily, the code compilation apparatus may be, but is not limited to, a chip.

[0029] It is understood that the beneficial effects of aspects two through eight above can be found in the relevant descriptions in aspect one above, and will not be repeated here. Attached Figure Description

[0030] Figure 1 is a schematic diagram of a scenario for calling a split graph and a subgraph according to an embodiment of this application;

[0031] Figure 2 is a schematic diagram of a code compilation process in a subgraph-split graph scenario provided by an embodiment of this application;

[0032] Figure 3 is a schematic diagram of a technical concept for compiling source code according to an embodiment of this application;

[0033] Figure 4 is a schematic diagram of a process for compiling multiple subgraphs into a computation graph according to an embodiment of this application;

[0034] Figure 5 is a schematic diagram of the structure of bytecode obtained by integrating multiple bytecodes according to an embodiment of this application;

[0035] Figure 6 is a flowchart illustrating a code compilation method provided in an embodiment of this application;

[0036] Figure 7 is a schematic diagram of the steps of compiling bytecode in the first bytecode that is executed later than the bytecode at the split point in the present application into the second bytecode;

[0037] Figure 8 is a schematic diagram of another step in compiling the bytecode in the first bytecode that is executed later than the bytecode at the split point, according to an embodiment of this application, into the second bytecode;

[0038] Figure 9 is a schematic diagram of a function object for constructing the portion after the split point provided in an embodiment of this application;

[0039] Figure 10 is a schematic diagram of another function object for constructing the portion after the split point provided in an embodiment of this application;

[0040] Figure 11 is a schematic diagram of an implementation of a code compilation method provided in an embodiment of this application;

[0041] Figure 12 is a schematic diagram of a code compilation process provided in an embodiment of this application;

[0042] Figure 13 is a schematic diagram of the structure of a code compilation device provided in an embodiment of this application;

[0043] Figure 14 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0044] Figure 15 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0045] Figure 16 is a schematic diagram of another computing device cluster structure provided in an embodiment of this application;

[0046] Figure 17 is a schematic diagram of the structure of a code compilation device provided in an embodiment of this application. Detailed Implementation

[0047] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0048] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0049] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0050] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0051] First, the relevant technical terms involved in the technical solution provided in this application will be introduced.

[0052] (1) Source code

[0053] Source code is the original code written by programmers, usually in text form. Source code is written in high-level programming languages ​​such as Python, Java, and C++. Source code is human-readable, easy to understand, and easy to modify.

[0054] (2) Bytecode

[0055] Bytecode is an intermediate representation of source code generated after compilation or interpretation. Bytecode is typically a low-level, platform-independent binary code that can be executed by a virtual machine or other interpreter.

[0056] (3) Crack diagram

[0057] Split graph refers to a strategy employed by the compiler during compilation when it encounters syntax that cannot be directly converted into a static graph. Specifically, the compiler converts the parts of the code that can be explicitly analyzed and compiled into a static computation graph into a static graph, while the parts that cannot be converted into a static graph are retained in a dynamic graph and built and executed at runtime. In this way, split graph technology allows deep learning frameworks to combine the performance advantages of static graphs with the flexibility of dynamic graphs to adapt to different programming needs and hardware optimizations.

[0058] (4) Subgraph call

[0059] In static graph mode, the compiler converts user-written code into a computation graph. If a function in the user-written code calls another function, the compiler generates a subgraph for the callee. The caller can then invoke the subgraph corresponding to the callee; this phenomenon is called a subgraph call. For example, as shown in Figure 1(C), suppose the user-written code is converted into bytecode. Bytecode A4 in code block A calls block B. We can then try compiling the bytecode in block B into a computation graph. Bytecode A4 can then call the computation graph compiled from block B. This call from bytecode A4 to the computation graph compiled from block B is called a "subgraph call."

[0060] (5) Cracked Image Scene

[0061] A split-graph scenario refers to a situation where, during compilation, the source code contains functions that can be converted to both static and dynamic graphs. In this case, the compiler can convert some functions in the source code to static graphs and retain others as dynamic graphs. For example, as shown in Figure 1(A), after converting the source code, we get 10 bytecodes A1 to A10. Assuming that only bytecode A5 cannot be converted to a static graph, when analyzing each bytecode individually, as shown in Figure 5(B), bytecodes A1 to A4 can be compiled into a single static graph, while bytecode A5 can be run as a dynamic graph. Bytecodes A6 to A10 can be re-analyzed to determine which can be run as static graphs and which as dynamic graphs. In Figure 1(B), if only bytecode A10 cannot be converted to a static graph, then bytecodes A6 to A9 can be compiled into a single static graph, while bytecode A10 can be run as a dynamic graph.

[0062] (6) Subgraph calling scenario

[0063] A subgraph call scenario refers to a situation during compilation where a function in user-written code calls another function. For example, as shown in Figure 2(A), assuming the user-written code is converted into bytecode, bytecode A4 in block A calls block B, and bytecode B5 in block B calls block C. Such a scenario can be called a "subgraph call scenario".

[0064] (7) Subgraph-Split Graph Scenario

[0065] A subgraph-split graph scenario refers to a situation during compilation where user-written code simultaneously contains functions that can be converted into both static and dynamic graphs, and where arbitrary call relationships exist between these functions. For example, continuing with Figure 2(A), if bytecode C3 in block C is a split graph point (meaning it cannot be added to a static graph), then the code corresponding to these bytecodes contains functions that can be converted into both static and dynamic graphs, and where call relationships exist between these functions. This scenario can be understood as a subgraph-split graph scenario. Exemplarily, the user-written code can be written in a high-level programming language and used to process machine learning models. For example, the user-written code can indicate the structure of the machine learning model, or it can indicate training or inference on the machine learning model. In some embodiments, the user-written code can include one or more of the following: code characterizing the model structure, code for training the model, and code for using the model for inference.

[0066] (8) Active variables

[0067] Active variables are variables generated from bytecode preceding a certain split point that are needed during the execution of bytecode after a split point.

[0068] The technical solution of this application will be described below.

[0069] In related technologies, an alternative compilation mode has been explored, distinct from traditional dynamic graph and static graph modes. In this new mode, the compiler parses bytecode to determine which code segments are suitable for static graph execution, while unsuitable segments are retained for dynamic graph execution. This allows the compiler to optimize the code using static graphs as much as possible while maintaining user-friendliness, thereby improving performance. However, the inventors discovered that in subgraph-split graph scenarios, this compilation mode compiles bytecode at each call level into a separate computation graph, resulting in a low optimization ceiling and significant performance overhead. See the analysis below for details.

[0070] Referring to Figure 2, the analysis process for the bytecode shown in Figure 2(A) can be as follows: First, analyze bytecodes A1 to A3 in block A to determine if they can be compiled into a static graph (i.e., a computation graph). Bytecode A4 calls block B, so after analyzing bytecode A4, block B can be analyzed. In block B, it is determined that bytecodes B1 to B4 can be compiled into a static graph. Bytecode B5 calls block C, so after analyzing bytecode B5, block C can be analyzed. In block C, it is determined that bytecodes C1 to C2 can be compiled into a static graph. However, bytecode C3 is a split point, so bytecode C3 cannot be compiled into a static graph. At this point, as shown in Figure 2(B), bytecodes A1 to A3 can be compiled into a static graph fg A, and bytecodes A5 to A10 can be integrated into the same function. Furthermore, a scheduling bytecode A can be generated, which can sequentially schedule fg A, bytecode A4, and the bytecode obtained by integrating bytecodes A5 to A10. Then, the scheduled bytecode A can be executed. For example, bytecode A4 can be referred to as the split point of block A.

[0071] When bytecode A4 is executed, it calls block B. At this point, block B needs to be re-analyzed. Within block B, bytecodes B1 to B4 are determined to be compileable into a static graph. Bytecode B5 calls block C, so after analyzing bytecode B5, block C needs to be re-analyzed. Within block C, bytecodes C1 to C2 are determined to be compileable into a static graph. However, bytecode C3 is a split point, so it cannot be compiled into a static graph. At this point, as shown in Figure 2(C), bytecodes B1 to B4 can be compiled into a static graph fg B, and bytecodes B6 to B8 can be integrated into the same function. Furthermore, a scheduling bytecode B is generated, which can sequentially schedule fg B, bytecode B5, and the bytecode obtained by integrating bytecodes B6 to B8. Then, the scheduling bytecode B can be executed. For example, bytecode B5 can be referred to as the split point of block B.

[0072] When bytecode B5 is executed, it calls block C. At this point, block C needs to be re-analyzed. Within block C, bytecodes C1 to C2 are determined to be compileable into a static graph. However, bytecode C3 is a split point, so it cannot be compiled into a static graph. Therefore, as shown in Figure 2(D), bytecodes C1 to C2 can be compiled into a static graph fg C, and bytecodes C4 to C5 can be integrated into the same function. This generates a scheduling bytecode C, which can sequentially schedule fg C, bytecode C3, and the bytecode obtained by integrating bytecodes C4 to C5. Then, the scheduling bytecode C can be executed.

[0073] When executing scheduled bytecode C, since block C is at the deepest call level in this call relationship, after executing bytecode C3, the bytecode obtained by integrating bytecodes C4 to C5 can be analyzed directly. Assuming that bytecodes C4 and C5 can both be compiled into static graphs, bytecodes C4 to C5 can be compiled into static graphs and executed to obtain the execution result of block C. After obtaining the execution result of block C, execution can return to continue with scheduled bytecode B. At this point, the bytecode obtained by integrating bytecodes B6 to B8 can be analyzed. Assuming that bytecodes B6 to B8 can both be compiled into static graphs, bytecodes B6 to B8 can be compiled into static graphs and executed to obtain the execution result of block B. After obtaining the execution result of block B, execution can return to continue with scheduled bytecode A. At this point, the bytecode obtained by integrating bytecodes A5 to A10 can be analyzed. Assuming that bytecodes A5 to A10 can all be compiled into static graphs, then bytecodes A5 to A10 can be compiled into static graphs and executed to obtain the execution result of block A.

[0074] While the compilation method described above can compile bytecode during program execution in a subgraph-split graph scenario, this method creates separate static graphs for functions at each level, such as fg A, fg B, and fg C as shown in Figure 2. This necessitates the transfer of results between different static graphs during runtime through calls, reducing execution efficiency. Furthermore, as the subgraph call depth increases, the number of small subgraphs increases with a complexity of O(n), and each subgraph call incurs additional performance overhead. Moreover, bytecode below the split graph points in functions at each level, such as bytecode A5 to A10, B6 to B8, and C4 to C5 in Figure 2, cannot be transformed into a complete graph, further increasing the number of static graphs and causing additional performance degradation.

[0075] In view of this, embodiments of this application provide a compilation method that can compile the bytecode executed before the first dynamically executed bytecode in the bytecode that only plays the role of the callee (i.e., the split graph point bytecode) into a computation graph, and / or integrate the bytecodes of different calling levels after the split graph point bytecode into a single bytecode, so that calls between subgraphs can be avoided during operation, and the number of static graphs can also be avoided, thereby improving compilation and execution efficiency.

[0076] For example, Figure 3 illustrates a schematic diagram of a technical concept for compiling source code according to an embodiment of this application. As shown in Figure 3, the overall architecture for compiling source code includes: a graph capture part, a full graph compilation part, a bytecode integration part, a scheduled bytecode generation part, and a scheduled bytecode execution part.

[0077] The graph capture section is used to determine whether the bytecode obtained from the Python runtime (e.g., a machine learning program compiled from Python) can be compiled into a static computation graph during program execution. It also identifies split graph points in the bytecode at different call levels and compiles the bytecode that can be compiled into a static computation graph at different call levels to obtain at least one static computation graph. For example, referring to Figure 2, fg A can be compiled in block A, fg B in block B, and fg C in block C. Furthermore, in Figure 2, split graph point A4 can be found in block A, split graph point B5 in block B, and split graph point C3 in block C. Split graph point C3 is the first bytecode that needs to be dynamically executed and is called by other bytecode, but does not call any other bytecode. In other words, split graph point C3 plays the role of the callee, not the caller. For example, at least one static computation graph can be compiled within a function call level. For example, the code corresponding to the bytecode obtained by the graph capture part from the runtime can be specified by the user or other devices, depending on the actual situation, and is not limited here. In this embodiment, the graph capture part can transmit the computation graph it compiles (fg A, fg B and fg C in Figure 2(B)) to the whole graph compilation part, transmit the identifier of the first split graph point that needs to be dynamically executed and only plays the role of the callee (C3 in Figure 2(A)) to the scheduling bytecode generation part, and transmit the bytecode to be executed after the first split graph point that needs to be dynamically executed and only plays the role of the callee (A5-A10, B6-B8, C4-C5 in Figure 2(A)) to the bytecode integration part.

[0078] The whole graph compilation section is used to compile the computation graph output by the graph capture section into a complete computation graph, and transmits the identifier of the compiled computation graph to the scheduling bytecode generation section. Specifically, the whole graph compilation section can collect the active variables in each static computation graph, and use the collected active variables as the output of the final compiled complete computation graph, and compile each computation graph to obtain a complete computation graph. For example, as shown in Figure 4, if the computation graph compiled from function A is graph A1, the computation graph compiled from function B is graph B1, and the computation graph compiled from function C is graph C1, then the steps to compile them into a single graph can be: ① Analyze the top-level graph A1 to obtain the active variable set (a1,) of A1; ② Analyze graph B1 to obtain the active variable set (b1,b2) of B1; ③ Analyze graph C1 to obtain the active variable set (c,b2) of C1. 1,); ④ Return the active variable (c1,) of graph C1 to the parent graph B1; ⑤ Return the active variable (b1,b2,c1) of graph B1+C1 to the top-level graph A1; ⑥ Return the active variable (a1,b1,b2,c1) of graph A1+B1+C1 as the output of the entire graph to the caller; ⑦ Capture all the bytecode of graph A1+B1+C1 to obtain three subgraphs (i.e., A1, B1, C1), and compile these three subgraphs into a complete computation graph.

[0079] The bytecode integration section is used to integrate the bytecode executed after the first split point in the graph capture section that needs to be dynamically executed and only plays the role of the callee, to form a complete bytecode. It also transmits the identifier of the newly formed bytecode to the scheduling bytecode generation section. When this complete bytecode is invoked, it can re-enter the graph capture process. During the generation of new bytecode, some bytecode for data processing (such as parameter import, invocation, etc.) can be added before the original bytecode without modifying other parts of the original bytecode. For example, continuing to refer to Figure 4, if bytecodes A2, B2, and C2 in Figure 4 are integrated into a single bytecode, and the output of the computation graph compiled from bytecodes A1, B1, and C1 is (a1,…,an,b1,…,bn,c1,…,cn), and the variable on the stack after the split point in function C is d, then bytecodes A2, B2, and C2 can be integrated into the bytecode shown in Figure 5 (A) or (B). In Figure 5(A), bytecode a and a bytecode calling bytecode B2 are added before bytecode A2. Bytecode b and a bytecode calling bytecode C2 are added before bytecode B2. Bytecode c is added before bytecode C2. Bytecodes a, b, and c are used to process the input parameters of their respective bytecodes, such as restoring the stack and local variables. Each of bytecodes a, b, and c can include at least one sub-bytecode. For example, bytecode a can consist of a series of sub-bytecodes. In Figure 5(B), a new bytecode is created containing three bytecodes that call bytecodes C2, B2, and A2 sequentially, arranged from top to bottom. In addition, the newly created bytecode in Figure 5(B) also includes bytecodes 1, 2, and 3. Bytecode 1 can process the input parameters (i.e., (a1,…,an,b1,…,bn,c1,…,cn,d)). Bytecode 2 can be used to save the return value of bytecode C2 as a local variable. Bytecode 3 can be used to save the return value of bytecode B2 as a local variable. In Figure 5(A) and (B), the bytecode used to call bytecode C2 can pass the parameters required by bytecode C2 to bytecode C2 and call bytecode C2; ​​the bytecode used to call bytecode B2 can pass the parameters required by bytecode B2 to bytecode B2 to call bytecode B2; and the bytecode used to call bytecode A2 can pass the parameters required by bytecode A2 to bytecode A2 and call bytecode A2.

[0080] The scheduling bytecode generation section generates scheduling bytecode based on the identifiers of the computation graph from the whole graph compilation section, the identifiers of the split graph points from the graph capture section, and the identifiers of the bytecode from the bytecode integration section. The scheduling bytecode may contain bytecode that calls the latest output computation graph from the whole graph compilation section (i.e., CALL_FUNCTION 1 shown in Figure 3), bytecode that calls the split graph point in the deepest call level determined by the graph capture section (i.e., split graph point 1 shown in Figure 3), and bytecode that calls the latest complete bytecode generated by the bytecode integration section (i.e., CALL_FUNCTION 2 shown in Figure 3), and these three bytecodes are arranged sequentially. Of course, the scheduling bytecode may also contain bytecode used for data processing, such as bytecode used for parameter import.

[0081] The scheduled bytecode execution section executes the scheduled bytecode generated by the scheduled bytecode generation section. During execution, the scheduled bytecode execution section can first call and execute the latest output computation graph from the whole graph compilation section, then execute the first split graph point identified by the graph capture section as needing dynamic execution and only acting as the callee, and finally call the latest generated complete bytecode from the bytecode integration section for processing, such as analysis or execution. After calling the latest generated complete bytecode from the bytecode integration section, the bytecode can re-enter the graph capture section for analysis. In some embodiments, executing the latest output computation graph from the whole graph compilation section and executing the bytecode corresponding to the split graph point can also be performed in parallel with compiling the latest generated bytecode.

[0082] Through the coordinated processing of the above parts, the computation graphs built at different function levels before the first split point that needs to be dynamically executed and only plays the role of the callee can be combined into a single graph. The functions at different levels after the split point can be arranged on the same function, so that calls between subgraphs can be avoided during the operation. At the same time, the bytecode after the first split point that needs to be dynamically executed and only plays the role of the callee can be turned into a single graph, avoiding further increase in the number of computation graphs and improving compilation efficiency.

[0083] The following describes the specific implementation process of the above technical concept.

[0084] For example, Figure 6 shows a schematic flowchart of a code compilation method provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. As shown in Figure 6, the code compilation method includes:

[0085] S601. Obtain the first bytecode corresponding to the target code from the runtime, wherein the target code is used to process the machine learning model.

[0086] In this embodiment, runtime refers to the time when a program (such as a machine learning program compiled from Python) is running, that is, a stage in the program's lifecycle. Target code can be code specified by the user or other devices for processing the machine learning model. For example, after a user writes code related to a machine learning model, the user can identify certain code within that code. The code identified by the user can be understood as target code. When identifying code, the user can select to enable just-in-time (JIT) compilation for the corresponding code to identify the target code.

[0087] During the execution of code related to the machine learning model, the first bytecode corresponding to the target code can be obtained from runtime (e.g., accessing data structures within the Python interpreter, i.e., frame objects). For example, the first bytecode can contain call relationships between bytecode segments, and includes bytecode that can be compiled into a static graph and bytecode that needs to be dynamically executed; that is, the first bytecode conforms to the aforementioned subgraph-split graph scenario. For example, the first bytecode can contain multiple subbytecode segments. For instance, the first bytecode can be the bytecode shown in Figure 2(A). In Figure 2(A), the bytecode segments in blocks A, B, and C together constitute the first bytecode.

[0088] S602. Compile the bytecode in the first bytecode that is executed before the split point bytecode into a first computation graph. The split point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed. The split point bytecode is called in the first bytecode and does not call other bytecode in the first bytecode.

[0089] In this embodiment, after obtaining the first bytecode, the compiler can parse the first bytecode to determine which bytecode in the first bytecode can be compiled into a static graph, and which bytecode in the first bytecode cannot be compiled into a static graph. The bytecode in the first bytecode that cannot be compiled into a static graph needs to be executed dynamically. In the call relationships contained in the first bytecode, when it is determined that a split-graph bytecode (also called a "split-graph point") in a certain call level cannot be compiled into a static graph (i.e., needs to be executed dynamically), and this split-graph point bytecode only plays the role of the callee, the bytecode in the first bytecode that executes before this split-graph point bytecode can be compiled into a computation graph (i.e., the first computation graph). For example, please refer to Figure 12(A), bytecode A4 in block A calls block B, bytecode B5 in block B calls block C, and bytecode C3 in block C is the first bytecode that cannot be compiled into a computation graph and only plays the role of the callee; therefore, bytecode C3 can be called a split-graph point bytecode. At this point, the bytecodes A1 to A3, B1 to B4, and C1 to C2 can be compiled into a single computation graph (i.e., the computation graph fg D shown in (C) of Figure 12). This eliminates the need to compile the bytecodes A1 to A3, B1 to B4, and C1 to C2 into separate computation graphs and execute each graph individually, as described in Figure 2.

[0090] As one possible implementation, when compiling the first computation graph, its output can be determined first. Then, the bytecode executed before the split-point bytecode can be compiled into a computation graph. Specifically, when determining the output of the first computation graph, the bytecode executed before the split-point bytecode and located in each call level can be compiled into a separate computation graph. One computation graph can be compiled from the bytecode in each call level. Then, the active variables in each computation graph are analyzed, and the analyzed active variables are used as the output of the first computation graph. For details, please refer to the description in the aforementioned bytecode integration section; it will not be repeated here.

[0091] S603. Combine the bytecode in the first bytecode that is executed later than the bytecode of the split point into the second bytecode.

[0092] In this embodiment, the compiler can combine the bytecode executed later than the split-graph point bytecode in the first bytecode into a second bytecode. This reorders the unanalyzed bytecode after the split-graph point bytecode, allowing the bytecode after the split-graph point bytecode to potentially be carried on a single computation graph, thus increasing the upper limit of compilation optimization for subsequent code. For example, the input to the second bytecode can be the execution result of the first computation graph and / or the split-graph point bytecode. For instance, continuing to refer to Figure 12(A), since bytecode C3 is a split-graph point, bytecodes A5 to A10, B6 to B8, and C4 to C5 can be combined into a single bytecode (i.e., the bytecode shown in Figure 12(D)) so that they can be analyzed together later.

[0093] As one possible implementation, as shown in Figure 7, the process of combining the bytecode executed later than the split-point bytecode in the first bytecode into the second bytecode may include: in S701, adding variable processing bytecode before the bytecode that is at the same call level as the split-point bytecode and executed later than the split-point bytecode. The variable processing bytecode is used to perform stack restoration and local variable operations. Since the split-point bytecode only acts as the callee, it does not call other bytecodes. Therefore, adding variable processing bytecode before the bytecode executed later than the split-point bytecode in the call level is sufficient to restore the variables in the stack of the dynamically executed split-point bytecode, and to restore the results obtained from the bytecode executed before the split-point bytecode in this call level. For example, referring to Figure 5(A), bytecode C2 is at the same call level as the split-point bytecode and executed later than the split-point bytecode. In this case, bytecode c is the variable processing bytecode mentioned in S701.

[0094] In S702, within the first bytecode, in each call level that has a calling relationship with the crack point bytecode, and before the bytecode executed later than the crack point bytecode, variable handling bytecode and call bytecode are added. The call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. Since all call levels except the one containing the crack point bytecode may call bytecode in other call levels, variable handling bytecode and call bytecode need to be added before the bytecode (which executes later than the crack point bytecode) in each call level except the one containing the crack point bytecode. The call bytecode allows the bytecode in the next call level to be called. For example, continuing to refer to Figure 5(A), bytecode B2 is executed later than the split point bytecode and is located in the previous call level of bytecode C2. Therefore, bytecode b (i.e., variable handling bytecode) and CALL_FUNCTION (i.e. call bytecode) need to be added before bytecode B2; similarly, bytecode a (i.e., variable handling bytecode) and CALL_FUNCTION (i.e. call bytecode) need to be added before bytecode A2.

[0095] In S703, according to the call hierarchy related to the split point bytecode in the first bytecode, the bytecode with added variable handling bytecode and call bytecode, as well as the bytecode with added variable handling bytecode, are combined to obtain the second bytecode. After adding the variable handling bytecode and call bytecode, these bytecodes can be combined according to the call hierarchy related to the split point bytecode, for example, establishing the caller-callee relationship, to obtain the second bytecode. For example, continuing to refer to Figure 5(A), the bytecode A2 with added bytecode a and CALL_FUNCTION, the bytecode B2 with added bytecode b and CALL_FUNCTION, and the bytecode C2 with added bytecode c are combined to obtain a new bytecode.

[0096] As another possible implementation, as shown in Figure 8, the process of combining bytecodes executed later than the split-map bytecode in the first bytecode to form the second bytecode may include: in S801, constructing the second sub-bytecode, which includes: multiple calling bytecodes, one of which is used to call a bytecode within the call hierarchy that has a calling relationship with the first bytecode and is executed later than the split-map bytecode, or, it is used to call a bytecode in the first bytecode that is at the same call hierarchy as the split-map bytecode and is executed later than the split-map bytecode. The execution order of the calling bytecodes is the reverse of the call hierarchy in which the calling bytecode is located. The second sub-bytecode may also contain variable handling bytecodes to restore the stack and local variable operations, or to store the execution results of other bytecodes, etc. For example, the structure of the second sub-bytecode may be the same as or similar to the structure of the bytecode containing bytecodes 1, 2, 3 and CALL_FUNCTION shown in Figure 5(B). In S802, the second sub-bytecode is combined with the bytecode in the first bytecode that is executed later than the split point bytecode, for example, to establish a caller-callee relationship, so as to obtain the second bytecode.

[0097] S604, execute the first computation graph and split graph point bytecode, and compile the second bytecode.

[0098] In this embodiment, after obtaining the first computation graph, the split graph point bytecode, and the second bytecode, the first computation graph and the split graph point bytecode can be executed sequentially. After execution, the second bytecode can be compiled. Of course, the "analysis of the second bytecode" step can be omitted in this step, depending on the actual situation, and is not limited here. When analyzing the second bytecode, the code compilation method shown in Figure 6 can be used for analysis. In addition, when compiling the second bytecode, it can be considered that the second bytecode is also obtained from the runtime. As a possible implementation, a scheduling bytecode can be constructed first and executed to execute the first computation graph and the split graph point bytecode, and to compile the second bytecode. The scheduling bytecode includes a first call bytecode, a second call bytecode, and a third call bytecode arranged in sequence. The first call bytecode is used to call the first computation graph, the second call bytecode is used to call the split graph point bytecode, and the third call bytecode is used to call the second bytecode, so as to facilitate the compilation of the second bytecode.

[0099] In this way, during code compilation, when the first bytecode that needs to be dynamically executed and only acts as a callee is identified, the overhead of calling multiple static graphs can be avoided by compiling the bytecode that executes before this bytecode into a computation graph and integrating the bytecode that executes after this bytecode into a single bytecode. Furthermore, forming a large computation graph gives the compiler a broader global perspective, allowing it more opportunities to perform compilation optimizations and further improve performance. Additionally, it provides an opportunity to incorporate subsequent unanalyzed code into a computation graph, raising the upper limit of compilation optimization for subsequent code.

[0100] It should be understood that S602 and S603 described in Figure 6 can be executed selectively, in addition to being executed as shown in Figure 6. For example, S602 can be executed instead of S603. In this case, S604 can be changed to execute the first computation graph and the bytecode of the split graph points, and to compile the bytecode in the first bytecode that is executed later than the bytecode of the split graph points. Of course, the step of "compiling the bytecode in the first bytecode that is executed later than the bytecode of the split graph points" can also be omitted in S604, depending on the actual situation, and is not limited here. When S603 is executed instead of S602, S604 can be changed to: execute the bytecode in the first bytecode that is executed before the bytecode of the split graph points and the bytecode of the split graph points, and to compile the second bytecode. Of course, the step of "compiling the second bytecode" can also be omitted in S604, depending on the actual situation, and is not limited here. The choice of which method to use depends on the actual situation, and is not limited here. In some embodiments, the first computation graph and the split graph point bytecode can be executed sequentially. For bytecode that is compiled later than the split graph point bytecode, it can be executed after the split graph point bytecode is executed, or before the split graph point bytecode is executed, or it can be executed in parallel with the split graph point bytecode or the first computation graph. The specific execution method depends on the actual situation and is not limited here.

[0101] The following sections will introduce two methods for constructing function objects for the part after the split point.

[0102] Method 1:

[0103] As shown in Figure 9, 1. The bytecode of the innermost split graph point is moved to the position of the split graph point shown in the dynamic graph execution part of the figure after the graph is executed.

[0104] 2. Pass the required parameters (a1…an, b1…bn, c1…cn…) of each layer to bytecode A2 in the order of stack-local variables (or any other order). This generates the operation that calls bytecode A2 (i.e., the last CALL_FUNCTION shown in the execution part of the dynamic diagram).

[0105] 3. For a portion of bytecode A2, add stack restoration, local variable operations, and CALL operations to call the portion of bytecode B2, generating a callable function. Pass the remaining required parameters (b1…bn, c1…cn…) to bytecode B2.

[0106] 4. Repeat step 3 until the last function call.

[0107] 5. For the last layer of functions, a stack restoration and local variable operation can be added to calculate the stack changes at the split point and restore the changed stack (that is, to restore the variables in the stack after the split point to the correct order). Because the split point of the innermost function has been moved to be executed before the operation of calling bytecode A2, and the result of the execution is ultimately passed in through the parameters, it is necessary to restore the correct order.

[0108] In Figure 9, “New_fn_1” shown in the dynamic graph execution section can be understood as the scheduling bytecode described above, “A1->B1->C1” shown in the static graph execution section can be understood as a computation graph obtained by compilation, and “New_fn_2” shown in the graph capture section can be understood as the second bytecode mentioned above.

[0109] Method 2:

[0110] As shown in Figure 10, compared to the scheme shown in Figure 9, this scheme eliminates the hierarchical call relationship, expanding the calls of bytecode A2, B2, and C2 to the same level. Compared to the scheme shown in Figure 9, this scheme reduces the parameter passing required for each level of call, but adds the processing of the return values ​​of preceding function calls. Generating such code only requires the following adjustments based on the code generation of the scheme shown in Figure 9:

[0111] 1. When passing parameters, only the parameters required by the current layer need to be passed.

[0112] 2. The return value generated by the preceding call is stored in the corresponding stack element.

[0113] 3. Repeat steps 1 and 2 in the execution order to generate function calls, eventually generating New fn_2, which is captured again.

[0114] In Figure 10, “New_fn_1” shown in the dynamic graph execution section can be understood as the scheduling bytecode described above, “A1->B1->C1” shown in the static graph execution section can be understood as a computation graph obtained by compilation, and “New_fn_2” shown in the graph capture section can be understood as the second bytecode mentioned above.

[0115] Both the schemes shown in Figure 9 and Figure 10 only add some processed bytecode without modifying other parts of the original function. Therefore, the remaining bytecode still generates function calls, and there are no issues such as variable name conflicts or ambiguous namespaces. As can be seen from Figures 9 and 10, the code compilation method provided in this embodiment can maintain the original function call relationship to generate reentrant code (i.e., the second bytecode), and finally re-enable the newly generated function to re-enter the graph capture process.

[0116] In this embodiment, the data required to generate the aforementioned second bytecode (e.g., “New_fn_2” shown in Figure 9 or Figure 10) may include one or more of the following:

[0117] 1. Active variables of the subgraph generated before the split point, and variables on the stack after the split point is executed.

[0118] 2. The bytecode of the split point is used to calculate stack changes. Specifically, by using the CPython runtime application programming interface to calculate the bytecode of the split point, the increase or decrease of variables on the stack can be determined.

[0119] 3. Bytecode executed later than the split point.

[0120] 4. Call the function of the split point to inherit the free variables, namespace and other attributes of the function corresponding to the split point.

[0121] It should be understood that the aforementioned code compilation method can be implemented, but is not limited to, within programs that analyze and optimize Python code, and can be applied to software such as machine learning or deep learning network optimization. This method achieves reentrancy by expanding each sub-function layer, reducing context recovery during graph splitting and the overhead of graph execution switching. The system architecture implementing this method can be shown in Figure 11, primarily located within the CPython runtime (or other runtimes), specifically in the graph capture and code generation section. In Figure 11, the runtime (Just-in-Time, JIT) refers to the runtime analysis system. Graph capture refers to converting the bytecode analyzed by JIT into an IR (In-Reference Language), referred to here as a graph, and then handing it over to a mature optimizer and compiler to generate efficient running code. Graph splitting occurs when graph capture encounters a scenario that the IR cannot represent, causing graph capture to cease; this is called graph splitting. At this point, it is necessary to restore the Python (or other programming language) context (restoring the context of the split graph's points) and return from optimized code execution to Python execution. Reentrancy refers to the process of returning to Python after the split graph has executed one or more unsupported Python statements (or functions) and then re-entering JIT.

[0122] The above is an introduction to the code compilation method provided in the embodiments of this application. For ease of understanding, the code compilation method described in Figure 6 will be explained below with an example.

[0123] For example, as shown in Figure 12, in Figure 12(A), the bytecode in blocks A, B, and C is analyzed. If bytecodes A1 to A3, B1 to B4, and C1 to C2 can all be compiled into static graphs, then when analyzing bytecode C3, as shown in Figure 12(B), A1 to A3 in block A can be compiled into graph fg A, B1 to B4 in block B can be compiled into graph fg B, and C1 to C2 in block C can be compiled into graph fg C. Then, as shown in Figure 12(C), graphs fg A, B, and C are compiled into a computation graph, fg D. In fg D, the result of executing fg C can be returned to fg B. After executing fg B, the result of executing fg B and the result from fg C can be returned to fg A. After executing fg A, the results of executing fg C, fg B, and fg A can be output together.

[0124] Alternatively, as shown in Figure 12(D), other bytecodes executed later than bytecode C3 can be integrated together to form a new bytecode. Of course, they can also be integrated as shown in Figure 5(B).

[0125] Next, the scheduling bytecode shown in Figure 12(E) can be constructed. Finally, as shown in Figure 12(F), the scheduling bytecode can be executed. When executing the scheduling bytecode, fg D can be called first, then bytecode C3 can be called, and finally the integrated bytecode shown in Figure 12(D) can be called. After calling the integrated bytecode shown in Figure 12(D), the integrated bytecode can be re-analyzed for processes such as graph capture.

[0126] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments or technical features involved in the embodiments described above can be combined according to the actual situation, and the combined solution is still within the protection scope of this application.

[0127] Based on the methods in the above embodiments, this application also provides a code compilation apparatus.

[0128] For example, Figure 13 shows a schematic diagram of a code compilation device provided in an embodiment of this application. As shown in Figure 13, the code compilation device 1300 includes an acquisition module 1301 and a processing module 1302. The acquisition module 1301 is used to acquire a first bytecode corresponding to a target code from runtime, the target code being used to process a machine learning model. The processing module 1302 is used to compile the bytecode in the first bytecode that is executed before the split-map point bytecode into a first computation graph, wherein the split-map point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed, the split-map point bytecode is called in the first bytecode, and the split-map point bytecode does not call other bytecode in the first bytecode. The processing module 1302 is also used to execute the first computation graph and the split-map point bytecode.

[0129] In some embodiments, the output of the first computation graph is the active variables of a plurality of second computation graphs; wherein, a second computation graph is obtained by compiling bytecode that is executed in the first bytecode before the split graph point bytecode and is in the same call level.

[0130] In some embodiments, the processing module 1302 is further configured to: combine the bytecode in the first bytecode that is executed later than the split point bytecode into a second bytecode, and compile the second bytecode.

[0131] In some embodiments, when the processing module 1302 compiles the bytecode in the first bytecode that is executed later than the crack point bytecode into the second bytecode, it specifically performs the following: adding variable processing bytecode before the bytecode that is at the same call level as the crack point bytecode and is executed later than the crack point bytecode. The variable processing bytecode is used to perform stack restoration and local variable operations. In the first bytecode, within each call level that has a call relationship with the crack point bytecode, adding variable processing bytecode and call bytecode before the bytecode that is executed later than the crack point bytecode, wherein the call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. According to the call level related to the crack point bytecode in the first bytecode, the bytecode with added variable processing bytecode and call bytecode, and the bytecode with added variable processing bytecode are combined to obtain the second bytecode.

[0132] In some embodiments, when the processing module 1302 combines bytecode executed later than the split-map point bytecode in the first bytecode to form the second bytecode, it is specifically used to: construct a second sub-bytecode, the second sub-bytecode including: multiple calling bytecodes, one calling bytecode being used to call a bytecode that has a calling relationship with the first bytecode in the calling hierarchy and is executed later than the split-map point bytecode, or being used to call a bytecode that is in the same calling hierarchy as the split-map point bytecode in the first bytecode and is executed later than the split-map point bytecode; the execution order of the calling bytecodes is the opposite of the calling hierarchy of the bytecode called by the calling bytecode; and combining the second sub-bytecode with the bytecode executed later than the split-map point bytecode in the first bytecode to obtain the second bytecode.

[0133] In some embodiments, after obtaining the first computation graph and the second bytecode, the processing module 1302 is further configured to: construct scheduling bytecode and execute the scheduling bytecode to execute the first computation graph and the split graph point bytecode, and compile the second bytecode. The scheduling bytecode includes a first call bytecode, a second call bytecode, and a third call bytecode arranged sequentially; the first call bytecode is used to call the first computation graph, the second call bytecode is used to call the split graph point bytecode, and the third call bytecode is used to call the second bytecode.

[0134] In some embodiments, both the acquisition module 1301 and the processing module 1302 shown in FIG. 13 can be implemented in software or in hardware. For example, the implementation of the acquisition module 1301 will be described below. Similarly, the implementation of the processing module 1302 can refer to the implementation of the acquisition module 1301.

[0135] As an example of a software functional unit, module 1301 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 1301 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0136] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0137] As an example of a hardware functional unit, the acquisition module 1301 may include at least one computing device, such as a server. Alternatively, the acquisition module 1301 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0138] The multiple computing devices included in the acquisition module 1301 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 1301 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 1301 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0139] It should be noted that, in other embodiments, the acquisition module 1301 can be used to execute any step in the code compilation method described in the above embodiments, and the processing module 1302 can also be used to execute any step in the code compilation method described in the above embodiments. Furthermore, the steps implemented by the acquisition module 1301 and the processing module 1302 can be specified as needed. By implementing different steps in the code compilation method described in the above embodiments through the acquisition module 1301 and the processing module 1302, all the functions of the code compilation device 1300 shown in FIG13 can be achieved.

[0140] This application also provides a computing device 1400. As shown in FIG14, the computing device 1400 includes: a bus 1402, a processor 1404, a memory 1406, and a communication interface 1408. The processor 1404, the memory 1406, and the communication interface 1408 communicate with each other via the bus 1402. The computing device 1400 may be a server or an electronic device. It should be understood that this application does not limit the number of processors and memories in the computing device 1400.

[0141] Bus 1402 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 14, but this does not imply that there is only one bus or one type of bus. Bus 1404 can include pathways for transmitting information between various components of computing device 1400 (e.g., memory 1406, processor 1404, communication interface 1408).

[0142] The processor 1404 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0143] The memory 1406 may include volatile memory, such as random access memory (RAM). The processor 1404 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0144] The memory 1406 stores executable program code, and the processor 1404 executes the executable program code to implement the functions of the acquisition module 1301 and the processing module 1302 shown in FIG. 13, thereby implementing the code compilation method described in the above embodiments. That is, the memory 1406 stores instructions for executing the code compilation method described in the above embodiments.

[0145] Alternatively, the memory 1406 stores executable code, and the processor 1404 executes the executable code to implement the functions of the code compilation apparatus 1300 shown in FIG. 13, thereby implementing the code compilation method described in the above embodiments. That is, the memory 1406 stores instructions for executing the code compilation method described in the above embodiments.

[0146] The communication interface 1403 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1400 and other devices or communication networks.

[0147] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. This computing device can be a server, such as a central server, an edge server, or a local server in a local data center.

[0148] As shown in Figure 15, the computing device cluster includes at least one computing device 1400. The memory 1406 of one or more computing devices 1400 in the computing device cluster may store the same instructions for executing the code compilation method described in the above embodiments.

[0149] In some possible implementations, the memory 1406 of one or more computing devices 1400 in the computing device cluster may also store partial instructions for executing the code compilation method described in the above embodiments. In other words, a combination of one or more computing devices 1400 can jointly execute instructions for executing the code compilation method described in the above embodiments.

[0150] It should be noted that the memory 1406 in different computing devices 1400 within the computing device cluster can store different instructions, which are used to execute some of the functions of the code compilation device 1300 shown in Figure 13. That is, the instructions stored in the memory 1406 of different computing devices 1400 can implement the functions of one or more modules in the acquisition module 1301 and the processing module 1302.

[0151] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 16 illustrates one possible implementation. As shown in Figure 16, two computing devices 1400A and 1400B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1406 in computing device 1400A stores instructions for executing the functions of the acquisition module 1301. Simultaneously, the memory 1406 in computing device 1400B stores instructions for executing the functions of the processing module 1302.

[0152] It should be understood that the functions of computing device 1400A shown in Figure 16 can also be performed by multiple computing devices 1400. Similarly, the functions of computing device 1400B can also be performed by multiple computing devices 1400.

[0153] This application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device clusters shown in Figures 15 and 16. The difference is that the memory 1406 of one or more computing devices 1400 in this computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0154] In some possible implementations, the memory 1406 of one or more computing devices 1400 in the computing device cluster may also store partial instructions for executing the aforementioned methods. In other words, a combination of one or more computing devices 1400 can jointly execute the instructions for executing the aforementioned methods.

[0155] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When the computer program instructions are executed by a computing device, the computing device performs the methods in the above embodiments; or, when the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices performs the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0156] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when executed by a computing device, cause the computing device to perform the methods in the above embodiments, or, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the methods in the above embodiments.

[0157] Based on the methods in the above embodiments, this application also provides a code compilation apparatus. Please refer to FIG17, which is a schematic diagram of another code compilation apparatus provided in this application. As shown in FIG17, the code compilation apparatus 1700 includes one or more processors 1701 and interface circuits 1702. Optionally, the code compilation apparatus 1700 may further include a bus 1703. Wherein:

[0158] Processor 1701 can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 1701 or through software instructions. Processor 1701 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods and steps disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0159] The interface circuit 1702 can be used to send or receive data, instructions or information. The processor 1701 can use the data, instructions or other information received by the interface circuit 1702 to process the data, instructions or other information, and can send the processed information out through the interface circuit 1702.

[0160] Optionally, the code compilation apparatus 1700 also includes memory, which may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0161] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which may be stored in the operating system).

[0162] Optionally, the interface circuit 1702 can be used to output the execution results of the processor 1701.

[0163] It should be noted that the functions of processor 1701 and interface circuit 1702 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.

[0164] It should be understood that each step of the above method embodiments can be completed by hardware logic circuits or software instructions in a processor.

[0165] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0166] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0167] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, cloud computing platform, or data center to another website, computer, cloud computing platform, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a cloud computing platform or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0168] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A code compiling method characterized by, The method includes: Obtain the first bytecode corresponding to the target code from the runtime, the target code being used to process the machine learning model; The bytecode that executes before the split point bytecode in the first bytecode is compiled into a first computation graph, wherein the split point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed, the split point bytecode is called in the first bytecode, and the split point bytecode does not call other bytecode in the first bytecode; Execute the first computation graph and the split graph point bytecode.

2. The method of claim 1, wherein, The output of the first computation graph is the active variables of multiple second computation graphs; One of the second computation graphs is obtained by compiling the bytecode that is executed before the split graph point bytecode in the first bytecode and is in the same call level.

3. The method according to claim 1 or 2, characterized in that, Also includes: The bytecodes in the first bytecode that are executed later than the bytecodes of the split point are combined into the second bytecode; Compile the second bytecode.

4. The method of claim 3, wherein, The step of combining the bytecode in the first bytecode that is executed later than the bytecode of the split point into the second bytecode includes: Add variable processing bytecode before the bytecode that is at the same call level as the split point bytecode and is executed later than the split point bytecode. The variable processing bytecode is used to perform stack recovery and local variable operations. In the first bytecode, in each call level that has a call relationship with the split point bytecode, and before the bytecode executed later than the split point bytecode, the variable processing bytecode and the call bytecode are added. The call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. According to the call hierarchy related to the split point bytecode in the first bytecode, the bytecode with added variable processing bytecode and the call bytecode, as well as the bytecode with added variable processing bytecode, are combined to obtain the second bytecode.

5. The method of claim 3, wherein, The step of combining the bytecode in the first bytecode that is executed later than the bytecode of the split point into the second bytecode includes: Construct a second sub-bytecode, which includes: multiple calling bytecodes, one of which is used to call a bytecode that is in the same calling hierarchy as the first bytecode and is executed later than the split point bytecode, or is used to call a bytecode in the first bytecode that is in the same calling hierarchy as the split point bytecode and is executed later than the split point bytecode; the execution order of the calling bytecodes is the opposite of the calling hierarchy of the bytecodes they call. The second sub-bytecode is combined with the bytecode in the first bytecode that is executed later than the split point bytecode to obtain the second bytecode.

6. The method according to any of claims 3-5, characterized by, After obtaining the first computation graph and the second bytecode, the process also includes: Construct scheduling bytecode and execute the scheduling bytecode to execute the first computation graph and the split graph point bytecode, and compile the second bytecode; The scheduling bytecode includes a first call bytecode, a second call bytecode, and a third call bytecode arranged in sequence. The first calling bytecode is used to call the first computation graph, the second calling bytecode is used to call the split graph point bytecode, and the third calling bytecode is used to call the second bytecode.

7. A code compiling method characterized by, The method includes: Obtain the first bytecode corresponding to the target code from the runtime, wherein the target code is used to process the machine learning model; The bytecodes in the first bytecode that are executed later than the split point bytecode are combined into the second bytecode. The split point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed. The split point bytecode is called in the first bytecode, and the split point bytecode does not call other bytecodes in the first bytecode. Compile the second bytecode.

8. The method of claim 7, wherein, The step of combining the bytecode in the first bytecode that is executed later than the bytecode of the split point into the second bytecode includes: Add variable processing bytecode before the bytecode that is at the same call level as the split point bytecode and is executed later than the split point bytecode. The variable processing bytecode is used to perform stack recovery and local variable operations. In the first bytecode, within each call level that has a call relationship with the split point bytecode, and before the bytecode executed later than the split point bytecode, the variable processing bytecode and the call bytecode are added. The call bytecode added to the bytecode in one call level is used to call the bytecode in the next call level. According to the call hierarchy related to the split point bytecode in the first bytecode, the bytecode with added variable processing bytecode and the call bytecode, as well as the bytecode with added variable processing bytecode, are combined to obtain the second bytecode.

9. The method of claim 7, wherein, The step of combining the bytecode in the first bytecode that is executed later than the bytecode of the split point into the second bytecode includes: Construct a second sub-bytecode, which includes: multiple calling bytecodes, one of which is used to call a bytecode that is in the same calling hierarchy as the first bytecode and is executed later than the split point bytecode, or is used to call a bytecode in the first bytecode that is in the same calling hierarchy as the split point bytecode and is executed later than the split point bytecode; the execution order of the calling bytecodes is the opposite of the calling hierarchy of the bytecodes they call. The second sub-bytecode is combined with the bytecode in the first bytecode that is executed later than the split point bytecode to obtain the second bytecode.

10. A code compiling apparatus characterized by comprising: include: An acquisition module is used to acquire the first bytecode corresponding to the target code from the runtime, wherein the target code is used to process the machine learning model; The processing module is used to compile the bytecode that is executed before the split point bytecode in the first bytecode into a first computation graph, wherein the split point bytecode is the first bytecode that needs to be dynamically executed in the first bytecode, the split point bytecode is called in the first bytecode, and the split point bytecode does not call other bytecode in the first bytecode; The processing module is also used to execute the first computation graph and the split graph point bytecode.

11. The apparatus of claim 10, wherein, The output of the first computation graph is the active variables of multiple second computation graphs; One of the second computation graphs is obtained by compiling the bytecode that is executed before the split graph point bytecode in the first bytecode and is in the same call level.

12. The apparatus of claim 10 or 11, wherein, The processing module is further configured to: combine the bytecode in the first bytecode that is executed later than the bytecode of the split point into a second bytecode, and compile the second bytecode.

13. The apparatus of claim 12, wherein, When the processing module combines the bytecode executed later than the split point bytecode in the first bytecode into the second bytecode, it is specifically used for: Add variable processing bytecode before the bytecode that is at the same call level as the split point bytecode and is executed later than the split point bytecode. The variable processing bytecode is used to perform stack recovery and local variable operations. In the first bytecode, in each call level that has a call relationship with the split point bytecode, and before the bytecode executed later than the split point bytecode, the variable processing bytecode and the call bytecode are added. The call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. According to the call hierarchy related to the split point bytecode in the first bytecode, the bytecode with added variable processing bytecode and the call bytecode, as well as the bytecode with added variable processing bytecode, are combined to obtain the second bytecode.

14. The apparatus of claim 12, wherein, When the processing module combines the bytecode executed later than the split point bytecode in the first bytecode into the second bytecode, it is specifically used for: Construct a second sub-bytecode, which includes: multiple calling bytecodes, one of which is used to call a bytecode that is in the same calling hierarchy as the first bytecode and is executed later than the split point bytecode, or is used to call a bytecode in the first bytecode that is in the same calling hierarchy as the split point bytecode and is executed later than the split point bytecode; the execution order of the calling bytecodes is the opposite of the calling hierarchy of the bytecodes they call. The second sub-bytecode is combined with the bytecode in the first bytecode that is executed later than the split point bytecode to obtain the second bytecode.

15. The apparatus of any of claims 12-14, wherein, After obtaining the first computation graph and the second bytecode, the processing module is further configured to: Construct scheduling bytecode and execute the scheduling bytecode to execute the first computation graph and the split graph point bytecode, and compile the second bytecode; The scheduling bytecode includes a first call bytecode, a second call bytecode, and a third call bytecode arranged in sequence. The first calling bytecode is used to call the first computation graph, the second calling bytecode is used to call the split graph point bytecode, and the third calling bytecode is used to call the second bytecode.

16. A code compiling apparatus characterized by comprising: include: An acquisition module is used to acquire the first bytecode corresponding to the target code from the runtime, wherein the target code is used to process the machine learning model; The processing module is used to combine the bytecode in the first bytecode that is executed later than the crack point bytecode into a second bytecode, wherein the crack point bytecode is the first bytecode in the first bytecode that needs to be dynamically executed, the crack point bytecode is called in the first bytecode, and the crack point bytecode does not call other bytecode in the first bytecode; The processing module is also used to compile the second bytecode.

17. The apparatus of claim 16, wherein, When the processing module combines the bytecode executed later than the split point bytecode in the first bytecode into the second bytecode, it is specifically used for: Add variable processing bytecode before the remaining bytecode that is at the same call level as the split point bytecode and is executed later than the split point bytecode. The variable processing bytecode is used to perform stack recovery and local variable operations. In the first bytecode, in each call level that has a call relationship with the split point bytecode, and before the bytecode executed later than the split point bytecode, the variable processing bytecode and the call bytecode are added. The call bytecode added before the bytecode in one call level is used to call the bytecode in the next call level. According to the call hierarchy related to the split point bytecode in the first bytecode, the bytecode with added variable processing bytecode and the call bytecode, as well as the bytecode with added variable processing bytecode, are combined to obtain the second bytecode.

18. The apparatus of claim 16, wherein, When the processing module combines the bytecode executed later than the split point bytecode in the first bytecode into the second bytecode, it is specifically used for: Construct a second sub-bytecode, which includes: multiple calling bytecodes, one of which is used to call a bytecode that is in the same calling hierarchy as the first bytecode and is executed later than the split point bytecode, or is used to call a bytecode in the first bytecode that is in the same calling hierarchy as the split point bytecode and is executed later than the split point bytecode; the execution order of the calling bytecodes is the opposite of the calling hierarchy of the bytecodes they call. The second sub-bytecode is combined with the bytecode in the first bytecode that is executed later than the split point bytecode to obtain the second bytecode.

19. A cluster of computing devices, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-6, or to perform the method as described in any one of claims 7-9.

20. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-6, or to perform the method as described in any one of claims 7-9, wherein the cluster of computing devices includes at least one computing device.

21. A computer program product comprising instructions, wherein: When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-6, or to perform the method as described in any one of claims 7-9, wherein the computing device cluster includes at least one computing device.

22. A code compiling apparatus characterized by comprising: include: Includes at least one processor and interface; The at least one processor obtains program instructions through the interface; The at least one processor is configured to execute the program line instructions to implement the method as described in any one of claims 1-6, or the method as described in any one of claims 7-9.

Images