Code compiling method and device, electronic equipment and storage medium
Patent Information
- Application Number
- CN202211246270.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-12
- Publication Date
- 2026-09-25
- Estimated Expiration
- 2042-10-12
AI Technical Summary
[0003]在相关技术中,对于代码的编译,通常是通过单一的代码编译机对待编译代码进行编译,这样,导致代码的编译效率极低,代码的编译速度极低
通过响应于代码编译请求,确定待编译代码的编译类型,并确定编译类型对应的目标编译集群,并基于运行信息,从目标编译集群中的代码编译机中,选择目标代码编译机,将待编译代码对应的编译任务下发至目标代码编译机,以使目标代码编译机对待编译代码进行编译。如此,通过将不同编译类型的待编译代码对应的编译任务,下发至不同编译集群中的目标代码编译机中进行编译,通过针对不同编译类型的待编译代码,部署差异化的代码编译机集群,并基于运行信息从相应的编译集群中确定目标代码编译机,从而更加准确的确定出待编译代码最适合的编译集群和编译集群中的目标代码编译机,从而有效提高了代码编译机的编译效率,从而有效提高编译代码的效率和速度。
Smart Images

Figure CN116991409B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a code compilation method, apparatus, electronic device, and storage medium. Background Technology
[0002] Code compilation is the process of translating source code written in a high-level language into binary code that a computer can recognize. During the development of software programs, multiple rounds of modification, compilation, and debugging are often required.
[0003] In related technologies, code compilation is usually performed by a single code compiler, which results in extremely low compilation efficiency and speed. Summary of the Invention
[0004] This application provides a code compilation method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the efficiency and speed of code compilation.
[0005] The technical solution of this application embodiment is implemented as follows: This application provides a code compilation method, including: A code compilation request is received, instructing that code to be compiled be compiled, wherein the code to be compiled is code written in a compiled language; In response to the code compilation request, the compilation type of the code to be compiled is determined, and the target compilation cluster corresponding to the compilation type is determined, wherein the target compilation cluster includes at least two code compilation machines, and different compilation types correspond to different target compilation clusters; Obtain the running information of each code compiler in the target compilation cluster, and determine the target code compiler from the at least two code compilers based on the running information; A compilation task corresponding to the code to be compiled is generated, and the compilation task is sent to the target code compiler so that the target code compiler can compile the code to be compiled.
[0006] This application provides a code compilation apparatus, including: A receiving module is configured to receive a code compilation request that instructs the compilation of code to be compiled, wherein the code to be compiled is code written in a compiled language; A response module is used to respond to the code compilation request, determine the compilation type of the code to be compiled, and determine the target compilation cluster corresponding to the compilation type, wherein the target compilation cluster includes at least two code compilation machines, and different compilation types correspond to different target compilation clusters; The determination module is used to obtain the running information of each code compiler in the target compilation cluster, and determine the target code compiler from the at least two code compilers based on the running information; The sending module is used to generate a compilation task corresponding to the code to be compiled, and send the compilation task to the target code compiler so that the target code compiler can compile the code to be compiled.
[0007] In some embodiments, the running information includes compilation task information indicating whether a compilation task exists; the determining module is further configured to filter each code compiler in the target compilation cluster based on the compilation task information of each code compiler in the target compilation cluster to obtain at least one candidate code compiler, wherein the candidate code compiler is a code compiler without a compilation task; and select a code compiler from the at least one candidate code compiler as the target code compiler.
[0008] In some embodiments, the running information further includes processing resource information and storage resource information; the determining module is further configured to determine the performance size of each candidate code compiler based on at least one of the processing resource information and storage resource information; and based on the determined performance size, select the candidate code compiler with the highest performance from the at least one candidate code compiler as the target code compiler.
[0009] In some embodiments, the determining module is further configured to perform the following processes for each of the candidate code compilers: determining the processing performance score of the candidate code compiler based on the processing resource information of the candidate code compiler; determining the storage performance score of the candidate code compiler based on the storage resource information of the candidate code compiler; and performing a weighted sum of the processing performance score and the storage performance score to obtain the performance magnitude of the candidate code compiler.
[0010] In some embodiments, the running information further includes processing resource information and storage resource information, wherein the storage resource information includes the cache rate of the code compiler for historical compilation tasks; the determining module is further configured to determine the processing performance score of each candidate code compiler based on the processing resource information of each candidate code compiler, and sort each candidate code compiler according to the processing performance score to obtain an initial sorting queue, wherein the candidate code compilers in the initial sorting queue are sorted in descending order of the processing performance score; starting from the first candidate code compiler in the initial sorting queue, candidate code compilers are selected sequentially to obtain a target number of candidate code compilers, wherein the target number is less than the total number of candidate code compilers in the first sorting queue; and the candidate code compiler with the highest cache rate among the target number of candidate code compilers is determined as the target code compiler.
[0011] In some embodiments, the determining module is used to obtain the runtime information of the code compiler from the agent corresponding to each code compiler in the target compilation cluster; wherein the agent is an application deployed in the code compiler for collecting the runtime information.
[0012] In some embodiments, the response module is configured to respond to a code compilation request, parse the code compilation request, and obtain the task parameters of the code compilation request; when the task parameters include a compilation type identifier, the compilation type of the code to be compiled is determined to be an instrumentation compilation type; when the task parameters do not include a compilation type identifier, the compilation type of the code to be compiled is determined to be a regular compilation type; wherein, the compilation type identifier is used to identify the instrumentation compilation type, and the code to be compiled of the instrumentation compilation type is used to modify the functionality of the code being compiled.
[0013] In some embodiments, the sending module is further configured to obtain the system compilation file corresponding to the compilation type and the target platform identifier associated with the code to be compiled; wherein, the system compilation file is used to record the library functions in the development system, and the target platform identifier is used to identify the development platform to which the code to be compiled is applicable; and a compilation task is generated based on the system compilation file, the target platform identifier and the code to be compiled.
[0014] In some embodiments, when the compilation type is an instrumentation compilation type, the sending module is further configured to obtain a system file list, which records system files in the development system; determine a compilation file list corresponding to the code to be compiled, which records code files corresponding to the code to be compiled, including system files corresponding to the code to be compiled; compare the system files in the system file list with each code file in the compilation file list to obtain overlapping files, wherein the overlapping files are files that exist in both the full system file list and the compilation file list; and determine the overlapping files as system compilation files corresponding to the instrumentation compilation type.
[0015] In some embodiments, when the compilation type is a regular compilation type, the sending module is further configured to obtain a system file list, wherein the system file list is used to record system files in the development system; and to determine each system file in the system file list as a system compilation file corresponding to the regular compilation type.
[0016] In some embodiments, the above-described code compilation apparatus further includes: a clustering module, configured to obtain the number of types of the compilation type and a number of code compilers to be clustered that is greater than or equal to a reference number, wherein the reference number is twice the number of types; and to cluster the code compilers to be clustered based on the compilation type to obtain a compilation cluster corresponding to each compilation type.
[0017] In some embodiments, the response module is further configured to select a target compilation cluster corresponding to the compilation type of the code to be compiled from the compilation clusters corresponding to each compilation type.
[0018] This application provides an electronic device, including: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the code compilation method provided in the embodiments of this application.
[0019] This application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the code compilation method provided in this application.
[0020] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the code compilation method described in this application.
[0021] The embodiments of this application have the following beneficial effects: By responding to code compilation requests, the compilation type of the code to be compiled is determined, and the target compilation cluster corresponding to the compilation type is identified. Based on runtime information, a target code compiler is selected from the code compilers within the target compilation cluster, and the compilation task corresponding to the code to be compiled is distributed to the target code compiler so that it can compile the code. In this way, by distributing compilation tasks for code to be compiled with different compilation types to target code compilers in different compilation clusters, and by deploying differentiated code compiler clusters for code to be compiled with different compilation types, and by selecting the target code compiler from the corresponding compilation cluster based on runtime information, the most suitable compilation cluster and target code compiler within that cluster are more accurately determined, thereby effectively improving the compilation efficiency of the code compiler and ultimately increasing the efficiency and speed of code compilation. Attached Figure Description
[0022] Figure 1 This is a schematic diagram of the code compilation system architecture provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the code compilation device provided in the embodiments of this application; Figures 3 to 4 This is a flowchart illustrating the code compilation method provided in an embodiment of this application; Figures 5 to 7 This is a schematic diagram illustrating the principle of the code compilation method provided in the embodiments of this application; Figure 8 This is a schematic diagram illustrating the effect of the code compilation method provided in the embodiments of this application. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0024] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0025] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.
[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0027] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.
[0028] 1) Compilation: Compilation is the process of using a compiler to generate a target program from a source program written in a source language. It's the act of using a compiler to produce the target program. Compilation transforms a high-level language into a binary language that a computer can understand. Computers only recognize 1s and 0s; the compiler converts the familiar human language into binary. The process of translating a source program into a target program by a compiler consists of five stages: lexical analysis; syntax analysis; semantic checking and intermediate code generation; code optimization; and target code generation. The main stages are lexical and syntax analysis, also known as source program analysis. During this analysis, syntax errors are detected and error messages are provided. A compiled language is a programming language implemented using a compiler. Unlike interpreted languages, which are executed line by line by an interpreter, compiled languages use a compiler to first compile the code into machine code before execution. Theoretically, any programming language can be either compiled or interpreted. The difference lies only in the application of the program.
[0029] 2) Compiler instrumentation: This is the process of modifying existing code or generating new code during code compilation.
[0030] 3) Code: This refers to source files written in a language supported by development tools. It is a set of explicit rules for representing information in discrete form using characters, symbols, or signal elements. Code design principles include uniqueness, standardization and universality, extensibility and stability, ease of identification and memorization, brevity and consistent formatting, and ease of modification. Source code is a branch of code; in a sense, source code is equivalent to code. In modern programming languages, source code can appear in the form of books or tapes, but the most common format is a text file. This typical format is designed for compiling computer programs. The ultimate goal of computer source code is to translate human-readable text into executable binary instructions; this process is called compilation, and it is accomplished by a compiler.
[0031] 4) Pipeline: This refers to a quasi-parallel processing technique where multiple instructions overlap during program execution. Simply put, it decomposes a sequential process into several sub-processes, each of which can be executed concurrently with other sub-processes. The aim is to improve processor efficiency and complete one instruction within one clock cycle.
[0032] During the implementation of the embodiments of this application, the applicant discovered the following problems with the related technology: In related technologies, code can be compiled either on a cloud machine or on a local development machine.
[0033] In related technologies, code compilation can be performed on cloud machines. However, regardless of whether compilation is performed on cloud machines or local development machines, it's impossible to distinguish between versions with coverage instrumentation and normal business versions. Compilation with coverage instrumentation is mixed with normal compilation, rendering cached content inapplicable, essentially requiring a full recompile. In scenarios involving multiple system compilation files, duplicate linking exists, and the compilation files of systems affected by code changes are not analyzed, further increasing processing time. Furthermore, it's difficult for compilation machine distribution to maximize cache utilization.
[0034] While some technologies allow compilation on local development machines, they suffer from limitations such as inability to dynamically scale and insufficient concurrency. When the compilation volume is large, local compilation cannot handle the workload, often resulting in queuing. Furthermore, the lack of incremental code analysis leads to insufficient compilation time. This makes them unsuitable for high-concurrency, high-frequency, and time-sensitive scenarios like pipelines.
[0035] The code compilation method provided in this application distributes different compilation tasks to different compilation clusters by splitting requests for normal versions and coverage instrumentation versions, making full use of compilation caching. This application uses command-line parameters to distinguish whether a compilation task is an instrumentation version; if so, the request is distributed to the instrumentation version compilation cluster; otherwise, it is forwarded to the normal version compilation cluster. Automatic incremental analysis identifies which system compilation files are affected by the code, eliminating omissions and errors. A rich distribution strategy is established, avoiding random distribution and fully utilizing the load and caching of each machine in the compilation cluster. Dynamic scaling is supported. The compiled cluster has a unified entry point, and the backend machines are all cloud development machines. When insufficient resources are detected, machine resource request-initialization-joining to the build cluster is automatically triggered, achieving dynamic scaling. Efficient caching is utilized. The maintenance cost of the local compilation environment for R&D engineers is reduced. Since all compilation actions are executed on the remote compilation cluster, developers do not need to maintain their local compilation environment, greatly reducing daily R&D costs.
[0036] This application provides a code compilation method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can effectively improve the efficiency and speed of code compilation. The exemplary application of the code compilation system provided in this application is described below.
[0037] See Figure 1 , Figure 1 This is a schematic diagram of the architecture of the code compilation system 100 provided in the embodiments of this application. The terminal (terminal 400-1 is shown as an example) connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.
[0038] Terminal 400 is used by users to access client 401 and display code on graphical interface 401-1 (graphical interface 401-1 is shown as an example). Terminal 400 and server 200 are interconnected via wired or wireless network.
[0039] Server 200 can deploy multiple compilation clusters, each containing at least one code compiler machine. See also Figure 1 Compilation cluster 1 includes compiler (code compiler) 1-1, compiler 1-2, compiler 1-3, and compiler 1-n; compilation cluster 2 includes compiler (code compiler) 2-1, compiler 2-2, compiler 2-3, and compiler 2-n.
[0040] In some embodiments, server 200 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smart TV, smartwatch, in-vehicle terminal, etc., but is not limited to these. The electronic device provided in this application embodiment can be implemented as a terminal or a server. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited in this application embodiment.
[0041] In some embodiments, see Figure 1 Server 200 receives a code compilation request instructing the compilation of the code to be compiled, and determines the target compilation cluster from compilation cluster 1 and compilation cluster 2. It then determines the target code compiler from the target compilation cluster and sends the compilation task to the target code compiler, enabling the target code compiler to compile the code to be compiled and send the compiled code to terminal 400.
[0042] In other embodiments, terminal 400 receives a code compilation request that instructs the code to be compiled, determines the target code compiler, and sends the compilation task to the target code compiler.
[0043] In other embodiments, the embodiments of this application can be implemented with the aid of cloud technology, which refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize the computation, storage, processing, and sharing of data.
[0044] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, allowing for on-demand use with flexibility and convenience. Cloud computing technology will become a crucial support. The backend services of cloud computing systems require substantial computing and storage resources.
[0045] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device 500 for code compilation provided in an embodiment of this application, wherein, Figure 2 The electronic device 500 shown can be Figure 2 Server 200 or terminal 400 in the middle, Figure 2The illustrated electronic device 500 includes at least one processor 410, a memory 450, and at least one network interface 420. The various components in the electronic device 500 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.
[0046] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.
[0047] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.
[0048] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.
[0049] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.
[0050] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks; The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, such as Bluetooth, WiFi, and Universal Serial Bus.
[0051] In some embodiments, the code compilation apparatus provided in this application can be implemented in software. Figure 2 A code compilation device 455 stored in memory 450 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: a receiving module 4551, a responding module 4552, a determining module 4553, and a sending module 4554. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.
[0052] In other embodiments, the code compilation apparatus provided in this application can be implemented in hardware. As an example, the code compilation apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the code compilation method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.
[0053] In some embodiments, the terminal or server can implement the code compilation method provided in this application by running a computer program or computer-executable instructions. For example, the computer program can be a native program in the operating system (e.g., a dedicated code compiler) or a software module, such as a code compilation module that can be embedded in any program (e.g., an instant messaging client, a photo album program, an electronic map client, a navigation client); for example, it can be a native application (APP), that is, a program that needs to be installed in the operating system to run. In summary, the above-mentioned computer program can be any form of application, module, or plugin.
[0054] The code compilation method provided in this application will be described in conjunction with exemplary applications and implementations of the server or terminal provided in the embodiments of this application.
[0055] See Figure 3 , Figure 3 This is a flowchart illustrating the code compilation method provided in the embodiments of this application, which will be combined with... Figure 3Steps 101 to 104 are described below. The code compilation method provided in this application embodiment can be implemented by the server or the terminal alone, or by the server and the terminal working together. The following description will take the implementation by the server alone as an example.
[0056] In step 101, a code compilation request is received, which instructs the code to be compiled to be compiled.
[0057] In some embodiments, the code to be compiled is code written in a compiled language.
[0058] In some embodiments, the code language includes compiled languages and interpreted languages. Compiled languages require compilation before execution, transforming the code into an executable machine language file. Because the translation is done only once and doesn't need to be repeated, execution efficiency is higher. Interpreted languages, on the other hand, do not pre-compile the code; they are typically stored as text and translated only during execution. Each time the program is executed, the code is translated.
[0059] In some embodiments, the code to be compiled is code written in a compiled language. Therefore, the code to be compiled obtained in the embodiments of this application is code that needs to be compiled by a code compiler.
[0060] In some embodiments, a code compilation request is used to instruct the code to be compiled. Receiving a code compilation request can be performed by the server or the terminal alone, or by the server and the terminal working together.
[0061] In some embodiments, the code compilation request may carry the code to be compiled and its compilation type. The compilation type includes instrumentation compilation type and regular compilation type. Instrumentation compilation type refers to code used to modify existing code or generate new code during code compilation. Regular compilation type refers to code used to modify existing code or generate new code outside of code compilation.
[0062] In step 102, in response to the code compilation request, the compilation type of the code to be compiled is determined, and the target compilation cluster corresponding to the compilation type is determined.
[0063] In some embodiments, the target compilation cluster includes at least two code compilers, and different compilation types correspond to different target compilation clusters.
[0064] In some embodiments, the number of code compilers included in the target compilation cluster can be dynamically adjusted according to the amount of real-time compilation tasks. When the amount of real-time compilation tasks is greater than a set task threshold, the number of code compilers in the target compilation cluster can be increased. When the amount of real-time compilation tasks is less than the set task threshold, the number of code compilers in the target compilation cluster can be reduced or not changed.
[0065] In some embodiments, the computers and operating systems on which the code compilers reside are different, and the code compiler can be object code used to run in the environment of the computer and operating system.
[0066] In some embodiments, the code compilation request may carry the code to be compiled and its compilation type. The compilation type includes instrumentation compilation type and regular compilation type. Instrumentation compilation type refers to code used to modify existing code or generate new code during code compilation. Regular compilation type refers to code used to modify existing code or generate new code outside of code compilation.
[0067] As an example, see Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of the code compilation method provided in this application embodiment. The target compilation cluster corresponding to the instrumentation compilation type includes compiler 1 and compiler 2, while the target compilation cluster corresponding to the regular compilation type includes compiler 3 to compiler n.
[0068] In some embodiments, see Figure 4 , Figure 4 This is a flowchart illustrating the code compilation method provided in the embodiments of this application. Figure 4 In step 102 shown, the determination of the compilation type of the code to be compiled in response to the code compilation request is achieved through steps 1021 to 1023.
[0069] In step 1021, in response to the code compilation request, the code compilation request is parsed to obtain the task parameters of the code compilation request.
[0070] In some embodiments, the task parameters of a code compilation request include various parameters for generating the corresponding compilation task to be compiled. The code compilation request carries these task parameters. Thus, the task parameters can be obtained by parsing the code compilation request.
[0071] In step 1022, when the task parameters include a compilation type identifier, the compilation type of the code to be compiled is determined to be an instrumentation compilation type.
[0072] In some embodiments, a compilation type identifier is used to identify the instrumentation compilation type, and the code to be compiled for the instrumentation compilation type is used to modify the functionality of the code being compiled.
[0073] As an example, the compile type identifier can be the identifier "cov" in the task parameters or a task parameter in a pipeline scenario.
[0074] In some embodiments, a compilation type identifier is used to identify the instrumentation compilation type. When the task parameters include a compilation type identifier, it indicates that the compilation type of the code to be compiled is an instrumentation compilation type.
[0075] In step 1023, when the task parameters do not include a compilation type identifier, the compilation type of the code to be compiled is determined to be a regular compilation type.
[0076] In some embodiments, a compilation type identifier is used to identify the instrumentation compilation type. When the task parameters do not include a compilation type identifier, it indicates that the compilation type of the code to be compiled is a regular compilation type.
[0077] In some embodiments, code to be compiled of a regular compilation type refers to code used to modify existing code or to generate new code outside of the code compilation process.
[0078] Thus, by determining whether the task parameters include a compilation type identifier, the compilation type of the code to be compiled can be accurately determined. The compilation type identifier can accurately identify the compilation type of the code to be compiled, which facilitates the subsequent determination of the target compilation cluster corresponding to the compilation type, thereby effectively improving the accuracy of the determined target compilation cluster.
[0079] In some embodiments, prior to step 102 above, the compilation clusters corresponding to each compilation type can be obtained by: obtaining the number of compilation types and the number of code compilers to be clustered that is greater than or equal to the number of references, wherein the number of references is twice the number of types; and clustering the code compilers to be clustered based on the compilation type to obtain the compilation clusters corresponding to each compilation type.
[0080] In some embodiments, the code compilers to be grouped do not belong to any compilation cluster. By grouping the code compilers to be grouped, each code compiler to be grouped is assigned to a corresponding compilation cluster, and the number of code compilers in each compilation cluster is greater than or equal to 2.
[0081] In some embodiments, since the number of code compilers in each compilation cluster is greater than or equal to 2, that is, the total number of code compilers needs to be greater than or equal to twice the number of types, in order to ensure that there are at least two code compilers in each compilation cluster.
[0082] In this way, by grouping the compilers of the code to be grouped based on the compilation type, a compilation cluster corresponding to each compilation type is obtained, thus obtaining a compilation cluster corresponding to each compilation type. This facilitates the subsequent compilation of the compiled code of each compilation type through the compilation cluster, thereby effectively improving the efficiency and speed of code compilation.
[0083] In some embodiments, the determination of the target compilation cluster corresponding to the compilation type in step 102 above can be achieved in the following way: select the target compilation cluster corresponding to the compilation type of the code to be compiled from the compilation clusters corresponding to each compilation type.
[0084] In some embodiments, one compilation type corresponds to one compilation cluster. By determining the compilation type of the code to be compiled, the compilation type of the code to be compiled is compared with the compilation type corresponding to each compilation cluster. The compilation cluster that matches the compilation type of the code to be compiled is determined as the target compilation cluster corresponding to the compilation type of the code to be compiled.
[0085] As an example, compilation type 1 corresponds to compilation cluster 1, compilation type 2 corresponds to compilation cluster 2, and compilation type 3 corresponds to compilation cluster 3. When the compilation type of the code to be compiled is compilation type 1, compilation cluster 1 corresponding to compilation type 1 is determined as the target compilation cluster for the code to be compiled. When the compilation type of the code to be compiled is compilation type 2, compilation cluster 2 corresponding to compilation type 2 is determined as the target compilation cluster for the code to be compiled. When the compilation type of the code to be compiled is compilation type 3, compilation cluster 3 corresponding to compilation type 3 is determined as the target compilation cluster for the code to be compiled.
[0086] In step 103, the running information of each code compiler in the target compilation cluster is obtained, and based on the running information, the target code compiler is determined from at least two code compilers.
[0087] In some embodiments, the runtime information includes compilation task information indicating whether a compilation task exists, and also includes processing resource information and storage resource information, wherein the storage resource information includes the cache rate of the code compiler for historical compilation tasks.
[0088] In some embodiments, the cache ratio can be calculated as follows: obtain the total amount of data from all historical compilation tasks and the total amount of data from cached historical compilation tasks, and determine the cache ratio by the ratio of the total amount of data from cached historical compilation tasks to the total amount of data from all historical compilation tasks. The total amount of historical compilation tasks includes both cached and uncached historical compilation tasks.
[0089] In some embodiments, obtaining the runtime information of each code compiler in the target compilation cluster in step 103 above can be achieved in the following way: obtaining the runtime information of the code compiler from the agent corresponding to each code compiler in the target compilation cluster; wherein, the agent is an application deployed in the code compiler to collect runtime information.
[0090] In some embodiments, an agent is an application deployed in the code compiler to collect runtime information. Agents are also called intelligent agents or intelligent entities. Traditional computing systems are closed and require consistency; however, social mechanisms are open and cannot satisfy consistency conditions. In such mechanisms, some individuals, when faced with conflicting situations, need to reach an acceptable solution through some negotiation mechanism. These individuals in a computing society are called agents. The organic combination of these individuals constitutes a computing society—a multi-agent system. An agent refers to a computing entity that resides in a certain environment, can continuously and autonomously function, and possesses characteristics such as residency, responsiveness, sociality, and initiative.
[0091] In some embodiments, since the agent deployed in the code compiler is an application used to collect runtime information, the server or terminal can obtain the runtime information of the code compiler from the agent.
[0092] In this way, by collecting runtime information through an agent deployed in the code compiler and obtaining runtime information of the code compiler from the agent, it is easier to determine the target code compiler from at least two code compilers based on the runtime information, thereby effectively improving the accuracy of the determined target code compiler and thus effectively improving the efficiency and speed of code compilation.
[0093] In some embodiments, the above runtime information includes compilation task information indicating whether a compilation task exists, see [link to documentation]. Figure 4 , Figure 4 This is a flowchart illustrating the code compilation method provided in the embodiments of this application. Figure 4 The determination of the target code compiler from at least two code compilers based on runtime information in step 103 is achieved through steps 1031 to 1032.
[0094] In step 1031, based on the compilation task information of each code compiler in the target compilation cluster, each code compiler in the target compilation cluster is screened to obtain at least one candidate code compiler.
[0095] In some embodiments, step 1031 above can be implemented as follows: For each code compiler in the target compilation cluster, the following processing is performed: when the compilation task information of a code compiler indicates that a compilation task exists, the code compiler is identified as a candidate code compiler; when the compilation task information of a code compiler indicates that no compilation task exists, the code compiler is identified as a candidate code compiler. From the candidate code compilers, a candidate code compiler with fewer compilation tasks than the task threshold is selected and identified as a candidate code compiler.
[0096] In some embodiments, the candidate code compiler is a code compiler with no compilation tasks or a code compiler with fewer compilation tasks than the task threshold.
[0097] In step 1032, a code compiler is selected as the target code compiler from at least one candidate code compiler.
[0098] In some embodiments, the above-mentioned operating information further includes processing resource information and storage resource information, and the above-mentioned step 1032 can be implemented in the following manner: based on at least one of the processing resource information and storage resource information, determine the performance size of each candidate code compiler; based on the determined performance size, select the candidate code compiler with the highest performance from at least one candidate code compiler as the target code compiler.
[0099] In some embodiments, determining the performance of each candidate code compiler based on at least one of processing resource information and storage resource information can be implemented by performing the following processing for each candidate code compiler: determining the processing performance score of the candidate code compiler based on its processing resource information; determining the storage performance score of the candidate code compiler based on its storage resource information; and weighted summing the processing performance score and the storage performance score to obtain the performance of the candidate code compiler.
[0100] In some embodiments, processing resource information may include the processing performance of the candidate code compiler's central processing unit, the memory storage performance of the candidate code compiler, and the disk space size of the candidate code compiler. Storage resource information may include the cache rate of the code compiler for historical compilation tasks, and the effective cache rate divided by dimensions such as module / committer / compilation task.
[0101] In some embodiments, the processing resource information of the candidate code compiler, including the processing performance of the central processing unit, the memory storage performance of the candidate code compiler, and the disk space size of the candidate code compiler, is proportional to the processing performance score.
[0102] In some embodiments, the storage resource information of the candidate code compiler, including the cache rate of the code compiler for historical compilation tasks and the effective cache rate divided by dimensions such as module / committer / compilation task, is proportional to the size of the storage performance score.
[0103] As an example, when the memory storage performance of candidate code compiler 1 is 512GB and the memory storage performance of candidate code compiler 1 is 1024GB, the processing performance score of candidate code compiler 1 can be 91 points, and the processing performance score of candidate code compiler 2 can be 98 points.
[0104] As an example, the effective cache ratio of candidate code compiler 1 is 98%, and the effective cache ratio of candidate code compiler 2 is 91%. The storage performance score of candidate code compiler 1 can be 99 points, and the storage performance score of candidate code compiler 2 can be 92 points.
[0105] In some embodiments, the above-mentioned weighted summation of processing performance score and storage performance score to obtain the performance size of candidate code compiler can be achieved as follows: obtain the first weight corresponding to the processing performance score and the second weight of the storage performance score, and sum the product of the processing performance score and the first weight with the product of the storage performance score and the second weight to determine the performance size of candidate code compiler.
[0106] In this way, by determining the processing performance score and storage performance score of the candidate code compiler, and then weighting and summing the processing performance score and storage performance score, the performance size of the candidate code compiler is obtained, thereby accurately determining the performance size of the candidate code compiler. Based on the performance size, the target code compiler is then accurately determined, thus effectively improving the accuracy of the determined target code compiler.
[0107] In some embodiments, the above-mentioned operational information further includes processing resource information and storage resource information, wherein the storage resource information includes the cache rate of the code compiler for historical compilation tasks.
[0108] In some embodiments, step 1032 above can be implemented as follows: based on the processing resource information of each candidate code compiler, determine the processing performance score of the candidate code compiler, and sort each candidate code compiler according to the processing performance score to obtain an initial sorting queue, wherein the candidate code compilers in the initial sorting queue are sorted in descending order of processing performance score; starting from the first candidate code compiler in the initial sorting queue, candidate code compilers are selected sequentially to obtain a target number of candidate code compilers, the target number being less than the total number of candidate code compilers in the first sorting queue; the candidate code compiler with the highest cache rate among the target number of candidate code compilers is determined as the target code compiler.
[0109] Thus, by sorting the candidate code compilers according to their processing performance scores to obtain an initial sorting queue, and then selecting a number of candidate code compilers from the initial sorting queue that is less than the total number of candidate code compilers in the first sorting queue, since the processing performance scores of the candidate code compilers at the beginning of the initial sorting queue are higher than those at the end, it is possible to select candidate code compilers with relatively better processing performance scores from the initial sorting queue. Then, from the selected candidate code compilers, the candidate code compiler with the highest cache rate is selected as the target code compiler. This ensures that the selected target code compiler comprehensively considers both processing performance and cache rate, making the selected target code compiler more accurate and better suited to the compilation task of the corresponding code to be compiled.
[0110] In step 104, a compilation task corresponding to the code to be compiled is generated and sent to the target code compiler so that the target code compiler can compile the code to be compiled.
[0111] In some embodiments, a compilation task corresponding to the code to be compiled can be generated by a server and sent to the target code compiler so that the target code compiler can compile the code to be compiled. Alternatively, a compilation task corresponding to the code to be compiled can be generated by a terminal and sent to the target code compiler so that the target code compiler can compile the code to be compiled.
[0112] In some embodiments, compilation is the process of generating a target program from a source program written in a source language using a compiler. Compilation is the act of generating a target program using a compiler. It transforms a high-level language into a binary language that a computer can understand. Computers only recognize 1s and 0s; the compiler converts a familiar language into binary. The process of translating a source program into a target program by a compiler consists of five stages: lexical analysis; syntax analysis; semantic checking and intermediate code generation; code optimization; and target code generation. The main stages are lexical and syntax analysis, also known as source program analysis. During this analysis, syntax errors are detected and warning messages are provided. A compiled language is a programming language implemented using a compiler. Unlike interpreted languages, which are executed line by line by an interpreter, compiled languages use a compiler to first compile the code into machine code before execution. Theoretically, any programming language can be either compiled or interpreted. The difference lies only in the application of the program.
[0113] In some embodiments, see Figure 4 , Figure 4 This is a flowchart illustrating the code compilation method provided in the embodiments of this application. Figure 4 The compilation task in step 104 shown, which generates the corresponding code to be compiled, can be achieved through the following steps 1041 to 1042.
[0114] In step 1041, the system compilation file corresponding to the compilation type and the target platform identifier associated with the code to be compiled are obtained.
[0115] In some embodiments, the system compilation file is used to record library functions in the development system, and the target platform identifier is used to identify the development platform to which the code to be compiled is applicable.
[0116] In some embodiments, when the compilation type is an instrumentation compilation type, obtaining the system compilation file corresponding to the compilation type in step 1041 above can be achieved as follows: obtaining a system file list, which records system files in the development system; determining a compilation file list corresponding to the code to be compiled, which records code files corresponding to the code to be compiled, including system files corresponding to the code to be compiled; comparing the system files in the system file list with each code file in the compilation file list to obtain overlapping files, wherein the overlapping files are files that exist in both the full system file list and the compilation file list; and determining the overlapping files as the system compilation files corresponding to the instrumentation compilation type.
[0117] In some embodiments, the system files in the development system recorded in the system file list may be all the system files in the development system.
[0118] In this way, by comparing the system files in the system file list with the code files in the compilation file list, overlapping files that exist in both the full system file list and the compilation file list can be identified. These overlapping files are the system files corresponding to the code to be compiled. In other words, the system files corresponding to the code to be compiled can be accurately identified, thereby accurately identifying the system compilation files corresponding to the instrumentation compilation type. As a result, when compiling the compilation task in the future, only the system compilation files associated with the code to be compiled need to be compiled, instead of compiling all the system compilation files. This significantly reduces the amount of data to be compiled, thereby effectively improving compilation efficiency and speed.
[0119] In some embodiments, when the compilation type is a regular compilation type, obtaining the system compilation file corresponding to the compilation type in step 1041 above can be achieved as follows: obtaining a system file list, wherein the system file list is used to record the system files in the development system; and determining each system file in the system file list as the system compilation file corresponding to the regular compilation type.
[0120] Thus, when the compilation type is the regular compilation type, each system file in the system file list is identified as the corresponding system compilation file for the regular compilation type. This facilitates the subsequent generation of compilation tasks based on the system compilation files, which correspond to the regular compilation type, and makes it easier to compile the code of the regular type. By distinguishing the type of the code to be compiled, the efficiency and speed of code compilation are effectively improved.
[0121] In step 1042, a compilation task is generated based on the system compilation file, the target platform identifier, and the code to be compiled.
[0122] In this way, by responding to code compilation requests, the compilation type of the code to be compiled is determined, and the target compilation cluster corresponding to the compilation type is identified. Based on runtime information, a target code compiler is selected from the code compilers within the target compilation cluster, and the compilation task corresponding to the code to be compiled is distributed to the target code compiler so that it can compile the code. By distributing compilation tasks for different compilation types of code to be compiled to target code compilers in different compilation clusters, and by deploying differentiated code compiler clusters for different compilation types of code, and by selecting the target code compiler from the corresponding compilation cluster based on runtime information, the most suitable compilation cluster and target code compiler within that cluster are more accurately determined. This effectively improves the compilation efficiency of the code compiler, thereby significantly increasing the efficiency and speed of code compilation.
[0123] The following will describe an exemplary application of the embodiments of this application in a real-world code compilation scenario.
[0124] Compilation is the process of using a compiler to generate a target program from a source program written in a source language. It's the act of using a compiler to produce the target program. Compilation transforms a high-level language into a binary language that a computer can understand. Computers only recognize 1s and 0s; the compiler converts the familiar human language into binary. The process of translating a source program into a target program by a compiler consists of five stages: lexical analysis; syntax analysis; semantic checking and intermediate code generation; code optimization; and target code generation. The main stages are lexical and syntax analysis, also known as source program analysis. During this analysis, syntax errors are detected and error messages are provided. A compiled language is a programming language implemented using a compiler. Unlike interpreted languages, which are executed line by line by an interpreter, compiled languages use a compiler to first compile the code into machine code before execution. Theoretically, any programming language can be either compiled or interpreted. The difference lies only in the application of the program.
[0125] The code compilation method provided in this application's embodiments establishes a dedicated compilation cluster for the system, providing customized support for coverage instrumentation versions and ensuring compilation in a fixed environment. Simultaneously, incremental analysis is performed before compilation to identify which public libraries are affected by the code changes, ensuring that each compilation is based on actual needs and reducing redundancy. In build machine distribution, random distribution is avoided; a flexible distribution strategy ensures full utilization of caching, thereby improving the system's compilation efficiency.
[0126] In some embodiments, see Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of the code compilation method provided in the embodiments of this application. The code compilation method provided in the embodiments of this application can... Figure 5 The compiler module implementation shown is as follows: Figure 5 The compilation module shown includes a compilation analysis module, a compilation environment optimization module, and a compilation cluster. The compilation cluster includes compiler 1, compiler 2, and compiler n. The internal components of each of these modules will be explained below.
[0127] In some embodiments, see Figure 5The compilation cluster comprises sub-clusters for compiling coverage instrumentation versions and sub-clusters for normal business compilation, each responsible for specific compilation actions. To improve compilation caching efficiency, this compilation cluster employs two main steps: scheduled full compilation. During off-peak hours in the early morning, each machine on both sub-compilation clusters performs a full compilation of each module. This ensures that before developers begin their work each day, the compilation machines have sufficient local caches of coverage instrumentation versions and normal compilation, accelerating the compilation process for daily tasks. The compilation machines report compilation data and local resource information to the environment optimization module in real time. Since the environment optimization module heavily relies on real-time information from the compilation machines for decision-making, each machine in the compilation cluster deploys a data collection module specifically for collecting the compilation machine's resource information (CPU / memory / disk space, etc.), compilation task information (task progress / task time, etc.), and caching information (effective caching rate differentiated by module / submitter, by compilation task, etc.).
[0128] In some embodiments, see Figure 6 , Figure 6 This is a schematic diagram illustrating the principle of the code compilation method provided in this application embodiment. The Git Diff command is used to obtain the list of code files A affected by the current change. The project files of the business module are parsed to obtain the system compilation file B for that business module. For each element in system compilation file B, the command `bazelquery --noimplicit_deps target_i` is used to analyze the list of source files that each element in system compilation list B depends on, thus obtaining a system compilation list map C with key-value pairs `{target_i:source_file_List}`. This system compilation list map C represents the mapping between each element `target_i` in system compilation file B and the list of dependent files. The code file list A and system compilation list map C are parsed iteratively to identify duplicate system compilation files in both lists, thereby analyzing the system compilation files affected by the current code change.
[0129] In some embodiments, see Figure 7 , Figure 7This is a schematic diagram illustrating the principle of the code compilation method provided in this application embodiment. It determines whether the current compilation task is a coverage instrumentation version or a normal business version. If the command-line parameter includes `--cov` or it's in a CI pipeline scenario, it's considered a coverage instrumentation version; otherwise, it's considered a normal business version. Based on the compilation type, the required compilation cluster type `build_set_type` is obtained, where the compilation cluster type includes sub-clusters for instrumentation version compilation and sub-clusters for normal business compilation. The machine list A under the compilation cluster is obtained, and the current load, compilation tasks, cache information, etc., of each machine are queried. Combining the load, compilation tasks, cache percentage, the history of submitted compilation machines, and the compilation history of modules with the same name, the optimal machine identifier for executing the compilation task in the compilation cluster is determined. Based on the machine identifier and the system compilation files affected by this code change, a compilation task is initiated on that machine, and the file is distributed to that machine for parallel compilation: `Bazel build target1 target2`.
[0130] In some embodiments, see Figure 8 , Figure 8 This is a schematic diagram illustrating the principle of the code compilation method provided in this application embodiment. The code compilation method provided in this application embodiment can effectively improve the compilation speed of developers and the execution speed of the pipeline, reduce personnel waiting costs, and achieve cost reduction and efficiency improvement. In terms of practical results, the compilation cache hit rate of the main modules has increased from 8% to 85%, and the compilation time has decreased from an extreme of 30 minutes to an average of about 6 minutes. Developers do not need to maintain a local compilation environment; they can easily initiate compilation tasks using the scaffolding / pipeline tools provided in this solution. It distinguishes between coverage instrumentation and normal compilation traffic, eliminates compilation cache obfuscation, and reduces additional impacts caused by incorrect compiled binaries.
[0131] In this way, by responding to code compilation requests, the compilation type of the code to be compiled is determined, and the target compilation cluster corresponding to the compilation type is identified. Based on runtime information, a target code compiler is selected from the code compilers within the target compilation cluster, and the compilation task corresponding to the code to be compiled is distributed to the target code compiler so that it can compile the code. By distributing compilation tasks for different compilation types of code to be compiled to target code compilers in different compilation clusters, and by deploying differentiated code compiler clusters for different compilation types of code, and by selecting the target code compiler from the corresponding compilation cluster based on runtime information, the most suitable compilation cluster and target code compiler within that cluster are more accurately determined. This effectively improves the compilation efficiency of the code compiler, thereby significantly increasing the efficiency and speed of code compilation.
[0132] It is understood that in the embodiments of this application, data related to the code to be compiled is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0133] The following description continues to illustrate the exemplary structure of the code compilation device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 3 As shown, the software modules stored in the code compilation device 455 of the memory 450 may include: a receiving module 4551, configured to receive a code compilation request instructing the compilation of code to be compiled, wherein the code to be compiled is code written in a compiled language; a response module 4552, configured to, in response to the code compilation request, determine the compilation type of the code to be compiled, and determine a target compilation cluster corresponding to the compilation type, wherein the target compilation cluster includes at least two code compilers, and different compilation types correspond to different target compilation clusters; a determining module 4553, configured to obtain the running information of each code compiler in the target compilation cluster, and determine a target code compiler from the at least two code compilers based on the running information; and a sending module 4554, configured to generate a compilation task corresponding to the code to be compiled, and send the compilation task to the target code compiler so that the target code compiler compiles the code to be compiled.
[0134] In some embodiments, the running information includes compilation task information indicating whether a compilation task exists; the determining module 4553 is further configured to filter each code compiler in the target compilation cluster based on the compilation task information of each code compiler in the target compilation cluster to obtain at least one candidate code compiler, wherein the candidate code compiler is a code compiler without a compilation task; and select a code compiler from the at least one candidate code compiler as the target code compiler.
[0135] In some embodiments, the running information further includes processing resource information and storage resource information; the determining module 4553 is further configured to determine the performance size of each candidate code compiler based on at least one of the processing resource information and storage resource information; and based on the determined performance size, select the candidate code compiler with the highest performance from the at least one candidate code compiler as the target code compiler.
[0136] In some embodiments, the determining module 4553 is further configured to perform the following processes for each of the candidate code compilers: determining the processing performance score of the candidate code compiler based on the processing resource information of the candidate code compiler; determining the storage performance score of the candidate code compiler based on the storage resource information of the candidate code compiler; and performing a weighted sum of the processing performance score and the storage performance score to obtain the performance magnitude of the candidate code compiler.
[0137] In some embodiments, the running information further includes processing resource information and storage resource information, wherein the storage resource information includes the cache rate of the code compiler for historical compilation tasks; the determining module 4553 is further configured to determine the processing performance score of each candidate code compiler based on the processing resource information of each candidate code compiler, and sort each candidate code compiler according to the processing performance score to obtain an initial sorting queue, wherein the candidate code compilers in the initial sorting queue are sorted in descending order of the processing performance score; starting from the first candidate code compiler in the initial sorting queue, candidate code compilers are selected sequentially to obtain a target number of candidate code compilers, wherein the target number is less than the total number of candidate code compilers in the first sorting queue; and the candidate code compiler with the highest cache rate among the target number of candidate code compilers is determined as the target code compiler.
[0138] In some embodiments, the determining module 4553 is used to obtain the running information of the code compiler from the agent corresponding to each code compiler in the target compilation cluster; wherein the agent is an application deployed in the code compiler for collecting the running information.
[0139] In some embodiments, the response module 4552 is configured to respond to a code compilation request, parse the code compilation request, and obtain the task parameters of the code compilation request; when the task parameters include a compilation type identifier, the compilation type of the code to be compiled is determined to be an instrumentation compilation type; when the task parameters do not include a compilation type identifier, the compilation type of the code to be compiled is determined to be a regular compilation type; wherein, the compilation type identifier is used to identify the instrumentation compilation type, and the code to be compiled of the instrumentation compilation type is used to modify the functionality of the code being compiled.
[0140] In some embodiments, the sending module 4554 is further configured to obtain the system compilation file corresponding to the compilation type and the target platform identifier associated with the code to be compiled; wherein, the system compilation file is used to record the library functions in the development system, and the target platform identifier is used to identify the development platform to which the code to be compiled is applicable; and a compilation task is generated based on the system compilation file, the target platform identifier and the code to be compiled.
[0141] In some embodiments, when the compilation type is an instrumentation compilation type, the sending module 4554 is further configured to obtain a system file list, wherein the system file list is used to record system files in the development system; determine a compilation file list corresponding to the code to be compiled, wherein the compilation file list is used to record code files corresponding to the code to be compiled, and the code files include system files corresponding to the code to be compiled; compare the system files in the system file list with each code file in the compilation file list to obtain overlapping files, wherein the overlapping files are files that exist in both the full system file list and the compilation file list; and determine the overlapping files as system compilation files corresponding to the instrumentation compilation type.
[0142] In some embodiments, when the compilation type is a regular compilation type, the sending module 4554 is further configured to obtain a system file list, wherein the system file list is used to record system files in the development system; and to determine each system file in the system file list as a system compilation file corresponding to the regular compilation type.
[0143] In some embodiments, the above-described code compilation apparatus further includes: a clustering module, configured to obtain the number of types of the compilation type and a number of code compilers to be clustered that is greater than or equal to a reference number, wherein the reference number is twice the number of types; and to cluster the code compilers to be clustered based on the compilation type to obtain a compilation cluster corresponding to each compilation type.
[0144] In some embodiments, the response module 4552 is further configured to select a target compilation cluster corresponding to the compilation type of the code to be compiled from the compilation clusters corresponding to each compilation type.
[0145] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the code compilation method described in this application.
[0146] This application provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are executed by a processor, they cause the processor to execute the code compilation method provided in this application. For example, ... Figure 4 The code compilation method is shown.
[0147] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of electronic devices including one or any combination of the above-mentioned memories.
[0148] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.
[0149] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).
[0150] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.
[0151] In summary, the embodiments of this application have the following beneficial effects: (1) By responding to the code compilation request, the compilation type of the code to be compiled is determined, and the target compilation cluster corresponding to the compilation type is determined. Based on the runtime information, a target code compiler is selected from the code compilers in the target compilation cluster, and the compilation task corresponding to the code to be compiled is sent to the target code compiler so that the target code compiler can compile the code to be compiled. In this way, by sending the compilation tasks corresponding to the code to be compiled for different compilation types to the target code compilers in different compilation clusters, by deploying differentiated code compiler clusters for the code to be compiled for different compilation types, and by determining the target code compiler from the corresponding compilation cluster based on the runtime information, the most suitable compilation cluster and the target code compiler in the compilation cluster are more accurately determined, thereby effectively improving the compilation efficiency of the code compiler and thus effectively improving the efficiency and speed of code compilation.
[0152] (2) By determining whether the task parameters include a compilation type identifier, the compilation type of the code to be compiled can be accurately determined. The compilation type identifier can accurately identify the compilation type of the code to be compiled, which facilitates the subsequent determination of the target compilation cluster corresponding to the compilation type, thereby effectively improving the accuracy of the determined target compilation cluster.
[0153] (3) By grouping the compilers of the code to be grouped based on the compilation type, a compilation cluster corresponding to each compilation type is obtained, thereby obtaining a compilation cluster corresponding to each compilation type. This facilitates the subsequent compilation of the compilation code of each compilation type through the compilation cluster, thereby effectively improving the efficiency and speed of code compilation.
[0154] (4) By collecting runtime information through an agent deployed in the code compiler and obtaining runtime information of the code compiler from the agent, it is easier to determine the target code compiler from at least two code compilers based on runtime information, thereby effectively improving the accuracy of the determined target code compiler and thus effectively improving the efficiency and speed of code compilation.
[0155] (5) By determining the processing performance score and storage performance score of the candidate code compiler, and by weighting and summing the processing performance score and storage performance score, the performance size of the candidate code compiler is obtained, thereby accurately determining the performance size of the candidate code compiler. Based on the performance size, the target code compiler is accurately determined, thereby effectively improving the accuracy of the determined target code compiler.
[0156] (6) By sorting each candidate code compiler according to its processing performance score, an initial sorting queue is obtained. Then, a number of candidate code compilers less than the total number of candidate code compilers in the first sorting queue are selected from the initial sorting queue. Since the processing performance scores of the candidate code compilers at the beginning of the initial sorting queue are higher than those at the end, the candidate code compilers with relatively good processing performance scores can be selected from the initial sorting queue. Then, the candidate code compiler with the highest cache rate is selected from the selected candidate code compilers as the target code compiler. This makes the selected target code compiler combine processing performance and cache rate, making the selected target code compiler more accurate and better suited to the compilation task of the corresponding code to be compiled.
[0157] (7) By comparing the system files in the system file list with the code files in the compilation file list, the overlapping files that exist in both the full system file list and the compilation file list are identified. The overlapping files are the system files corresponding to the code to be compiled. In other words, the system files corresponding to the code to be compiled can be accurately identified, thereby accurately identifying the system compilation files corresponding to the instrumentation compilation type. Therefore, when compiling the compilation task in the future, only the system compilation files associated with the code to be compiled need to be compiled, instead of compiling all the system compilation files. This significantly reduces the amount of data to be compiled, thereby effectively improving compilation efficiency and speed.
[0158] (8) When the compilation type is the regular compilation type, each system file in the system file list is identified as the system compilation file corresponding to the regular compilation type. This facilitates the subsequent generation of compilation tasks corresponding to the regular compilation type based on the system compilation files, and makes it easier to compile the code to be compiled of the regular type. By distinguishing the type of the code to be compiled, the efficiency and speed of compiling the code can be effectively improved.
[0159] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.
Claims
1. A code compilation method, characterized in that, The method includes: A code compilation request is received, instructing that code to be compiled be compiled, wherein the code to be compiled is code written in a compiled language; In response to the code compilation request, the compilation type of the code to be compiled is determined, and the target compilation cluster corresponding to the compilation type is determined, wherein the target compilation cluster includes at least two code compilation machines, and different compilation types correspond to different target compilation clusters; Obtain the running information of each code compiler in the target compilation cluster, and determine the target code compiler from the at least two code compilers based on the running information; When the compilation type is an instrumentation compilation type, a system file list is obtained, which records system files in the development system; a compilation file list corresponding to the code to be compiled is determined, which records code files corresponding to the code to be compiled, including system files corresponding to the code to be compiled; the system files in the system file list are compared with each code file in the compilation file list to obtain overlapping files, wherein the overlapping files are files that exist in both the system file list and the compilation file list; the overlapping files are determined as the system compilation files corresponding to the instrumentation compilation type. When the compilation type is a regular compilation type, obtain the system file list; determine the system files in the system file list as the system compilation files corresponding to the regular compilation type; Obtain the target platform identifier associated with the code to be compiled; Based on the system compilation file corresponding to the compilation type, the target platform identifier, and the code to be compiled, a compilation task corresponding to the code to be compiled is generated, and the compilation task is sent to the target code compiler so that the target code compiler can compile the code to be compiled.
2. The method according to claim 1, characterized in that, The runtime information includes compilation task information used to indicate whether a compilation task exists; The step of determining the target code compiler from the at least two code compilers based on the runtime information includes: Based on the compilation task information of each code compiler in the target compilation cluster, each code compiler in the target compilation cluster is screened to obtain at least one candidate code compiler, wherein the candidate code compiler is a code compiler that does not have a compilation task; From the at least one candidate code compiler, select one code compiler as the target code compiler.
3. The method according to claim 2, characterized in that, The operational information also includes processing resource information and storage resource information; Selecting a code compiler from the at least one candidate code compiler as the target code compiler includes: Based on at least one of the processing resource information and storage resource information, determine the performance size of each of the candidate code compilers; Based on a determined performance level, the candidate code compiler with the highest performance is selected from the at least one candidate code compilers as the target code compiler.
4. The method according to claim 3, characterized in that, Determining the performance of each candidate code compiler based on at least one of the processing resource information and storage resource information includes: For each of the candidate code compilers, the following processing is performed: Based on the processing resource information of the candidate code compiler, the processing performance score of the candidate code compiler is determined; Based on the storage resource information of the candidate code compiler, the storage performance score of the candidate code compiler is determined; The performance of the candidate code compiler is obtained by weighted summing of the processing performance score and the storage performance score.
5. The method according to claim 2, characterized in that, The operational information also includes processing resource information and storage resource information, wherein the storage resource information includes the cache rate of the code compiler for historical compilation tasks; Selecting a code compiler from the at least one candidate code compiler as the target code compiler includes: Based on the processing resource information of each candidate code compiler, the processing performance score of each candidate code compiler is determined, and each candidate code compiler is sorted according to the processing performance score to obtain an initial sorting queue, wherein the candidate code compilers in the initial sorting queue are sorted in descending order of the processing performance score; Starting from the first candidate code compiler in the initial sorting queue, candidate code compilers are selected sequentially to obtain a target number of candidate code compilers, where the target number is less than the total number of candidate code compilers in the first sorting queue. The candidate code compiler with the highest cache rate among the target number of candidate code compilers is determined as the target code compiler.
6. The method according to claim 1, characterized in that, The step of obtaining the runtime information of each code compiler in the target compilation cluster includes: Obtain the runtime information of the code compiler from the agent corresponding to each code compiler in the target compilation cluster; The agent is an application deployed in the code compiler to collect the runtime information.
7. The method according to claim 1, characterized in that, In response to the code compilation request, determining the compilation type of the code to be compiled includes: In response to the code compilation request, the code compilation request is parsed to obtain the task parameters of the code compilation request; When the task parameters include a compilation type identifier, the compilation type of the code to be compiled is determined to be an instrumentation compilation type; When the task parameters do not include the compilation type identifier, the compilation type of the code to be compiled is determined to be the regular compilation type; The compilation type identifier is used to identify the instrumentation compilation type, and the code to be compiled for the instrumentation compilation type is used to modify the functionality of the code being compiled.
8. The method according to claim 1, characterized in that, The system compilation file is used to record the library functions in the development system, and the target platform identifier is used to identify the development platform to which the code to be compiled is applicable.
9. The method according to claim 1, characterized in that, Before determining the target compilation cluster corresponding to the compilation type, the method further includes: Obtain the number of types of the compilation type, and the number of code compilers to be partitioned that is greater than or equal to the number of references, wherein the number of references is twice the number of types; Based on the compilation type, the code compilers to be grouped are grouped to obtain compilation clusters corresponding to each compilation type.
10. The method according to claim 9, characterized in that, Determining the target compilation cluster corresponding to the compilation type includes: Select the target compilation cluster corresponding to the compilation type of the code to be compiled from the compilation clusters corresponding to each of the compilation types.
11. A code compilation apparatus, characterized in that, The device includes: A receiving module is configured to receive a code compilation request that instructs the compilation of code to be compiled, wherein the code to be compiled is code written in a compiled language; A response module is used to respond to the code compilation request, determine the compilation type of the code to be compiled, and determine the target compilation cluster corresponding to the compilation type, wherein the target compilation cluster includes at least two code compilation machines, and different compilation types correspond to different target compilation clusters; The determination module is used to obtain the running information of each code compiler in the target compilation cluster, and determine the target code compiler from the at least two code compilers based on the running information; The sending module is configured to: obtain a system file list (which records system files in the development system) when the compilation type is an instrumentation compilation type; determine a compilation file list corresponding to the code to be compiled (which records code files, including system files); compare the system files in the system file list with the code files in the compilation file list to obtain overlapping files (files that exist in both the system file list and the compilation file list); and identify the overlapping files as system compilation files corresponding to the instrumentation compilation type. When the compilation type is a regular compilation type, the module obtains the system file list; identifies the system files in the system file list as system compilation files corresponding to the regular compilation type; obtain a target platform identifier associated with the code to be compiled; and, based on the system compilation files corresponding to the compilation type, the target platform identifier, and the code to be compiled, generate a compilation task corresponding to the code to be compiled and send the compilation task to the target code compiler to compile the code to be compiled.
12. The apparatus according to claim 11, characterized in that, The runtime information includes compilation task information used to indicate whether a compilation task exists; The determining module is further configured to filter each code compiler in the target compilation cluster based on the compilation task information of each code compiler in the target compilation cluster, and obtain at least one candidate code compiler, wherein the candidate code compiler is a code compiler that does not have a compilation task; From the at least one candidate code compiler, select one code compiler as the target code compiler.
13. The apparatus according to claim 12, characterized in that, The operational information also includes processing resource information and storage resource information; The determining module is further configured to determine the performance size of each of the candidate code compilers based on at least one of the processing resource information and storage resource information; Based on a determined performance level, the candidate code compiler with the highest performance is selected from the at least one candidate code compilers as the target code compiler.
14. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the code compilation method according to any one of claims 1 to 10.
15. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a processor, they implement the code compilation method according to any one of claims 1 to 10.
16. A computer program product comprising a computer program or computer-executable instructions, characterized in that, When the computer program or computer-executable instructions are executed by a processor, they implement the code compilation method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Automatic compiling method and device adopting Jupyter
CN112650500A