Compilation task processing method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a compiling task processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: receiving a compiling request initiated by a user, and identifying a compiling resource type indicated by the user and used for executing a compiling task according to the compiling request; in response to the fact that the compiling resource type is identified as a private compiling resource type, calling a private compiling node exclusive to the user from a private compiling node set in a compiling node cluster, and executing the compiling task by utilizing a private cache, exclusive to the user, of the private compiling node; and in response to the fact that the compiling node type is identified as a public compiling resource type, distributing a public compiling node for the user from a public compiling node cluster which is shared by multiple users in the compiling node cluster, calling the public compiling node, and executing the compiling task by utilizing a public cache of the public compiling node. By adopting the method, the problem of resource waste can be avoided.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a compilation task processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] A compilation cluster is a computing cluster that connects multiple compilation nodes through a network and is dedicated to large-scale software compilation tasks. These nodes work together to process multiple compilation jobs at the same time, significantly improving the speed and efficiency of compiling large software projects.

[0003] In the prior art, most compilation clusters are public compilation clusters, that is, the compilation nodes in the compilation cluster are all compilation nodes used by multiple users, and most compilation tasks are also executed using these compilation nodes used by multiple users.

[0004] However, there is a problem of resource waste when using existing compilation clusters to process compilation tasks. Summary of the invention

[0005] Based on this, it is necessary to provide a compilation task processing method, device, computer equipment, computer-readable storage medium and computer program product that can avoid resource waste problems in response to the above technical problems.

[0006] In a first aspect, the present application provides a compilation task processing method, comprising:

[0007] receiving a compilation request initiated by a user, and identifying, according to the compilation request, a compilation resource type indicated by the user for executing a compilation task;

[0008] In response to identifying that the compilation resource type is a private compilation resource type, calling a private compilation node dedicated to the user from a set of private compilation nodes in the compilation node cluster, and executing the compilation task using a private cache of the private compilation node that is dedicated to the user;

[0009] In response to identifying that the compiling node type is a public compiling resource type, a public compiling node is allocated to the user from a public compiling node cluster shared by multiple users in the compiling node cluster, the public compiling node is called and a public cache of the public compiling node is used to execute the compiling task.

[0010] In a second aspect, the present application further provides a compilation task processing device, comprising:

[0011] A receiving module, configured to receive a compilation request initiated by a user, and identify, according to the compilation request, a type of compilation resource indicated by the user for executing a compilation task;

[0012] a first execution module, configured to, in response to identifying that the compilation resource type is a private compilation resource type, call a private compilation node dedicated to the user from a set of private compilation nodes in a compilation node cluster, and execute the compilation task by using a private cache of the private compilation node that is dedicated to the user;

[0013] The second execution module is used for allocating a public compiling node to the user from a public compiling node cluster shared by multiple users in the compiling node cluster in response to identifying that the compiling node type is a public compiling resource type, calling the public compiling node and utilizing a public cache of the public compiling node to execute the compiling task.

[0014] In one embodiment, the receiving module is specifically configured to determine whether the compilation request carries a private compilation resource parameter value; in response to the compilation request carrying the private compilation resource parameter value, determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type.

[0015] In one embodiment, the receiving module is specifically used to identify the user type of the user in response to the compilation request not carrying the private compilation resource parameter value; in response to the user type being the first user type, determine that the compilation resource type indicated by the user for executing the compilation task is the private compilation resource type; in response to the user type being the second user type, determine that the compilation resource type indicated by the user for executing the compilation task is the public compilation resource type.

[0016] In one embodiment, the first execution module is specifically configured to, in response to identifying that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, call, from a set of private compilation nodes in the compilation node cluster, a private compilation node that is pre-connected by the user to the compilation node cluster and is exclusive to the user; in response to identifying that the compilation resource type is a private compilation resource type and the compilation request does not carry the private compilation resource parameter value, call, from the set of private compilation nodes in the compilation node cluster, a private compilation node that is allocated to the user by the compilation node cluster and is exclusive to the user.

[0017] In one embodiment, the first execution module is further used to receive a private compiling node creation request initiated by the user; determine the compiling machine indicated by the private compiling node creation request according to the private compiling node creation request; and add the compiling machine as a private compiling node exclusively for the user to the set of private compiling nodes in the compiling node cluster.

[0018] In one embodiment, the second execution module is specifically configured to, in response to identifying that the compiling node type is a public compiling resource type, determine a service level for the user according to service configuration information for the user; in response to the service level being a first service level, determine a first public compiling node set for use by users of the first service level from a public compiling node cluster for use by multiple users in the compiling node cluster, and allocate a public compiling node to the user from the first public compiling node set; in response to the service level being a second service level, determine a second public compiling node set for use by users of the second service level from a public compiling node cluster for use by multiple users in the compiling node cluster, and allocate a public compiling node to the user from the second public compiling node set; wherein, the efficiency of responding to the compiling task using the first public compiling node set is greater than the efficiency of responding to the compiling task using the second public compiling node set.

[0019] In one embodiment, the second execution module is further configured to receive a private compiling node creation request initiated by the user; determine, according to the private compiling node creation request, a compiling node in an idle state from a public compiling node cluster in the compiling node cluster; and add the compiling node in an idle state to a set of private compiling nodes in the compiling node cluster as a private compiling node exclusively for the user.

[0020] In one embodiment, the second execution module is specifically configured to, in response to identifying that the compilation node type is a public compilation resource type, determine a predicted compilation time consumption of the compilation task; in response to the predicted compilation time consumption satisfying a preset high time consumption judgment condition, allocate a public compilation node to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and configure the public compilation node as a private compilation node exclusively for the user during a period in which the public compilation node executes the compilation task.

[0021] In one embodiment, the second execution module is specifically used to preprocess the compilation task through the master node in the compilation node cluster to generate subtasks; determine multiple slave nodes for executing the subtasks on the public compilation node allocated to the user; schedule the multiple slave nodes, and use the public cache of the public compilation node to concurrently execute the subtasks preprocessed by the master node.

[0022] In one embodiment, the second execution module is specifically used to determine, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing a plurality of first-category subtasks among the subtasks preprocessed by the master node; and to determine, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing a plurality of second-category subtasks among the subtasks preprocessed by the master node; wherein the resource requirement of the first-category subtasks is greater than the resource requirement of the second-category subtasks, and a preset number of the plurality of first-category subtasks concurrently executed on a single slave node is less than a preset number of the plurality of second-category subtasks concurrently executed on a single slave node.

[0023] In one embodiment, the second execution module is specifically used to determine, on a public compilation node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of second-category subtasks, based on the constraint condition, the number of subtasks of the concurrent plurality of second-category subtasks and a preset number of second-category subtasks concurrently executed on a single slave node, with the concurrency number of subtasks limited by the preprocessing speed of the master node as a constraint condition.

[0024] In one embodiment, the second execution module is specifically used to split each subtask of the multiple first-category subtasks into multiple concurrent second-category subtasks; and determine multiple slave nodes for concurrently executing the second-category subtasks split from the multiple first-category subtasks according to the total number of second-category subtasks split from the multiple first-category subtasks and the preset number of first-category subtasks concurrently executed on a single slave node.

[0025] In one embodiment, the second execution module is specifically used to obtain a permission management account for accessing the data disk of the public compilation node; synchronize the permission management account to the multiple slave nodes used to execute the subtask; and use the permissions of the permission management account to perform read and write operations on the data disk during the process of calling the multiple slave nodes and concurrently executing the subtasks preprocessed by the master node using the public cache of the public compilation node.

[0026] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in any embodiment of the first aspect when executing the computer program.

[0027] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any embodiment of the first aspect above.

[0028] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the method described in any embodiment of the first aspect above.

[0029] The above-mentioned compilation task processing method, device, computer equipment, computer-readable storage medium and computer program product can, after receiving a compilation request initiated by a user, identify the compilation resource type indicated by the user for executing the compilation task according to the compilation request. When the compilation resource type is a private compilation resource type, the user's exclusive private compilation node can be called from the private compilation node set in the compilation node cluster, and the compilation task can be executed by using the private cache of the private compilation node that is exclusive to the user; when the compilation resource type is a public compilation resource type, a public compilation node can be allocated to the user from the public compilation node cluster that is shared by multiple users in the compilation node cluster, and the public compilation node can be called and the public cache of the public compilation node can be used to execute the compilation task. When the user selects the private compilation resource type, since the cache used is a private cache that is exclusive to the user, the problem of cache invalidation caused by the update of the cache by other users can be effectively avoided. When the user selects the public compilation resource type, since the compilation node used is a compilation node shared by multiple users, the compilation task can be executed efficiently and concurrently, and since the cache used is a public cache, the cache hit rate can also be effectively improved. The present application provides different types of compilation resources to users for them to choose according to their actual needs, so as to avoid the problem of only providing one compilation resource to users in the prior art, resulting in resource waste when the compilation resource is not compatible with the compilation task. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 An application environment diagram of a compiling task processing method in one embodiment;

[0032] Figure 2 A schematic diagram of a flow chart of a compiling task processing method in one embodiment;

[0033] Figure 3 A schematic diagram of a compilation node cluster in one embodiment;

[0034] Figure 4 A schematic diagram of a private compilation node set in one embodiment;

[0035] Figure 5 is a schematic diagram of a common compilation node cluster in one embodiment;

[0036] Figure 6 A flowchart of a compiling task processing method in another embodiment;

[0037] Figure 7 is a schematic diagram of a common compilation node in one embodiment;

[0038] Figure 8 is a schematic diagram of a common compilation node in another embodiment;

[0039] Fig. 9 A schematic diagram of mounting a data disk to a slave node in one embodiment;

[0040] Fig.10 is a schematic diagram of running a container in a container in one embodiment;

[0041] Fig.11 is a schematic diagram of a public cache in one embodiment;

[0042] Fig.12 A schematic diagram of a flow chart of a compiling task processing method in another embodiment;

[0043] Fig.13 is a schematic diagram of the number of concurrent subtasks in one embodiment;

[0044] Fig.14 A schematic diagram of version update in one embodiment;

[0045] Fig.15 is a structural block diagram of a compiling task processing device in one embodiment;

[0046] Fig.16 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] In order to clearly describe the technical solution of the present application and facilitate understanding of the technical solution of the present application, the key concepts involved in the present application are explained below.

[0049] Compilation refers to the process of converting source code written in a high-level programming language into machine language that can be directly executed by a computer device. A compile request refers to a request for instructing the execution of a compile task, and a compile task refers to a task of compiling a specific source code.

[0050] A compilation resource refers to a resource used by a compilation task when the compilation task is executed. The compilation resource type may be information used to characterize the usage rights of the compilation resource.

[0051] A compilation node cluster refers to a collection of multiple compilation nodes. A compilation node is a compilation resource provided by a compilation node cluster for users to perform compilation tasks. A private compilation node is a private compilation resource provided by a compilation node cluster for a specific user and is only used by that specific user. A public compilation node is a public compilation resource provided by a compilation node cluster for multiple users and is used by multiple users.

[0052] A private cache is a cache that can be used to store data generated by a specific private compilation node when executing a compilation task instructed by a specific user. A public cache is a cache that can be used to store data generated by any public compilation node when executing a compilation task instructed by any user.

[0053] The compilation task processing method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the user can communicate with the server through the terminal to initiate a compilation request. The server can be a server cluster composed of multiple servers, that is, a compilation node cluster in this application.

[0054] Exemplarily, when a user has a compilation requirement for a compilation task, a compilation request can be sent to a server through a terminal. The server can determine the type of compilation resource indicated by the user for executing the compilation task based on the compilation requirement sent by the user. If the server determines that the compilation resource type is a private compilation resource type, the server calls a private compilation node dedicated to the user from a set of private compilation nodes included therein, and uses a private cache of the private compilation node that is dedicated to the user to execute the compilation task. If the server determines that the compilation resource type is a public compilation resource type, the server allocates a public compilation node to the user from a cluster of public compilation nodes shared by multiple users, calls the public compilation node, and uses a public cache of the public compilation node to execute the compilation task.

[0055] In an exemplary embodiment, Figure 2 As shown, a compilation task processing method is provided, the method comprising the following steps:

[0056] Step 201: Receive a compilation request initiated by a user, and identify the type of compilation resources indicated by the user for executing a compilation task according to the compilation request.

[0057] Compilation refers to the process of converting source code written in a high-level programming language into machine language that can be directly executed by computer devices.

[0058] The high-level programming language may be a process-oriented programming language, such as C, or an object-oriented programming language, such as Java and C++, or a scripting language, such as Python and JavaScript.

[0059] Machine language refers to binary code composed of 0s and 1s that can be directly recognized and executed by computer devices.

[0060] A compile request refers to a request for instructing to execute a compile task, and a compile task refers to a task for compiling specific source code.

[0061] Compilation resources may be resources used by a compilation task when the compilation task is executed. Compilation resources may include hardware resources and software resources.

[0062] The compilation resource type may be information used to characterize the usage rights of the compilation resource, and may include private compilation resources and public compilation resources.

[0063] A private compilation resource type refers to a compilation resource type that is only used by a specific user. A public compilation resource refers to a compilation resource type that is used by multiple users.

[0064] Exemplarily, the compilation request may be manually initiated by a user through a terminal. After the user writes or modifies the source code, the user may manually initiate a compilation request to the server through the terminal to compile the written or modified source code. In this process, compiling the source code written or modified by the user is a compilation task.

[0065] In some exemplary embodiments, when there is source code that needs to be compiled, the user can determine the type of compilation resources used to perform the compilation task according to the compilation requirements, that is, the type of compilation resources used to compile the source code, and initiate a compilation request to the server through the terminal to instruct the execution of the compilation task.

[0066] Furthermore, after receiving the compilation request initiated by the user, the server may identify the type of compilation resource for the compilation task to be performed by the user indicated by the user according to the compilation request.

[0067] Specifically, since the compilation request is initiated by the user, the compilation resource type can be determined according to the user's user ID. For example, the server can obtain a list of compilation resource types for each user from a database, and the list can be shown in Table 1. The compilation resource type can be determined using the user ID and the list.

[0068] Table 1

[0069]

[0070] The compilation resource type may also be determined based on the compilation resource type information carried in the compilation request. For example, the terminal may display an operation control for the user to select the compilation resource type, and the user may determine the compilation resource type used to perform the compilation task based on the compilation requirements and operate the operation control. The terminal responds to the user operation completion, determines the compilation resource type information based on the user operation, generates a compilation request based on the compilation resource information, and initiates the compilation request to the server. After receiving the compilation request, the server may obtain the compilation resource type information carried in the compilation request, and determine the compilation resource type indicated by the compilation resource type information as the compilation resource type indicated by the user for performing the compilation task.

[0071] Step 202: In response to identifying that the compilation resource type is a private compilation resource type, calling a private compilation node dedicated to the user from a set of private compilation nodes in a compilation node cluster, and using a private cache of the private compilation node dedicated to the user to execute the compilation task.

[0072] A compilation node cluster refers to a collection of multiple compilation nodes. The compilation node cluster can use compilation nodes to execute compilation tasks, that is, the compilation node is a compilation resource provided by the compilation node cluster for users to execute compilation tasks.

[0073] A compile node refers to a device that can provide compile functions. A compile node can be a physical device, such as a server or terminal, or a virtual device running on a physical device, such as a virtual machine or container.

[0074] Furthermore, according to the usage rights of the compilation nodes, the compilation nodes in the compilation node cluster can be divided into private compilation nodes and public compilation nodes. The private compilation node corresponds to the private compilation resource type in the compilation resource type, that is, the private compilation node is a private compilation resource provided by the compilation node cluster for a specific user and is only used by the specific user. A set consisting of multiple private compilation nodes is a private compilation node set.

[0075] The cache can be used to store data generated when the compile node executes the compile task. For example, data of compiler-related resources, such as the compiler's executable file and the compiler-related tool chain. Data of header files and library files, such as the standard header files and standard library files required for the compilation process. Intermediate files, precompiled files, and compilation results of the compilation task. Precompiled files refer to files obtained after preprocessing the source files.

[0076] The private cache is used to store data generated by the private compiling node when executing a compiling task. Exemplarily, each private compiling node corresponds to a private cache, and the private cache is only used to store data generated by the private compiling node when executing a compiling task.

[0077] Furthermore, since the private cache and the private compilation node are one-to-one, and the private compilation node and the user are one-to-one, it can be understood that the private cache and the user are also one-to-one, that is, the private cache can be used to store data generated by a specific private compilation node when executing a compilation task instructed by a specific user.

[0078] In some exemplary embodiments, Figure 3 As shown, after receiving the compilation request initiated by the user, the server can identify the type of compilation resource indicated by the user for executing the compilation task according to the compilation request.

[0079] Furthermore, when the server recognizes that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type, it may first determine a private compilation node set from the compilation node cluster, and then determine a private compilation node exclusive to the user from the private compilation node set.

[0080] Specifically, the private compiling node exclusive to the user may be determined from the private compiling node set according to the identification information of the user. For example, the private compiling node set includes three private compiling nodes, namely, private compiling node A exclusive to user A, private compiling node B exclusive to user B, and private compiling node C exclusive to user C. Among them, the name of private compiling node A may include the identification information of user A, the name of private compiling node B may include the identification information of user B, and the name of private compiling node C may include the identification information of user C. In this way, after determining the identification information of the user, the private compiling node exclusive to the user and including the identification information of the user may be determined from the private compiling node set according to the identification information of the user.

[0081] Furthermore, after the private compiling node dedicated to the user is determined, the private compiling node dedicated to the user and the private cache of the private compiling node that is dedicated to the user can be used to execute the compiling task.

[0082] Step 203: In response to identifying that the compiling node type is a public compiling resource type, a public compiling node is allocated to the user from a public compiling node cluster shared by multiple users in the compiling node cluster, the public compiling node is called, and a public cache of the public compiling node is used to execute the compiling task.

[0083] The public compiling node corresponds to the public compiling resource type in the compiling resource type, that is, the public compiling node is a public compiling resource provided by the compiling node cluster for multiple users and used by multiple users. A collection of multiple public compiling nodes is a private compiling node cluster.

[0084] The public cache is used to store data generated by the public compiling node when executing a compiling task. Exemplarily, all public compiling nodes in the public compiling node cluster can share a cache, which can be used to store data generated by each public compiling node in the public compiling node cluster when executing a compiling task.

[0085] Furthermore, since the public cache and public compiling nodes are one-to-many, and the public compiling nodes and users are many-to-many, it can be understood that the public cache and users are also one-to-many. That is, the public cache can be used to store data generated by any public compiling node when executing a compiling task instructed by any user.

[0086] In an optional embodiment, the public compilation node cluster may also include multiple groups, each group corresponds to a cache, each group includes multiple public compilation nodes, and the public compilation nodes belonging to the same group share the public cache corresponding to their group, and the public cache is used to store data generated by this group of public compilation nodes when executing compilation tasks.

[0087] In some exemplary embodiments, Figure 3 As shown, after receiving the compilation request initiated by the user, the server can identify the type of compilation resource indicated by the user for executing the compilation task according to the compilation request.

[0088] Furthermore, when the server recognizes that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type, it may first determine a public compilation node set from the compilation node cluster, and then allocate a public compilation node to the user from the public compilation node set.

[0089] Specifically, a public compiling node may be allocated to a user from a public compiling node set according to the idle information of each public compiling node. For example, the idle information of each public compiling node may be obtained first, and a public compiling node may be allocated to a user from public compiling nodes whose idle information indicates an idle state.

[0090] Furthermore, after the public compiling node is determined, the public compiling node and the public cache of the public compiling node may be used to execute the compiling task.

[0091] The above-mentioned compilation task processing method, after receiving the compilation request initiated by the user, can identify the compilation resource type indicated by the user for executing the compilation task according to the compilation request. When the compilation resource type is a private compilation resource type, the private compilation node exclusive to the user can be called from the private compilation node set in the compilation node cluster, and the private cache of the private compilation node exclusive to the user can be used to execute the compilation task; when the compilation resource type is a public compilation resource type, a public compilation node can be allocated to the user from the public compilation node cluster shared by multiple users in the compilation node cluster, and the public compilation node can be called and the public cache of the public compilation node can be used to execute the compilation task. When the user selects the private compilation resource type, since the cache used is a private cache exclusive to the user, the problem of cache invalidation caused by the update of the cache by other users can be effectively avoided. When the user selects the public compilation resource type, since the compilation node used is a compilation node shared by multiple users, the compilation task can be executed efficiently and concurrently, and since the cache used is a public cache, the cache hit rate can also be effectively improved. The present application provides different types of compilation resources to users for them to choose according to their actual needs, so as to avoid the problem of only providing one compilation resource to users in the prior art, resulting in resource waste when the compilation resource is not compatible with the compilation task.

[0092] In an exemplary embodiment, the identifying, based on the compilation request, the compilation resource type indicated by the user for executing the compilation task includes: determining whether the compilation request carries a private compilation resource parameter value; and determining, in response to the compilation request carrying the private compilation resource parameter value, that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type.

[0093] The private compiling resource parameter value refers to a parameter value used to indicate that the compiling resource type indicated by the user for executing the compiling task is a private compiling resource type.

[0094] Exemplarily, if the user indicates that the compilation resource type used to execute the compilation task is a private compilation resource type, when the terminal generates a compilation request, it can set a specific field in the compilation request to the private compilation resource parameter value.

[0095] Furthermore, after receiving the compilation request initiated by the user, the server can directly identify whether the specific field of the compilation request is the private compilation resource parameter value. If the server determines that the specific field of the compilation request is the private compilation resource parameter value, it can determine that the compilation resource type indicated by the user for executing the compilation task is the private compilation resource type.

[0096] If the server determines that the specific field of the compilation request is not the private compilation resource parameter value, it can determine that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0097] Exemplarily, if the user indicates that the compilation resource type used to execute the compilation task is a private compilation resource type, then after generating a compilation request, the terminal may send the compilation request together with the private compilation resource parameter value to the server.

[0098] Furthermore, if the server receives the compilation request and the private compilation resource parameter value, it can determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type. If the server only receives the compilation request, it can determine that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0099] The above-mentioned method for judging whether the compilation request carries a private compilation resource parameter value; in response to the compilation request carrying the private compilation resource parameter value, determining that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type, and judging the compilation resource type indicated by the user for executing the compilation task by judging whether the compilation request carries a private compilation resource parameter value, can accurately judge the compilation resource type indicated by the user for executing the compilation task, and avoid using a compilation resource of a compilation resource type that is not suitable for the compilation task to execute the compilation task, resulting in a waste of resources.

[0100] In an exemplary embodiment, the method of identifying, based on the compilation request, the type of compilation resource indicated by the user for performing the compilation task, further includes: in response to the compilation request not carrying the private compilation resource parameter value, identifying the user type of the user; in response to the user type being a first user type, determining that the type of compilation resource indicated by the user for performing the compilation task is a private compilation resource type; in response to the user type being a second user type, determining that the type of compilation resource indicated by the user for performing the compilation task is a public compilation resource type.

[0101] The user type refers to an identifier used to distinguish the type of compilation resource indicated by the user for executing the compilation task. The user type may include a first user type and a second user type, wherein the first user type is used to indicate that the type of compilation resource indicated by the user for executing the compilation task is a private compilation resource type, and the second user type is used to indicate that the type of compilation resource indicated by the user for executing the compilation task is a public compilation resource type.

[0102] In some exemplary embodiments, after receiving a compilation request initiated by a user, the server may first determine whether the compilation request carries a private compilation resource parameter value. If the server determines that the compilation request does not carry a private compilation resource parameter value, the server may identify the user type of the user.

[0103] Specifically, since the compilation request is initiated by the user through the terminal, the server can determine the user who initiated the compilation request through the compilation request, then obtain the user information of the user from the database, and determine the user type of the user according to the user information of the user. For example, the service level of the user can be determined according to the user information, and then the user type of the user can be determined according to the user service level. It can also be determined according to the user information whether the user has activated the private compilation resource function, and then the user type of the user can be determined according to the determination result.

[0104] Further, when the server determines that the user type of the user is the first user type, it can determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type; when the server determines that the user type of the user is the second user type, it can determine that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0105] The method of identifying the user type of the user in response to identifying that the compilation resource type is a private compilation resource type; determining that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type in response to the user type being a first user type; and determining that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type in response to the user type being a second user type. When the compilation request does not carry a private compilation resource parameter value, the compilation resource type indicated by the user for executing the compilation task can also be determined according to the user type of the user, thereby effectively improving the accuracy of determining the compilation resource type.

[0106] In an exemplary embodiment, in response to identifying that the compilation resource type is a private compilation resource type, calling a private compilation node exclusive to the user from a set of private compilation nodes in a compilation node cluster includes: in response to identifying that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, calling, from the set of private compilation nodes in the compilation node cluster, a private compilation node exclusive to the user that is pre-connected to the compilation node cluster by the user; in response to identifying that the compilation resource type is a private compilation resource type and the compilation request does not carry the private compilation resource parameter value, calling, from the set of private compilation nodes in the compilation node cluster, a private compilation node exclusive to the user that is allocated to the user by the compilation node cluster.

[0107] For example, Figure 4As shown, the private compilation node set may include two types of private compilation nodes, one is a private compilation node that the user pre-connects to the compilation node cluster and is only used by the user, and the other is a private compilation node that the compilation node cluster selects from the public compilation nodes included therein and is allocated to the user and is only used by the user.

[0108] In some exemplary embodiments, after receiving a resource request initiated by a user, the server may identify the type of compilation resource indicated by the user for executing the compilation task according to the compilation request.

[0109] Furthermore, if the server recognizes that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, the server calls a private compilation node that is pre-connected to the compilation node cluster and is exclusive to the user from the private compilation node set in the compilation node cluster to execute the compilation task. If the server recognizes that the compilation resource type is a private compilation resource type and the compilation request does not carry a private compilation resource parameter value, the server calls a private compilation node that is allocated to the user and is exclusive to the user by the compilation node cluster from the private compilation node set in the compilation node cluster to execute the compilation task.

[0110] That is to say, the same user can have two types of private compilation nodes in the compilation node cluster at the same time, one is the one that the user pre-connects to the compilation node cluster, and the other is the one that the compilation node cluster allocates to the user. After selecting the type of private compilation resource according to actual needs, the user can also select the type of private compilation node according to actual needs.

[0111] In the above method, in response to identifying that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, calling a private compilation node that is pre-connected to the compilation node cluster by the user and is exclusive to the user from the private compilation node set in the compilation node cluster; in response to identifying that the compilation resource type is a private compilation resource type and the compilation request does not carry the private compilation resource parameter value, calling a private compilation node that is allocated to the user by the compilation node cluster and is exclusive to the user from the private compilation node set in the compilation node cluster, after determining that the compilation resource type is a private compilation resource type, the type of the private compilation node can also be determined according to whether the compilation request carries the private compilation resource parameter value, thereby improving the flexibility of private compilation node selection.

[0112] In an exemplary embodiment, the method further includes: receiving a private compiling node creation request initiated by the user; determining, according to the private compiling node creation request, a compiling machine indicated by the private compiling node creation request; and adding the compiling machine as a private compiling node exclusively for the user to a set of private compiling nodes in the compiling node cluster.

[0113] A private compile node creation request refers to a request for instructing to create a user-exclusive private compile node in a compile node cluster.

[0114] A compiler machine refers to a device with compiling functions. A compiler machine can be a physical compiler machine or a cloud compiler machine. A physical compiler machine can be a compiler machine that exists in the form of physical hardware, such as a computer device with a compiling application installed. A cloud compiler machine can be a compiler machine built based on cloud computing technology.

[0115] In some exemplary embodiments, when a user has a need to create a private compiling node dedicated to him / herself in a compiling node cluster, a private compiling node creation request may be initiated. After receiving the private compiling node creation request initiated by the user, the server may determine, according to the private compiling node creation request, a compiling machine to be connected to the compiling node cluster indicated by the private compiling node creation request.

[0116] Specifically, the private compiling node creation request may carry the identification information of the compiling machine, and the server may determine the compiling machine according to the identification information of the compiling machine. The server may also determine the compiling machine that has a binding relationship with the user after receiving the private compiling node creation request initiated by the user, and determine the compiling machine that has a binding relationship with the user as the compiling machine.

[0117] Furthermore, after determining the compiler, the server may use the compiler as a private compilation node dedicated to the user and add the compiler to the set of private compilation nodes in the compilation node cluster.

[0118] In an optional embodiment of the present application, the method further includes: receiving a private compilation node set creation request initiated by a user; determining a compilation machine set indicated by the private compilation set creation request according to the private compilation set creation request; and adding the compilation machine set to the compilation node cluster as a private compilation set exclusive to the user.

[0119] The private compiling node set creation request refers to a request for instructing to create a user-exclusive private compiling node set in a compiling node cluster.

[0120] In some exemplary embodiments, when a team composed of multiple users has a need to create a private compilation node set dedicated to the team in a compilation node cluster, a private compilation node set creation request may be initiated. After receiving the private compilation node set creation request initiated by the user, the server response may determine, according to the private compilation node set creation request, a compilation machine set to be connected to the compilation node cluster indicated by the private compilation node set creation request.

[0121] Furthermore, after determining the compilation machine set, the server may add the compilation machine set as a private compilation node set exclusively for the team to the compilation node cluster.

[0122] The method of receiving a private compiling node creation request initiated by the user; determining, according to the private compiling node creation request, a compiling machine indicated by the private compiling node creation request; and adding the compiling machine as a private compiling node exclusive to the user to a set of private compiling nodes in the compiling node cluster can connect the compiling machine indicated by the user to the compiling node cluster as a private compiling node exclusive to the user for use by the user, thereby improving the flexibility of creating private compiling nodes.

[0123] In an exemplary embodiment, in response to identifying that the type of the compiling node is a public compiling resource type, allocating a public compiling node to the user from a public compiling node cluster for common use by multiple users in the compiling node cluster includes: in response to identifying that the type of the compiling node is a public compiling resource type, determining a service level for the user according to service configuration information for the user; in response to the service level being a first service level, determining a first set of public compiling nodes for common use by users of the first service level from a public compiling node cluster for common use by multiple users in the compiling node cluster, and allocating a public compiling node to the user from the first set of public compiling nodes; in response to the service level being a second service level, determining a second set of public compiling nodes for common use by users of the second service level from a public compiling node cluster for common use by multiple users in the compiling node cluster, and allocating a public compiling node to the user from the second set of public compiling nodes.

[0124] The efficiency of using the first public compiling node set to respond to the compiling task is greater than the efficiency of using the second public compiling node set to respond to the compiling task.

[0125] The service configuration information can be used to characterize the relevant information of the service provided by the compilation node cluster to the user. The service configuration information includes the service level of the user. The service enjoyed by the user in the compilation node cluster is related to the service level of the user. The service level can include a first service level and a second service level, and the first service level is higher than the second service level.

[0126] Exemplarily, the compilation node cluster is used to provide compilation resources to execute the compilation tasks indicated by the user, and the compilation resources enjoyed by users with different service levels may be different. Specifically, the higher the service level of the user, the higher the priority of its compilation task. For example, when the compilation node cluster receives compilation requests initiated by multiple users, it can preferentially allocate compilation nodes to users with higher service levels to preferentially execute the compilation tasks of users with higher service levels. The higher the service level of the user, the higher the performance of the compilation node allocated to execute its compilation task. For example, when the compilation node cluster receives compilation requests initiated by multiple users, it can allocate compilation nodes with higher compilation performance to users with higher service levels to efficiently execute the compilation tasks of users with higher service levels.

[0127] For example, Figure 5 As shown, in the public compiling node cluster, two types of public compiling nodes may also be included, one is a public compiling node for public use by users of the first service level, and the other is a public compiling node for public use by users of the second service level. The first public compiling node set is a set consisting of a plurality of public compiling nodes for public use by users of the first service level, and the second public compiling node set is a set consisting of a plurality of public compiling nodes for public use by users of the second service level.

[0128] In some exemplary embodiments, after receiving a resource request initiated by a user, the server may identify the type of compilation resource indicated by the user for executing the compilation task according to the compilation request.

[0129] Furthermore, if the server identifies that the compiled resource type is a public compiled resource type, it determines the service level of the user according to the service configuration information for the user. Specifically, the server can obtain the user's service configuration information from the database according to the user's user ID, and then determine the user's service level according to the user's service configuration information.

[0130] If the server determines that the service level of the user is the first service level, the server determines a first public compiling node set for common use by users of the first service level from a public compiling node cluster for common use by multiple users in the compiling node cluster, and allocates a public compiling node to the user from the first public compiling node set.

[0131] If the server determines that the service level of the user is the second service level, the server determines a second public compiling node set for common use by users of the second service level from a public compiling node cluster for common use by multiple users in the compiling node cluster, and allocates a public compiling node to the user from the second public compiling node set.

[0132] It should be noted that the first public compiling node set and the second public compiling node set may share a common cache, or may respectively share a common cache, that is, the public compiling nodes in the first public compiling node set share a common cache, and the public compiling nodes in the second public compiling node set share a common cache.

[0133] In the above method, in response to identifying that the type of the compiling node is a public compiling resource type, the service level of the user is determined according to the service configuration information for the user; in response to the service level being a first service level, a first public compiling node set for use by users of the first service level is determined from a public compiling node cluster for use by multiple users in the compiling node cluster, and a public compiling node is allocated to the user from the first public compiling node set; in response to the service level being a second service level, a second public compiling node set for use by users of the second service level is determined from a public compiling node cluster for use by multiple users in the compiling node cluster, and a public compiling node is allocated to the user from the second public compiling node set. After determining that the type of the compiling resource is a public compiling resource type, a public compiling node can also be allocated to the user from the public compiling node set corresponding to the service level according to the service level of the user, thereby improving the flexibility of selecting the public compiling node.

[0134] In an exemplary embodiment, the method further includes: receiving a private compiling node creation request initiated by the user; determining, according to the private compiling node creation request, a compiling node in an idle state from a public compiling node cluster in the compiling node cluster; and adding the compiling node in an idle state as a private compiling node exclusively for the user to a set of private compiling nodes in the compiling node cluster.

[0135] In some exemplary embodiments, when a user needs to create a private compiling node dedicated to him / herself in a compiling node cluster, a private compiling node creation request may be initiated. After receiving the private compiling node creation request initiated by the user, the server may determine an idle compiling node from the public compiling node cluster in the compiling node cluster.

[0136] Furthermore, after determining the compilation node in the idle state, the server may add the compilation node in the idle state as a private compilation node dedicated to the user to the set of private compilation nodes in the compilation node cluster.

[0137] The method of receiving a private compiling node creation request initiated by the user; determining a compiling node in an idle state from a public compiling node cluster in the compiling node cluster according to the private compiling node creation request; and adding the compiling node in an idle state to a set of private compiling nodes in the compiling node cluster as a private compiling node exclusively for the user can determine a compiling node in an idle state from the public compiling node cluster as a private compiling node exclusively for the user for use by the user, thereby improving the flexibility of creating private compiling nodes.

[0138] In an optional embodiment of the present application, Figure 6 As shown, the method further includes: receiving a compilation request initiated by a user, determining whether the compilation request carries a private compilation resource parameter value; in response to the compilation request carrying the private compilation resource parameter value, determining that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type; in response to identifying that the compilation resource type is a private compilation resource type, calling a private compilation node dedicated to the user from a set of private compilation nodes in a compilation node cluster, and using a private cache (that is, a private cache) of the private compilation node that is dedicated to the user Figure 5 the compilation task is executed by using a private compilation resource in the compilation request; in response to the compilation request not carrying the private compilation resource parameter value, determining that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type, and determining the service level of the user according to the service configuration information for the user; in response to the service level being a first service level, determining a first public compilation node set for use by users of the first service level from a public compilation node cluster for use by multiple users in the compilation node cluster, and allocating a public compilation node to the user from the first public compilation node set; in response to the service level being a second service level, determining a second public compilation node set for use by users of the second service level from a public compilation node cluster for use by multiple users in the compilation node cluster, and allocating a public compilation node to the user from the second public compilation node set; wherein the efficiency of responding to the compilation task using the first public compilation node set is greater than the efficiency of responding to the compilation task using the second public compilation node set.

[0139] In an exemplary embodiment, in response to identifying that the compilation node type is a public compilation resource type, allocating a public compilation node to the user from a public compilation node cluster shared by multiple users in the compilation node cluster includes: in response to identifying that the compilation node type is a public compilation resource type, determining a predicted compilation time consumption of the compilation task; in response to the predicted compilation time consumption satisfying a preset high time consumption judgment condition, allocating a public compilation node to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and configuring the public compilation node as a private compilation node exclusively for the user during a period when the public compilation node executes the compilation task.

[0140] The predicted compilation time is the predicted value of the time required for the compilation node cluster to call the compilation node to execute the compilation task.

[0141] The high time-consuming judgment condition may be a condition pre-set by a technician according to actual needs and used to distinguish the time consumption of a compilation task.

[0142] Exemplarily, the high time consumption condition can be a specific time value. If the predicted compilation time is greater than the time value, it can be determined that the predicted compilation time meets the preset high time consumption condition. If the predicted compilation time is less than the time value, it can be determined that the predicted compilation time does not meet the preset high time consumption condition.

[0143] In some exemplary embodiments, after determining that the compilation node type is a public compilation resource type, the server may first determine the predicted compilation time consumption of the compilation task by using a pre-trained compilation time consumption prediction model.

[0144] Furthermore, if the server determines that the predicted compilation time meets a preset high-time-consuming judgment condition, a public compilation node is allocated to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and the public compilation node is configured as a private compilation node exclusively for the user during the period when the public compilation node executes the compilation task.

[0145] If the server determines that the predicted compilation time does not meet the preset high time consumption judgment condition, it allocates a public compilation node to the user from the public compilation node cluster shared by multiple users in the compilation node cluster to execute the compilation task.

[0146] The method of determining the predicted compilation time of the compilation task in response to identifying that the compilation node type is a public compilation resource type; in response to the predicted compilation time satisfying a preset high-time-consumption judgment condition, allocating a public compilation node to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and configuring the public compilation node as a private compilation node exclusively for the user during a period when the public compilation node executes the compilation task, if it is determined that the predicted compilation time of the compilation task satisfies the preset high-time-consumption judgment condition, allocating a public compilation node to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and configuring the public compilation node as a private compilation node exclusively for the user during the period when the public compilation node executes the compilation task, so as to efficiently execute the compilation task and improve the execution efficiency of high-time-consuming compilation tasks.

[0147] In an exemplary embodiment, calling the public compiling node and utilizing the public cache of the public compiling node to execute the compiling task includes: preprocessing the compiling task to generate subtasks through a master node in the compiling node cluster; determining a plurality of slave nodes for executing the subtasks on the public compiling node allocated to the user; and scheduling the plurality of slave nodes to concurrently execute the subtasks obtained by preprocessing by the master node using the public cache of the public compiling node.

[0148] Preprocessing refers to the preparation operations for the compilation task before the compilation task is officially executed. The preparation operations may include code analysis operations, environment preparation operations, and task decomposition operations.

[0149] Specifically, the code analysis operation can be used to perform syntax checking on the source code of the compilation task, analyze the structure of the source code, and check the references of the source code to external resources. The environment preparation operation can be used to determine the compiler version adapted to the compilation task, load the configuration file related to the compilation task, and download the referenced external resources. The task decomposition operation is used to decompose the compilation task to generate multiple subtasks.

[0150] Slave nodes can be containers or virtual machines running on a common compilation node. Figure 7 As shown, a common compilation node can run multiple slave nodes, that is, Figure 7 The public compile node can allocate computing resources to the container so that the container has the ability to execute the compile task. In an optional embodiment, the resource configuration of the slave node can be 8 cores and 32G. The number of CPU cores available for the slave node is 8, and the available memory capacity is 32G.

[0151] In some exemplary embodiments, after determining that the compiling node type is a public compiling resource type, the server may allocate a public compiling node to the user from a public compiling node cluster in the compiling node cluster that is shared by multiple users.

[0152] Furthermore, the server can pre-process the compilation task through the master node in the compilation node cluster to obtain multiple sub-tasks, and determine multiple slave nodes for executing the multiple sub-tasks on the public compilation node allocated to the user, and schedule the multiple slave nodes to use the public cache of the public compilation node to concurrently execute the sub-tasks pre-processed by the master node.

[0153] It should be noted that the number of multiple slave nodes determined to execute multiple subtasks should be less than a preset number, which can be preset by a technician according to actual needs. Since there may be subtasks that occupy a large amount of memory among the multiple subtasks, it is necessary to limit the number of subnodes that execute subtasks concurrently to avoid memory anomalies.

[0154] The method of preprocessing the compilation task through the master node in the compilation node cluster to generate subtasks; determining multiple slave nodes for executing the subtasks on the public compilation node allocated to the user; scheduling the multiple slave nodes to concurrently execute the subtasks obtained by the preprocessing of the master node using the public cache of the public compilation node, divides the compilation task into multiple subtasks to achieve concurrent execution of multiple subtasks based on multiple slave nodes, and effectively improves the processing efficiency of the compilation task.

[0155] In an exemplary embodiment, the multiple slave nodes for executing the subtask are determined on the common compilation node allocated to the user, including: for multiple concurrent first-category subtasks among the subtasks preprocessed by the master node, multiple slave nodes for concurrently executing the multiple first-category subtasks are determined on the common compilation node allocated to the user; for multiple concurrent second-category subtasks among the subtasks preprocessed by the master node, multiple slave nodes for concurrently executing the multiple second-category subtasks are determined on the common compilation node allocated to the user.

[0156] Among them, the resource requirement of the first type of subtask is greater than the resource requirement of the second type of subtask, and the preset number of the multiple first type subtasks executed concurrently on a single slave node is less than the preset number of the multiple second type subtasks executed concurrently on a single slave node.

[0157] The resource requirement can be used to characterize the amount of resources required to execute a subtask. Specifically, it can be the memory, number of CPU cores, disk I / O bandwidth, etc. required to execute a subtask. In this application, the resource requirement can be the memory required to execute a subtask.

[0158] In some exemplary embodiments, after the server pre-processes the compilation task through the master node in the compilation node cluster and generates subtasks, the server may determine the subtask type according to the resource demand of the subtask.

[0159] Furthermore, when the server determines that the subtask is a first-category subtask, it can determine multiple slave nodes for concurrently executing the multiple first-category subtasks on the public compiling node allocated by the user, and schedule the multiple slave nodes to concurrently execute the subtasks preprocessed by the master node using the public cache of the public compiling node. When the server determines that the subtask is a second-category subtask, it can determine multiple slave nodes for concurrently executing the multiple second-category subtasks on the public compiling node allocated to the user, and schedule the multiple slave nodes to concurrently execute the subtasks preprocessed by the master node using the public cache of the public compiling node.

[0160] Specifically, Figure 7 and Figure 8 As shown, since the resource requirement of the first category subtask is greater than the resource requirement of the second category subtask, that is, the memory occupied by the first category subtask during execution is greater than the memory occupied by the second category subtask during execution, and the memory of each common compilation node is fixed, therefore, the preset number of multiple first category subtasks executed concurrently on a single slave node is less than the preset number of multiple second category subtasks executed concurrently on a single slave node. Figure 7 shows the number of concurrent executions of the first type of subtasks on a single slave node. Figure 7 The number of concurrent executions of the first type of subtasks on a single slave node is 4. Figure 7 Take slave a in the example, there are four first-class subtasks in slave a, namely gcc a1, cpp, gcc a2, cpp, gcc a3, cpp and gcc a4, cpp. Figure 8 shows the number of second-class subtasks executed concurrently on a single slave node. Figure 8 The number of concurrent executions of the second type of subtasks on a single slave node is 8. Figure 8Taking slave a as an example, there are 8 second-category subtasks in slave a, namely gcc a1, cpp, gcc a2, cpp, gcc a3, cpp, gcc a4, cpp, gcca5, cpp, gcc a6, cpp, gcc a7, cpp and gcc a4, cpp.

[0161] The method for determining, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of first-category subtasks in the subtasks preprocessed by the master node; and for determining, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of second-category subtasks in the subtasks preprocessed by the master node, classifies the subtasks to determine the number of the plurality of slave nodes for concurrently executing the subtasks, thereby avoiding memory exceptions.

[0162] In an exemplary embodiment, the multiple slave nodes for concurrently executing the multiple second-category subtasks are determined on the common compilation node allocated for the user, including: on the common compilation node allocated for the user, taking the concurrent number of subtasks limited by the preprocessing speed of the master node as a restriction condition, and determining the multiple slave nodes for concurrently executing the multiple second-category subtasks based on the restriction condition, the number of subtasks of the concurrent multiple second-category subtasks, and the preset number of second-category subtasks concurrently executed on a single slave node.

[0163] The preprocessing speed refers to the speed at which the master node preprocesses the compilation task. Specifically, the preprocessing speed may be the code analysis operation speed, the environment preparation operation speed, the task decomposition operation speed, and the like.

[0164] In some exemplary embodiments, when the server determines that a subtask is a second-category subtask, it may, on a common compilation node allocated to the user, use the concurrent number of subtasks limited by the preprocessing speed of the master node as a restriction condition, and determine multiple slave nodes for concurrently executing the multiple second-category subtasks based on the restriction condition, the number of subtasks of the concurrent multiple second-category subtasks, and the preset number of second-category subtasks concurrently executed on a single slave node.

[0165] When determining the multiple slave nodes for concurrently executing the multiple second-category subtasks, it is necessary not only to consider the preset number of second-category subtasks concurrently executed on a single slave node, but also to consider the preprocessing speed of the master node. Since the second-category subtasks are generated when the master node preprocesses the compilation tasks, the second-category subtasks can only be executed after the second-category subtasks are generated. Therefore, when determining the multiple slave nodes for concurrently executing the multiple second-category subtasks, it is also necessary to consider the number of second-category subtasks that the master node can preprocess and generate within a unit time, so as to avoid the situation where the number of slave nodes is greater than the number of second-category subtasks and there are idle slave nodes, resulting in a waste of resources.

[0166] The above method is to determine the multiple slave nodes for concurrently executing the multiple second-category subtasks on the common compilation node allocated to the user, with the concurrent number of subtasks limited by the preprocessing speed of the master node as the limiting condition, based on the limiting condition, the number of subtasks of the multiple second-category subtasks in parallel, and the preset number of second-category subtasks concurrently executed on a single slave node. When determining the multiple slave nodes for concurrently executing the multiple second-category subtasks, not only the preset number of second-category subtasks concurrently executed on a single slave node is taken into consideration, but also the preprocessing speed of the master node is taken into consideration, thereby ensuring that the number of slave nodes matches the number of second-category subtasks and avoiding the occurrence of resource waste.

[0167] In an exemplary embodiment, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of first-category subtasks are determined, including: splitting each of the plurality of first-category subtasks into a plurality of concurrent second-category subtasks; and determining a plurality of slave nodes for concurrently executing the second-category subtasks split from the plurality of first-category subtasks according to the total number of second-category subtasks split from the plurality of first-category subtasks and a preset number of first-category subtasks concurrently executed on a single slave node.

[0168] In some exemplary embodiments, for first-category subtasks with relatively large resource requirements, the server may split each of the plurality of first-category subtasks into a plurality of concurrent second-level subtasks.

[0169] Furthermore, after splitting, the server can determine multiple slave nodes for concurrently executing the secondary subtasks split from the multiple first-class subtasks according to the total number of secondary subtasks split from the multiple first-class subtasks and the preset number of primary subtasks concurrently executed on a single slave node.

[0170] The above method splits each subtask of the multiple first-category subtasks into multiple concurrent second-category subtasks; determines the multiple slave nodes for concurrently executing the second-category subtasks split from the multiple first-category subtasks according to the total number of second-category subtasks split from the multiple first-category subtasks and the preset number of first-category subtasks concurrently executed on a single slave node. For first-category subtasks with large resource requirements, they can be split first to obtain multiple second-category subtasks, and then the multiple second-category subtasks can be processed in parallel, which can effectively improve the processing efficiency of the first-category subtasks.

[0171] In an exemplary embodiment, the scheduling of the multiple slave nodes, using the public cache of the public compiling node, and concurrently executing the subtasks preprocessed by the master node includes: obtaining a permission management account for accessing the data disk of the public compiling node; synchronizing the permission management account to the multiple slave nodes used to execute the subtasks; and performing read and write operations on the data disk using the permissions of the permission management account in the process of calling the multiple slave nodes and using the public cache of the public compiling node to concurrently execute the subtasks preprocessed by the master node.

[0172] The permission management account refers to an identity credential used to access the data disk of a public compilation node. The permission management account can specifically be a unibuild account.

[0173] In some exemplary embodiments, the public compilation node can mount the data disk into the slave node so that the slave node can perform read and write operations on the data disk during the execution of the subtask, that is, read the relevant data of the subtask in the data disk, or write the result data of executing the subtask to the data disk.

[0174] Therefore, the server can first obtain the permission management account for accessing the data disk of the public compilation node, and then synchronize the permission management account to multiple slave nodes for executing the subtask. Specifically, the uid (User Identifier) ​​and gid (Group Identifier) ​​of the permission management account can be synchronized to multiple slave nodes for executing the subtask.

[0175] After performing the above operations, the server can enable the slave nodes to use the permissions of the permission management account to read and write operations on the data disk to execute compilation tasks while calling the multiple slave nodes and using the public cache of the public compilation node to concurrently execute the subtasks preprocessed by the master node.

[0176] The above-mentioned method of obtaining a permission management account for accessing the data disk of the public compilation node; synchronizing the permission management account to the multiple slave nodes used to execute the subtask; and using the permissions of the permission management account to perform read and write operations on the data disk in the process of calling the multiple slave nodes and concurrently executing the subtasks preprocessed by the master node using the public cache of the public compilation node can avoid confusion in data disk permissions, resulting in limited read and write operations of the slave nodes on the data disk and inability to execute compilation tasks.

[0177] In an optional embodiment of the present application, the method further includes: obtaining a preset base image; and using the base image to pull up multiple slave nodes for executing the subtask on a public compilation node allocated to the user.

[0178] A base image is an operating system template. Specifically, a base image can be an operating system of a commonly used compiler with a build toolchain installed.

[0179] A build toolchain refers to a set of tools used to convert source code into executable programs or target files. It includes the tools required for the entire process from code compilation and linking to the final generation of target files. The build toolchain can include gcc, clang, binutils, git, etc.

[0180] In some exemplary embodiments, the server can obtain a preset basic image from a database. In the process of determining multiple slave nodes for executing the subtask on a public compilation node allocated to the user, the server can use the basic image to directly pull up multiple slave nodes for executing the subtask, without having to download the operating system, install the build tool chain, etc., thereby effectively improving the processing efficiency of the compilation task.

[0181] In an optional embodiment of the present application, the method also includes: in the process of pulling up multiple slave nodes for executing the subtask, mounting the data disk of the common compilation node to the preset mount point of the multiple slave nodes; in the process of calling the multiple slave nodes and using the common cache of the common compilation node to concurrently execute the subtasks preprocessed by the master node, using the preset mount point and the permissions of the permission management account to perform read and write operations on the data disk.

[0182] A mount point is a directory location in an operating system that is used to connect different file systems. A preset mount point is a mount point that a technician pre-sets on a slave node based on actual needs.

[0183] In some exemplary embodiments, the server may mount the data disk of the common compilation node to a preset mount point of the plurality of slave nodes when pulling up the plurality of slave nodes for executing the subtask.

[0184] Furthermore, in the process of calling the multiple slave nodes and using the public cache of the public compilation node to concurrently execute the subtasks preprocessed by the master node, the server can use the preset mount point to determine the access path of the data disk, and then use the permissions of the permission management account and the access path of the data disk to perform read and write operations on the data disk.

[0185] For example, Fig. 9 As shown, Fig. 9 Three methods of mounting the data disk of the public compiling node to the slave node are shown. As described above, the slave node can be a container running in the public compiling node. Figure 8 The containers in the can be considered as slave nodes. Fig. 9 The three methods are management volume mode, binding data volume mode and temporary data volume mode. Among them, the container can be connected to the Docker area in the file system of the public compilation node through the management volume mode. The Docker area refers to the area where Docker stores data such as images, containers, and network configurations on the public compilation node. The container can also be connected to the file system of the public compilation node through the binding data volume mode. The container can also be connected to the memory of the public compilation node through the temporary data volume mode.

[0186] In an optional embodiment of the present application, the method further includes: acquiring an external daemon service of the slave node, and mounting the external daemon service of the slave node into the slave node.

[0187] In some exemplary embodiments, since the slave node itself may be a container, when executing a subtask on the slave node, a subtask environment needs to be built, which is equivalent to building a container in a container. Therefore, the external daemon service of the slave node can be obtained first, and then the external daemon service of the slave node can be mounted to the subtask environment in the slave node, so that the subtask environment can receive related operations instructed by the user through the external daemon service of the slave node.

[0188] Specifically, Fig.10 As shown, in Fig.10 In the example, by mounting the communication interface of the slave node to the subtask environment, the subtask environment can receive related operations instructed by the user through the external daemon service of the slave node. Fig.10 The command line interface in is used to receive user operation instructions for the subtask environment. The user can send a request to the external daemon service through the command line interface so that the subtask environment responds to the request through the external daemon service, and can also send a request directly to the slave node through the command line interface so that the slave node responds to the request.

[0189] In an optional embodiment of the present application, the method further includes: extending the expiration time of the public cache; and using the extended expiration time to invalidate the public cache.

[0190] The expiration time can be used to indicate the time for cleaning the public cache. The expiration time can specifically be the expiration time of the daemon process of the public cache. When the expiration time is reached, the daemon process of the public cache will become invalid and the public cache will be cleaned.

[0191] In some exemplary embodiments, the server may extend the expiration time of the public cache. Further, after adjusting the expiration time of the public cache, the server may use the adjusted expiration time to invalidate the public cache.

[0192] For example, the expiration time of the public cache is 6 hours, that is, if the server does not use the public cache to perform a compilation task for 6 hours, the public cache can be invalidated, that is, the public cache is cleared. After adjusting the expiration time of the public cache, the expiration time of the public cache can be extended from 6 hours to 12 hours, that is, if the server does not use the public cache to perform a compilation task for 12 hours, the public cache can be invalidated, that is, the public cache is cleared.

[0193] The above-mentioned method of adjusting the expiration time of the public cache; using the adjusted expiration time to invalidate the public cache; the expiration time after adjustment is greater than the expiration time, by extending the method of the public cache, by extending the expiration time of the public cache, the cache hit rate in the process of executing compilation tasks can be effectively improved.

[0194] In an optional embodiment of the present application, the public compilation node is called and the public cache of the public compilation node is used to execute the compilation task, including: according to the compilation task, a cache hit operation is performed from the memory-level cache in the public cache of the compilation node; in response to the failure of the cache hit operation, a cache hit operation is performed from the first-level cache in the disk-level cache in the public cache of the compilation node; in response to the failure of the cache hit operation, a cache hit operation is performed from the second-level cache in the disk-level cache in the public cache of the compilation node; in response to the failure of the cache hit operation, the compilation task is executed.

[0195] Memory-level cache refers to a cache mechanism that temporarily stores frequently accessed or recently used data in the computer's memory (RAM).

[0196] Disk-level cache refers to a cache mechanism that temporarily stores data on disk. Level 1 cache refers to the high-speed cache located inside the CPU, closest to the CPU core. Level 2 cache is also located inside the CPU. It is slightly farther from the CPU core than the level 1 cache, but much closer than the memory. It is located between the level 1 cache and the memory, and the capacity of the level 2 cache is greater than that of the level 1 cache.

[0197] In some exemplary embodiments, Fig.11 As shown, when the server calls the public compilation node and uses the public cache of the public compilation node to execute the compilation task, it can first use bazel to parse the compilation file indicated by the compilation task, and then perform a cache hit operation from the memory-level cache in the public cache of the compilation node. If the cache hit operation is successful, the compilation task is executed using the hit cache.

[0198] Furthermore, if the cache hit operation fails, a cache hit operation is performed from the first-level cache in the disk-level cache in the public cache of the compilation node. If the cache hit operation is successful, the compilation task is executed using the hit cache. If the cache hit operation fails, a cache hit operation is performed from the second-level cache in the disk-level cache in the public cache of the compilation node. If the cache hit operation is successful, the compilation task is executed using the hit cache. If the cache hit operation fails, the compilation task is executed directly.

[0199] Furthermore, in an optional embodiment of the present application, different versions of third-party libraries use different cache directories.

[0200] Since most compilation tasks contain multiple different business module codes, these different business modules may rely on third-party libraries such as protobuf (a tool library for serializing structured data) and gperftools (a library for performance analysis and optimization), and the version requirements of these third-party libraries may not be consistent among business modules. For example, business module A may need to use protobuf version 3.0, while business module A may need to use protobuf version 3.5.

[0201] This situation of different version requirements can easily lead to cache invalidation. Specifically, because the cache mechanism usually determines whether to reuse cached data based on identifiers such as file content, if the caches corresponding to different versions of third-party libraries are not properly distinguished, caches may overlap with each other.

[0202] For example, when business module A is compiled, the relevant compilation result data may be cached. When business module B is compiled, since the cache does not distinguish between versions, the corresponding operations of business module B may mistakenly overwrite the previous cache content of business module A, resulting in the next time business module A is used, it cannot obtain the correct cache data from the cache, which is a problem of cache invalidation.

[0203] Therefore, different cache directories are used for different versions of third-party libraries to avoid the above-mentioned cache invalidation problem.

[0204] The above method of performing a cache hit operation from the memory-level cache in the public cache of the compilation node according to the compilation task; in response to the failure of the cache hit operation, performing a cache hit operation from the first-level cache in the disk-level cache in the public cache of the compilation node; in response to the failure of the cache hit operation, performing a cache hit operation from the second-level cache in the disk-level cache in the public cache of the compilation node; and executing the compilation task in response to the failure of the cache hit operation can effectively improve the cache hit rate.

[0205] In an optional embodiment of the present application, the method further includes: when the compilation node cluster is in an idle state, obtaining the compilation record of the compilation node cluster; according to the compilation record, compiling the compilation files in the basic library according to the compilation number, and storing the data obtained after the compilation process as a cache in a public cache.

[0206] The base library refers to the database used to store source files for compilation tasks.

[0207] In some exemplary embodiments, as the compilation task changes, the source files stored in the basic library may change, the version of the dependent third-party library may also change, and the code of the compilation environment may also change, then the cache stored in the public cache may no longer be applicable.

[0208] Therefore, when the compilation node cluster is in an idle state, the compilation records of the compilation node cluster can be obtained, the compilation times of each source file in the basic library can be determined according to the compilation records, and each source file can be sorted according to the compilation times. Each source file is compiled in the sorted order, and the data obtained after the compilation process is used as a cache and stored in the public cache.

[0209] Specifically, the compilation count may be the number of times the source file is compiled, so that source files that are used more often can be compiled first to obtain caches of these source files, thereby ensuring the cache hit rate. The compilation count may also be the order in which the source files are compiled, so that the compilation order of which source files has priority can be determined based on the compilation count, and the compilation results of these source files with priority may need to be used when compiling other source files, so these source files can be compiled first to facilitate the compilation of other source files that rely on their compilation results.

[0210] In an optional embodiment of the present application, the method further includes: in the process of pulling up multiple slave nodes for executing the subtask, mapping the network ports of the multiple slave nodes to a common compilation node to which the multiple slave nodes belong.

[0211] In some exemplary embodiments, when the server pulls up multiple slave nodes for executing the subtask, it can also map the network ports of the multiple slave nodes to the common compilation node to which the multiple slave nodes belong, so that the master node can access the slave nodes based on the network ports in the common compilation node.

[0212] Furthermore, in an optional embodiment of the present application, the slave nodes in each public compilation node are all in the pre-started state, and only specific subtasks need to be assigned to the slave nodes, without other pre-operations. Moreover, the server can monitor each slave node in real time, and when a slave node abnormality is detected, the corresponding slave node will be automatically restarted.

[0213] In an exemplary embodiment of the present application, Fig.12 As shown, another compilation task processing method is provided, the method comprising the following steps:

[0214] Step 1201: receiving a private compiling node creation request initiated by the user; determining a compiling machine indicated by the private compiling node creation request according to the private compiling node creation request; and adding the compiling machine as a private compiling node exclusively for the user to a set of private compiling nodes in the compiling node cluster.

[0215] In some exemplary embodiments, when a user has a need to create a private compiling node exclusively for himself in a compiling node cluster, a private compiling node creation request may be initiated. After receiving the private compiling node creation request initiated by the user, the server may determine, according to the private compiling node creation request, a compiling machine to be connected to the compiling node cluster indicated by the private compiling node creation request. After determining the compiling machine, the server may add the compiling machine as a private compiling node exclusively for the user to a set of private compiling nodes in the compiling node cluster.

[0216] Step 1202: receiving a private compiling node creation request initiated by the user; determining, according to the private compiling node creation request, a compiling node in an idle state from a public compiling node cluster in the compiling node cluster; and adding the compiling node in an idle state as a private compiling node exclusively for the user to a set of private compiling nodes in the compiling node cluster.

[0217] In some exemplary embodiments, when a user has a need to create a private compiling node exclusively for himself in a compiling node cluster, a private compiling node creation request may be initiated. After receiving the private compiling node creation request initiated by the user, in addition to adding the compiling machine indicated by the private compiling node creation request as the private compiling node exclusively for the user to the private compiling node set in the compiling node cluster, the server may also determine an idle compiling node from the public compiling node cluster in the compiling node cluster.

[0218] Furthermore, after determining the compilation node in the idle state, the server may add the compilation node in the idle state as a private compilation node dedicated to the user to the set of private compilation nodes in the compilation node cluster.

[0219] Step 1203: Receive a compilation request initiated by the user, and determine whether the compilation request carries a private compilation resource parameter value; in response to the compilation request carrying the private compilation resource parameter value, determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type.

[0220] In some exemplary embodiments, after receiving a compilation request initiated by a user, the server may first determine whether the compilation request carries a private compilation resource parameter value. If the server determines that the compilation request carries the private compilation resource parameter value, the server determines that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type; otherwise, the server determines that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0221] Step 1204: In response to the compilation request not carrying the private compilation resource parameter value, identifying the user type of the user; in response to the user type being the first user type, determining that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type; in response to the user type being the second user type, determining that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0222] In some exemplary embodiments, after receiving a compilation request initiated by a user, the server may first determine whether the compilation request carries a private compilation resource parameter value. If the server determines that the compilation request does not carry the private compilation resource parameter value, in addition to directly determining that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type, the server may also identify the user type of the user.

[0223] Further, when the server determines that the user type of the user is the first user type, it can determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type; when the server determines that the user type of the user is the second user type, it can determine that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0224] Step 1205: In response to identifying that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, a private compilation node exclusive to the user and pre-connected to the compilation node cluster by the user is called from the set of private compilation nodes in the compilation node cluster, and the compilation task is executed by using a private cache of the private compilation node exclusive to the user.

[0225] In some exemplary embodiments, if the server recognizes that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, it calls a private compilation node exclusive to the user that is pre-connected to the compilation node cluster from the set of private compilation nodes in the compilation node cluster to execute the compilation task.

[0226] Step 1206: In response to identifying that the compilation resource type is a private compilation resource type, and the compilation request does not carry the private compilation resource parameter value, call a private compilation node allocated to the user by the compilation node cluster and dedicated to the user from the set of private compilation nodes in the compilation node cluster, and use a private cache of the private compilation node that is dedicated to the user to execute the compilation task.

[0227] In some exemplary embodiments, if the server identifies that the compilation resource type is a private compilation resource type, in addition to the above compilation request carrying the private compilation resource parameter value, there is also a case where the compilation request does not carry the private compilation resource parameter value.

[0228] In this case, the server may call a private compiling node allocated to the user by the compiling node cluster and dedicated to the user from the private compiling node set in the compiling node cluster to execute the compiling task.

[0229] Step 1207: In response to identifying that the compilation node type is a public compilation resource type, determining a predicted compilation time consumption of the compilation task; in response to the predicted compilation time consumption satisfying a preset high time consumption judgment condition, allocating a public compilation node to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, configuring the public compilation node as a private compilation node exclusively used by the user during a period in which the public compilation node executes the compilation task, and executing the compilation task by using a private cache of the private compilation node exclusively used by the user.

[0230] In some exemplary embodiments, after determining that the compilation node type is a public compilation resource type, the server may first determine the predicted compilation time consumption of the compilation task by using a pre-trained compilation time consumption prediction model.

[0231] Furthermore, if the server determines that the predicted compilation time meets a preset high-time-consuming judgment condition, a public compilation node is allocated to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and the public compilation node is configured as a private compilation node exclusively for the user during the period when the public compilation node executes the compilation task.

[0232] If the server determines that the predicted compilation time does not meet the preset high time consumption judgment condition, it allocates a public compilation node to the user from the public compilation node cluster shared by multiple users in the compilation node cluster to execute the compilation task.

[0233] Step 1208: In response to identifying that the compilation node type is a public compilation resource type, determining a service level for the user according to service configuration information for the user; in response to the service level being a first service level, determining a first public compilation node set for use by users of the first service level from a public compilation node cluster for use by multiple users in the compilation node cluster, and allocating a public compilation node to the user from the first public compilation node set.

[0234] In some exemplary embodiments, if the server identifies that the compiled resource type is a public compiled resource type, the server may also determine the service level of the user according to the service configuration information for the user.

[0235] If the server determines that the service level of the user is the first service level, the server determines a first public compiling node set for common use by users of the first service level from a public compiling node cluster for common use by multiple users in the compiling node cluster, and allocates a public compiling node to the user from the first public compiling node set.

[0236] Step 1209: In response to the service level being the second service level, determine a second public compiling node set for use by users of the second service level from a public compiling node cluster for use by multiple users in the compiling node cluster, and allocate a public compiling node to the user from the second public compiling node set.

[0237] In some exemplary embodiments, if the server determines that the service level of the user is the second service level based on the service configuration information for the user, it can determine a second public compilation node set for use by users of the second service level from a public compilation node cluster for use by multiple users in the compilation node cluster, and allocate a public compilation node to the user from the second public compilation node set.

[0238] Step 1210: preprocess the compilation task through the master node in the compilation node cluster to generate subtasks; for the concurrent multiple first-category subtasks in the subtasks obtained by the preprocessing of the master node, split each subtask in the multiple second-category subtasks into multiple concurrent second-category subtasks; determine the multiple slave nodes used to concurrently execute the second-category subtasks split from the multiple second-category subtasks according to the total number of second-category subtasks split from the multiple second-category subtasks and the preset number of second-category subtasks concurrently executed on a single slave node.

[0239] In some exemplary embodiments, after determining that the compiling node type is a public compiling resource type, the server may allocate a public compiling node to the user from a public compiling node cluster in the compiling node cluster that is shared by multiple users.

[0240] Furthermore, the server may pre-process the compilation task through the master node in the compilation node cluster to obtain multiple subtasks. When the server determines that the subtask is a first-class subtask, each of the multiple first-class subtasks may be split into multiple concurrent second-class subtasks.

[0241] After splitting, the server can determine multiple slave nodes for concurrently executing the secondary subtasks split from the multiple first-class subtasks according to the total number of secondary subtasks split from the multiple first-class subtasks and the preset number of primary subtasks concurrently executed on a single slave node.

[0242] Step 1211, for the multiple concurrent second-category subtasks in the subtasks obtained by preprocessing the master node, on the common compilation node allocated to the user, the concurrent number of subtasks limited by the preprocessing speed of the master node is used as a restriction condition, and based on the restriction condition, the number of subtasks of the multiple concurrent second-category subtasks and the preset number of second-category subtasks concurrently executed on a single slave node, a plurality of slave nodes for concurrently executing the multiple second-category subtasks are determined.

[0243] In some exemplary embodiments, when the server determines that a subtask is a second-category subtask, it may, on a common compilation node allocated to the user, use the concurrent number of subtasks limited by the preprocessing speed of the master node as a restriction condition, and determine multiple slave nodes for concurrently executing the multiple second-category subtasks based on the restriction condition, the number of subtasks of the concurrent multiple second-category subtasks, and the preset number of second-category subtasks concurrently executed on a single slave node.

[0244] Step 1212, obtain the permission management account used to access the data disk of the public compilation node; synchronize the permission management account to the multiple slave nodes used to execute the subtask; in the process of calling the multiple slave nodes and using the public cache of the public compilation node to concurrently execute the subtasks preprocessed by the master node, use the permissions of the permission management account to perform read and write operations on the data disk.

[0245] In some exemplary embodiments, the public compilation node can mount the data disk into the slave node so that the slave node can perform read and write operations on the data disk during the execution of the subtask, that is, read the relevant data of the subtask in the data disk, or write the result data of executing the subtask to the data disk.

[0246] Therefore, the server can first obtain the permission management account for accessing the data disk of the public compilation node, and then synchronize the permission management account to the multiple slave nodes used to execute the subtask. After performing the above operations, the server can enable the slave nodes to use the permissions of the permission management account to read and write operations on the data disk to execute the compilation task in the process of calling the multiple slave nodes and concurrently executing the subtasks preprocessed by the master node using the public cache of the public compilation node.

[0247] In some exemplary embodiments, when a user has a need to create a private compiling node dedicated to him / herself in a compiling node cluster, a private compiling node creation request may be initiated. After receiving the private compiling node creation request initiated by the user, the server may determine, according to the private compiling node creation request, a compiling machine to be connected to the compiling node cluster indicated by the private compiling node creation request.

[0248] Specifically, the private compiling node creation request may carry the identification information of the compiling machine, and the server may determine the compiling machine according to the identification information of the compiling machine. The server may also determine the compiling machine that has a binding relationship with the user after receiving the private compiling node creation request initiated by the user, and determine the compiling machine that has a binding relationship with the user as the compiling machine.

[0249] Furthermore, after determining the compiler, the server may use the compiler as a private compilation node dedicated to the user and add the compiler to the set of private compilation nodes in the compilation node cluster.

[0250] When a user needs to create a private compile node in a compile node cluster, a private compile node creation request can be initiated. After receiving the private compile node creation request initiated by the user, the server can determine an idle compile node from the public compile node cluster in the compile node cluster.

[0251] Furthermore, after determining the compilation node in the idle state, the server may add the compilation node in the idle state as a private compilation node dedicated to the user to the set of private compilation nodes in the compilation node cluster.

[0252] When a user needs to compile the source code of a software project, the compilation of the source code of the software project is a compilation task. It can be determined whether to use private compilation resources to perform the compilation task based on the compilation requirements of the software project, that is, to determine the type of compilation resources used to compile the source code, and after the determination, a compilation request for instructing the execution of the compilation task is initiated to the server through the terminal. Alternatively, when a user needs to update a developed software project, if there is an update to the source code of the software project, the updated source code needs to be compiled, and the compilation of the updated source code is a compilation task. It can be determined whether to use private compilation resources to perform the compilation task based on the compilation requirements of the compilation task, that is, to determine the type of compilation resources used to compile the source code, and after the determination, a compilation request for instructing the execution of the compilation task is initiated to the server through the terminal.

[0253] Further, after receiving the compilation request initiated by the user, the server can determine whether the compilation request carries the private compilation resource parameter value. If the server determines that the compilation request carries the private compilation resource parameter value, it can determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type. If the server determines that the compilation request does not carry the private compilation resource parameter value, it can further identify the user type of the user. Specifically, since the compilation request is initiated by the user through the terminal, the server can determine the user who initiated the compilation request through the compilation request, and then obtain the user information of the user from the database, and determine the user type of the user according to the user information. For example, the service level of the user can be determined according to the user information, and then the user type of the user can be determined according to the user service level. It can also be determined according to the user information whether the user has opened the private compilation resource function, and then the user type of the user can be determined according to the determination result. Further, when the server determines that the user type of the user is the first user type, it can determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type. When the server determines that the user type of the user is the second user type, it can determine that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0254] Furthermore, if the server recognizes that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, the server calls a private compilation node that is pre-connected to the compilation node cluster and is exclusive to the user from the private compilation node set in the compilation node cluster to execute the compilation task. If the server recognizes that the compilation resource type is a private compilation resource type and the compilation request does not carry a private compilation resource parameter value, the server calls a private compilation node that is allocated to the user and is exclusive to the user by the compilation node cluster from the private compilation node set in the compilation node cluster to execute the compilation task.

[0255] That is to say, the same user can have two types of private compilation nodes in the compilation node cluster at the same time, one is the one that the user pre-connects to the compilation node cluster, and the other is the one that the compilation node cluster allocates to the user. After selecting the type of private compilation resource according to actual needs, the user can also select the type of private compilation node according to actual needs.

[0256] Furthermore, after determining that the compilation node type is a public compilation resource type, the server may first determine the predicted compilation time of the compilation task through a pre-trained compilation time prediction model. If the server determines that the predicted compilation time meets the preset high-time discrimination condition, a public compilation node is allocated to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and the public compilation node is configured as a private compilation node exclusively for the user during the period when the public compilation node executes the compilation task. If the server determines that the predicted compilation time does not meet the preset high-time discrimination condition, a public compilation node is allocated to the user from a public compilation node cluster shared by multiple users in the compilation node cluster to execute the compilation task.

[0257] If the server identifies that the compilation resource type is a public compilation resource type, the server determines the service level of the user according to the service configuration information for the user. Specifically, the server can obtain the service configuration information of the user from the database according to the user ID of the user, and then determine the service level of the user according to the service configuration information of the user. If the server determines that the service level of the user is the first service level, a first public compilation node set for users of the first service level is determined from the public compilation node cluster for multiple users in the compilation node cluster, and a public compilation node is allocated to the user from the first public compilation node set. If the server determines that the service level of the user is the second service level, a second public compilation node set for users of the second service level is determined from the public compilation node cluster for multiple users in the compilation node cluster, and a public compilation node is allocated to the user from the second public compilation node set.

[0258] After the server preprocesses the compilation task through the master node in the compilation node cluster and generates a subtask, the subtask type can be determined according to the resource demand of the subtask. When the server determines that the subtask is a first-type subtask, it can determine multiple slave nodes for concurrently executing the multiple first-type subtasks on the public compilation node allocated by the user, and schedule the multiple slave nodes to concurrently execute the subtasks obtained by the preprocessing of the master node using the public cache of the public compilation node. When the server determines that the subtask is a second-type subtask, it can determine multiple slave nodes for concurrently executing the multiple second-type subtasks on the public compilation node allocated for the user, and schedule the multiple slave nodes to concurrently execute the subtasks obtained by the preprocessing of the master node using the public cache of the public compilation node.

[0259] When the server determines that the subtask is a second-category subtask, the server can use the number of subtask concurrency limited by the preprocessing speed of the master node as a limiting condition on the public compilation node allocated to the user, and determine the multiple slave nodes used to concurrently execute the multiple second-category subtasks based on the limiting condition, the number of subtasks of the multiple concurrent second-category subtasks, and the preset number of second-category subtasks executed concurrently on a single slave node. When the server determines that the subtask is a first-category subtask, each subtask in the multiple first-category subtasks can be split into multiple concurrent secondary subtasks. After the splitting, the server can determine the multiple slave nodes used to concurrently execute the secondary subtasks split from the multiple first-category subtasks according to the total number of secondary subtasks split from the multiple first-category subtasks and the preset number of primary subtasks executed concurrently on a single slave node.

[0260] It should be noted that after conducting relevant practices using the compilation task processing method provided in this application, it can be found that for the same compilation task, the compilation time required using the compilation task processing method provided in this application is shorter, and the specific data can be shown in Table 2.

[0261] Table 2

[0262]

[0263] Furthermore, the total number of compiled files included in the compilation task is 12141, and the statistical distribution of memory required for compiling each file can be shown in Table 3.

[0264] Table 3

[0265]

[0266] From this, we can see that only a few compiled files in the compilation task have large memory requirements (compiled files that take a long time to compile).

[0267] In the prior art, during the execution of a compilation task, the number of concurrent subtasks is fixed, and is mostly determined based on the compilation file with the largest memory requirement. In the present application, the number of concurrent subtasks is different according to different compilation files (subtasks). During the execution of a compilation task, the number of concurrent subtasks changes dynamically, so as to execute as many subtasks concurrently as possible. Specifically, Fig.13 As shown, in Fig.13 2 shows the number of concurrent subtasks in the process of executing a compilation task in the prior art, and the number of concurrent subtasks in the process of executing a compilation task in the method provided by the present application.

[0268] Furthermore, if the prior art is used, there is still a problem that only part of the source files in the compilation task are modified, but the compilation task needs to be re-executed. Fig.14 As shown, after a compiling node A1 stops executing compiling task B1, it may be assigned to other compiling tasks B2 for use. When compiling task B1 needs to use compiling node A again, two situations may occur. One is that compiling node A1 is in an idle state, then compiling task B1 can continue to be executed by compiling node A1, but since compiling node A1 may update the code warehouse, clean up the environment cache of the compiling task, and switch the code warehouse branch when executing other compiling tasks B2, these will cause that even if compiling task B1 can continue to be executed by compiling node A, other source files need to be recompiled. The other is that compiling node A1 is in a non-idle state, that is, compiling node A1 is occupied by other compiling tasks B2, then compiling task B1 can only be assigned to other compiling nodes A2 for processing, since compiling node A2 may also have the problem of updating the code warehouse, cleaning up the environment cache of the compiling task, and switching the code warehouse branch when executing other compiling tasks, and since compiling node A2 has not compiled the compiling task B1, the cache hit rate is lower, which may result in the need to compile more source files.

[0269] In the solution provided by this application, since users can select private compilation nodes to perform compilation tasks according to actual needs, and since private compilation nodes and the cache of private compilation nodes are only for users, the above problems can be avoided. Fig. 9 , for a compilation project indicated by a compilation task, it may include component A, component B, component C and component D. After completing version 2.3.6 of the compilation project, if component A needs to be updated, that is, component A needs to be updated to version 3.5.1, only the source code of the updated component A can be compiled, and after the compilation is completed, the 3.5.1 version of the project can be quickly obtained based on the cache of historical components B, C and D and the compilation results for the updated component A, avoiding the problem of having to recompile the source codes of components B, C and D. Fig. 9 The parallel grayscale in this article refers to parallel development and grayscale release. That is, by using the method provided by this application, it is possible to modify one source file and then compile only one source file.

[0270] Furthermore, if the existing technology is used, there is still a problem that the compilation takes a long time. Specifically, it can be shown in Table 4.

[0271] Table 4

[0272]

[0273] Table 4 shows multiple cc1plus processes, such as processes 23507, 23314, 23400, 23443, 23406, etc. Their shared memory is about 5.2g, and 6G memory is allocated to meet the cc1plus process. Judging from the memory usage of these processes, 6G memory can basically meet the running requirements of a single cc1plus process. It is known that in the prior art, the configuration of a single slave compilation node is 8 cores and 16G. Since each cc1plus process consumes about 5.2G memory, 16G memory can accommodate up to about 16÷5.2≈3.08 such processes. But in fact, since the memory usage cannot exceed the total memory of the node to avoid memory overflow problems, each slave compilation node can only accommodate 2 such cc1plus processes.

[0274] The number of cc1plus processes is not only large, but also consumes a lot of memory. Therefore, on a single slave compilation node, due to memory limitations, only two cc1plus processes can be run concurrently. For example, if three cc1plus processes are run at the same time, 3×5.2=15.6G, which is likely to exceed the total memory of 16G, resulting in a memory overflow problem.

[0275] When there are 100+ large cpp files that need to be compiled, since each slave compilation node can only compile 2 cpp files concurrently (due to the memory limit of the cc1plus process), multiple rounds of compilation are required. For example, if there are 100 cpp files, each node can compile 2 files per round. Assuming that there are enough nodes in the system to process 14 files in one round, it will take about 100÷14≈7.14, that is, at least 8 rounds of compilation to complete the compilation of all files. Assuming that each round takes about 150 seconds, it will take 8×150=1200 seconds to complete the compilation of these files.

[0276] When a large module has tens of thousands of source files, if only two source files are updated but 1% of the source files need to be recompiled, due to the above concurrency limit, even if only 100 files are recompiled, it will still be affected by low concurrency. Based on 14 files per round and 150 seconds per round, 100 files will take about 1200 seconds.

[0277] The method provided by the present application splits the first type of sub-files with large resource requirements to obtain multiple secondary sub-tasks, and executes multiple secondary sub-tasks based on multiple slave nodes, thereby improving the efficiency of compilation. That is, by adopting the method provided by the present application, the average compilation time for large modules can be reduced by more than 70%, the median can be reduced by more than 80%, and the average compilation time can be reduced from 4000 seconds to about 2000 seconds, and the average rate of large module online can be increased by 60%.

[0278] Moreover, since the method provided in the present application can effectively improve the cache hit rate, the number of cache misses can be reduced from 500 to 100, and the median can be reduced to single digits, thereby increasing the compilation speed by more than 40%, and reducing the proportion of memory overflow exceptions to 0, making the resulting compilation failure rate 0.

[0279] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0280] Based on the same inventive concept, the embodiment of the present application also provides a compiling task processing device for implementing the compiling task processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more compiling task processing device embodiments provided below can refer to the limitations of the compiling task processing method above, and will not be repeated here.

[0281] In an exemplary embodiment, Fig.15 As shown, a compiling task processing device 1500 is provided, comprising: a receiving module 1501, a first execution module 1502 and a second execution module 1503, wherein:

[0282] The receiving module 1501 is used to receive a compilation request initiated by a user, and identify the type of compilation resource indicated by the user for executing the compilation task according to the compilation request;

[0283] A first execution module 1502 is configured to, in response to identifying that the compilation resource type is a private compilation resource type, call a private compilation node dedicated to the user from a set of private compilation nodes in the compilation node cluster, and use a private cache of the private compilation node that is dedicated to the user to execute the compilation task;

[0284] The second execution module 1503 is used for allocating a public compiling node to the user from a public compiling node cluster shared by multiple users in the compiling node cluster in response to identifying that the compiling node type is a public compiling resource type, calling the public compiling node and utilizing a public cache of the public compiling node to execute the compiling task.

[0285] In one embodiment, the receiving module 1501 is specifically used to determine whether the compilation request carries a private compilation resource parameter value; in response to the compilation request carrying the private compilation resource parameter value, determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type.

[0286] In one embodiment, the receiving module 1501 is specifically used to identify the user type of the user in response to identifying that the compilation resource type is a private compilation resource type; in response to the user type being the first user type, determine that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type; in response to the user type being the second user type, determine that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

[0287] In one embodiment, the first execution module 1502 is specifically configured to, in response to identifying that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, call, from a set of private compilation nodes in the compilation node cluster, a private compilation node that is pre-connected by the user to the compilation node cluster and is exclusive to the user; in response to identifying that the compilation resource type is a private compilation resource type and the compilation request does not carry the private compilation resource parameter value, call, from the set of private compilation nodes in the compilation node cluster, a private compilation node that is allocated to the user by the compilation node cluster and is exclusive to the user.

[0288] In one embodiment, the first execution module 1502 is further used to receive a private compiling node creation request initiated by the user; determine the compiling machine indicated by the private compiling node creation request according to the private compiling node creation request; and add the compiling machine as a private compiling node dedicated to the user to the set of private compiling nodes in the compiling node cluster.

[0289] In one embodiment, the second execution module 1503 is specifically configured to, in response to identifying that the compiling node type is a public compiling resource type, determine the service level of the user according to the service configuration information for the user; in response to the service level being the first service level, determine a first public compiling node set for use by users of the first service level from a public compiling node cluster for use by multiple users in the compiling node cluster, and allocate a public compiling node to the user from the first public compiling node set; in response to the service level being the second service level, determine a second public compiling node set for use by users of the second service level from a public compiling node cluster for use by multiple users in the compiling node cluster, and allocate a public compiling node to the user from the second public compiling node set; wherein the efficiency of responding to the compiling task using the first public compiling node set is greater than the efficiency of responding to the compiling task using the second public compiling node set.

[0290] In one embodiment, the second execution module 1503 is further configured to receive a private compilation node creation request initiated by the user; determine, according to the private compilation node creation request, a compilation node in an idle state from a public compilation node cluster in the compilation node cluster; and add the compilation node in an idle state as a private compilation node exclusively for the user to a set of private compilation nodes in the compilation node cluster.

[0291] In one embodiment, the second execution module 1503 is specifically configured to, in response to identifying that the compilation node type is a public compilation resource type, determine a predicted compilation time consumption of the compilation task; in response to the predicted compilation time consumption satisfying a preset high time consumption judgment condition, allocate a public compilation node to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and configure the public compilation node as a private compilation node exclusively for the user during a period in which the public compilation node executes the compilation task.

[0292] In one embodiment, the second execution module 1503 is specifically used to preprocess the compilation task through the master node in the compilation node cluster to generate subtasks; determine multiple slave nodes for executing the subtasks on the public compilation node allocated to the user; schedule the multiple slave nodes, and use the public cache of the public compilation node to concurrently execute the subtasks preprocessed by the master node.

[0293] In one embodiment, the second execution module 1503 is specifically used to determine, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing a plurality of first-category subtasks in the subtasks preprocessed by the master node; and to determine, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing a plurality of second-category subtasks in the subtasks preprocessed by the master node; wherein the resource requirement of the first-category subtasks is greater than the resource requirement of the second-category subtasks, and a preset number of the plurality of first-category subtasks concurrently executed on a single slave node is less than a preset number of the plurality of second-category subtasks concurrently executed on a single slave node.

[0294] In one embodiment, the second execution module 1503 is specifically used to determine, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of second-category subtasks, based on the constraint condition, the number of subtasks of the plurality of concurrent second-category subtasks and a preset number of second-category subtasks concurrently executed on a single slave node, with the concurrency number of subtasks limited by the preprocessing speed of the master node as the constraint condition.

[0295] In one embodiment, the second execution module 1503 is specifically used to split each subtask of the multiple first-category subtasks into multiple concurrent second-category subtasks; and determine multiple slave nodes for concurrently executing the second-category subtasks split from the multiple first-category subtasks according to the total number of second-category subtasks split from the multiple first-category subtasks and the preset number of first-category subtasks concurrently executed on a single slave node.

[0296] In one embodiment, the second execution module 1503 is specifically used to obtain a permission management account for accessing the data disk of the public compilation node; synchronize the permission management account to the multiple slave nodes used to execute the subtask; and use the permissions of the permission management account to perform read and write operations on the data disk during the process of calling the multiple slave nodes and concurrently executing the subtasks preprocessed by the master node using the public cache of the public compilation node.

[0297] Each module in the above-mentioned compilation task processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0298] In an exemplary embodiment, a server cluster is provided, and the internal structure diagram of any server in the server cluster can be as follows: Fig.16As shown. The server includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the server is used to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the server is used to store data. The input / output interface of the server is used to exchange information between the processor and an external device. The communication interface of the server is used to communicate with an external terminal through a network connection. When the server in the server cluster is executed by the processor, a compilation task processing method is implemented.

[0299] Those skilled in the art will understand that Fig.16 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0300] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps described in the above-mentioned compilation task processing method embodiments when executing the computer program.

[0301] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps described in the above-mentioned compilation task processing method embodiments are implemented.

[0302] In one embodiment, a computer program product is provided, including a computer program, which implements the steps described in the above-mentioned compilation task processing method embodiments when executed by a processor.

[0303] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0304] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0305] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0306] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A compilation task processing method, characterized in that: The method comprises: Receiving a compilation request initiated by a user, and identifying, according to the compilation request, a compilation resource type indicated by the user for executing a compilation task; In response to identifying that the compilation resource type is a private compilation resource type, calling a private compilation node dedicated to the user from a set of private compilation nodes in a compilation node cluster, and using a private cache of the private compilation node dedicated to the user to execute the compilation task; In response to identifying that the compiling node type is a public compiling resource type, a public compiling node is allocated to the user from a public compiling node cluster shared by multiple users in the compiling node cluster, the public compiling node is called and a public cache of the public compiling node is used to execute the compiling task.

2. The method according to claim 1, characterized in that: The step of identifying, according to the compilation request, the type of compilation resource indicated by the user for executing the compilation task comprises: Determining whether the compilation request carries a private compilation resource parameter value; In response to the compilation request carrying the private compilation resource parameter value, it is determined that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type.

3. The method according to claim 2, characterized in that The step of identifying the type of compilation resource indicated by the user for executing the compilation task according to the compilation request further includes: In response to the compilation request not carrying the private compilation resource parameter value, identifying a user type of the user; In response to the user type being the first user type, determining that the compilation resource type indicated by the user for executing the compilation task is a private compilation resource type; In response to the user type being the second user type, it is determined that the compilation resource type indicated by the user for executing the compilation task is a public compilation resource type.

4. The method according to claim 1, characterized in that In response to identifying that the compilation resource type is a private compilation resource type, calling the user-specific private compilation node from a set of private compilation nodes in a compilation node cluster includes: In response to identifying that the compilation resource type is a private compilation resource type and the compilation request carries a private compilation resource parameter value, calling, from a set of private compilation nodes in the compilation node cluster, a private compilation node that is pre-connected by the user to the compilation node cluster and is exclusive to the user; In response to identifying that the compilation resource type is a private compilation resource type and the compilation request does not carry the private compilation resource parameter value, calling a private compilation node allocated to the user by the compilation node cluster and dedicated to the user from the set of private compilation nodes in the compilation node cluster.

5. The method according to claim 4, characterized in that The method further comprises: Receiving a private compilation node creation request initiated by the user; Determining, according to the private compiling node creation request, a compiling machine indicated by the private compiling node creation request; The compiler is used as a private compiling node dedicated to the user and added to a set of private compiling nodes in the compiling node cluster.

6. The method according to claim 1, characterized in that In response to identifying that the compiling node type is a public compiling resource type, allocating a public compiling node to the user from a public compiling node cluster shared by multiple users in the compiling node cluster includes: In response to identifying that the compiling node type is a public compiling resource type, determining a service level for the user according to service configuration information for the user; In response to the service level being the first service level, determining a first public compiling node set for common use by users of the first service level from a public compiling node cluster for common use by multiple users in the compiling node cluster, and allocating a public compiling node to the user from the first public compiling node set; In response to the service level being a second service level, determining a second set of public compiling nodes for use by users of the second service level from a public compiling node cluster for use by multiple users in the compiling node cluster, and allocating a public compiling node to the user from the second set of public compiling nodes; The efficiency of using the first set of public compiling nodes to respond to the compiling task is greater than the efficiency of using the second set of public compiling nodes to respond to the compiling task.

7. The method according to claim 6, characterized in that The method further comprises: Receiving a private compilation node creation request initiated by the user; Determining, according to the private compiling node creation request, a compiling node in an idle state from a public compiling node cluster in the compiling node cluster; The compiling node in the idle state is used as a private compiling node dedicated to the user, and is added to a set of private compiling nodes in the compiling node cluster.

8. The method according to claim 1, characterized in that In response to identifying that the compiling node type is a public compiling resource type, allocating a public compiling node to the user from a public compiling node cluster shared by multiple users in the compiling node cluster includes: In response to identifying that the compiling node type is a public compiling resource type, determining a predicted compiling time consumption of the compiling task; In response to the predicted compilation time satisfying a preset high time consumption judgment condition, a public compilation node is allocated to the user from a public compilation node cluster shared by multiple users in the compilation node cluster, and the public compilation node is configured as a private compilation node exclusively for the user during a period in which the public compilation node executes the compilation task.

9. The method according to claim 1, characterized in that: The calling the public compiling node and utilizing the public cache of the public compiling node to execute the compiling task includes: Preprocessing the compilation task through a master node in the compilation node cluster to generate subtasks; On a common compiling node allocated to the user, determining a plurality of slave nodes for executing the subtask; The multiple slave nodes are scheduled to concurrently execute the subtasks obtained by preprocessing by the master node using the common cache of the common compiling node.

10. The method according to claim 9, characterized in that The step of determining, on the common compiling node allocated to the user, a plurality of slave nodes for executing the subtask comprises: For a plurality of concurrent first-category subtasks among the subtasks preprocessed by the master node, determining, on a common compiling node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of first-category subtasks; For a plurality of concurrent second-category subtasks among the subtasks preprocessed by the master node, determining, on a common compilation node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of second-category subtasks; Among them, the resource requirement of the first type of subtask is greater than the resource requirement of the second type of subtask, and the preset number of the multiple first type subtasks executed concurrently on a single slave node is less than the preset number of the multiple second type subtasks executed concurrently on a single slave node.

11. The method according to claim 10, characterized in that The step of determining, on the common compiling node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of second-category subtasks comprises: On the common compilation node allocated to the user, the concurrent number of subtasks limited by the preprocessing speed of the master node is used as a restriction condition, and based on the restriction condition, the number of subtasks of the concurrent multiple second-category subtasks and the preset number of second-category subtasks concurrently executed on a single slave node, a plurality of slave nodes for concurrently executing the multiple second-category subtasks are determined.

12. The method according to claim 11, characterized in that The step of determining, on the common compiling node allocated to the user, a plurality of slave nodes for concurrently executing the plurality of first-category subtasks comprises: Splitting each of the plurality of first-category subtasks into a plurality of concurrent second-level subtasks; According to the total number of secondary subtasks split from the multiple first-category subtasks and the preset number of primary subtasks concurrently executed on a single slave node, multiple slave nodes for concurrently executing the secondary subtasks split from the multiple first-category subtasks are determined.

13. The method according to any one of claims 9 to 12, characterized in that: The scheduling of the plurality of slave nodes, using the common cache of the common compiling node, to concurrently execute the subtasks obtained by the preprocessing of the master node, includes: Obtaining a permission management account for accessing the data disk of the public compilation node; Synchronizing the rights management account to the plurality of slave nodes for executing the subtasks; In the process of calling the multiple slave nodes and concurrently executing the subtasks obtained by preprocessing by the master node using the public cache of the public compiling node, the data disk is read and written using the authority of the authority management account.

14. A compilation task processing device, characterized in that: The device comprises: A receiving module, configured to receive a compilation request initiated by a user, and identify, according to the compilation request, a type of compilation resource indicated by the user for executing a compilation task; a first execution module, configured to, in response to identifying that the compilation resource type is a private compilation resource type, call a private compilation node dedicated to the user from a set of private compilation nodes in a compilation node cluster, and execute the compilation task by using a private cache of the private compilation node that is dedicated to the user; The second execution module is configured to, in response to identifying that the compiling node type is a public compiling resource type, allocate a public compiling node to the user from a public compiling node cluster shared by multiple users in the compiling node cluster, call the public compiling node and use a public cache of the public compiling node to execute the compiling task.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 13 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images