Continuous integration scheduling method and device, equipment, storage medium and program product

Abstract

The invention relates to the technical field of continuous integration, and particularly provides a continuous integration scheduling method and device, equipment, a storage medium and a program product. The method comprises the steps of receiving an operation request of a to-be-operated task; in response to a running request of a to-be-run task, sending a node information acquisition request carrying an application type of the to-be-run task to the engine scheduling module; receiving node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the to-be-operated task, the current operation state data of the plurality of continuous integration engine slave nodes, the preset operation data and the predicted operation data; and scheduling the to-be-run task from the node to the target continuous integration engine by using the node information. According to the method, the tasks can be reasonably scheduled to the corresponding nodes, the task execution efficiency is improved, the node downtime risk is reduced, and continuous integration engine resources are reasonably utilized.

Description

Technical Field

[0001] The present disclosure relates to the technical field of continuous integration, and in particular to a scheduling method, apparatus, device, storage medium, and program product for continuous integration. Background Art

[0002] Continuous Integration (CI) is part of the DevOps (Development Operations, software development and information technology operations) practice. It accelerates defect discovery and resolution, improves software quality, and shortens release time through automated building and testing.

[0003] In related technologies, the resource utilization rate of the automated operation and maintenance deployment system for scheduling continuous integration engine nodes is low, which easily causes load imbalance problems and leads to a high risk of continuous integration engine node downtime. Summary of the Invention

[0004] In view of the above problems, the present disclosure is proposed. The present disclosure provides a continuous integration scheduling method, apparatus, device, storage medium and program product.

[0005] According to one aspect of the present disclosure, a continuous integration scheduling method is provided, which is applied to a continuous integration engine master node in an automated operation and maintenance deployment system, wherein the automated operation and maintenance deployment system further includes an engine scheduling module and a plurality of continuous integration engine slave nodes associated with the continuous integration engine master node. The method comprises:

[0006] Receive a request to run a task to be run;

[0007] In response to the running request of the task to be run, sending a node information acquisition request to the engine scheduling module; wherein the node information acquisition request carries the application type of the task to be run;

[0008] Receiving node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the task to be run, current running status data of the multiple continuous integration engine slave nodes, preset running data, and predicted running data;

[0009] The node information is used to schedule the to-be-run task to the target continuous integration engine slave node.

[0010] According to another aspect of the present disclosure, a continuous integration scheduling device is provided, which is applied to a continuous integration engine master node in an automated operation and maintenance deployment system. The automated operation and maintenance deployment system also includes an engine scheduling module and a plurality of continuous integration engine slave nodes associated with the continuous integration engine master node. The device includes:

[0011] A communication module, used for receiving a request to execute a task to be executed;

[0012] The communication module is further configured to send a node information acquisition request to the engine scheduling module in response to the execution request of the task to be executed; wherein the node information acquisition request carries the application type of the task to be executed;

[0013] The communication module is further configured to receive node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the task to be run, the current running status data of the plurality of continuous integration engine slave nodes, the preset running data, and the predicted running data;

[0014] A processing module is used to schedule the to-be-run task to the target continuous integration engine slave node by using the node information.

[0015] According to another aspect of the exemplary embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described in the exemplary embodiments of the present disclosure.

[0016] According to another aspect of the exemplary embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method described in the exemplary embodiments of the present disclosure is implemented.

[0017] According to another aspect of the exemplary embodiments of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method described in the exemplary embodiments of the present disclosure is implemented.

[0018] As will be described in detail below, according to the continuous integration scheduling method of the embodiment of the present disclosure, a running request for a task to be run is received; in response to the running request for the task to be run, a node information acquisition request is sent to the engine scheduling module; wherein the node information acquisition request carries the application type of the task to be run; and node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request is received; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the task to be run, current running status data of multiple continuous integration engine slave nodes, preset running data, and predicted running data; and the node information is used to schedule the task to be run to the target continuous integration engine slave node. When determining the target continuous integration engine slave node from multiple continuous integration engine slave nodes, the application type of the task to be run can be combined with the current running status data, preset running data, and predicted running data of each continuous integration engine slave node to reasonably schedule tasks of different application types in complex scenarios to corresponding nodes, thereby avoiding the situation where some nodes are overloaded and some nodes are idle, effectively improving the execution efficiency of tasks, reducing the risk of downtime of the continuous integration engine slave node, and reasonably utilizing the continuous integration engine resources.

[0019] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other purposes, features, and advantages of the present disclosure will become more apparent through a more detailed description of the embodiments of the present disclosure in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and are not intended to limit the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 A schematic structural diagram of a continuous integration engine system provided by an exemplary embodiment of the present disclosure is shown;

[0022] Figure 2 The main flow chart of the continuous integration complete service provided by the embodiment of the present disclosure is shown;

[0023] Figure 3 A schematic diagram of a process for determining a target continuous integration engine slave node according to an embodiment of the present disclosure is shown;

[0024] Figure 4 A timing diagram of a process for determining a target continuous integration engine slave node provided by an embodiment of the present disclosure is shown;

[0025] Figure 5shows a timing diagram of creating a task configuration file in the related art;

[0026] Figure 6 Shows a creation sequence diagram and a running sequence diagram of a task configuration file provided by an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram of the process of multi-version plug-in task control provided by an embodiment of the present disclosure is shown;

[0028] Figure 8 A flow chart of a continuous integration scheduling method provided by an exemplary embodiment of the present disclosure is shown;

[0029] Figure 9 A schematic structural diagram of a continuous integration scheduling device provided by an exemplary embodiment of the present disclosure is shown;

[0030] Figure 10 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure is shown;

[0031] Figure 11 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present disclosure more apparent, the following will describe in detail exemplary embodiments of the present disclosure with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0033] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0034] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0035] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0036] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0037] Cloud native is a software development and deployment methodology that combines cloud computing, containers, microservices, and DevOps (software development and information technology operations) technologies to create highly scalable, reliable, and maintainable applications. This methodology improves development efficiency and application stability by automating the build, testing, and deployment processes. Cloud native applications typically adopt a distributed architecture to ensure high availability and load balancing.

[0038] Continuous Integration (CI) is part of DevOps practices. It accelerates defect discovery and resolution, improves software quality, and shortens release time through automated builds and testing. CI engines simplify the integration process, allowing developers to focus on business logic. In cluster mode, CI tasks are scheduled by the master node, executed by slave nodes, and accessed by users through a web interface. Dynamic parameter passing increases the flexibility of the CI engine, allowing parameters to be passed at runtime, eliminating the need to modify engine templates due to parameter changes. Plug-in version control ensures smooth upgrades of plug-ins in CI tasks without affecting existing functionality. Together, these technologies promote automation and efficiency in software development.

[0039] In the field of continuous integration, a single worker node often cannot meet the needs of large-scale concurrent scenarios, which requires multiple nodes to work in a cluster. Therefore, a reasonable cluster load balancing solution can greatly improve the efficiency of the entire continuous integration process. Related existing technical solutions are as follows:

[0040] In one approach, the automated operation and maintenance deployment system first checks the health of the Jenkins cluster, creates and assigns a build task to a designated master node. If the master node is unable to handle the task, the automated operation and maintenance deployment system automatically migrates the task to another master node. Both master nodes are pre-designated in the Jenkins cluster by the administrator or elected through an election mechanism that considers the number of idle executors and memory utilization.

[0041] This solution has the following flaws: 1) The load balancing algorithm simply combines the node's CPU (Central Processing Unit) memory usage for scheduling; 2) It cannot properly schedule certain tasks that rely on node cache; 3) Even for nodes with the same configuration, different processor models and network IO (Input / Output) may lead to differences in node performance. In this scenario, existing solutions cannot distinguish and schedule all nodes uniformly; 4) It cannot implement scheduling of specific nodes. For example, tasks under the ARM (Advanced RISC Machines) architecture can only be scheduled to work on ARM structure nodes.

[0042] In another solution, the automated operation and maintenance deployment system obtains build trigger data from the DevOps continuous integration platform and displays it on the pipeline configuration page. Then, based on the preset scheduling strategy, the automated operation and maintenance deployment system will bind and schedule the selected build machine and implement a queuing mechanism to optimize resource utilization and meet concurrency requirements. The build machine scheduler is responsible for scheduling the pipeline execution engine, generating task queues, driving pipeline execution, arranging task execution, and dynamically displaying the build machine's operating status and task progress. This process is designed to improve the efficiency, transparency, responsiveness, and resource utilization of the continuous integration platform, ensure that build tasks can be executed efficiently, and thus maximize the performance of the automated operation and maintenance deployment system.

[0043] This solution has the following defects: the resource scheduling of this solution is also combined with the hardware resources of the server node, and the scheduling solution is not accurate; there is no control over resource consumption for special tasks in characteristic scenarios, which can easily lead to the exhaustion of resources of the entire node and thus downtime.

[0044] In summary, scheduling continuous integration tasks in a cloud-native environment faces multiple challenges. First, when using round-robin scheduling, resource allocation can be uneven due to differences in cluster node resource configurations, impacting efficiency. Second, specific tasks, such as Android builds, rely on specific node caches, and ARM architecture tasks must run on ARM nodes, requiring specialized scheduling strategies.

[0045] Furthermore, the execution results of the same task may vary under different hardware resources and network IO conditions, making improving task success rates a major challenge. Furthermore, different application types, such as Java, Python, and Vue (Vue.js, pronounced similarly to "view," meaning "view"), have different resource requirements, making proper resource allocation to prevent node overload crucial. Task build times vary, and resource consumption fluctuates, making it difficult to accurately predict. Finally, unrestricted tasks can exhaust node resources, impacting the execution of other tasks and even causing node downtime.

[0046] In response to the above problems, the embodiments of the present disclosure provide a scheduling method for continuous integration, which optimizes the scheduling of container-based continuous integration tasks in complex cloud-native scenarios. It is a load balancing solution that combines hardware resources, caching mechanism, and scoring mechanism, a scheduling solution that estimates resource usage, and a solution that limits resource consumption when tasks are run. It can reasonably schedule tasks of different application types in complex scenarios to corresponding nodes, avoid excessive load on some nodes and idle operation of some nodes, effectively improve the execution efficiency of tasks, reduce the risk of continuous integration engine node downtime, and reasonably utilize the continuous integration engine resources.

[0047] Figure 1 FIG. 1 shows a schematic diagram of the structure of a continuous integration engine system provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the continuous integration engine system includes an engine scheduling module and a continuous integration engine cluster, and the continuous integration engine cluster includes a continuous integration engine master node and multiple continuous integration engine slave nodes associated with the continuous integration engine master node. The number of continuous integration engine slave nodes can be two, three, or even more, depending on the actual application scenario, and the embodiment of the present disclosure does not specifically limit this. In the method of the embodiment of the present disclosure, the number of continuous integration engine slave nodes can be m, where m is a natural number greater than or equal to 2.

[0048] like Figure 1 As shown, the disclosed embodiment can run all scheduling services in Docker containers, maintaining the uniformity of the application runtime environment. The continuous integration engine slave node (such as Slave, Execution Machine) uses a virtual machine environment, and other system service components all run in Docker containers.

[0049] The engine scheduling module is a continuous integration task workflow management and continuous integration engine cluster control management module. Before the continuous integration task to be run is executed, the continuous integration engine master node will call back the relevant interface of the engine scheduling module to determine the target continuous integration engine slave node ( Figure 1 1.1 in ), and then return the node information of the target continuous integration engine slave node to the continuous integration engine master node ( Figure 1 1.2 in the target continuous integration engine master node schedules the task to be run to the target continuous integration engine slave node for execution ( Figure 1 2.1) and returns the result of the task to be run ( Figure 1 2.2).

[0050] Figure 2 The main flow chart of the continuous integration complete service provided by the embodiment of the present disclosure is shown. Figure 2As shown, in the embodiment of the present disclosure, tasks to be run can be divided according to application type, and tasks to be run of multiple application types can be obtained, such as type A tasks, type B tasks, type C tasks, type D tasks, type E tasks, and type F tasks. For example, the application type can include but is not limited to multiple types of Java, Python, Golang, Vue, Android, and C.

[0051] Before determining the target continuous integration engine slave node, the disclosed embodiment sets the following preconditions: First, set the upper limit of containerized running resources (including the number of running cores and running memory) for the tasks to be run of different application types. For example, the upper limit of containerized running resources for Python type applications is 1H1G (i.e., 1 CPU core and 1GB of memory), the upper limit of containerized running resources for Java type applications is 1H2G, and the upper limit of containerized running resources for Vue type applications is 2H4G.

[0052] Second, set the preset upper limit of containerized running resources (including the preset upper limit of running cores and the preset upper limit of running memory) and the corresponding preset upper limit of running tasks for each continuous integration engine slave node. For example, when the preset upper limit of containerized running resources of the continuous integration engine slave node is set to 4H8G, the corresponding preset upper limit of running tasks is set to 5; when the preset upper limit of containerized running resources of the continuous integration engine slave node is set to 8H16G, the corresponding preset upper limit of running tasks is set to 15.

[0053] Third, set separate tags for tasks of a preset application type. For example, if a task requires cache or ARM architecture, all tasks of the preset application type are run on the same continuous integration engine slave node. The type of preset application type can be selected based on the actual application scenario, and this embodiment of the disclosure does not specifically limit this. The continuous integration engine slave node that runs tasks of the preset application type can be pre-set to the preset continuous integration engine slave node corresponding to the preset application type.

[0054] Based on this, the multiple continuous integration engine slave nodes in the embodiment of the present disclosure may include at least one preset continuous integration engine slave node and multiple non-preset continuous integration engine slave nodes; wherein, at least one preset continuous integration engine slave node is used to run tasks of a corresponding number of preset application types, and the number of non-preset continuous integration engine slave nodes is at least two. The specific number is selected according to actual conditions, and the embodiment of the present disclosure does not make any specific restrictions on this.

[0055] After receiving the execution instruction of the task to be run, the continuous integration engine master node requests the engine scheduling module to determine the target continuous integration engine slave node from the multiple continuous integration engine slave nodes associated with the continuous integration engine master node, so as to schedule the task to be run to the target continuous integration engine slave node for execution.

[0056] Figure 3 FIG. 1 shows a flow chart of determining a target continuous integration engine slave node provided by an embodiment of the present disclosure. Figure 3 As shown, the running instruction of the task to be run can carry the application type of the task to be run. The continuous integration engine master node can send a node information acquisition request generated based on the application type of the task to be run to the engine scheduling module. The engine scheduling module responds to the node information acquisition request and determines the target continuous integration engine slave node in the following steps:

[0057] Step 301: The engine scheduling module receives a node information acquisition request and determines the target continuous integration engine slave node that is most suitable for running the task to be run from multiple continuous integration engine slave nodes associated with the continuous integration engine master node.

[0058] Step 302: The engine scheduling module determines whether the application type of the task to be run matches one of at least one preset application type, so as to determine whether it needs to be directed to a fixed continuous integration engine slave node (i.e., the preset continuous integration engine slave node corresponding to the preset application type) according to the task requirements.

[0059] Here, whether the application type of the task to be run matches one of the at least one preset application types can be understood as: when the number of preset application types is one, whether the application type of the task to be run is the same as the preset application type; when the number of preset application types is multiple, whether the application type of the task to be run is one of the multiple preset application types.

[0060] Step 303: If the application type of the task to be executed matches one of the at least one preset application type, the preset continuous integration engine slave node corresponding to the preset application type is determined as the target continuous integration engine slave node. Because the resources of these preset continuous integration engine slave nodes are limited, the tasks to be executed by scheduling them to these preset continuous integration engine slave nodes will be queued. In this case, the preset continuous integration engine slave node corresponding to the preset application type can be determined as the target continuous integration engine slave node, and step 307 is executed.

[0061] Step 304: If the application type of the task to be run does not match one of the at least one preset application type, obtain the current containerized running resource value (including the current number of running cores and the current running memory), the current number of running tasks and the current running failure rate of each non-preset continuous integration engine slave node, and then execute step 305.

[0062] Step 305: Based on each current containerized operation resource value, combined with the predicted containerized operation resource usage rate (including the predicted operating core ratio and the predicted operating memory ratio) after each non-preset continuous integration engine slave node runs the task to be run, and based on the current containerized operation resource value, the predicted containerized operation resource usage rate and the current operation failure rate, calculate the fault weight score of each non-preset continuous integration engine slave node; based on the fault weight scores of multiple non-preset continuous integration engine slave nodes, select the most suitable candidate continuous integration engine slave node from multiple non-preset continuous integration engine slave nodes. For example, the non-preset continuous integration engine slave node with the lowest fault weight score among multiple non-preset continuous integration engine slave nodes is determined as the candidate continuous integration engine slave node. Then, execute step 306.

[0063] Here, if the number of currently running tasks of the non-preset continuous integration engine slave node is equal to the preset running task upper limit, the non-preset continuous integration engine slave node does not participate in the determination of the candidate continuous integration engine slave node.

[0064] Step 306: Determine whether the candidate continuous integration engine slave node is the target continuous integration engine slave node. If the candidate continuous integration engine slave node is the target continuous integration engine slave node, execute step 307; otherwise, execute step 308.

[0065] Here, the embodiment of the present disclosure can determine the candidate continuous integration engine slave node whose predicted containerized operation resource utilization rate (including the predicted operating core ratio and the predicted operating memory ratio) is less than or equal to the preset containerized operation resource utilization rate as the target continuous integration engine slave node. The above-mentioned preset containerized operation resource utilization rate can be selected according to the actual application scenario, and the embodiment of the present disclosure does not specifically limit this. In the method of the embodiment of the present disclosure, the preset containerized operation resource utilization rate is 80%.

[0066] Step 307: Return the node information of the target continuous integration engine slave node to the continuous integration engine master node, so that the continuous integration engine master node dynamically schedules the to-be-run task to the continuous integration engine slave node for execution based on the node information of the target continuous integration engine slave node.

[0067] Step 308: If the candidate continuous integration engine slave node is not the target continuous integration engine slave node, that is, the predicted containerized operation resource usage of the candidate continuous integration engine slave node is greater than the preset containerized operation resource usage, then it enters the queue waiting and performs round-robin calculation until the target continuous integration engine slave node is determined, and then executes step 307.

[0068] Here, the target continuous integration engine slave node may be determined according to steps 304 to 306 above, or a non-preset continuous integration engine slave node may be randomly designated as the target continuous integration engine slave node.

[0069] The continuous integration engine master node can also score the running results of the target continuous integration engine slave node running the to-be-run task, obtain the running scoring results of the target continuous integration engine slave node running the to-be-run task, and store the running scoring results in the scheduling engine module.

[0070] The process sequence diagram of determining the target continuous integration engine slave node involved in the above steps 301 to 308 is as follows: Figure 4 As shown, Figure 4 FIG. 1 shows a flow chart of determining a target continuous integration engine slave node provided by an embodiment of the present disclosure. Figure 4 As shown, the user initiates a task to be run (step 401). When the continuous integration engine master node receives the execution command of the task to be run, it requests the engine scheduling module to calculate the optimal scheduling node (i.e., the target continuous integration engine slave node). It can send a node information acquisition request to the engine scheduling module (step 402). The engine scheduling module calculates the target continuous integration engine slave node (steps 4021 to 4023, which can be combined with Figure 3 After calculating the content of the target continuous integration engine slave node, the node information of the target continuous integration engine slave node is returned to the continuous integration engine master node; and the continuous integration engine master node schedules the task to be run to the target continuous integration engine slave node for execution (step 403).

[0071] When the engine scheduling module calculates the target continuous integration engine slave node, the engine scheduling module can query the physical status information of the continuous integration engine slave node in real time, that is, the current containerized running resource value (including the current running core number and the current running memory), the current number of running tasks and the current running failure rate mentioned above. At the same time, it will also calculate the maximum value of the resources that the task may occupy based on the application type of the task to be run (that is, the preset containerized running resource value corresponding to the application type of the task to be run, including the preset number of running cores and the preset running memory); the engine scheduling module will predict the containerized running resource utilization rate and failure rate of each continuous integration engine slave node after running the task to be run, and obtain the predicted containerized running resource utilization rate (including the predicted running core ratio and the predicted running memory ratio). Then, based on the current containerized running resource value, the predicted containerized running resource utilization rate and the current running failure rate, the fault weight score of each non-preset continuous integration engine slave node is calculated. Among them, the calculation formula for the fault weight score is as follows:

[0072]

[0073] Wherein, Ci represents the predicted running core ratio of the non-preset continuous integration engine from node i, Mi represents the predicted running memory ratio of the non-preset continuous integration engine from node i, Si represents the current running failure rate of the non-preset continuous integration engine from node i, and x represents the dynamic threshold, which is equal to 60%.

[0074] Taking the task scheduling of the Vue application type as an example for analysis, the preset number of running cores for the Vue application type task is 2H and the preset running memory is 4G. Table 1 shows the real-time running data of the four non-preset continuous integration engine slave nodes of the embodiment of the present disclosure. As shown in Table 1, the engine scheduling module obtains the real-time running data of the four non-preset continuous integration engine slave nodes, which may include the current number of running cores, the current running memory and the current number of running tasks. It may also include the preset upper limit of the number of running cores, the preset upper limit of running memory and the preset upper limit of running tasks of the non-preset continuous integration engine slave node, as well as the predicted running core ratio, predicted running memory ratio and current running failure rate after running the Vue application type task.

[0075] Table 1 Real-time running data of 4 non-preset continuous integration engine slave nodes

[0076] Slave Node 1 Slave Node 2 Slave Node 3 Slave Node 4 Current number of running cores 8H 2H 6H 16H Preset upper limit of running cores 16H 32H 8H 64H Current running memory 15G 4G 8G 24G Preset running memory limit 32G 64G 16G 128G The number of currently running tasks 2 1 3 5 Preset running task limit 5 8 3 15 Current operating failure rate 10％ 20％ 15％ 17％ Predicted running core ratio 62.5％ 12.5％ The task is full, not attempting to load 28.125％ Predicted running memory usage 59.4％ 12.5％ The task is full, not trying to load 21.875％ Fault Weight Scoring 56.35 14 Not included in calculation 23.71

[0077] As can be seen from Table 1, the number of currently running tasks of slave node 3 is 3, which is equal to the preset upper limit of running tasks. Therefore, the Vue application type task is not attempted to be loaded on the slave node 3. At the same time, according to the above calculation formula for fault weight score, it can be calculated that the fault weight score of slave node 1 is 56.35, the fault weight score of slave node 2 is 14, and the fault weight score of slave node 4 is 23.71. It can be seen that the fault weight score of slave node 2 is the lowest. Therefore, slave node 2 is determined to be the running Vue application type task. e Candidate continuous integration engine slave nodes for application type tasks; and since the predicted running core ratio and the predicted running memory ratio of slave node 2 are both less than 80%, slave node 2 can be determined as the target continuous integration engine slave node.

[0078] Figure 5 FIG. 1 shows a timing diagram of creating a task configuration file in the related art. Figure 5As shown, first, the user sends a creation request for the task to be run to the business platform (step 501). After receiving the creation request, the business platform queries the business parameters of the task to be run (step 502); then, it sends a generation instruction for the task to be run to the continuous integration engine master node (step 503). In the process of generating the task to be run, the continuous integration engine master node deeply binds the business parameters with the preset task configuration template (pre-stored in the continuous integration engine master node), obtains the task configuration file of the task to be run (step 504) and returns after it is stored on the disk (step 505), and the creation of the task to be run is completed; when the task run request initiated by the user is obtained, the task to be run is run and the running result is returned.

[0079] This method of creating task configuration files for tasks to be run has the following drawbacks: business parameters are deeply bound to the preset task configuration templates. Once there are changes in business parameters on the business platform side, it is necessary to call the task engine interface to adjust the preset task configuration template. This not only increases the access traffic to the continuous integration engine master node, but also makes the integration of various middleware on the business platform more difficult. At the same time, the task configuration files will eventually be stored on the disk space. Over time, if the disk space is insufficient, you must be particularly cautious when you want to clean up the disk space, because once the task configuration files written to the disk are deleted, it is difficult to recover from the source data. Furthermore, the long-term storage of task configuration files on the disk will greatly affect the scheduling efficiency of the continuous integration engine master node, because the continuous integration engine master node will scan the disk to obtain relevant data during the scheduling process.

[0080] Based on this, the embodiment of the present disclosure introduces a dynamic parameter transmission mechanism, which eliminates the need for business parameters to be deeply bound to the preset task configuration template, thereby streamlining the generation process of the task configuration file of the continuous integration engine master node.

[0081] Figure 6 FIG1 shows a creation sequence diagram and a running sequence diagram of a task configuration file provided by an embodiment of the present disclosure. Figure 6 As shown, the user sends a request to create a task to be run to the business platform (step 601). After receiving the request, the business platform directly forwards the request to the continuous integration engine master node (step 602). Based on the request, the continuous integration engine master node generates a task configuration file for the task to be run (step 603), writes the task configuration template to the disk as the task configuration file for the task to be run (step 604), and returns it.

[0082] The user sends a request to the business platform to execute the task to be run (step 605). Based on the request, the business platform dynamically generates the business parameters for the task to be run (step 606) and sends the business parameters to the continuous integration engine master node (step 607). The continuous integration engine master node receives the business parameters for the task to be run and retrieves the task configuration file for the task to be run from disk (step 608). It then runs the task to be run using the business parameters and the task configuration template for the task to be run.

[0083] It can be seen that the embodiment of the present disclosure stores the business parameters in the business platform, and dynamically transmits the business parameters to the continuous integration engine master node when the task to be run is running. Figure 5 Compared with the method shown, the method of the embodiment of the present disclosure has the following advantages: it reduces the coupling degree between the business platform and the continuous integration engine main node. Even if the business parameters of the task to be run on the business platform side are modified and adjusted, there is no need to change the task configuration template generated by the continuous integration engine main node and stored on the disk; since the business parameters are not written to the disk, the task configuration template can be quickly generated by the continuous integration engine main node. Therefore, in the case of insufficient disk space, the disk can be cleaned up at will; since a large number of disk input and output operations are reduced during engine scheduling, the operating efficiency of the continuous integration engine main node is improved.

[0084] In related technologies, automated operations and deployment systems use Jenkins plugins to initialize the build environment, receive and parse application configuration files to determine the required plugin version. The system then retrieves and installs the corresponding version of the business module code into the build environment, and executes the corresponding business logic to build and output the application file package. This approach allows different versions of business modules to be managed and built within a single plugin, enhancing the plugin's adaptability and reusability while reducing maintenance costs.

[0085] However, this related solution has the following defects: the plug-in in this related solution is based on the Jenkins continuous integration engine, and lacks task-level plug-ins; in other words, this related solution is useless on other non-Jenkins continuous integration engines; at the multi-version control level, this related solution receives the plug-in version number in the task configuration file to load the plug-in while the environment is being built and initialized, and cannot dynamically obtain the plug-in, which also causes many inconveniences in the later expansion of this solution.

[0086] As can be seen, in the traditional task orchestration process, all steps are not managed using a plug-in approach. This means that even the slightest change to a pending task requires adjusting the task configuration template. In other words, if a step in a pending task needs to be upgraded or adjusted, the entire task configuration template must be adjusted to adapt. Therefore, existing tasks must be modified before they can be run. This brings great inconvenience to task orchestration and maintenance, and dramatically increases maintenance costs.

[0087] In the method of the embodiment of the present disclosure, a plug-in approach is adopted to orchestrate management tasks. In the configuration process of the plug-in task, each step in the task is completed by a corresponding plug-in, and the entire task can be understood as the product of the orchestration of multiple different plug-ins. On the one hand, this solution simplifies the task orchestration process, and on the other hand, it reduces the degree of coupling between tasks and businesses. In the process of upgrading certain plug-ins, the embodiment of the present disclosure introduces a multi-version plug-in control solution to ensure the compatibility of old and new versions of plug-ins in both the original tasks and the new tasks.

[0088] The principle behind this approach is to manage GitLab repositories using multiple branches. A plugin is a specific function within a branch of a GitLab repository. By leveraging the multi-branch concept of repository management, multiple versions can be managed. Plugin upgrades can be thought of as creating new branches based on improvements to the old version.

[0089] In the method of the present embodiment, the configuration process of the multi-version plug-in task occurs after the business platform receives a request from the user to create a task to be run.

[0090] Figure 7 A flow chart of multi-version plug-in task control provided by an embodiment of the present disclosure is shown as follows: Figure 7 As shown in the figure, the entire steps of the multi-version plug-in task control process are summarized as follows:

[0091] After receiving a request from a user to create a pending task, the business platform initiates the task creation operation. When a pending task is first created, it orchestrates the task and selects plugins A and B. The version information for the pending task, plugins A, and plugin B are all v1. This information is then stored in the GitLab repository. After the continuous integration engine master node obtains the version information for the pending task, plugins A, and plugin B, it requests the corresponding versions of plugins A and B from the GitLab repository (i.e., v1 of plugin A and v1 of plugin B). It then assembles the v1 version of the pending task and runs it.

[0092] If the business scenario changes, the business platform needs to upgrade plugin A, but not plugin B. After upgrading plugin A from v1 to v2, you can store the v2 version of plugin A in the GitLab repository.

[0093] When the business platform can create v2 version of the pending tasks, you can request the GitLab repository to obtain the latest version information of plug-in A and plug-in B, that is, v2 version of plug-in A and v1 version of plug-in B, and then assemble the v2 version of the pending tasks and run them.

[0094] It is not difficult to see from the above steps that by combining Gitlab's multi-branch to manage multiple versions of plug-ins, a smooth upgrade of the plug-in can be achieved. Existing online tasks using old versions of plug-ins can continue to run, and newly created tasks based on upgraded plug-ins can also run without obstacles. It can be seen that the embodiment of the present disclosure can achieve the normal operation of new tasks and the unaffected operation of old tasks after the plug-in upgrade through multi-version plug-in management.

[0095] Based on this, the continuous integration scheduling method provided by the embodiment of the present disclosure has the following advantages:

[0096] (1) In the related art, the Jenkins load balancing algorithm is simply combined with the CPU memory usage of the node for scheduling. In comparison, the embodiment of the present disclosure can combine the application type of the task to be run, and comprehensively consider the current running status data, preset running data and predicted running data of each continuous integration engine slave node to perform reasonable node scheduling in certain complex scenarios, thereby achieving the most balanced scheduling, ensuring that the load of each continuous integration engine slave node is normal and that the resources of each continuous integration engine slave node can be utilized to the greatest extent.

[0097] (2) In the embodiment of the present disclosure, certain tasks that can be accelerated by caching can be scheduled to nodes that have been cached before, thereby improving operating efficiency.

[0098] (3) Considering the different performance of each machine (even if they have the same configuration but different processor models, etc.), the efficiency of the same task running on nodes with the same configuration will also be different. The embodiment of the present disclosure introduces a node scoring mechanism (i.e., fault weight scoring) into the scheduling strategy for optimization, which can more effectively utilize cluster resources, reduce task completion time, and improve overall operating efficiency. At the same time, it helps to balance the load and avoid the situation where some nodes are overloaded while other node resources are idle.

[0099] (4) The embodiment of the present disclosure introduces a dynamic parameter transmission mechanism to streamline the engine task configuration process and realize dynamic reuse. Under this mechanism, even if the engine task is cleared, it can be quickly restored through the data stored on the edge side without affecting the actual task. The introduction of plug-in task control can isolate the task into multiple subtasks (i.e., plug-ins) that do not interfere with each other, reducing the coupling degree of task scheduling. Through multi-version plug-in management, it is possible to ensure the normal operation of new tasks after the plug-in upgrade while ensuring that old tasks are not affected.

[0100] Based on the above embodiments, the embodiments of the present disclosure provide a continuous integration scheduling method, which is applied to the continuous integration engine master node in the automated operation and maintenance deployment system. The automated operation and maintenance deployment system also includes an engine scheduling module and multiple continuous integration engine slave nodes associated with the continuous integration engine master node. Figure 8 FIG. 1 shows a flow chart of a continuous integration scheduling method provided by an exemplary embodiment of the present disclosure. Figure 8 As shown, the continuous integration scheduling method includes:

[0101] S801, receiving a request for executing a task to be executed;

[0102] S802, in response to the execution request of the task to be executed, sending a node information acquisition request to the engine scheduling module; wherein the node information acquisition request carries the application type of the task to be executed;

[0103] S803, receiving node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the task to be executed, current running status data of the plurality of continuous integration engine slave nodes, preset running data, and predicted running data;

[0104] S804: Schedule tasks to be run from the node to the target continuous integration engine using the node information.

[0105] Specifically, this can be combined with the above Figures 1 to 4 The number of continuous integration engine slave nodes associated with the continuous integration engine master node can be two, three, or even more, depending on the actual application scenario, and is not specifically limited in this embodiment of the present disclosure.

[0106] Upon receiving a request to execute a task to be executed, the continuous integration engine master node may send a node information acquisition request to the engine scheduling module to request the engine scheduling module to return node information of the target continuous integration engine slave node for executing the task to be executed.

[0107] Here, the node information acquisition request may be generated based on the application type of the task to be run. For example, the engine scheduling module may obtain the application type of the task to be run from a related business interface, and then generate the node information acquisition request based on the application type of the task to be run.

[0108] After receiving the information, the engine scheduling module can Figure 4 Steps 4021 to 4023 and Figure 3 The engine scheduling module can determine the target continuous integration engine slave node based on the application type of the task to be run, the current running status data of multiple continuous integration engine slave nodes, preset running data, and predicted running data. Ultimately, the continuous integration engine master node can use the node information to schedule the task to be run to the target continuous integration engine slave node.

[0109] It can be seen that when determining the target continuous integration engine slave node from multiple continuous integration engine slave nodes, the embodiment of the present disclosure can combine the application type of the task to be run, and comprehensively consider the current running status data, preset running data and predicted running data of each continuous integration engine slave node, and reasonably schedule tasks of different application types in complex scenarios to corresponding nodes, avoiding the situation where some nodes are overloaded and some nodes are idle, effectively improving the execution efficiency of tasks, reducing the risk of continuous integration engine slave node downtime, and reasonably utilizing continuous integration engine resources, solving the problem of unbalanced scheduling of continuous integration engine nodes in special cloud-native scenarios, and greatly improving the efficiency of continuous integration.

[0110] In some embodiments, the plurality of continuous integration engine slave nodes include a preset continuous integration engine slave node corresponding to at least one preset application type, and a plurality of non-preset continuous integration engine slave nodes;

[0111] If the application type of the task to be executed matches one of the at least one preset application type, the target continuous integration engine slave node is a preset continuous integration engine slave node corresponding to the one of the at least one preset application type;

[0112] If the application type of the task to be run does not match one of the at least one preset application type, the target continuous integration engine slave node is determined by the engine scheduling module to be one of the multiple non-preset continuous integration engine slave nodes; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the current running status data, preset running data and predicted running data of the multiple non-preset continuous integration engine slave nodes.

[0113] Specifically, the multiple continuous integration engine slave nodes in the embodiment of the present disclosure may include at least one preset continuous integration engine slave node and multiple non-preset continuous integration engine slave nodes; wherein, at least one preset continuous integration engine slave node is used to run tasks of a corresponding number of preset application types, and the number of non-preset continuous integration engine slave nodes is at least two. The specific number is selected according to actual conditions, and the embodiment of the present disclosure does not specifically limit this.

[0114] For the understanding of whether the application type of the task to be executed matches one of the at least one preset application type, please refer to the above text and will not be repeated here.

[0115] If the application type of the task to be run matches one of the at least one preset application types, see the relevant content of step 303 above. The engine scheduling module determines the preset continuous integration engine slave node corresponding to the one of the at least one preset application types as the target continuous integration engine slave node, so that the task of the specific application type runs on the specified continuous integration engine slave node.

[0116] If the application type of the task to be executed does not match one of the at least one preset application type, as described above in step 304, the engine scheduling module determines one of the multiple non-preset continuous integration engine slave nodes as the target continuous integration engine slave node. The target continuous integration engine slave node is determined by the engine scheduling module based on the current operating status data, preset operating data, and predicted operating data of the multiple non-preset continuous integration engine slave nodes. This ensures the efficient use of continuous integration engine resources, effectively improves task execution efficiency, and reduces the risk of continuous integration engine slave node downtime.

[0117] In some embodiments, the current operation status data includes the current containerized operation resource value, the current number of running tasks, and the current operation failure rate; the preset operation data includes the preset upper limit of the containerized operation resource value and the preset upper limit of the running tasks; the predicted operation data includes the predicted containerized operation resource usage rate;

[0118] When the current containerized operation resource value meets the preset upper limit of the containerized operation resource value and the current number of running tasks meets the preset upper limit of the running tasks, the engine scheduling module determines the predicted containerized operation resource utilization rate based on the current containerized operation resource value, the current number of running tasks and the preset containerized operation resource value of the tasks to be run.

[0119] Specifically, the preset running data of each continuous integration engine slave node may include a preset containerized running resource value upper limit and a preset running task upper limit, the values of which can be determined according to the actual application scenario, and the embodiments of the present disclosure do not specifically limit this.

[0120] The current running status data of each continuous integration engine slave node may include the current containerized running resource value, the current number of running tasks and the current running failure rate. The engine scheduling module can obtain it in real time through the relevant interface of the continuous integration engine master node.

[0121] The current containerized operation resource value meets the preset containerized operation resource value upper limit, which can be understood as: the current containerized operation resource value is less than the preset containerized operation resource value upper limit; the current number of running tasks meets the preset running task upper limit, which can be understood as: the current number of running tasks is less than the preset running task upper limit.

[0122] The preset containerized operation resource value for the task to be run corresponds to the application type of the task to be run. When the current containerized operation resource value meets the preset upper limit of the containerized operation resource value and the number of currently running tasks meets the preset upper limit of the running tasks, the engine scheduling module can obtain the preset containerized operation resource value corresponding to the application type of the task to be run from the business interface, and then determine the predicted containerized operation resource utilization rate based on the current containerized operation resource value, the current number of running tasks, and the preset containerized operation resource value for the task to be run.

[0123] Here, the current containerized operation resource value may include the current number of operating cores and the current operating memory; the preset upper limit of the containerized operation resource value may include the preset upper limit of the number of operating cores and the preset upper limit of the operating memory; the preset containerized operation resource value may include the preset number of operating cores and the preset operating memory; and the predicted containerized operation resource usage rate may include the predicted operating core ratio and the predicted operating memory ratio. In the embodiment of the present disclosure, the above parameters may be counted in a real-time operation data table as shown in Table 1.

[0124] In some embodiments, if the application type of the task to be executed does not match one of the at least one preset application type, the target continuous integration engine slave node is the non-preset continuous integration engine slave node with the lowest fault weight score among the multiple non-preset continuous integration engine slave nodes;

[0125] Among them, the predicted containerized operation resource usage rate of the non-preset continuous integration engine slave node with the lowest fault weight score among the multiple non-preset continuous integration engine slave nodes meets the predicted containerized operation resource usage rate;

[0126] The failure weight score is calculated based on the predicted containerized operation resource usage and the current operation failure rate.

[0127] Specifically, referring to steps 305 to 306 in the previous text, if the application type of the task to be run does not match one of the at least one preset application type, the engine scheduling module can determine the non-preset continuous integration engine slave node with the lowest fault weight score among multiple non-preset continuous integration engine slave nodes as a candidate continuous integration engine slave node.

[0128] Then, determine whether the predicted containerized operation resource usage of the candidate continuous integration engine slave node meets the predicted containerized operation resource usage. If the predicted containerized operation resource usage of the candidate continuous integration engine slave node meets the predicted containerized operation resource usage, then determine the candidate continuous integration engine slave node as the target continuous integration engine slave node and execute step 307; otherwise, execute step 308.

[0129] The fault weight score is calculated based on the predicted containerized operation resource utilization and the current operation failure rate. The calculation formula is described above and will not be repeated here.

[0130] In some embodiments, the automated operation and maintenance deployment system further includes a business platform, and the method may further include:

[0131] Receive the creation request of the pending task sent by the business platform;

[0132] In response to the creation request, generate a task configuration file for the task to be run and write the task configuration file to disk;

[0133] Receive business parameters generated based on the running request of the task to be run, sent by the business platform;

[0134] Get the task configuration file after disk processing;

[0135] After receiving the node information of the target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request, the task configuration file, business parameters and node information are used to schedule the task to be run to the target continuous integration engine slave node.

[0136] Specifically, see the above Figure 6 Regarding the relevant content, when the continuous integration engine master node receives a request from the business platform to create a task to be run, it can respond to the request, generate a task configuration file for the task to be run, and save the task configuration file to disk. At this time, the business parameters in the task configuration file are dynamically adjustable and are not fixed.

[0137] When a user needs to run a pending task, the business platform generates business parameters for the pending task in response to the user's operation on the pending task and sends these business parameters to the continuous integration engine master node. Upon receiving these business parameters, the continuous integration engine master node can retrieve the processed task configuration file from disk. Then, after receiving the node information of the target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request, it can use the task configuration file, business parameters, and node information to schedule the pending task to the target continuous integration engine slave node.

[0138] It can be seen that the method of the embodiment of the present invention introduces a dynamic parameter transmission mechanism when scheduling the tasks to be run to the target continuous integration engine slave node. Compared with the existing technology, the coupling degree between the business platform and the continuous integration engine master node is reduced. Even if the business parameters of the tasks to be run on the business platform side are modified and adjusted, there is no need to change the task configuration template generated by the continuous integration engine master node and stored on the disk; since the business parameters are not written to the disk, the task configuration template can be quickly generated by the continuous integration engine master node. Therefore, in the case of insufficient disk space, the disk can be cleaned up at will because; since a large number of disk input and output operations are reduced during engine scheduling, the operating efficiency of the continuous integration engine master node is improved.

[0139] In some embodiments, the task to be run includes multiple plug-ins; in response to the creation request, generating a task configuration file for the task to be run may include:

[0140] In response to the creation request, obtain the target version information and historical version information of each plug-in;

[0141] If the historical version information of each plug-in meets the target version information, a task configuration file for the task to be run is generated based on the historical version information of multiple plug-ins;

[0142] If the historical version information of any plug-in does not meet the target version information, the business platform is used to update the historical version information of any plug-in to the target version information until the historical version information of each plug-in meets the target version information.

[0143] Specifically, the disclosed embodiment can perform orchestration processing on the tasks to be run to obtain multiple plug-ins. These plug-ins can be understood as subtasks obtained after the tasks to be run are orchestrated. In actual applications, the multiple plug-ins included in the tasks to be run and the version information corresponding to each plug-in can be stored in the Gitlab warehouse so that the subsequent continuous integration engine master node can obtain it.

[0144] Due to changes in application scenarios, some plug-ins need to be upgraded to adapt to complex application scenarios. This can avoid upgrading the entire task to be run (i.e., all plug-ins), reducing the maintenance cost of the task.

[0145] For details, please refer to the previous article Figure 7 The continuous integration engine master node responds to the creation request and obtains the target version information and historical version information of each plug-in; the target version information can come from the business platform, and the historical version information can come from the Gitlab warehouse.

[0146] Then, determine whether the historical version information of each plug-in meets the target version information. If the historical version information of each plug-in meets the target version information, it means that the versions of each plug-in for the task to be run stored in the GitLab warehouse are all the latest versions. At this time, a task configuration file for the task to be run can be generated based on the historical version information of multiple plug-ins.

[0147] If the task configuration file is not in the previous Figure 6 In step 604 shown in FIG, the task configuration file of the task to be run is written to the disk; if the task configuration file is as described above, Figure 6 In step 604 shown, the task configuration file of the task to be run has been written to the disk, so the continuous integration engine master node can directly call the task configuration file that has been written to the disk.

[0148] If the historical version information of any plug-in does not meet the target version information, it indicates that the business platform needs to upgrade the plug-in. At this time, the business platform can update the historical version information of any plug-in to the target version information until the historical version information of all plug-ins meets the target version information. After the historical version information of all plug-ins meets the target version information, the business platform can also save the version information of each upgraded plug-in to the GitLab repository.

[0149] Based on this, the disclosed embodiment can manage multiple versions of plug-in tasks by combining Gitlab's multiple branches, and can achieve smooth upgrades of plug-ins. Existing online tasks using old versions of plug-ins can continue to run, and newly created tasks based on upgraded plug-ins can also run without obstacles. It can be seen that the disclosed embodiment can achieve the normal operation of new tasks and the unaffected operation of old tasks after plug-in upgrades through multi-version plug-in management.

[0150] The above mainly introduces the solutions provided by the embodiments of the present disclosure. It is understandable that in order to implement the above functions, the electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present disclosure.

[0151] The embodiments of the present disclosure can divide the functional units of the electronic device according to the above method examples. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present disclosure is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0152] In the case of dividing each functional module according to each function, an exemplary embodiment of the present disclosure provides a continuous integration scheduling device, which is applied to a continuous integration engine master node in an automated operation and maintenance deployment system. The automated operation and maintenance deployment system also includes an engine scheduling module and multiple continuous integration engine slave nodes associated with the continuous integration engine master node. Figure 9 FIG. 1 shows a schematic diagram of the structure of a continuous integration scheduling device provided by an exemplary embodiment of the present disclosure. Figure 9 As shown, the apparatus 900 includes:

[0153] Communication module 901, used to receive a request to execute a task to be executed;

[0154] The communication module 901 is further configured to send a node information acquisition request to the engine scheduling module in response to the execution request of the task to be executed; wherein the node information acquisition request carries the application type of the task to be executed;

[0155] The communication module 901 is further configured to receive node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the task to be run, the current running status data of the multiple continuous integration engine slave nodes, the preset running data, and the predicted running data;

[0156] The processing module 902 is configured to schedule the to-be-run task to the target continuous integration engine slave node using the node information.

[0157] In some embodiments, the plurality of continuous integration engine slave nodes include a preset continuous integration engine slave node corresponding to at least one preset application type, and a plurality of non-preset continuous integration engine slave nodes;

[0158] If the application type of the task to be run matches one of the at least one preset application type, the target continuous integration engine slave node is a preset continuous integration engine slave node corresponding to the one of the at least one preset application type;

[0159] If the application type of the task to be executed does not match one of the at least one preset application type, the target continuous integration engine slave node is determined by the engine scheduling module to be one of the multiple non-preset continuous integration engine slave nodes; wherein, the target continuous integration engine slave node is determined by the engine scheduling module based on current running status data, preset running data and predicted running data of the multiple non-preset continuous integration engine slave nodes.

[0160] In some embodiments, the current operating state data includes the current containerized operating resource value, the current number of operating tasks, and the current operating failure rate; the preset operating data includes the preset upper limit of the containerized operating resource value and the preset upper limit of the operating tasks; the predicted operating data includes the predicted containerized operating resource usage rate;

[0161] The engine scheduling module determines the predicted containerized operation resource usage rate based on the current containerized operation resource value, the current number of running tasks and the preset containerized operation resource value of the task to be run, when the current containerized operation resource value meets the preset containerized operation resource value upper limit and the current number of running tasks meets the preset running task upper limit.

[0162] In some embodiments, if the application type of the task to be run does not match one of the at least one preset application type, the target continuous integration engine slave node is the non-preset continuous integration engine slave node with the lowest fault weight score among the multiple non-preset continuous integration engine slave nodes;

[0163] The predicted containerized operation resource usage rate of the non-preset continuous integration engine slave node with the lowest fault weight score among the multiple non-preset continuous integration engine slave nodes satisfies the predicted containerized operation resource usage rate;

[0164] The fault weight score is calculated based on the predicted containerized operation resource usage and the current operation failure rate.

[0165] In some embodiments, the automated operation and maintenance deployment system further includes a business platform;

[0166] The communication module 901 is further configured to receive a request for creating the task to be run sent by the service platform;

[0167] The processing module 902 is further configured to generate a task configuration file of the task to be executed in response to the creation request, and write the task configuration file to a disk;

[0168] The communication module 901 is further configured to receive the service parameters generated based on the execution request of the task to be executed and sent by the service platform;

[0169] The processing module 902 is further configured to obtain the task configuration file after disk processing;

[0170] The processing module 902 is further configured to schedule the to-be-run task to the target continuous integration engine slave node using the task configuration file, the business parameters, and the node information after receiving the node information of the target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request.

[0171] In some embodiments, the task to be run includes multiple plug-ins;

[0172] The processing module 902 is also used to respond to the creation request and obtain the target version information and historical version information of each plug-in; if the historical version information of each plug-in meets the target version information, generate the task configuration file of the task to be run based on the historical version information of multiple plug-ins; if the historical version information of any of the plug-ins does not meet the target version information, use the business platform to update the historical version information of any of the plug-ins to the target version information until the historical version information of each of the plug-ins meets the target version information.

[0173] An embodiment of the present disclosure further provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the above method disclosed in the embodiment of the present disclosure.

[0174] Figure 10 FIG. 1 shows a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present disclosure. Figure 10As shown, the electronic device 1000 includes at least one processor 1001 and a memory 1002 coupled to the processor 1001. The processor 1001 can execute corresponding steps in the above method disclosed in the embodiment of the present disclosure.

[0175] The processor 1001 may also be referred to as a central processing unit (CPU), which may be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in the embodiment of the present disclosure may be performed by hardware integrated logic circuits in the processor 1001 or by software instructions. The processor 1001 may be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiment of the present disclosure may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in the memory 1002, such as a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The processor 1001 reads the information in the memory 1002 and performs the steps of the method in combination with its hardware.

[0176] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, they can be transferred from a storage medium or a network to a computer system having a dedicated hardware structure, for example, Figure 11 The computer system 1100 shown is installed with the programs constituting the software. When the various programs are installed, the computer system can perform various functions, including the functions described above. Figure 11 A schematic diagram of the structure of a computer system provided by an exemplary embodiment of the present disclosure is shown.

[0177] Computer system 1100 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are intended to be examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0178] like Figure 11 As shown, the computer system 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. Various programs and data required for the operation of the computer system 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0179] Several components within computer system 1100 are connected to I / O interface 1105, including an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. Input unit 1106 can be any type of device capable of inputting information into computer system 1100. Input unit 1106 can receive input numeric or character information and generate key input signals related to user settings and / or function control of an electronic device. Output unit 1107 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. Storage unit 1108 may include, but is not limited to, a magnetic disk or an optical disk. Communication unit 1109 allows computer system 1100 to exchange information / data with other devices over a network, such as the Internet, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0180] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units for running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned method disclosed in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1102 and / or communication unit 1109. In some embodiments, the computing unit 1101 may be configured to perform the above-mentioned method disclosed in the embodiments of the present disclosure by any other appropriate means (e.g., by means of firmware).

[0181] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method disclosed in the embodiment of the present disclosure.

[0182] The computer-readable storage medium in the embodiments of the present disclosure can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. The above-mentioned computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specifically, the above-mentioned computer-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0183] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0184] The embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor, the method disclosed in the embodiments of the present disclosure is implemented.

[0185] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer.

[0186] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0187] The modules, components, or units described in the embodiments of the present disclosure may be implemented in software or hardware. The names of the modules, components, or units do not necessarily limit the modules, components, or units themselves.

[0188] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, and without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0189] The above descriptions are merely some embodiments of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the present disclosure.

[0190] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that modifications may be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A continuous integration scheduling method, characterized in that: A continuous integration engine master node is applied to an automated operation and maintenance deployment system, wherein the automated operation and maintenance deployment system further comprises an engine scheduling module and a plurality of continuous integration engine slave nodes associated with the continuous integration engine master node; the method comprises: Receive a request to run a task to be run; In response to the running request of the task to be run, sending a node information acquisition request to the engine scheduling module; wherein the node information acquisition request carries the application type of the task to be run; Receiving node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the task to be run, current running status data of the multiple continuous integration engine slave nodes, preset running data, and predicted running data; The node information is used to schedule the to-be-run task to the target continuous integration engine slave node.

2. The method according to claim 1, wherein The plurality of continuous integration engine slave nodes include a preset continuous integration engine slave node corresponding to at least one preset application type, and a plurality of non-preset continuous integration engine slave nodes; If the application type of the task to be run matches one of the at least one preset application type, the target continuous integration engine slave node is a preset continuous integration engine slave node corresponding to the one of the at least one preset application type; If the application type of the task to be run does not match one of the at least one preset application type, the target continuous integration engine slave node is determined by the engine scheduling module to be one of the multiple non-preset continuous integration engine slave nodes; wherein, the target continuous integration engine slave node is determined by the engine scheduling module based on current running status data, preset running data and predicted running data of the multiple non-preset continuous integration engine slave nodes.

3. The method according to claim 2, wherein The current running state data includes the current containerized running resource value, the current running task number and the current running failure rate; the preset running data includes the preset containerized running resource value upper limit and the preset running task upper limit; The predicted operation data includes predicted containerized operation resource usage; The engine scheduling module determines the predicted containerized operation resource usage rate based on the current containerized operation resource value, the current number of running tasks and the preset containerized operation resource value of the task to be run, when the current containerized operation resource value meets the preset containerized operation resource value upper limit and the current number of running tasks meets the preset running task upper limit.

4. The method according to claim 3, wherein If the application type of the to-be-run task does not match one of the at least one preset application type, the target continuous integration engine slave node is the non-preset continuous integration engine slave node with the lowest fault weight score among the multiple non-preset continuous integration engine slave nodes; The predicted containerized operation resource usage rate of the non-preset continuous integration engine slave node with the lowest fault weight score among the multiple non-preset continuous integration engine slave nodes satisfies the predicted containerized operation resource usage rate; The fault weight score is calculated based on the predicted containerized operation resource usage and the current operation failure rate.

5. The method according to any one of claims 1 to 4, wherein The automated operation and maintenance deployment system further includes a business platform, and the method further includes: Receiving a request for creating the task to be run sent by the business platform; In response to the creation request, generating a task configuration file for the task to be run, and writing the task configuration file to a disk; Receiving the business parameters generated based on the running request of the task to be run and sent by the business platform; Obtaining the task configuration file after disk processing; After receiving the node information of the target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request, the to-be-run task is scheduled to the target continuous integration engine slave node using the task configuration file, the business parameters and the node information.

6. The method according to claim 5, wherein The task to be run includes multiple plug-ins; The step of generating a task configuration file for the task to be executed in response to the creation request includes: In response to the creation request, obtaining target version information and historical version information of each plug-in; If the historical version information of each of the plug-ins meets the target version information, generating a task configuration file for the task to be run based on the historical version information of the plurality of plug-ins; If the historical version information of any of the plug-ins does not meet the target version information, the historical version information of any of the plug-ins is updated to the target version information using the service platform until the historical version information of each of the plug-ins meets the target version information.

7. A continuous integration scheduling device, characterized in that: A continuous integration engine master node is applied to an automated operation and maintenance deployment system, wherein the automated operation and maintenance deployment system further comprises an engine scheduling module and a plurality of continuous integration engine slave nodes associated with the continuous integration engine master node; the device comprises: A communication module, used for receiving a request to execute a task to be executed; The communication module is further configured to send a node information acquisition request to the engine scheduling module in response to the execution request of the task to be executed; wherein the node information acquisition request carries the application type of the task to be executed; The communication module is further configured to receive node information of a target continuous integration engine slave node returned by the engine scheduling module based on the node information acquisition request; wherein the target continuous integration engine slave node is determined by the engine scheduling module based on the application type of the task to be run, the current running status data of the plurality of continuous integration engine slave nodes, the preset running data, and the predicted running data; A processing module is used to schedule the to-be-run task to the target continuous integration engine slave node by using the node information.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to claim 1.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claim 1 are implemented.

Images