A method and device for accelerating a continuous integration pipeline

Abstract

The present application discloses a method and device for accelerating a continuous integration pipeline. The method calculates the earliest execution time corresponding to each task based on the time when each task in the pipeline generates a build product when executed multiple times and the time when the build product of the previous task is requested. When the pipeline is executed this time, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the time when the corresponding previous task generates the build product. The method enables the tasks in the pipeline to be executed flexibly in advance, reduces unnecessary waiting time during task execution, effectively shortens the execution time of the pipeline, and thus improves the R&D efficiency of the product. Moreover, by executing each task in advance within the corresponding earliest execution time, it is possible to ensure that the basic process of the build is logically maintained during the execution of the pipeline and has no impact on the final build result.

Description

Technical Field

[0001] The present application relates to the technical field of software continuous integration, and in particular to a method and device for accelerating a continuous integration pipeline. Background Art

[0002] With the rapid growth in the scale and complexity of software development, ensuring software development efficiency and quality has become increasingly important. Continuous integration (CI) is the process by which software developers continuously integrate their code into a development mainline for code compliance checks, compilation, deployment, and automated testing. CI helps developers quickly identify problems during software development and integration, enabling them to resolve code issues as quickly as possible. After completing code development or modifications, developers typically want to quickly see the results of the CI pipeline so they can proceed to the next stage of work based on the results, ensuring on-time delivery of deliverable software.

[0003] In related technologies, when a continuous integration pipeline executes a task, subsequent tasks require the build artifacts of the previous task. Therefore, each task in the pipeline is typically executed serially, requiring the previous task to complete before the next task begins to ensure successful builds. This serial execution of the continuous integration pipeline involves many complex steps, and developers sometimes have to wait a long time before seeing the pipeline execution results of their code modifications, which seriously affects code development efficiency.

[0004] Therefore, how to shorten the execution time of the continuous integration pipeline to achieve acceleration of the continuous integration pipeline is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of this application is to provide a method and device for accelerating a continuous integration pipeline, aiming to solve the technical problem in related technologies of serial execution of various tasks during pipeline integration and low integration efficiency.

[0006] In a first aspect, the present application provides a method for accelerating a continuous integration pipeline, the method comprising the following steps:

[0007] Calculate the earliest execution time for each task in the pipeline based on the time it takes to generate build artifacts when executing multiple tasks and the time it takes to request the build artifacts of the previous task.

[0008] When the pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product.

[0009] In some embodiments, when the current pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product. Specifically, the following steps are included:

[0010] Execute each task in advance according to the corresponding earliest execution time, so that the time when each task requests the build product of the previous task coincides with the time when the corresponding previous task generates the build product.

[0011] In some embodiments, when the current pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product, and further includes the following steps:

[0012] The lead execution time of each task receiving input;

[0013] Determine whether the start execution time of each task is after the corresponding earliest execution time if the corresponding task is executed in advance according to the advance execution time;

[0014] If so, the corresponding task is executed in advance according to the advance execution time; otherwise, each task is executed in advance according to the corresponding earliest execution time.

[0015] In some embodiments, before executing the corresponding task in advance according to the advance execution time, the following steps are further included:

[0016] Determine whether the advance execution time is a positive integer time;

[0017] If so, the corresponding task is executed in advance according to the advance execution time; otherwise, the corresponding task is not executed in advance according to the advance execution time.

[0018] In some embodiments, calculating the earliest execution time corresponding to each task based on the time when the build product is generated when each task in the pipeline is executed multiple times and the time when the build product of the previous task is requested specifically includes the following steps:

[0019] Determine the average time it takes for each task to generate a build artifact and the average time it takes to request a build artifact from a previous task over multiple executions.

[0020] The earliest execution time of each task is obtained by subtracting the average time it takes for each task to request the build product of the previous task from the average time it takes for the previous task to generate the build product. The earliest execution time of each task includes the earliest time at which each task can start execution after the previous task corresponding to each task starts execution.

[0021] In some embodiments, determining the average time for each task to generate a build artifact and the average time for requesting a build artifact of a previous task during multiple executions specifically includes the following steps:

[0022] Based on the average time and standard deviation of the build artifact generation time of multiple executions of each task before the most recent execution, determine whether there is any abnormality in the time it takes to generate the build artifact for each task's most recent execution;

[0023] Store the time for generating build products without exceptions and the time for requesting the build products of the previous task, and update the average time for generating build products of the corresponding task based on the stored time for generating build products. Update the average time and standard deviation of the time for requesting the build products of the previous task for the corresponding task based on the stored time for requesting the build products of the previous task.

[0024] Delete the time when the build artifact with the exception was generated and the time when the build artifact of the previous task was requested.

[0025] In some embodiments, determining whether there is an abnormality in the time taken to generate a build artifact during each latest execution of each task based on the average time and the standard deviation of the time taken to generate a build artifact during multiple executions before the latest execution of each task specifically includes the following steps:

[0026] Set the time interval for generating build products based on the average time and standard deviation of the time for generating build products during multiple executions before the most recent execution of each task.

[0027] Determine whether the time of generating the build product when each task is most recently executed is within the judgment interval;

[0028] If so, it is determined that there is no abnormality in the time when the corresponding task is most recently executed to generate the build product; otherwise, it is determined that there is an abnormality in the time when the corresponding task is most recently executed to generate the build product.

[0029] In some embodiments, before storing the time of generating the build artifact without exception and the time of requesting the build artifact of the previous task, the following steps are further included:

[0030] Determine whether the number of times for generating a build product stored for the corresponding task and the time for requesting a build product for the previous task reaches a preset threshold;

[0031] If so, delete the time of generating the build product when the pipeline is executed for the earliest time stored in the corresponding task and the time of requesting the build product of the previous task, and then store the time of generating the build product and the time of requesting the build product of the previous task without any exception. Otherwise, store it directly.

[0032] In a second aspect, the present application further provides an acceleration device for a continuous integration pipeline, the device comprising:

[0033] An information management module, which is used to calculate the earliest execution time corresponding to each task in the pipeline based on the time when the build product is generated when each task is executed multiple times and the time when the build product of the previous task is requested;

[0034] The configuration management module is used to execute each task in advance within the corresponding earliest execution time during the execution of this pipeline, so as to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product.

[0035] In some embodiments, the configuration management module is further configured to:

[0036] Execute each task in advance according to the corresponding earliest execution time, so that the time when each task requests the build product of the previous task coincides with the time when the corresponding previous task generates the build product.

[0037] The present application provides a method and device for accelerating a continuous integration pipeline. The method calculates the earliest execution time corresponding to each task based on the time when each task in the pipeline generates a build product when it is executed multiple times and the time when the build product of the previous task is requested. When the pipeline is executed this time, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the time when the corresponding previous task generates the build product. This method enables the tasks in the pipeline to be executed flexibly in advance, reduces unnecessary waiting time during task execution, and can effectively shorten the execution time of the pipeline and thus improve the R&D efficiency of the product. Moreover, by executing each task in advance within the corresponding earliest execution time, it can ensure that the basic process of the build is logically maintained during the execution of the pipeline and has no impact on the final build result. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 A flowchart of a method for accelerating a continuous integration pipeline provided in an embodiment of the present application;

[0040] Figure 2 This is a flowchart of the first execution method for each task in the pipeline;

[0041] Figure 3Schematic diagram of the second execution method of each task in the pipeline;

[0042] Figure 4 A schematic block diagram of an acceleration device for a continuous integration pipeline provided in an embodiment of the present application;

[0043] Figure 5 This is a schematic diagram of the execution flow of the monitoring management module;

[0044] Figure 6 Schematic diagram of the execution flow of the information management module;

[0045] Figure 7 This is a schematic diagram of the execution flow of the configuration management module.

[0046] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0049] The present invention provides a method and apparatus for accelerating a continuous integration pipeline, wherein the method can be applied to a computer device, such as a laptop computer, a desktop computer, or other electronic device.

[0050] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0051] The general idea of ​​this application is as follows:

[0052] A method for accelerating a continuous integration pipeline, the method comprising the following steps:

[0053] Step S101: Calculate the earliest execution time corresponding to each task according to the time when the build product is generated when each task in the pipeline is executed multiple times and the time when the build product of the previous task is requested.

[0054] Step S102: When the current pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product.

[0055] Please refer to Figure 1 , Figure 1 A flowchart of a method for accelerating a continuous integration pipeline provided in an embodiment of the present application.

[0056] like Figure 1 As shown, the method includes steps S101 to S102.

[0057] Step S101: Calculate the earliest execution time corresponding to each task according to the time when the build product is generated when each task in the pipeline is executed multiple times and the time when the build product of the previous task is requested.

[0058] For illustrative purposes, this embodiment is described by taking an example in which a pipeline includes three tasks A, B, and C. This example is only for the convenience of description and explanation and does not limit the number of tasks in the pipeline.

[0059] Monitor the time it takes to generate build products and the time it takes to request the build products of the previous task each time tasks A, B, and C in the pipeline are executed. That is, at each execution, monitor the time from the start of task A's execution to the generation of build products; monitor the time from the start of task B's execution to the request for task A's build products, and the time from the start of task B's execution to the generation of build products; monitor the time from the start of task C's execution to the request for task B's build products, and the time from the start of task C's execution to the generation of build products. This way, you can get the execution time of tasks A, B, and C when they are executed multiple times. For the sake of convenience, the time it takes for each task to generate build products and the time it takes to request the build products of the previous task are collectively referred to as the execution time of each task.

[0060] In some embodiments, the earliest execution time corresponding to each task is calculated based on the time for generating build products when each task in the pipeline is executed multiple times and the time for requesting the build products of the previous task, specifically including: determining the average time for each task to generate build products and the average time for requesting the build products of the previous task when executed multiple times; using the average time for the previous task corresponding to each task to generate build products minus the average time for each task to request the build products of the previous task, to obtain the earliest execution time of each task, wherein the earliest execution time of each task includes the earliest time that each task can start execution after the previous task corresponding to each task starts execution.

[0061] In this embodiment, the average time for each task to generate a build product and the average time for requesting the build product of the previous task are calculated to estimate the time for each task to generate a build product and the time for requesting the build product of the previous task during the next pipeline execution.

[0062] For example, each task in the pipeline is executed multiple times, and the execution time of each task A, B, and C during the multiple executions is stored. For example, the time for generating a build product during 20 executions of task B and the time for 20 requests for task A to generate a build product are stored. Then, the average time for task B to generate a build product during the 20 executions and the average time for requesting task A's build product can be calculated as the estimated execution time of task B during the next pipeline execution. Similarly, the average time for task A to generate a build product can be calculated as the estimated time for task A to generate a build product. The estimated time for task B to request task A's build product is subtracted from the estimated time for task A to generate a build product to obtain the earliest execution time of task B during the next pipeline execution. The earliest execution time of task B is the earliest time that task B can start executing after task A starts executing. For example, it is estimated that it takes 5 minutes for task A to generate a build product from the start of execution, and it is estimated that it takes 2 minutes for task B to request task A's build product from the start of execution. Then, the earliest execution time of task B can be calculated as starting 3 minutes after task A starts executing. Therefore, it is understandable that the earliest execution time of the current task is based on the start time of the previous task. This allows Task B to be executed earlier based on the earliest execution time. By the time Task A generates a build artifact, Task B has just executed the build artifact requested by Task A. At this time, Task B can directly use Task A's build artifact for execution without waiting for Task A to complete. This reduces unnecessary waiting or preparation time for Task B, effectively shortening the execution time of the pipeline and thereby improving product R&D efficiency.

[0063] It's worth noting that the earliest execution time for each task is the earliest time it can start after the previous task starts. That is, after Task A starts, Task B can start, and after Task B starts, Task C can start.

[0064] As a preferred real-time method, in addition to monitoring the execution time of each task, it also includes: determining whether the execution status of each task is a successful build state. If the execution status of a task is a failed build state, the time for generating the build product during the current execution of the monitored task and the time for requesting the build product of the previous task are deleted. If the execution status of a task is a successful build state, the time for generating the build product during the current execution of the monitored task and the time for requesting the build product of the previous task are retained. In this embodiment, the execution time of the task that failed to build is deleted so that the retained execution time has guiding significance for estimating the execution time of the corresponding task.

[0065] Preferably, the determining of the average time for each task to generate a build product and the average time for requesting a build product of the previous task during multiple executions specifically includes: determining whether there is an abnormality in the time for generating a build product each time the task is most recently executed based on the average time and the standard deviation of the time for generating a build product during multiple executions before the most recent execution of each task; storing the time for generating a build product and the time for requesting a build product of the previous task when there is no abnormality, and updating the average time for generating a build product for the corresponding task according to the stored time for generating a build product, and updating the average time and the standard deviation of the time for requesting a build product of the previous task for the corresponding task according to the stored time for requesting a build product of the previous task; and deleting the time for generating a build product and the time for requesting a build product of the previous task when there is an abnormality.

[0066] Furthermore, the method determines whether there is an abnormality in the time of generating the build product each time each task is executed for the latest time based on the average time and standard deviation of the time of generating the build product when each task is executed multiple times before the latest execution, specifically including: setting a judgment interval for the time of generating the build product according to the average time and standard deviation of the time of generating the build product when each task is executed multiple times before the latest execution; determining whether the time of generating the build product when each task is executed for the latest time is within the judgment interval; if so, determining that there is no abnormality in the time of generating the build product when the corresponding task is executed for the latest time; otherwise, determining that there is an abnormality in the time of generating the build product when the corresponding task is executed for the latest time.

[0067] It can be understood that in this embodiment, before a new execution of a task is performed, a reasonable interval for the time to generate a build product is set based on the stored average time for generating a build product when the task is executed and the standard deviation of the corresponding time. The execution time of the task whose time to generate a build product is within the interval is stored, and the execution time of the task that is not within the interval is deleted, thereby deleting the execution time of the task whose time to generate a build product is too short or too long, thereby further ensuring the accuracy of the time for this task to generate a build product and the time for requesting the build product of the previous task during the next pipeline execution estimated based on the average time.

[0068] For example, taking Task B as an example, before the latest execution of Task B, the average time u and the standard deviation σ of Task B's build product generation are calculated based on the stored time of multiple Task B build products. Based on the average time u and the standard deviation σ, the judgment interval for whether there is an abnormality in the time of Task B's build product generation can be set to [u-3σ, u+3σ]. After the latest successful execution of Task B, determine whether the time of Task B's build product generation is within the interval [u-3σ, u+3σ]. If so, determine that there is no abnormality in the execution time of Task B this time, store the execution time of Task B this time, and update the average time u and the standard deviation σ of Task B's build product generation based on the execution time, as well as the average time for Task B to request Task A's build product. Otherwise, determine that there is an abnormality in the execution time of Task B this time, and do not store the execution time of Task B.

[0069] It is worth noting that each execution time of each task in each pipeline will generate a unique record id when it is stored.

[0070] After storing Task B's most recent execution time, the average time Task B takes to generate build artifacts can be updated using the preset average time calculation formula: (u*cnt+b) / (cnt+1), where u is the average time Task B took to generate build artifacts before its most recent execution, cnt is the number of times Task B has stored its build artifact generation times before its most recent execution, and b is the time Task B took to generate build artifacts during its most recent execution. The result of this formula is the updated average time Task B takes to generate build artifacts. If the result is a non-integer, round it up. The same method can be used to update the average time Task B takes to request the build artifacts of the previous task.

[0071] Furthermore, the standard deviation of the time for task B to generate a build product can be updated by using a preset formula and the stored time for task B to generate a build product. Updates the standard deviation of the time it takes for Task B to generate build artifacts. Where σ is the standard deviation of the time it takes for Task B to generate build artifacts stored before the most recent execution, cnt is the number of build artifacts stored before the most recent execution, u is the average time it takes for Task B to generate build artifacts before the most recent execution, and b is the time it takes for Task B to generate build artifacts during the most recent execution. The result calculated using this formula is the standard deviation of the task update. If the result is non-integer, round it down.

[0072] Preferably, a dictionary can be used to record information such as the standard deviation of each task in the pipeline after each update. For example, the key in the dictionary is the ID of each pipeline execution, and the value contains the dictionary variable for each task, where iid is the ID of each task in the pipeline, time is the average time for each task in the pipeline to generate a build product, std is the standard deviation of the time it takes for each task to generate a build product, and cnt is the number of times each task generates a build product. When the pipeline includes three tasks A, B, and C, the standard deviation of the time it takes to generate a build product and related information for each task are stored in a dictionary as follows:

[0073] {'id':{'iid1':['time1','std','cnt'],'iid2':['time','std','cnt2','iid3':['time3','std','cnt']}}.

[0074] Among them, iid1 is the record of the current execution of task A, iid2 is the record of the current execution of task B, and iid3 is the record of the current execution of task C.

[0075] As a preferred implementation, before storing the time of generating the build product without abnormalities and the time of requesting the build product of the previous task, it also includes: judging whether the number of the stored times of generating the build product and the time of requesting the build product of the previous task for the corresponding task reaches a preset quantity threshold; if so, deleting the time of generating the build product and the time of requesting the build product of the previous task when the pipeline is executed for the earliest time stored for the corresponding task, otherwise, not deleting.

[0076] Specifically, the system determines whether the number of stored execution times for each task has reached N (number of N) thousand. If so, the earliest stored execution time for that task is deleted, and the latest stored execution time is then stored. This allows the entire storage process to form a large circular buffer, with the write thread writing new information to the front and the purge thread removing old information from the back, thereby reducing system storage capacity and improving system performance.

[0077] Step S102: When the current pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product.

[0078] In some embodiments, when the current pipeline is executed, each task can be set to be executed in advance according to the corresponding earliest execution time, so that the time when each task requests the build product of the previous task coincides with the time when the corresponding previous task generates the build product.

[0079] For example, by executing Task B ahead of time based on its earliest execution time, Task A can generate a build artifact just as Task B is executing the requested build artifact. Task B can then directly use Task A's build artifact for subsequent execution. This allows the previous task to generate a build artifact and the next task to execute when it needs it, allowing the next task to directly use the previous task's build artifact for execution. This reduces unnecessary waiting or preparation time, effectively shortening the pipeline's execution time and ultimately improving product development efficiency.

[0080] As a preferred embodiment, in addition to directly executing each task in advance according to the corresponding earliest execution time, it is also possible to: receive the input of the advance execution time of each task; determine whether the start execution time of each task is after the corresponding earliest execution time if the corresponding task is executed in advance according to the advance execution time; if so, execute the corresponding task in advance according to the advance execution time, otherwise, execute each task in advance according to the corresponding earliest execution time. It should be understood that the advance execution time here is also based on the start execution time of the previous task corresponding to each task. For example, if the input task advance execution time is 3 minutes, it means that the task will start execution 3 minutes after the corresponding previous task starts execution.

[0081] It is understandable that if a task is executed ahead of schedule based on the advance execution time, and the task's start execution time is before the earliest execution time, the current task may have already reached the point where it needs to request the previous task's build artifacts, but the corresponding previous task's build artifacts have not yet been generated, causing the current task to wait for a long time. This may lead to too many simultaneous tasks and excessive system pressure. Therefore, when determining whether the task's start execution time is after the corresponding earliest execution time, the task should be executed ahead of schedule based on the advance execution time.

[0082] It is worth noting that when entering the advance execution time of a task, you can enter the advance execution time of all tasks so that all tasks can be executed in advance. The execution process is as follows: Figure 2Or you can enter the advance execution time of some tasks to enable them to be executed in advance. The execution process is as follows Figure 3 shown.

[0083] Preferably, before executing the corresponding task in advance according to the advance execution time, the method further includes: determining whether the advance execution time is a positive integer; if so, executing the corresponding task in advance according to the advance execution time; otherwise, not executing the corresponding task in advance according to the advance execution time. The positive integer time can be set to a whole hour, a whole minute, or a whole second.

[0084] It is worth noting that if the advance execution time entered is 0, it means that the corresponding task will not be executed in advance, and the default parameter can be set to 0. When the advance execution times entered are partial times, you can choose to execute the task in advance according to the earliest execution time, or not to execute the task in advance.

[0085] Moreover, after the execution of this pipeline is completed, the time when each task in this pipeline generates the build product and the build product time of the previous task corresponding to the request are stored according to the above method, and the average time and time standard deviation are updated according to the stored time.

[0086] In a specific embodiment, at the software level, a monitoring management module, an information management module, and a configuration management module can be set up, a total of three modules to implement the acceleration method of the continuous integration pipeline. The data interaction between the three modules is as follows: Figure 4 shown.

[0087] The monitoring and management module is used to trigger the execution of tasks in the pipeline, and monitor and count the time for generating the build product of each task and the time for requesting the build product of the previous task each time, and generate the corresponding pipeline id and task id, and pass the task execution time obtained by monitoring and the id information to the information management module. The information management module is used to determine whether there are any abnormalities in the execution time of the tasks obtained by the monitoring and management module, store the execution time of tasks without abnormalities, generate the pipeline id and task id, and update the average execution time of each task, and send the updated average execution time and id information of each task to the configuration management module. The configuration management module generates configuration information based on the average execution time of each task and the received advance execution time of each task, and sends the configuration information to the monitoring and management module so that the monitoring and management module controls the completion of each task according to the received configuration information when the pipeline is executed a new time.

[0088] like Figure 5 As shown, the specific steps performed by the monitoring management module include:

[0089] Step 201: Monitor the execution time of each task during pipeline execution.

[0090] Step 202: Determine whether each task is successfully constructed. If so, proceed to step 203; if not, proceed to step 204.

[0091] Step 203: Send the execution time of each monitored task to the information management module.

[0092] Step 204: Delete the execution time of the corresponding task.

[0093] The information management module generates a unique record ID upon receiving the time required for each task to successfully execute during each pipeline execution, sent by the monitoring management module. It also records the time each task generated its build artifact after a successful build, the time it requested the previous task to generate its build artifact, and the number of times the task line was executed (cnt). Based on the previously stored execution times for each task, the module calculates the average build artifact generation time u and standard deviation σ for each task. Based on these values, the module sets a judgment interval [u-3σ,u+3σ] to determine whether the build artifact generation time is abnormal. The module then updates the mean and standard deviation based on execution times falling within this interval. The specific method is not detailed here.

[0094] like Figure 6 As shown, the specific steps performed by the information management module include:

[0095] Step S301: Receive the pipeline information and the execution time of each task in the pipeline sent by the monitoring management module, and generate an internal unique record.

[0096] Step S302: Determine whether the time it takes to generate a build artifact for each task falls within the corresponding range [u-3σ,u+3σ]. If so, proceed to step S303; if not, proceed to step S304. This step determines whether the time it takes to generate a build artifact is abnormal; both too short and too long values ​​are considered abnormal.

[0097] Step S303: Update the average time u and standard deviation σ of generating the construction product of each task and the counting variable cnt.

[0098] Step S304: Delete the execution time of the corresponding task.

[0099] Step S305: Determine whether the number of executions of each task is less than N ten thousand. If so, proceed to step S306; if not, proceed to step S307.

[0100] Step S306: directly storing the execution time of the corresponding task.

[0101] Step S307: Delete the oldest stored execution time of the corresponding task and store the execution time record of the current execution of the task. This makes the entire historical execution record like a large circular buffer. The writing thread writes new records to the front, while the cleaning thread removes old records from the back, thereby reducing system storage capacity and improving system performance.

[0102] Step S308: Calculate the earliest execution time of each task and send it to the configuration management module.

[0103] The configuration management module is used to receive feedback from the monitoring management module and the information management sub-module, as well as the advance execution time of each task set by the user, and verify the received advance execution time of each task to prevent the input of illegal data, and send configuration information to the monitoring management module according to the advance execution time of each task, so that the monitoring management module can execute each task in advance according to the configuration information.

[0104] like Figure 7 As shown, the specific steps when the configuration management module is executed are as follows:

[0105] Step S401: receiving the advance execution time of each task input by the user.

[0106] Step S402: Determine whether the start execution time of the corresponding task is after the earliest execution time when executing the corresponding task according to the received advance execution time. If so, proceed to step S403; otherwise, proceed to step S404.

[0107] Step S403: Continue to determine whether the accepted advance execution time is a positive integer. If so, proceed to step S405; if not, proceed to step S406.

[0108] Step S404: issuing an illegal parameter input reminder;

[0109] Step S405: Save the received advance execution time of the task, generate configuration information based on the saved advance execution time, and synchronize the configuration information to the monitoring management module, so that the monitoring management module can execute each task in advance according to the configuration information.

[0110] Step S406: Issue an illegal parameter input reminder.

[0111] Preferably, the configuration management module also receives the task ID transmitted by the monitoring management module and the task ID transmitted by the information management module, and sends the generated configuration information to the monitoring management module after the IDs transmitted by the two modules match.

[0112] like Figure 4 As shown, Figure 4A schematic block diagram of an acceleration device for a continuous integration pipeline provided in an embodiment of the present application. The device includes:

[0113] An information management module, which is used to calculate the earliest execution time corresponding to each task in the pipeline based on the time when the build product is generated when each task is executed multiple times and the time when the build product of the previous task is requested;

[0114] The configuration management module is used to execute each task in advance within the corresponding earliest execution time during the execution of this pipeline, so as to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product.

[0115] Wherein, the configuration management module is also used for:

[0116] Execute each task in advance according to the corresponding earliest execution time, so that the time when each task requests the build product of the previous task coincides with the time when the corresponding previous task generates the build product.

[0117] Wherein, the configuration management module is also used for:

[0118] The lead execution time of each task receiving input;

[0119] Determine whether the start execution time of each task is after the corresponding earliest execution time if the corresponding task is executed in advance according to the advance execution time;

[0120] If so, the corresponding task is executed in advance according to the advance execution time; otherwise, each task is executed in advance according to the corresponding earliest execution time.

[0121] Wherein, the configuration management module is also used for:

[0122] Determine whether the advance execution time is a positive integer time;

[0123] If so, the corresponding task is executed in advance according to the advance execution time; otherwise, the corresponding task is not executed in advance according to the advance execution time.

[0124] Wherein, the information management module is also used for:

[0125] Determine the average time it takes for each task to generate a build artifact and the average time it takes to request a build artifact from a previous task over multiple executions.

[0126] The earliest execution time of each task is obtained by subtracting the average time it takes for each task to request the build product of the previous task from the average time it takes for the previous task to generate the build product. The earliest execution time of each task includes the earliest time at which each task can start execution after the previous task corresponding to each task starts execution.

[0127] Wherein, the information management module is also used for:

[0128] Based on the average time and standard deviation of the build artifact generation time of multiple executions of each task before the most recent execution, determine whether there is any abnormality in the time it takes to generate the build artifact for each task's most recent execution;

[0129] Store the time for generating build products without exceptions and the time for requesting the build products of the previous task, and update the average time and standard deviation of the corresponding task's build product generation time based on the stored time for generating build products. Update the average time for requesting the build products of the previous task for the corresponding task based on the stored time for requesting the build products of the previous task.

[0130] Delete the time when the build artifact with the exception was generated and the time when the build artifact of the previous task was requested.

[0131] Wherein, the information management module is also used for:

[0132] Set the time interval for generating build products based on the average time and standard deviation of the time for generating build products during multiple executions before the most recent execution of each task.

[0133] Determine whether the time of generating the build product when each task is most recently executed is within the judgment interval;

[0134] If so, it is determined that there is no abnormality in the time when the corresponding task is most recently executed to generate the build product; otherwise, it is determined that there is an abnormality in the time when the corresponding task is most recently executed to generate the build product.

[0135] Wherein, the information management module is also used for:

[0136] Determine whether the number of times for generating a build product stored for the corresponding task and the time for requesting a build product for the previous task reaches a preset threshold;

[0137] If so, delete the time of generating the build product when the pipeline is executed for the earliest time stored in the corresponding task and the time of requesting the build product of the previous task, and then store the time of generating the build product and the time of requesting the build product of the previous task without any exception. Otherwise, store it directly.

[0138] It should be noted that those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned embodiments and will not be repeated here.

[0139] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0140] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for accelerating a continuous integration pipeline, characterized in that: The following steps are involved: Calculate the earliest execution time for each task in the pipeline based on the time it takes to generate build artifacts when executing multiple tasks and the time it takes to request the build artifacts of the previous task. When the pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product.

2. The method for accelerating a continuous integration pipeline according to claim 1, characterized in that: When the pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product. Specifically, the following steps are included: Execute each task in advance according to the corresponding earliest execution time, so that the time when each task requests the build product of the previous task coincides with the time when the corresponding previous task generates the build product.

3. The method for accelerating a continuous integration pipeline according to claim 1, characterized in that: When the pipeline is executed, each task is executed in advance within the corresponding earliest execution time to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product, and further includes the following steps: The lead execution time of each task receiving input; Determine whether the start execution time of each task is after the corresponding earliest execution time if the corresponding task is executed in advance according to the advance execution time; If so, the corresponding task is executed in advance according to the advance execution time; otherwise, each task is executed in advance according to the corresponding earliest execution time.

4. The method for accelerating a continuous integration pipeline according to claim 3, characterized in that: Before executing the corresponding task in advance according to the advance execution time, the following steps are also included: Determine whether the advance execution time is a positive integer time; If so, the corresponding task is executed in advance according to the advance execution time; otherwise, the corresponding task is not executed in advance according to the advance execution time.

5. The method for accelerating a continuous integration pipeline according to claim 1, characterized in that: The method of calculating the earliest execution time corresponding to each task according to the time when the build product is generated when each task in the pipeline is executed multiple times and the time when the build product of the previous task is requested includes the following steps: Determine the average time it takes for each task to generate a build artifact and the average time it takes to request a build artifact from a previous task over multiple executions. The earliest execution time of each task is obtained by subtracting the average time it takes for each task to request the build product of the previous task from the average time it takes for the previous task to generate the build product. The earliest execution time of each task includes the earliest time at which each task can start execution after the previous task corresponding to each task starts execution.

6. The method for accelerating a continuous integration pipeline according to claim 5, characterized in that: Determining the average time for each task to generate a build product and the average time for requesting the build product of the previous task during multiple executions specifically includes the following steps: Based on the average time and standard deviation of the build artifact generation time of multiple executions of each task before the most recent execution, determine whether there is any abnormality in the time it takes to generate the build artifact for each task's most recent execution; Store the time for generating build products without exceptions and the time for requesting the build products of the previous task, and update the average time and standard deviation of the corresponding task's build product generation time based on the stored time for generating build products. Update the average time for requesting the build products of the previous task for the corresponding task based on the stored time for requesting the build products of the previous task. Delete the time when the build artifact with the exception was generated and the time when the build artifact of the previous task was requested.

7. The method for accelerating a continuous integration pipeline according to claim 6, characterized in that: The method of determining whether there is an abnormality in the time of generating the build product each time when each task is most recently executed, based on the average time and standard deviation of the time of generating the build product when each task is most recently executed, specifically includes the following steps: Set the time interval for generating build products based on the average time and standard deviation of the time for generating build products during multiple executions before the most recent execution of each task. Determine whether the time of generating the build product when each task is most recently executed is within the judgment interval; If so, it is determined that there is no abnormality in the time when the corresponding task is most recently executed to generate the build product; otherwise, it is determined that there is an abnormality in the time when the corresponding task is most recently executed to generate the build product.

8. The method for accelerating a continuous integration pipeline according to claim 6, characterized in that: Before storing the time of generating the build product without exception and the time of requesting the build product of the previous task, the following steps are also included: Determine whether the number of times for generating a build product stored for the corresponding task and the time for requesting a build product for the previous task reaches a preset threshold; If so, delete the time of generating the build product when the pipeline is executed for the earliest time stored in the corresponding task and the time of requesting the build product of the previous task, and then store the time of generating the build product and the time of requesting the build product of the previous task without any exception. Otherwise, store it directly.

9. An acceleration device for a continuous integration pipeline, characterized in that: include: An information management module, which is used to calculate the earliest execution time corresponding to each task in the pipeline based on the time when the build product is generated when each task is executed multiple times and the time when the build product of the previous task is requested; The configuration management module is used to execute each task in advance within the corresponding earliest execution time during the execution of this pipeline, so as to shorten the time difference between each task requesting the build product of the previous task and the corresponding previous task generating the build product.

10. The acceleration device for the continuous integration pipeline according to claim 9, characterized in that: The configuration management module is also used to: Execute each task in advance according to the corresponding earliest execution time, so that the time when each task requests the build product of the previous task coincides with the time when the corresponding previous task generates the build product.

Images