Task scheduling method and device, equipment, storage medium and program product

Abstract

The invention provides a task scheduling method and device, equipment, a storage medium and a program product, and relates to the technical field of computers. The task state data comprises task state information of the to-be-executed task, and the task state information is used for identifying whether the to-be-executed task can obtain corresponding task resources from the continuous integration platform or not; predicting a resource state of the continuous integration platform based on the task state data, and generating resource state information of the continuous integration platform; and performing task scheduling on the to-be-executed task based on the resource state information. Through the method, the resource state information of the continuous integration platform can be indirectly speculated without monitoring each task resource in real time by using different platforms and methods, so that a large amount of monitoring resources can be saved, the resource monitoring cost is reduced, and the resource monitoring efficiency is improved; and the subsequent task scheduling efficiency and the resource utilization rate can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular to a task scheduling method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] CI / CD (Continuous Integration / Continuous Deployment) is a software development technology, and developers can continuously integrate and deploy code through automated processes and tools on a continuous integration platform to perform different tasks.

[0003] In the process of software development, the continuous integration platform runs various tasks, such as MR (Merge Request) tasks of developers, high-priority verification tasks, and routine verification tasks. However, due to the needs of business testing, different tasks need to use different task resources during execution or testing, such as physical machine or virtual machine resources for some tasks, and cloud resources for some tasks. To achieve efficient task scheduling, the resource status of the continuous integration platform needs to be monitored in real time to schedule tasks according to the real-time resource status of the continuous integration platform.

[0004] In the prior art, different types of task resources on the continuous integration platform need to be monitored by different platforms or methods, such as cloud resources that need to be monitored by a cloud platform. Real-time monitoring of each type of task resource using different platforms and methods can ensure that the resource status information of the continuous integration platform obtained by monitoring is accurate, and then a FIFO (First-In First-Out) scheduling strategy can be used to schedule tasks to be executed on the continuous integration platform.

[0005] Although real-time monitoring of each type of task resource using different platforms and methods can obtain accurate resource status information of the continuous integration platform, this method consumes a large amount of monitoring resources, resulting in high resource monitoring costs, low resource monitoring efficiency, and easy impact on the efficiency and resource utilization of subsequent task scheduling. SUMMARY

[0006] The present application provides a task scheduling method, device, equipment, storage medium and program product to solve the defects of high resource monitoring cost, low resource monitoring efficiency, and easy impact on the efficiency and resource utilization of subsequent task scheduling of the existing task scheduling method.

[0007] The application provides a task scheduling method, comprising: obtaining task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used for identifying whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform; based on the task state data, the resource state of the continuous integration platform is predicted to generate resource state information of the continuous integration platform; and based on the resource state information, the to-be-executed task is scheduled.

[0008] According to the task scheduling method provided by the application, the task state data is generated in a dotting manner, the task state information comprises a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded when the to-be-executed task waits to obtain corresponding task resources, and the resource locking identification point is an identification point recorded when the to-be-executed task obtains corresponding task resources; based on the task state data, the resource state of the continuous integration platform is predicted to generate resource state information of the continuous integration platform, comprising: if the task state information of the to-be-executed task comprises the resource waiting identification point and does not comprise the resource locking identification point, the waiting time of the to-be-executed task is recorded; based on the waiting time, the resource state of the continuous integration platform is predicted to generate the resource state information of the continuous integration platform.

[0009] According to the task scheduling method provided by the application, after the resource state of the continuous integration platform is predicted based on the waiting time to generate the resource state information of the continuous integration platform, the method further comprises: if the waiting time is greater than or equal to a preset threshold, a reconnaissance task is generated; based on the reconnaissance task, corresponding task resources of the to-be-executed task are reacquired; and if the to-be-executed task successfully acquires the corresponding task resources, the resource state information is corrected.

[0010] According to the task scheduling method provided by the application, the number of to-be-executed tasks is at least two; based on the resource state information, the to-be-executed tasks are scheduled, comprising: determining a task score of each to-be-executed task; based on the resource state information and the task score of each to-be-executed task, at least one target to-be-executed task is selected from all to-be-executed tasks; and based on the continuous integration platform, the target to-be-executed task is executed.

[0011] According to the task scheduling method provided by the application, the to-be-executed tasks are stored in a task queue; the task score of each to-be-executed task is determined, comprising: respectively determining the priority of each to-be-executed task, the compilation time length of each to-be-executed task and the time when each to-be-executed task enters the task queue; based on the priority of each to-be-executed task, the compilation time length of each to-be-executed task, the time when each to-be-executed task enters the task queue and the dependency relationship between all to-be-executed tasks, the task score of each to-be-executed task is determined.

[0012] According to the task scheduling method provided by the application, a task score includes an integer part and a decimal part, the integer part is determined based on the priority and the compilation time of a to-be-executed task and the dependency relationship among all to-be-executed tasks, and the decimal part is determined based on the time when the to-be-executed task enters a task queue.

[0013] The application further provides a task scheduling device, comprising: an acquisition module, configured to acquire task state data; the task state data comprises task state information of to-be-executed tasks, and the task state information is used to identify whether the to-be-executed tasks can acquire corresponding task resources from a continuous integration platform; a resource state prediction module, configured to predict the resource state of the continuous integration platform based on the task state data, and generate resource state information of the continuous integration platform; and a task scheduling module, configured to perform task scheduling on the to-be-executed tasks based on the resource state information.

[0014] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements any of the above task scheduling methods when executing the computer program.

[0015] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement any of the above task scheduling methods.

[0016] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement any of the above task scheduling methods.

[0017] The task scheduling method, device, equipment, storage medium and program product provided by the present application avoid the mode of directly monitoring different types of task resources in real time, but first acquire task state data of a continuous integration platform, and the task state data includes task state information of each to-be-executed task on the continuous integration platform. Since the task state information is used to identify whether the to-be-executed task can acquire corresponding task resources from the continuous integration platform, if the to-be-executed task can acquire corresponding task resources from the continuous integration platform, it indirectly indicates that the corresponding task resources are sufficient, and if the to-be-executed task cannot acquire corresponding task resources from the continuous integration platform, it indirectly indicates that the corresponding task resources are insufficient. That is, the task state information of each to-be-executed task can indirectly reflect the task resource situation that can be currently provided by the continuous integration platform. Therefore, the resource state of the continuous integration platform can be indirectly inferred according to the task state information of each to-be-executed task, the current resource state information of the continuous integration platform is generated, and the to-be-executed task is scheduled according to the indirectly inferred resource state information. Since the resource state information of the continuous integration platform can be indirectly inferred without using different platforms and methods to monitor each type of task resource in real time, a large amount of monitoring resources can be saved, the resource monitoring cost is reduced, the resource monitoring efficiency is improved, and the efficiency and resource utilization rate of subsequent task scheduling are improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is one of the flowcharts of the task scheduling method provided by the present application.

[0020] Figure 2 is another flowchart of the task scheduling method provided by the present application.

[0021] Figure 3 is a structural schematic diagram of the task scheduling device provided by the present application.

[0022] Figure 4 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0023] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0024] Please refer to Figure 1 and Figure 2 , Figure 1 is one of flowcharts of the task scheduling method provided by the present application, Figure 2 is another flowchart of the task scheduling method provided by the present application.

[0025] As Figure 1 shown, in the present embodiment, the task scheduling method is applied to a scheduling system, and the task scheduling method comprises steps S110 to S130, and each step is specifically as follows: S110: obtaining task state data.

[0026] The task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform.

[0027] In the present embodiment, the basic principle of the task scheduling method is to determine the task state information of each task on the continuous integration platform (i.e. CI platform) by means of dotting in the task running process, and then to indirectly infer the current resource state information of the continuous integration platform according to the task state information of each task, and finally to select a corresponding number of tasks from the task queue and execute them according to the current resource state information of the continuous integration platform, so as to realize the task scheduling of the continuous integration platform.

[0028] Specifically, a task state database is maintained, and the task state database stores the task state information of all to-be-executed tasks on the continuous integration platform, and the task state data comprises the task state information of these to-be-executed tasks.

[0029] Among them, the task state data is generated based on the dotting method, that is, the task state information of each to-be-executed task is generated based on the dotting method.

[0030] Among them, the task state information comprises a resource waiting identification point and / or a resource locking identification point; the resource waiting identification point (i.e. “WAIT” point) is an identification point recorded by the to-be-executed task when it is waiting to obtain corresponding task resources; and the resource locking identification point (i.e. “LOCK” point) is an identification point recorded by the to-be-executed task when it obtains corresponding task resources.

[0031] The dotting during the task running refers to, for each task, setting a resource waiting identification point (i.e., a "WAIT" point) for the task before the task obtains the corresponding task resource from the continuous integration platform, setting a resource locking identification point (i.e., a "LOCK" point) for the task after the task obtains the corresponding task resource from the continuous integration platform, and storing the added identification point information as task state information and storing the task state information into the task state database each time a resource waiting identification point or a resource locking identification point is added.

[0032] It should be noted that, for each to-be-executed task, the task state information can indicate whether the to-be-executed task can currently obtain the corresponding task resource from the continuous integration platform: if the to-be-executed task can currently obtain the corresponding task resource from the continuous integration platform, it indirectly indicates that the corresponding task resource of the continuous integration platform is currently sufficient; if the to-be-executed task cannot currently obtain the corresponding task resource from the continuous integration platform, it indirectly indicates that the corresponding task resource of the continuous integration platform is currently insufficient. Based on the above principle, the task state information of each to-be-executed task can actually indirectly reflect the current task resource situation of the continuous integration platform, and thus the current resource state of the continuous integration platform can be indirectly inferred according to the task state information of each to-be-executed task.

[0033] Further, the scheduling system can periodically scan the task state database according to a preset period, obtain the current task state data of the continuous integration platform from the task state database, and use the data as a data basis for subsequent resource state inference.

[0034] S120: predicting the resource state of the continuous integration platform based on the task state data, and generating resource state information of the continuous integration platform.

[0035] Specifically, the resource state of the continuous integration platform is predicted based on the task state information of each to-be-executed task, and the resource state information of the continuous integration platform is generated.

[0036] It should be noted that, since the process of determining the current resource state of the continuous integration platform by dotting does not achieve real-time and accurate monitoring of different types of task resources, the resource state information determined by this method may have some errors compared with the actual situation, i.e., the resource state information indirectly inferred from the task state information of each to-be-executed task cannot completely reflect the current real resource state of the continuous integration platform, and the inferred resource state information is only a predicted value that is relatively close to the real resource state, so the indirectly inferred resource state information can only reflect the "resource ambiguous state" of the continuous integration platform.

[0037] S130: scheduling the to-be-executed task based on the resource state information.

[0038] Specifically, according to the resource state information capable of reflecting the current "resource ambiguous state" of the continuous integration platform, all the to-be-executed tasks on the continuous integration platform are scheduled.

[0039] The task scheduling method provided by the embodiment avoids directly monitoring different types of task resources in real time, but first acquires task state data of the continuous integration platform, the task state data including task state information of each to-be-executed task on the continuous integration platform. Since the task state information is used to identify whether the to-be-executed task can acquire the corresponding task resource from the continuous integration platform, if the to-be-executed task can acquire the corresponding task resource from the continuous integration platform, it means that the corresponding task resource is sufficient, and if the to-be-executed task cannot acquire the corresponding task resource from the continuous integration platform, it means that the corresponding task resource is insufficient. That is, the task state information of each to-be-executed task can indirectly reflect the current task resource situation of the continuous integration platform, so that the resource state of the continuous integration platform can be indirectly inferred according to the task state information of each to-be-executed task, the current resource state information of the continuous integration platform is generated, and the to-be-executed tasks are scheduled according to the indirectly inferred resource state information. Since the resource state information of the continuous integration platform can be indirectly inferred without using different platforms and methods to monitor each type of task resource in real time, a large amount of monitoring resources can be saved, the resource monitoring cost can be reduced, the resource monitoring efficiency can be improved, and the efficiency of subsequent task scheduling and the resource utilization rate can be improved.

[0040] In some embodiments, the task state data is generated based on a dotting method, the task state information includes a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded by the to-be-executed task when waiting to acquire the corresponding task resource, and the resource locking identification point is an identification point recorded by the to-be-executed task when acquiring the corresponding task resource. Based on the task state data, the resource state of the continuous integration platform is predicted to generate the resource state information of the continuous integration platform, including: if the task state information of the to-be-executed task includes the resource waiting identification point and does not include the resource locking identification point, the waiting time of the to-be-executed task is recorded; based on the waiting time, the resource state of the continuous integration platform is predicted to generate the resource state information of the continuous integration platform.

[0041] Specifically, a task state database is first maintained, and the task state database stores the task state information of all the to-be-executed tasks on the continuous integration platform. The task state data includes the task state information of these to-be-executed tasks.

[0042] The task state data is generated based on a dotting method, that is, the task state information of each to-be-executed task is generated based on a dotting method.

[0043] The task state information includes a resource waiting identification point and / or a resource locking identification point. The resource waiting identification point (i.e., a "WAIT" point) is an identification point recorded by a to-be-executed task when the to-be-executed task is waiting for obtaining a corresponding task resource. The resource locking identification point (i.e., a "LOCK" point) is an identification point recorded by the to-be-executed task when the to-be-executed task obtains the corresponding task resource.

[0044] The point setting in the task running process means that, for each task, a resource waiting identification point (i.e., a "WAIT" point) is set for the task before the task obtains a corresponding task resource from the continuous integration platform, a resource locking identification point (i.e., a "LOCK" point) is set for the task after the task obtains the corresponding task resource from the continuous integration platform, and the added identification point information is taken as the task state information each time the resource waiting identification point or the resource locking identification point is added, and the task state information is stored into the task state database.

[0045] Further, the scheduling system can periodically scan the task state database according to a preset period, and obtain the current task state data of the continuous integration platform from the task state database as a data basis for subsequent resource state speculation.

[0046] Further, for each to-be-executed task, if the task state information of the to-be-executed task only includes the resource waiting identification point (i.e., a "WAIT" point) but does not include the resource locking identification point (i.e., a "LOCK" point), it indicates that the current corresponding task resource of the continuous integration platform is insufficient, and at this time, the waiting time of the to-be-executed task can be recorded.

[0047] Meanwhile, for each to-be-executed task, if the task state information of the to-be-executed task includes both the resource waiting identification point (i.e., a "WAIT" point) and the resource locking identification point (i.e., a "LOCK" point), it indicates that the current corresponding task resource of the continuous integration platform is sufficient.

[0048] Further, the scheduling system can predict the resource state of the continuous integration platform according to the waiting time of all to-be-executed tasks whose task resources are insufficient (i.e., to-be-executed tasks whose task state information only includes the resource waiting identification point but does not include the resource locking identification point), according to a preconfigured relationship between the waiting time and the resource state, and generate resource state information of the continuous integration platform.

[0049] The resource state information of the continuous integration platform includes resource state levels corresponding to different types of task resources, and the lower the resource state level is, the more sufficient the corresponding task resource is.

[0050] For example, assuming that the continuous integration platform includes four task resources, denoted as RescourceA, RescourceB, RescourceC and RescourceD, the resource state information inferred from the task state information of each to-be-executed task can include the resource state levels of the four task resources, and the resource state information can be denoted as {“RescourceA”:“IDLE”,“RescourceB”:“LIGHT”,“RescourceC”:“MODERATE”,“RescourceD”:“HEAVY”}. Among them, the resource state level of RescourceA is IDLE, representing that the resource state level of RescourceA is the first level, and the task resource RescourceA is very sufficient; the resource state level of RescourceB is LIGHT, representing that the resource state level of RescourceB is the second level, and the task resource RescourceB is relatively sufficient; the resource state level of RescourceC is MODERATE, representing that the resource state level of RescourceC is the third level, and the task resource RescourceC is relatively nervous; and the resource state level of RescourceD is HEAVY, representing that the resource state level of RescourceD is the fourth level, and the task resource RescourceD is very nervous.

[0051] In this embodiment, before all to-be-executed tasks are formally executed by the continuous integration platform, the scheduling system can dynamically obtain the task resource type corresponding to each to-be-executed task according to the configuration of the continuous integration platform, and insert each to-be-executed task into different task queues according to the task resource type corresponding to each to-be-executed task.

[0052] Among them, the task resource types corresponding to all to-be-executed tasks in the same task queue are the same.

[0053] Optionally, the task queue is a Redis queue.

[0054] Further, the scheduling system can select a corresponding number of to-be-executed tasks from different task queues according to the resource state levels of different types of task resources inferred, and execute the to-be-executed tasks, thereby realizing the task scheduling of the continuous integration platform.

[0055] For example, the resource state level of the task resource RescourceA is the lowest, which indicates that the task resource RescourceA is the most sufficient, and at this time, multiple to-be-executed tasks of the task resource type RescourceA can be selected from the task queue and executed; the resource state level of the task resource RescourceD is the highest, which indicates that the task resource RescourceD is the least, and at this time, an individual to-be-executed task of the task resource type RescourceD can be selected from the task queue and executed, or even no to-be-executed task can be selected and executed.

[0056] In some embodiments, based on the waiting time, after generating the resource state information of the continuous integration platform, the method further includes: if the waiting time is greater than or equal to a preset threshold, generating a reconnaissance task; based on the reconnaissance task, reacquiring the corresponding task resource for the to-be-executed task; and if the to-be-executed task acquires the corresponding task resource successfully, correcting the resource state information.

[0057] It can be understood that, since the process of determining the current resource state of the continuous integration platform by the dotting method does not realize real-time and accurate monitoring of different types of task resources, the resource state information determined by this method may have some errors compared with the actual situation, that is, the resource state information indirectly inferred from the task state information of each to-be-executed task cannot completely reflect the current real resource state of the continuous integration platform, and the inferred resource state information is only a predicted value close to the real resource state, so the indirectly inferred resource state information can only reflect the “resource fuzzy state” of the continuous integration platform.

[0058] Since the resource state information can only reflect the “resource fuzzy state” of the continuous integration platform, when a certain task resource has a monitoring fault or the running logic of the task itself has a problem, the scheduling system will have a certain probability of resource state misjudgment, and therefore a resource state correction mechanism needs to be introduced.

[0059] Specifically, for each to-be-executed task, if the task state information of the to-be-executed task only includes a resource waiting identification point (i.e., a “WAIT” point) and does not include a resource locking identification point (i.e., a “LOCK” point), it indicates that the current corresponding task resource of the continuous integration platform is insufficient, and at this time, the waiting time of the to-be-executed task can be recorded.

[0060] Further, for each to-be-executed task, if the waiting time of the to-be-executed task is greater than or equal to a preset threshold, the scheduling system can trigger a corresponding reconnaissance task (also referred to as a scout task) for the to-be-executed task, and the scheduling system can execute the reconnaissance task to attempt to reacquire the corresponding task resource for the to-be-executed task.

[0061] If the task resource corresponding to the to-be-executed task is acquired successfully, it indicates that the current task resource of the continuous integration platform is sufficient, i.e., the previous resource state information of the scheduling system is misjudged, at this time, the to-be-executed task can be removed from the waiting state, and the resource state information is corrected.

[0062] If the task resource corresponding to the to-be-executed task fails to be acquired, it indicates that the previous resource state information of the scheduling system is not misjudged, and there is no need to correct the resource state information.

[0063] In some embodiments, the number of to-be-executed tasks is at least two; based on the resource state information, the task scheduling of the to-be-executed tasks comprises: determining the task score of each to-be-executed task; based on the resource state information and the task score of each to-be-executed task, selecting at least one target to-be-executed task from all to-be-executed tasks; based on the continuous integration platform, executing the target to-be-executed task.

[0064] As shown in Figure 2 Before all to-be-executed tasks are executed formally by the continuous integration platform, the scheduling system can dynamically acquire the task resource type corresponding to each to-be-executed task according to the configuration of the continuous integration platform, and insert each to-be-executed task into different task queues according to the task resource type corresponding to each to-be-executed task.

[0065] Among them, the task resource types corresponding to all to-be-executed tasks in the same task queue are the same.

[0066] Optionally, the task queue is a Redis queue.

[0067] Further, after generating the resource state information of the continuous integration platform, the task score of each to-be-executed task can be determined first, and then a corresponding number of to-be-executed tasks are selected as target to-be-executed tasks from different task queues according to the resource state information of the continuous integration platform and the task score of each to-be-executed task.

[0068] For example, the resource state level of the task resource RescourceA is the lowest, which indicates that the task resource RescourceA is the most sufficient, at this time, 10 to-be-executed tasks of the task resource type RescourceA can be selected and executed, if there are 20 to-be-executed tasks in the task queue of the task resource type RescourceA, 10 to-be-executed tasks with the highest task score can be selected as target to-be-executed tasks from the 20 to-be-executed tasks; the resource state level of the task resource RescourceD is the highest, which indicates that the task resource RescourceD is the least, at this time, 1 to-be-executed task with the highest task score of the task resource type RescourceD can be selected as a target to-be-executed task.

[0069] Further, all the selected target pending tasks are executed by the continuous integration platform, and the task status of all the target pending tasks is marked as "running", which means the task is being executed.

[0070] Optionally, if the scheduling system is pre-configured with a shared resource policy and a throttling policy, the target pending task is executed by referring to the shared resource policy and the throttling policy when the target pending task is selected and executed.

[0071] For example, assuming there are a task queue groupA of task resource type RescourceA and a task queue groupB of task resource type RescourceB, and the scheduling system is pre-configured to allow the pending tasks in groupA to use the task resource RescourceB of groupB, then when the continuous integration platform lacks the task resource RescourceA, and the number of pending tasks in groupA that use the task resource RescourceB of groupB does not reach the upper limit of throttling, the scheduling system can still select a certain number of pending tasks from groupA as target pending tasks and execute them.

[0072] Optionally, during the operation of the continuous integration platform, the scheduling system can run a daemon program to monitor all the target pending tasks marked as "running" on the continuous integration platform until the target pending tasks are executed, at which time the task status of the executed target pending tasks can be marked as "done", which means the task is executed.

[0073] In some embodiments, the pending tasks are stored in the task queue; determining the task score of each pending task comprises: respectively determining the priority of each pending task, the compilation time of each pending task, and the time when each pending task enters the task queue; and determining the task score of each pending task based on the priority of each pending task, the compilation time of each pending task, the time when each pending task enters the task queue, and the dependency relationship between all the pending tasks.

[0074] As Figure 2As shown, before all the to-be-executed tasks are executed by the continuous integration platform, the scheduling system can store each to-be-executed task to be triggered by the trigger into a task state database of the scheduling system through a REST API (Representational State Transfer Application Programming Interface, an application programming interface complying with the REST architecture specification), and mark the task state of each to-be-executed task as "wait", indicating that the task is waiting for execution.

[0075] Further, the pre-check center of the scheduling system can dynamically obtain the task resource type corresponding to each to-be-executed task according to the configuration of the continuous integration platform, and insert each to-be-executed task into different task queues according to the task resource type corresponding to each to-be-executed task, and mark the task state of all to-be-executed tasks inserted into the task queue as "ready", indicating that the task has completed the execution preparation.

[0076] Among them, the task resource types corresponding to all to-be-executed tasks in the same task queue are the same.

[0077] Optionally, the task queue is a Redis queue, and the data structure adopted by the Redis queue is an ordered set ZSet.

[0078] Further, the scheduling system needs to determine the task score of each to-be-executed task inserted into different task queues.

[0079] Specifically, the scheduling system can first determine the priority configured by the trigger for each to-be-executed task, the compilation time of each to-be-executed task, and the time when each to-be-executed task enters the corresponding task queue.

[0080] Further, the scheduling system can determine the task score of each to-be-executed task according to the priority of each to-be-executed task, the compilation time of each to-be-executed task, the time when each to-be-executed task enters the task queue, and the dependency relationship between all to-be-executed tasks.

[0081] Optionally, if there is a dependency relationship between multiple to-be-executed tasks, the to-be-executed task corresponding to the less dependency relationship, the higher priority, and the shorter compilation time with other to-be-executed tasks has a higher task score; the to-be-executed task corresponding to the more dependency relationship, the lower priority, and the longer compilation time with other to-be-executed tasks has a lower task score.

[0082] In some embodiments, the task score of one task includes an integer part and a decimal part, the integer part is determined based on the priority and the compilation time of the task and the dependency relationship between all tasks, and the decimal part is determined based on the time when the task enters the task queue.

[0083] In this embodiment, for each task to be executed, the task score of the task to be executed includes an integer part and a decimal part, the integer part is determined by the scheduling system based on the priority and the compilation time of the task and the dependency relationship between all tasks, and the decimal part is determined by the scheduling system based on the time when the task enters the task queue, which can ensure that the task with high priority has high score and the task that enters the task queue early has high score, and these tasks with high score can be executed preferentially in the subsequent task scheduling process.

[0084] The task scheduling method provided in this embodiment is a task scheduling method based on resource fuzzy state applicable to the continuous integration process, the basic principle of which is to determine the task state information of each task on the continuous integration platform by means of dotting during the task running process, and then indirectly infer the current resource state information of the continuous integration platform according to the task state information of each task, and finally select a corresponding number of tasks from different task queues according to the current resource state information of the continuous integration platform and execute them, thereby realizing the task scheduling of the continuous integration platform. This method avoids the way of directly monitoring different types of task resources in real time, which can save a lot of monitoring resources, reduce the resource monitoring cost, improve the resource monitoring efficiency, and is conducive to improving the efficiency of subsequent task scheduling and resource utilization, and has high universality and scalability.

[0085] To verify the effectiveness of the task scheduling method provided in this embodiment, part of the experimental data of the task scheduling method of this embodiment is provided here for reference. In the verification experiment, the task scheduling method of this embodiment is applied to the CI / CD project, and the experimental results are shown in Table 1.

[0086] Table 1 Time First Week Second Week Third Week Fourth Week Max latency (min) 593.93 417.53 495 181 80th percentile latency (min) 87.39 48.73 171.2 30.72 Average latency (min) 45.13 28.39 76.68 19.3 Resource utilization (%) 85.32 86.6 86.47 87.8 In the verification experiment, the task scheduling method provided in the prior art is used from the first week to the third week, and the task scheduling method of the embodiment is used from the fourth week. As shown in Table 1, after the task scheduling method of the embodiment is used, the maximum waiting time of the high-priority task, the 80% quantile waiting time, and the average waiting time are all decreased compared with the case where the task scheduling method of the embodiment is not used, and the maximum waiting time of the high-priority task is decreased by 82.97% compared with the first week. Moreover, without adjusting the task throughput of the continuous integration platform, the resource utilization of the continuous integration platform is increased, which fully verifies the effectiveness of the task scheduling method of the embodiment.

[0087] The application further provides a task scheduling device. Figure 3 , Figure 3 is a structural schematic diagram of the task scheduling device provided by the application. In the embodiment, the task scheduling device comprises an acquisition module 310, a resource state prediction module 320, and a task scheduling module 330.

[0088] The acquisition module 310 is configured to acquire task state data.

[0089] The task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can acquire corresponding task resources from the continuous integration platform.

[0090] The resource state prediction module 320 is configured to predict a resource state of the continuous integration platform based on the task state data, and generate resource state information of the continuous integration platform.

[0091] The task scheduling module 330 is configured to perform task scheduling on the to-be-executed task based on the resource state information.

[0092] In some embodiments, the task state data is generated in a dotting manner, the task state information comprises a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded by the to-be-executed task when the to-be-executed task is waiting to acquire corresponding task resources, and the resource locking identification point is an identification point recorded by the to-be-executed task when the to-be-executed task acquires corresponding task resources; the resource state of the continuous integration platform is predicted based on the task state data, and the resource state information of the continuous integration platform is generated, comprising: if the task state information of the to-be-executed task comprises the resource waiting identification point and does not comprise the resource locking identification point, recording a waiting time of the to-be-executed task; predicting the resource state of the continuous integration platform based on the waiting time, and generating the resource state information of the continuous integration platform.

[0093] In some embodiments, based on the latency, the method further comprises: generating a reconnaissance task if the latency is greater than or equal to a preset threshold; reacquiring the corresponding task resource for the to-be-executed task based on the reconnaissance task; and correcting the resource state information if the corresponding task resource for the to-be-executed task is successfully acquired.

[0094] In some embodiments, the number of to-be-executed tasks is at least two; the task scheduling based on the resource state information comprises: determining a task score of each to-be-executed task; selecting at least one target to-be-executed task from all to-be-executed tasks based on the resource state information and the task score of each to-be-executed task; and executing the target to-be-executed task based on the continuous integration platform.

[0095] In some embodiments, the to-be-executed tasks are stored in a task queue; the task score of each to-be-executed task is determined by: determining the priority of each to-be-executed task, the compilation time of each to-be-executed task, and the time when each to-be-executed task enters the task queue, respectively; and determining the task score of each to-be-executed task based on the priority of each to-be-executed task, the compilation time of each to-be-executed task, the time when each to-be-executed task enters the task queue, and the dependency relationship between all to-be-executed tasks.

[0096] In some embodiments, one task score comprises an integer part and a decimal part; the integer part is determined based on the priority and the compilation time of one to-be-executed task and the dependency relationship between all to-be-executed tasks; and the decimal part is determined based on the time when one to-be-executed task enters the task queue.

[0097] The present application also provides an electronic device. Figure 4 is a structural schematic diagram of the electronic device provided by the present application, as Figure 4 shown, the electronic device can include a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 complete mutual communication through the communication bus 440. The processor 410 can invoke the logical instructions in the memory 430 to execute a task scheduling method, which comprises: acquiring task state data; the task state data comprises task state information of to-be-executed tasks, and the task state information is used to identify whether the to-be-executed tasks can acquire corresponding task resources from a continuous integration platform; predicting the resource state of the continuous integration platform based on the task state data to generate resource state information of the continuous integration platform; and scheduling the to-be-executed tasks based on the resource state information.

[0098] In addition, the logic instructions in the memory 430 described above can be realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0099] The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the task scheduling method provided by the above methods, and the task scheduling method comprises: obtaining task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform; based on the task state data, predicting a resource state of the continuous integration platform to generate resource state information of the continuous integration platform; and based on the resource state information, performing task scheduling on the to-be-executed task.

[0100] The present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute the task scheduling method provided by the above methods, and the task scheduling method comprises: obtaining task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform; based on the task state data, predicting a resource state of the continuous integration platform to generate resource state information of the continuous integration platform; and based on the resource state information, performing task scheduling on the to-be-executed task.

[0101] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0102] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and of course, the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the method described in each embodiment or some parts of the embodiment.

[0103] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features thereof; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A task scheduling method, characterized in that, include: Get task status data; The task status data includes task status information of the task to be executed, which is used to identify whether the task to be executed can obtain the corresponding task resources from the continuous integration platform. Based on the task status data, the resource status of the continuous integration platform is predicted, and the resource status information of the continuous integration platform is generated. Based on the resource status information, the tasks to be executed are scheduled.

2. The task scheduling method according to claim 1, characterized in that, The task status data is generated based on a point-based method. The task status information includes resource waiting markers and / or resource locking markers. The resource waiting markers are markers recorded by the task to be executed while waiting to acquire the corresponding task resource. The resource locking markers are markers recorded by the task to be executed when it acquires the corresponding task resource. The step of predicting the resource status of the continuous integration platform based on the task status data and generating resource status information of the continuous integration platform includes: If the task status information of the task to be executed includes the resource waiting identifier but does not include the resource locking identifier, then the waiting time of the task to be executed is recorded. Based on the waiting time, the resource status of the continuous integration platform is predicted, and the resource status information of the continuous integration platform is generated.

3. The task scheduling method according to claim 2, characterized in that, After predicting the resource status of the continuous integration platform based on the waiting time and generating the resource status information of the continuous integration platform, the method further includes: If the waiting time is greater than or equal to a preset threshold, a reconnaissance mission is generated; Based on the reconnaissance mission, the corresponding mission resources are reacquired for the mission to be executed. If the task resource corresponding to the task to be executed is successfully acquired, the resource status information is corrected.

4. The task scheduling method according to claim 1, characterized in that, The number of tasks to be executed is at least two; The step of scheduling the tasks to be executed based on the resource status information includes: Determine the task score for each of the tasks to be performed; Based on the resource status information and the task score of each of the tasks to be executed, at least one target task to be executed is selected from all the tasks to be executed; Based on the continuous integration platform, the target task to be executed is performed.

5. The task scheduling method according to claim 4, characterized in that, The tasks to be executed are stored in a task queue; Determining the task score for each of the tasks to be executed includes: The priority of each task to be executed, the compilation time of each task to be executed, and the time when each task to be executed enters the task queue are determined respectively. The task score of each task to be executed is determined based on the priority of each task to be executed, the compilation time of each task to be executed, the time when each task to be executed enters the task queue, and the dependencies between all tasks to be executed.

6. The task scheduling method according to claim 5, characterized in that, A task score includes an integer part and a fractional part, the integer part being determined based on the priority and compilation time of a task to be executed and the dependencies between all tasks to be executed, and the fractional part being determined based on the time when a task to be executed enters the task queue.

7. A task scheduling device, characterized in that, include: The acquisition module is used to acquire task status data; The task status data includes task status information of the task to be executed, which is used to identify whether the task to be executed can obtain the corresponding task resources from the continuous integration platform. The resource status prediction module is used to predict the resource status of the continuous integration platform based on the task status data, and generate the resource status information of the continuous integration platform. The task scheduling module is used to schedule the tasks to be executed based on the resource status information.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the task scheduling method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the task scheduling method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the task scheduling method as described in any one of claims 1 to 6.

Images