Task scheduling method and device, equipment, storage medium and program product

Abstract

The application provides a task scheduling method and device, equipment, a storage medium and a program product, and relates to the technical field of computers. The method comprises the following steps: obtaining task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used for identifying whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform; based on the task state data, the resource state of the continuous integration platform is predicted to generate resource state information of the continuous integration platform; and based on the resource state information, the to-be-executed task is scheduled. In the foregoing manner, the resource state information of the continuous integration platform can be indirectly inferred without using different platforms and methods to perform real-time monitoring on each kind of task resource, a large amount of monitoring resources can be saved, the resource monitoring cost is reduced, the resource monitoring efficiency is improved, and then the efficiency of subsequent task scheduling and the resource utilization rate are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular to a task scheduling method and device, equipment, a storage medium and a program product. BACKGROUND

[0002] CI / CD (Continuous Integration / Continuous Deployment) is a software development technology, and developers can continuously integrate and deploy code through automated processes and tools on a continuous integration platform to perform different tasks.

[0003] In the process of software development, the continuous integration platform runs various tasks, such as MR (Merge Request) tasks of developers, high-priority verification tasks, and routine verification tasks. However, due to the needs of business testing, different tasks need to use different task resources during execution or testing, such as physical machine or virtual machine resources for some tasks, and cloud resources for some tasks. To achieve efficient task scheduling, the resource status of the continuous integration platform needs to be monitored in real time to schedule tasks according to the real-time resource status of the continuous integration platform.

[0004] In the prior art, different types of task resources on the continuous integration platform need to be monitored by different platforms or methods, such as cloud resources that need to be monitored by a cloud platform. Real-time monitoring of each type of task resource using different platforms and methods can ensure that the resource status information of the continuous integration platform obtained by monitoring is accurate, and then a FIFO (First-In First-Out) scheduling strategy can be used to schedule tasks to be executed on the continuous integration platform.

[0005] Although real-time monitoring of each type of task resource using different platforms and methods can obtain accurate resource status information of the continuous integration platform, this method consumes a large amount of monitoring resources, resulting in high resource monitoring costs, low resource monitoring efficiency, and easy impact on the efficiency and resource utilization of subsequent task scheduling. SUMMARY

[0006] The present application provides a task scheduling method, device, equipment, storage medium and program product to solve the defects of high resource monitoring cost, low resource monitoring efficiency, and easy impact on the efficiency and resource utilization of subsequent task scheduling of the existing task scheduling method.

[0007] The application provides a task scheduling method, comprising: obtaining task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used for identifying whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform; based on the task state data, the resource state of the continuous integration platform is predicted to generate resource state information of the continuous integration platform; and based on the resource state information, the to-be-executed task is scheduled.

[0008] According to the task scheduling method provided by the application, the task state data is generated in a dotting manner, the task state information comprises a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded when the to-be-executed task waits to obtain corresponding task resources, and the resource locking identification point is an identification point recorded when the to-be-executed task obtains corresponding task resources; based on the task state data, the resource state of the continuous integration platform is predicted to generate resource state information of the continuous integration platform, comprising: if the task state information of the to-be-executed task comprises the resource waiting identification point and does not comprise the resource locking identification point, the waiting time of the to-be-executed task is recorded; based on the waiting time, the resource state of the continuous integration platform is predicted to generate the resource state information of the continuous integration platform.

[0009] According to the task scheduling method provided by the application, after the resource state of the continuous integration platform is predicted based on the waiting time to generate the resource state information of the continuous integration platform, the method further comprises: if the waiting time is greater than or equal to a preset threshold, a reconnaissance task is generated; based on the reconnaissance task, corresponding task resources of the to-be-executed task are reacquired; and if the to-be-executed task successfully acquires the corresponding task resources, the resource state information is corrected.

[0010] According to the task scheduling method provided by the application, the number of to-be-executed tasks is at least two; based on the resource state information, the to-be-executed tasks are scheduled, comprising: determining a task score of each to-be-executed task; based on the resource state information and the task score of each to-be-executed task, at least one target to-be-executed task is selected from all to-be-executed tasks; and based on the continuous integration platform, the target to-be-executed task is executed.

[0011] According to the task scheduling method provided by the application, the to-be-executed tasks are stored in a task queue; the task score of each to-be-executed task is determined, comprising: respectively determining the priority of each to-be-executed task, the compilation time length of each to-be-executed task and the time when each to-be-executed task enters the task queue; based on the priority of each to-be-executed task, the compilation time length of each to-be-executed task, the time when each to-be-executed task enters the task queue and the dependency relationship between all to-be-executed tasks, the task score of each to-be-executed task is determined.

[0012] According to the task scheduling method provided by the application, a task score includes an integer part and a decimal part, the integer part is determined based on the priority and the compilation time of a to-be-executed task and the dependency relationship among all to-be-executed tasks, and the decimal part is determined based on the time when the to-be-executed task enters a task queue.

[0013] The application further provides a task scheduling device, comprising: an acquisition module, configured to acquire task state data; the task state data comprises task state information of to-be-executed tasks, and the task state information is used to identify whether the to-be-executed tasks can acquire corresponding task resources from a continuous integration platform; a resource state prediction module, configured to predict the resource state of the continuous integration platform based on the task state data, and generate resource state information of the continuous integration platform; and a task scheduling module, configured to perform task scheduling on the to-be-executed tasks based on the resource state information.

[0014] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements any of the above task scheduling methods when executing the computer program.

[0015] The application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement any of the above task scheduling methods.

[0016] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement any of the above task scheduling methods.

[0017] The task scheduling method, device, equipment, storage medium and program product provided by the present application avoid the mode of directly monitoring different types of task resources in real time, but first acquire task state data of a continuous integration platform, and the task state data includes task state information of each to-be-executed task on the continuous integration platform. Since the task state information is used to identify whether the to-be-executed task can acquire corresponding task resources from the continuous integration platform, if the to-be-executed task can acquire corresponding task resources from the continuous integration platform, it indirectly indicates that the corresponding task resources are sufficient, and if the to-be-executed task cannot acquire corresponding task resources from the continuous integration platform, it indirectly indicates that the corresponding task resources are insufficient. That is, the task state information of each to-be-executed task can indirectly reflect the task resource situation that can be currently provided by the continuous integration platform. Therefore, the resource state of the continuous integration platform can be indirectly inferred according to the task state information of each to-be-executed task, the current resource state information of the continuous integration platform is generated, and the to-be-executed task is scheduled according to the indirectly inferred resource state information. Since the resource state information of the continuous integration platform can be indirectly inferred without using different platforms and methods to monitor each type of task resource in real time, a large amount of monitoring resources can be saved, the resource monitoring cost is reduced, the resource monitoring efficiency is improved, and the efficiency and resource utilization rate of subsequent task scheduling are improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is one of the flowcharts of the task scheduling method provided by the present application.

[0020] Figure 2 is another flowchart of the task scheduling method provided by the present application.

[0021] Figure 3 is a structural schematic diagram of the task scheduling device provided by the present application.

[0022] Figure 4 is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0023] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0024] Please refer to Figure 1 and Figure 2 , Figure 1 is one of the flowcharts of the task scheduling method provided by the present application, Figure 2 is another flowchart of the task scheduling method provided by the present application.

[0025] As Figure 1 shown, in the present embodiment, the task scheduling method is applied to a scheduling system, and the task scheduling method comprises steps S110 to S130, and each step is specifically as follows:

[0026] S110: obtaining task state data.

[0027] The task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform.

[0028] In the present embodiment, the basic principle of the task scheduling method is to determine the task state information of each task on the continuous integration platform (i.e. CI platform) by means of dotting in the task running process, and then to indirectly infer the current resource state information of the continuous integration platform according to the task state information of each task, and finally to select a corresponding number of tasks from the task queue and execute them according to the current resource state information of the continuous integration platform, so as to realize the task scheduling of the continuous integration platform.

[0029] Specifically, a task state database is maintained, and the task state database stores the task state information of all to-be-executed tasks on the continuous integration platform, and the task state data comprises the task state information of these to-be-executed tasks.

[0030] Among them, the task state data is generated based on the dotting method, that is, the task state information of each to-be-executed task is generated based on the dotting method.

[0031] Among them, the task state information comprises a resource waiting identification point and / or a resource locking identification point; the resource waiting identification point (i.e. “WAIT” point) is an identification point recorded by the to-be-executed task when it is waiting to obtain corresponding task resources; and the resource locking identification point (i.e. “LOCK” point) is an identification point recorded by the to-be-executed task when it obtains corresponding task resources.

[0032] The dotting during the task running process means that for each task, a resource waiting identification point (i.e., a "WAIT" point) is set for the task before the task obtains the corresponding task resource from the continuous integration platform, a resource locking identification point (i.e., a "LOCK" point) is set for the task after the task obtains the corresponding task resource from the continuous integration platform, and each time a resource waiting identification point or a resource locking identification point is added, the added identification point information is taken as the task state information, and the task state information is stored in the task state database.

[0033] It should be noted that for each to-be-executed task, the task state information can indicate whether the to-be-executed task can currently obtain the corresponding task resource from the continuous integration platform: if the to-be-executed task can currently obtain the corresponding task resource from the continuous integration platform, it indirectly indicates that the corresponding task resource of the continuous integration platform is currently sufficient; if the to-be-executed task cannot currently obtain the corresponding task resource from the continuous integration platform, it indirectly indicates that the corresponding task resource of the continuous integration platform is currently insufficient. Based on the above principle, the task state information of each to-be-executed task can actually indirectly reflect the current task resource situation of the continuous integration platform, and thus the current resource state of the continuous integration platform can be indirectly inferred according to the task state information of each to-be-executed task.

[0034] Further, the scheduling system can periodically scan the task state database according to a preset period, obtain the current task state data of the continuous integration platform from the task state database, and use the data as a data basis for subsequent resource state inference.

[0035] S120: predicting the resource state of the continuous integration platform based on the task state data, and generating resource state information of the continuous integration platform.

[0036] Specifically, the resource state of the continuous integration platform is predicted based on the task state information of each to-be-executed task, and the resource state information of the continuous integration platform is generated.

[0037] It should be noted that since the process of determining the current resource state of the continuous integration platform by dotting does not achieve real-time and accurate monitoring of different types of task resources, the resource state information determined by this method may have some errors compared with the actual situation, that is, the indirectly inferred resource state information based on the task state information of each to-be-executed task cannot completely reflect the current real resource state of the continuous integration platform, and the inferred resource state information is only a predicted value that is relatively close to the real resource state, so the indirectly inferred resource state information can only reflect the "resource ambiguous state" of the continuous integration platform.

[0038] S130: scheduling the to-be-executed task based on the resource state information.

[0039] Specifically, the to-be-executed tasks on the continuous integration platform are scheduled based on the resource state information that can reflect the current "resource ambiguous state" of the continuous integration platform.

[0040] The task scheduling method provided in this embodiment avoids the way of directly monitoring different types of task resources in real time, but first acquires task state data of the continuous integration platform, and the task state data includes task state information of each to-be-executed task on the continuous integration platform. Since the task state information is used to identify whether the to-be-executed task can acquire the corresponding task resource from the continuous integration platform, if the to-be-executed task can acquire the corresponding task resource from the continuous integration platform, it means that the corresponding task resource is sufficient, and if the to-be-executed task cannot acquire the corresponding task resource from the continuous integration platform, it means that the corresponding task resource is insufficient. That is, the task state information of each to-be-executed task can indirectly reflect the current task resource situation of the continuous integration platform, so the resource state of the continuous integration platform can be indirectly inferred based on the task state information of each to-be-executed task, the current resource state information of the continuous integration platform is generated, and the to-be-executed task is scheduled based on the indirectly inferred resource state information. Since the resource state information of the continuous integration platform can be indirectly inferred without using different platforms and methods to monitor each type of task resource in real time, a large amount of monitoring resources can be saved, the resource monitoring cost can be reduced, the resource monitoring efficiency can be improved, and the efficiency of subsequent task scheduling and the resource utilization rate can be improved.

[0041] In some embodiments, the task state data is generated based on the dotting method, the task state information includes a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded by the to-be-executed task when waiting to acquire the corresponding task resource, and the resource locking identification point is an identification point recorded by the to-be-executed task when acquiring the corresponding task resource. Based on the task state data, the resource state of the continuous integration platform is predicted to generate the resource state information of the continuous integration platform, including: if the task state information of the to-be-executed task includes the resource waiting identification point and does not include the resource locking identification point, the waiting time of the to-be-executed task is recorded; based on the waiting time, the resource state of the continuous integration platform is predicted to generate the resource state information of the continuous integration platform.

[0042] Specifically, a task state database is first maintained, and the task state database stores the task state information of all to-be-executed tasks on the continuous integration platform, and the task state data includes the task state information of these to-be-executed tasks.

[0043] The task state data is generated based on the marking method, i.e., the task state information of each to-be-executed task is generated based on the marking method.

[0044] The task state information includes a resource waiting mark point and / or a resource locking mark point; the resource waiting mark point (i.e., a "WAIT" point) is a mark point recorded by the to-be-executed task when waiting for obtaining the corresponding task resource; and the resource locking mark point (i.e., a "LOCK" point) is a mark point recorded by the to-be-executed task when obtaining the corresponding task resource.

[0045] The marking in the task running process means that for each task, a resource waiting mark point (i.e., a "WAIT" point) is set for the task before the task obtains the corresponding task resource from the continuous integration platform, a resource locking mark point (i.e., a "LOCK" point) is set for the task after the task obtains the corresponding task resource from the continuous integration platform, and the added mark point information is taken as the task state information each time a resource waiting mark point or a resource locking mark point is added, and the task state information is stored in the task state database.

[0046] Further, the scheduling system can periodically scan the task state database according to the preset period, obtain the current task state data of the continuous integration platform from the task state database, and take the data as the data basis for subsequent resource state speculation.

[0047] Further, for each to-be-executed task, if the task state information of the to-be-executed task only includes the resource waiting mark point (i.e., a "WAIT" point) but does not include the resource locking mark point (i.e., a "LOCK" point), it indicates that the current corresponding task resource of the continuous integration platform is insufficient, and at this time, the waiting time of the to-be-executed task can be recorded.

[0048] Meanwhile, for each to-be-executed task, if the task state information of the to-be-executed task includes both the resource waiting mark point (i.e., a "WAIT" point) and the resource locking mark point (i.e., a "LOCK" point), it indicates that the current corresponding task resource of the continuous integration platform is sufficient.

[0049] Further, the scheduling system can predict the resource state of the continuous integration platform according to the waiting time of all to-be-executed tasks with insufficient resources (i.e., the to-be-executed tasks whose task state information only includes the resource waiting mark point but does not include the resource locking mark point), according to the preconfigured relationship between the waiting time and the resource state, and generate the resource state information of the continuous integration platform.

[0050] The resource state information of the continuous integration platform includes the resource state levels of different types of task resources, and the lower the resource state level is, the more sufficient the corresponding task resource is.

[0051] For example, assuming that the continuous integration platform includes four task resources, denoted as RescourceA, RescourceB, RescourceC and RescourceD, the resource state information inferred from the task state information of each to-be-executed task can include the resource state levels of the four task resources, and the resource state information can be denoted as {“RescourceA”:“IDLE”,“RescourceB”:“LIGHT”,“RescourceC”:“MODERATE”,“RescourceD”:“HEAVY”}. Among them, the resource state level of RescourceA is IDLE, representing that the resource state level of RescourceA is the first level, and the task resource RescourceA is very sufficient; the resource state level of RescourceB is LIGHT, representing that the resource state level of RescourceB is the second level, and the task resource RescourceB is relatively sufficient; the resource state level of RescourceC is MODERATE, representing that the resource state level of RescourceC is the third level, and the task resource RescourceC is relatively nervous; and the resource state level of RescourceD is HEAVY, representing that the resource state level of RescourceD is the fourth level, and the task resource RescourceD is very nervous.

[0052] In this embodiment, before all to-be-executed tasks are formally executed by the continuous integration platform, the scheduling system can dynamically obtain the task resource type corresponding to each to-be-executed task according to the configuration of the continuous integration platform, and insert each to-be-executed task into different task queues according to the task resource type corresponding to each to-be-executed task.

[0053] Among them, all to-be-executed tasks in the same task queue correspond to the same task resource type.

[0054] Optionally, the task queue is a Redis queue.

[0055] Further, the scheduling system can select a corresponding number of to-be-executed tasks from different task queues according to the resource state levels of different types of task resources inferred, and execute the to-be-executed tasks, thereby realizing the task scheduling of the continuous integration platform.

[0056] For example, the resource state level of the task resource RescourceA is the lowest, which means that the task resource RescourceA is the most sufficient, and at this time, multiple to-be-executed tasks of the task resource type RescourceA can be selected from the task queue and executed; the resource state level of the task resource RescourceD is the highest, which means that the task resource RescourceD is the least, and at this time, an individual to-be-executed task of the task resource type RescourceD can be selected from the task queue and executed, or even no to-be-executed task can be selected and executed.

[0057] In some embodiments, based on the waiting time, after generating the resource state information of the continuous integration platform, the method further comprises: if the waiting time is greater than or equal to a preset threshold, generating a reconnaissance task; based on the reconnaissance task, reacquiring the corresponding task resource for the to-be-executed task; and if the corresponding task resource for the to-be-executed task is acquired successfully, correcting the resource state information.

[0058] It can be understood that, since the process of determining the current resource state of the continuous integration platform by the dotting method does not realize real-time and accurate monitoring of different types of task resources, it may lead to a certain error in the resource state information determined by this method, that is, the resource state information indirectly inferred from the task state information of each to-be-executed task cannot completely reflect the current real resource state of the continuous integration platform, and the inferred resource state information is only a predicted value close to the real resource state, so the indirectly inferred resource state information can only reflect the “resource fuzzy state” of the continuous integration platform.

[0059] Since the resource state information can only reflect the “resource fuzzy state” of the continuous integration platform, when a certain task resource has a monitoring fault or the running logic of the task itself has a problem, the scheduling system will have a certain probability of resource state misjudgment, and therefore a resource state correction mechanism needs to be introduced.

[0060] Specifically, for each to-be-executed task, if the task state information of the to-be-executed task only includes a resource waiting identification point (i.e., a “WAIT” point) and does not include a resource locking identification point (i.e., a “LOCK” point), it means that the current corresponding task resource of the continuous integration platform is insufficient, and at this time, the waiting time of the to-be-executed task can be recorded.

[0061] Further, for each to-be-executed task, if the waiting time of the to-be-executed task is greater than or equal to a preset threshold, the scheduling system can trigger a corresponding reconnaissance task (also known as a scout task) for the to-be-executed task, and the scheduling system can execute the reconnaissance task to attempt to reacquire the corresponding task resource for the to-be-executed task.

[0062] If the task resource corresponding to the to-be-executed task is acquired successfully, it indicates that the current task resource of the continuous integration platform is sufficient, i.e., the previous resource state information of the scheduling system is misjudged, at this time, the to-be-executed task can be removed from the waiting state, and the resource state information is corrected.

[0063] If the task resource corresponding to the to-be-executed task is not acquired, it indicates that the previous resource state information of the scheduling system is not misjudged, and the resource state information does not need to be corrected.

[0064] In some embodiments, the number of to-be-executed tasks is at least two; based on the resource state information, the task scheduling of the to-be-executed tasks comprises: determining the task score of each to-be-executed task; based on the resource state information and the task score of each to-be-executed task, selecting at least one target to-be-executed task from all to-be-executed tasks; based on the continuous integration platform, executing the target to-be-executed task.

[0065] As shown in Figure 2 Before all to-be-executed tasks are executed formally by the continuous integration platform, the scheduling system can dynamically acquire the task resource type corresponding to each to-be-executed task according to the configuration of the continuous integration platform, and insert each to-be-executed task into different task queues according to the task resource type corresponding to each to-be-executed task.

[0066] Among them, the task resource types corresponding to all to-be-executed tasks in the same task queue are the same.

[0067] Optionally, the task queue is a Redis queue.

[0068] Further, after generating the resource state information of the continuous integration platform, the task score of each to-be-executed task can be determined first, and then a corresponding number of to-be-executed tasks are selected as target to-be-executed tasks from different task queues according to the resource state information of the continuous integration platform and the task score of each to-be-executed task.

[0069] For example, the resource state level of the task resource RescourceA is the lowest, which indicates that the task resource RescourceA is the most sufficient, at this time, 10 to-be-executed tasks of the task resource type RescourceA can be selected and executed, if there are 20 to-be-executed tasks in the task queue of the task resource type RescourceA, 10 to-be-executed tasks with the highest task score can be selected as target to-be-executed tasks from the 20 to-be-executed tasks; the resource state level of the task resource RescourceD is the highest, which indicates that the task resource RescourceD is the least, at this time, 1 to-be-executed task with the highest task score of the task resource type RescourceD can be selected as a target to-be-executed task.

[0070] Further, all the selected target pending tasks are executed by the continuous integration platform, and the task status of all the target pending tasks is marked as "running", which means the task is being executed.

[0071] Optionally, if the scheduling system is pre-configured with a shared resource policy and a throttling policy, the target pending task is executed by referring to the shared resource policy and the throttling policy when the target pending task is selected and executed.

[0072] For example, assuming there are a task queue groupA of task resource type RescourceA and a task queue groupB of task resource type RescourceB, and the scheduling system is pre-configured to allow the pending tasks in groupA to use the task resource RescourceB of groupB, then when the continuous integration platform lacks the task resource RescourceA, and the number of pending tasks in groupA that use the task resource RescourceB of groupB does not reach the upper limit of throttling, the scheduling system can still select a certain number of pending tasks from groupA as target pending tasks and execute them.

[0073] Optionally, during the operation of the continuous integration platform, the scheduling system can run a daemon program to monitor all the target pending tasks marked as "running" on the continuous integration platform until the target pending tasks are executed, at which time the task status of the executed target pending tasks can be marked as "done", which means the task is executed.

[0074] In some embodiments, the pending tasks are stored in the task queue; determining the task score of each pending task comprises: respectively determining the priority of each pending task, the compilation time of each pending task, and the time when each pending task enters the task queue; and determining the task score of each pending task based on the priority of each pending task, the compilation time of each pending task, the time when each pending task enters the task queue, and the dependency relationship between all the pending tasks.

[0075] As Figure 2As shown, before all the to-be-executed tasks are executed by the continuous integration platform, the scheduling system can store each to-be-executed task to be triggered by the trigger into a task state database of the scheduling system through a REST API (Representational State Transfer Application Programming Interface, an application programming interface complying with the REST architecture specification), and mark the task state of each to-be-executed task as "wait", indicating that the task is waiting for execution.

[0076] Further, the pre-check center of the scheduling system can dynamically obtain the task resource type corresponding to each to-be-executed task according to the configuration of the continuous integration platform, and insert each to-be-executed task into different task queues according to the task resource type corresponding to each to-be-executed task, and mark the task state of all to-be-executed tasks inserted into the task queue as "ready", indicating that the task has completed the preparation for execution.

[0077] Among them, the task resource types corresponding to all to-be-executed tasks in the same task queue are the same.

[0078] Optionally, the task queue is a Redis queue, and the data structure adopted by the Redis queue is an ordered set ZSet.

[0079] Further, the scheduling system needs to determine the task score of each to-be-executed task inserted into different task queues.

[0080] Specifically, the scheduling system can first determine the priority configured by the trigger for each to-be-executed task, the compilation time of each to-be-executed task, and the time when each to-be-executed task enters the corresponding task queue.

[0081] Further, the scheduling system can determine the task score of each to-be-executed task according to the priority of each to-be-executed task, the compilation time of each to-be-executed task, the time when each to-be-executed task enters the task queue, and the dependency relationship between all to-be-executed tasks.

[0082] Optionally, if there is a dependency relationship between multiple to-be-executed tasks, the to-be-executed task corresponding to the less dependency relationship with other to-be-executed tasks, the higher priority, and the shorter compilation time has a higher task score; the to-be-executed task corresponding to the more dependency relationship with other to-be-executed tasks, the lower priority, and the longer compilation time has a lower task score.

[0083] In some embodiments, the task score of one task includes an integer part and a decimal part, the integer part is determined based on the priority and compilation time of the task and the dependency relationship between all tasks, and the decimal part is determined based on the time when the task enters the task queue.

[0084] In this embodiment, for each task to be executed, the task score of the task to be executed includes an integer part and a decimal part, the integer part is determined by the scheduling system according to the priority and compilation time of the task to be executed and the dependency relationship between all tasks to be executed, and the decimal part is determined by the scheduling system according to the time when the task to be executed enters the task queue, which can ensure that the task score of the task to be executed with high priority and the task to be executed entering the task queue early is high, and these tasks to be executed with high scores can be executed preferentially in the subsequent task scheduling process.

[0085] The task scheduling method provided in this embodiment is a task scheduling method based on resource fuzzy state applicable to the continuous integration process, the basic principle of which is to determine the task state information of each task on the continuous integration platform by means of dotting in the task running process, and then indirectly infer the current resource state information of the continuous integration platform according to the task state information of each task, and finally select a corresponding number of tasks from different task queues according to the current resource state information of the continuous integration platform and execute them, thereby realizing the task scheduling of the continuous integration platform. This method avoids the way of directly monitoring different types of task resources in real time, which can save a lot of monitoring resources, reduce resource monitoring cost, improve resource monitoring efficiency, and is conducive to improving the efficiency and resource utilization rate of subsequent task scheduling, and has high universality and scalability.

[0086] To verify the effectiveness of the task scheduling method provided in this embodiment, part of the experimental data of the task scheduling method of this embodiment is provided here for reference. In the verification experiment, the task scheduling method of this embodiment is applied to the CI / CD project, and the experimental results are shown in Table 1.

[0087] Table 1

[0088] Time First Week Second Week Third Week Fourth Week Max latency (min) 593.93 417.53 495 181 80th percentile latency (min) 87.39 48.73 171.2 30.72 Average latency (min) 45.13 28.39 76.68 19.3 Resource utilization (%) 85.32 86.6 86.47 87.8

[0089] In the verification experiment, the task scheduling method provided in the prior art is used from the first week to the third week, and the task scheduling method of the embodiment is used from the fourth week. As shown in Table 1, after the task scheduling method of the embodiment is used, the maximum waiting time of the high-priority task, the 80% quantile waiting time, and the average waiting time are all decreased compared with the case where the task scheduling method of the embodiment is not used, and the maximum waiting time of the high-priority task is decreased by 82.97% compared with the first week. Moreover, without adjusting the task throughput of the continuous integration platform, the resource utilization of the continuous integration platform is increased, which fully verifies the effectiveness of the task scheduling method of the embodiment.

[0090] The application further provides a task scheduling device. Figure 3 , Figure 3 is a structural schematic diagram of the task scheduling device provided by the application. In the embodiment, the task scheduling device comprises an acquisition module 310, a resource state prediction module 320, and a task scheduling module 330.

[0091] The acquisition module 310 is configured to acquire task state data.

[0092] The task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can acquire corresponding task resources from the continuous integration platform.

[0093] The resource state prediction module 320 is configured to predict a resource state of the continuous integration platform based on the task state data, and generate resource state information of the continuous integration platform.

[0094] The task scheduling module 330 is configured to perform task scheduling on the to-be-executed task based on the resource state information.

[0095] In some embodiments, the task state data is generated in a dotting manner, the task state information comprises a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded by the to-be-executed task when the to-be-executed task is waiting to acquire corresponding task resources, and the resource locking identification point is an identification point recorded by the to-be-executed task when the to-be-executed task acquires corresponding task resources; the resource state of the continuous integration platform is predicted based on the task state data, and the resource state information of the continuous integration platform is generated, comprising: if the task state information of the to-be-executed task comprises the resource waiting identification point and does not comprise the resource locking identification point, recording a waiting time of the to-be-executed task; predicting the resource state of the continuous integration platform based on the waiting time, and generating the resource state information of the continuous integration platform.

[0096] In some embodiments, based on the latency, the method further comprises: generating a reconnaissance task if the latency is greater than or equal to a preset threshold; reacquiring the corresponding task resource for the to-be-executed task based on the reconnaissance task; and correcting the resource state information if the corresponding task resource for the to-be-executed task is successfully acquired.

[0097] In some embodiments, the number of to-be-executed tasks is at least two; the task scheduling based on the resource state information comprises: determining a task score of each to-be-executed task; selecting at least one target to-be-executed task from all to-be-executed tasks based on the resource state information and the task score of each to-be-executed task; and executing the target to-be-executed task based on the continuous integration platform.

[0098] In some embodiments, the to-be-executed tasks are stored in a task queue; the task score of each to-be-executed task is determined by: determining the priority of each to-be-executed task, the compilation time of each to-be-executed task, and the time when each to-be-executed task enters the task queue, respectively; and determining the task score of each to-be-executed task based on the priority of each to-be-executed task, the compilation time of each to-be-executed task, the time when each to-be-executed task enters the task queue, and the dependency relationship between all to-be-executed tasks.

[0099] In some embodiments, one task score comprises an integer part and a decimal part; the integer part is determined based on the priority and the compilation time of one to-be-executed task and the dependency relationship between all to-be-executed tasks; and the decimal part is determined based on the time when one to-be-executed task enters the task queue.

[0100] The present application also provides an electronic device. Figure 4 is a structural schematic diagram of the electronic device provided by the present application, as Figure 4 shown, the electronic device can include a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 complete mutual communication through the communication bus 440. The processor 410 can invoke the logical instructions in the memory 430 to execute a task scheduling method, which comprises: acquiring task state data; the task state data comprises task state information of to-be-executed tasks, and the task state information is used to identify whether the to-be-executed tasks can acquire corresponding task resources from a continuous integration platform; predicting the resource state of the continuous integration platform based on the task state data to generate resource state information of the continuous integration platform; and scheduling the to-be-executed tasks based on the resource state information.

[0101] In addition, the logic instructions in the memory 430 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0102] The present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the task scheduling method provided by the above methods, and the task scheduling method comprises: obtaining task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform; based on the task state data, predicting a resource state of the continuous integration platform to generate resource state information of the continuous integration platform; and based on the resource state information, performing task scheduling on the to-be-executed task.

[0103] The present application also provides a computer program product, which comprises a computer program, and the computer program can be stored on a non-transitory computer readable storage medium, and when the computer program is executed by a processor, the computer can execute the task scheduling method provided by the above methods, and the task scheduling method comprises: obtaining task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform; based on the task state data, predicting a resource state of the continuous integration platform to generate resource state information of the continuous integration platform; and based on the resource state information, performing task scheduling on the to-be-executed task.

[0104] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0105] Those skilled in the art can clearly understand the implementation of the embodiments by means of software and necessary general hardware platforms through the description of the above embodiments, and the embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the method described in each embodiment or some parts of the embodiment.

[0106] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A task scheduling method, characterized by, The method comprises: acquiring task state data; the task state data comprises task state information of a to-be-executed task, and the task state information is used for identifying whether the to-be-executed task can acquire corresponding task resources from a continuous integration platform; based on the task state data, predicting a resource state of the continuous integration platform to generate resource state information of the continuous integration platform; based on the resource state information, performing task scheduling on the to-be-executed task; the task state data is generated based on a dotting method, the task state information comprises a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded when the to-be-executed task is waiting to acquire corresponding task resources, and the resource locking identification point is an identification point recorded when the to-be-executed task acquires corresponding task resources; the method of predicting the resource state of the continuous integration platform based on the task state data to generate the resource state information of the continuous integration platform comprises: if the task state information of the to-be-executed task comprises the resource waiting identification point and does not comprise the resource locking identification point, recording a waiting time of the to-be-executed task; based on the waiting time, predicting the resource state of the continuous integration platform to generate the resource state information of the continuous integration platform.

2. The task scheduling method of claim 1, wherein, after the method of predicting the resource state of the continuous integration platform based on the waiting time to generate the resource state information of the continuous integration platform, the method further comprises: if the waiting time is greater than or equal to a preset threshold, generating a reconnaissance task; based on the reconnaissance task, reacquiring corresponding task resources for the to-be-executed task; if the to-be-executed task acquires the corresponding task resources successfully, correcting the resource state information.

3. The task scheduling method of claim 1, wherein, the number of the to-be-executed tasks is at least two; the method of performing task scheduling on the to-be-executed task based on the resource state information comprises: determining a task score of each to-be-executed task; based on the resource state information and the task score of each to-be-executed task, selecting at least one target to-be-executed task from all the to-be-executed tasks; based on the continuous integration platform, executing the target to-be-executed task.

4. The task scheduling method of claim 3, wherein, the to-be-executed tasks are stored in a task queue; the method of determining a task score of each to-be-executed task comprises: respectively determining a priority of each to-be-executed task, a compilation time of each to-be-executed task, and a time when each to-be-executed task enters the task queue; based on the priority of each to-be-executed task, the compilation time of each to-be-executed task, the time when each to-be-executed task enters the task queue, and a dependency relationship between all the to-be-executed tasks, determining the task score of each to-be-executed task.

5. The task scheduling method of claim 4, wherein, One of the task scores includes an integer part and a decimal part, the integer part is determined based on the priority and the compilation time of one of the to-be-executed tasks and the dependency relationship between all the to-be-executed tasks, and the decimal part is determined based on the time when one of the to-be-executed tasks enters the task queue.

6. A task scheduling apparatus characterized by comprising: Comprise: An acquisition module is configured to acquire task state data. The task state data includes task state information of a to-be-executed task, and the task state information is used to identify whether the to-be-executed task can obtain corresponding task resources from a continuous integration platform. A resource state prediction module is configured to predict a resource state of the continuous integration platform based on the task state data, and generate resource state information of the continuous integration platform. A task scheduling module is configured to perform task scheduling on the to-be-executed task based on the resource state information. The task state data is generated based on a dotting method, the task state information includes a resource waiting identification point and / or a resource locking identification point, the resource waiting identification point is an identification point recorded when the to-be-executed task is waiting to obtain corresponding task resources, and the resource locking identification point is an identification point recorded when the to-be-executed task obtains corresponding task resources. The prediction of the resource state of the continuous integration platform based on the task state data to generate the resource state information of the continuous integration platform includes: If the task state information of the to-be-executed task includes the resource waiting identification point and does not include the resource locking identification point, record the waiting time of the to-be-executed task. Based on the waiting time, the resource state of the continuous integration platform is predicted to generate the resource state information of the continuous integration platform.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to realize the task scheduling method in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the task scheduling method in any one of claims 1 to 5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the task scheduling method in any one of claims 1 to 5.

Images