Method, device, terminal and medium for scheduling tasks and training neural network models

Abstract

The embodiment of the present invention provides a method, device, terminal and medium for scheduling tasks and training neural network models. The method for scheduling tasks includes: determining whether the queue of subtasks to be constructed in the main task has changed, and if so, executing the following steps; obtaining the remaining construction time of each subtask being constructed and each subtask to be constructed in the main task based on the time-consuming prediction neural network model, and taking the largest one as the remaining construction time of the main task; determining whether the remaining construction time of the main task meets the time-consuming requirement, and if not, adjusting the queue order of the subtasks to be constructed in the main task. In this way, when the resources of the slave server are limited, the queue order of the subtasks to be constructed in each main task can be dynamically adjusted to rationally utilize the limited resources of the slave server and ensure that each main task is delivered quickly and on time.

Description

Technical Field

[0001] The present invention relates to the technical field of scheduling tasks, and in particular to a method, device, terminal and medium for scheduling tasks and training a neural network model. Background Art

[0002] Jenkins is an open source software project and a continuous integration tool developed based on Java. It aims to provide an open and easy-to-use software platform for monitoring repetitive tasks and making continuous integration of software possible.

[0003] Jenkins servers are mainly divided into two types: master server and slave server. The master server is mainly used for allocating build tasks, and the slave server is mainly used for executing build tasks.

[0004] Typically, when building a project, each project contains multiple projects that need to be built. These projects may be triggered collectively and need to be built concurrently. Therefore, when using Jenkins for project builds, dozens or even hundreds of slave servers need to be deployed. If the slave servers have sufficient resources, these projects can be built simultaneously, and the construction time of the project with the longest construction time will be the construction time of the entire project. However, it is not possible to deploy an unlimited number of slave servers, otherwise it will lead to resource waste and low utilization.

[0005] To ensure fast and on-time project build delivery, it's often desirable to employ a scheduling strategy to effectively utilize limited slave server resources. However, traditional resource scheduling strategies are typically static. That is, once a build task is initiated, its corresponding priority, slave server type, and other parameters are essentially determined, and the order in which projects are built is also fixed. If a higher-priority task jumps the queue, an originally queued task is dequeued, or other circumstances occur that extend the build time, there's no guarantee that the build task will be completed quickly and on time. Summary of the Invention

[0006] The technical problem solved by the embodiment of the present invention is how to reasonably utilize the slave server resources used to build a project to ensure that the project construction can be delivered quickly and on time.

[0007] To solve the above technical problems, embodiments of the present invention provide a method, device, terminal, and medium for scheduling tasks and training neural network models.

[0008] A method for scheduling tasks provided by an embodiment of the present invention includes: S110, determining whether the queue of subtasks to be constructed in a main task has changed, and if so, executing the following steps, wherein the change includes the entry of subtasks into the queue and / or the dequeueing of subtasks in the subtasks to be constructed; S120, obtaining the remaining construction time of each subtask being constructed and each subtask to be constructed in the main task based on a time consumption prediction neural network model, and taking the largest one as the remaining construction time of the main task; S130, determining whether the remaining construction time of the main task meets the time consumption requirement, and if not, adjusting the queuing order of the subtasks to be constructed in the main task.

[0009] Optionally, obtaining the remaining construction time of each subtask under construction in the main task based on the time prediction neural network model includes: predicting the predicted construction time of each subtask under construction in the main task through the time prediction neural network model; obtaining the constructed time of each subtask under construction in the main task; and taking the difference between the predicted construction time and the constructed time of each subtask under construction in the main task as the remaining construction time of each subtask under construction in the main task.

[0010] Optionally, obtaining the remaining construction time of each subtask to be constructed in the main task based on the time prediction neural network model includes: predicting the predicted construction time of each subtask to be constructed in the main task through the time prediction neural network model; obtaining the task release time of each subtask being constructed in the slave server that is suitable for constructing each subtask to be constructed in the main task; and taking the sum of the predicted construction time of each subtask to be constructed in the main task and the corresponding task release time as the remaining construction time of each subtask to be constructed in the main task.

[0011] Optionally, step S130 includes: determining whether the remaining construction time of the main task is less than or equal to the available construction time of the main task; if not, determining whether the queue level of each subtask to be constructed in the main task has reached the highest level; if it has not reached the highest level, increasing the queue level of the corresponding subtask to be constructed in the main task, and returning to step S120; wherein, the available construction time of the main task is the difference between the expected completion time of the main task and the current time.

[0012] Optionally, if the maximum level is reached, then subtasks in progress in the main task with a lower queue level than the main task are aborted.

[0013] Optionally, increasing the queue level of the corresponding subtask to be constructed includes: increasing the queue level of the subtask to be constructed that has the largest remaining construction time among the subtasks to be constructed that have not reached the highest level; and / or increasing the queue level of the subtask to be constructed that has the largest remaining construction time among the subtasks to be constructed that have not reached the highest level; and / or increasing the queue level of the subtask to be constructed that has the largest remaining construction time greater than the available construction time of the main task; and / or increasing the queue level of all subtasks to be constructed that have not reached the highest level.

[0014] An embodiment of the present invention also provides a device for scheduling tasks, including: a judgment module, which is suitable for judging whether the queue of subtasks to be constructed in the main task has changed, and whether the remaining construction time of the main task meets the time requirement, wherein the change includes the entry of subtasks into the queue and / or the dequeue of subtasks in the subtasks to be constructed; a calculation module, which is suitable for obtaining the remaining construction time of each subtask being constructed and each subtask to be constructed in the main task based on a time prediction neural network model when there is a change in the queue, and taking the largest one as the remaining construction time of the main task; an adjustment module, which is suitable for obtaining the time requirement of the main task, and adjusting the queue order of the subtasks to be constructed in the main task when the remaining construction time of the main task does not meet the time requirement.

[0015] Optionally, the calculation module includes: a prediction unit, which is suitable for predicting the construction predicted time of each subtask under construction and each subtask to be constructed in the main task through a time prediction neural network model; an acquisition unit, which is suitable for acquiring the construction time of each subtask under construction in the main task, and the task release time of each subtask under construction in the slave server suitable for constructing each subtask to be constructed in the main task; a calculation unit, which is suitable for taking the difference between the construction predicted time of each subtask under construction in the main task and the respective construction time as the remaining construction time of each subtask under construction in the main task, taking the sum of the construction predicted time of each subtask to be constructed in the main task and the corresponding task release time as the remaining construction time of each subtask to be constructed in the main task, and taking the remaining construction time of the one with the largest remaining construction time among each subtask under construction and each subtask to be constructed in the main task as the remaining construction time of the main task.

[0016] Optionally, the judgment module includes: a first judgment unit, which is suitable for obtaining the queue of subtasks to be constructed in the main task, and judging whether the queue has changed, wherein the change includes the entry of subtasks into the queue and / or the dequeueing of subtasks in the subtasks to be constructed; a second judgment unit, which is suitable for judging whether the remaining construction time of the main task is less than or equal to the available construction time of the main task, wherein the available construction time of the main task is the difference between the expected completion time of the main task and the current time; a third judgment unit, which is suitable for judging whether the queue level of each subtask to be constructed in the main task has reached the highest level when the remaining construction time of the main task is less than or equal to the available construction time of the main task.

[0017] Optionally, the adjustment module is adapted to increase the queue level of the corresponding subtask to be constructed when the queue level of the subtask to be constructed in the main task has not reached the highest level.

[0018] Optionally, the adjustment module is adapted to terminate the subtasks being constructed in the main task that have a lower queue level than the main task when the queue level of each subtask to be constructed in the main task reaches the highest level.

[0019] An embodiment of the present invention also provides a terminal for scheduling tasks, including: a main server, used to assign subtasks to be constructed in the main task to the slave server; a slave server, used to receive the subtasks to be constructed assigned by the main server and execute construction on them; wherein, the main server includes a first memory and a first processor, the first memory stores a first computer instruction that can be executed on the first processor, and when the first processor executes the first computer instruction, the steps of the method for scheduling tasks provided by the embodiment of the present invention are executed.

[0020] An embodiment of the present invention further provides a first storage medium on which a first computer instruction is stored. When the first computer instruction is executed, the steps of the method for scheduling tasks provided in the embodiment of the present invention are executed.

[0021] An embodiment of the present invention also provides a method for training a time-consuming prediction neural network model, including: S410, obtaining the configuration parameters of each subtask in each main task and the performance parameters of the slave server used to construct the corresponding subtask, wherein the configuration parameters and the performance parameters both include feature vectors related to the construction time of the corresponding subtask; S420, constructing a training sample set based on the configuration parameters and the performance parameters; S430, using the training sample set, with the construction time of the corresponding subtask as the target, to train a preset neural network and obtain a time-consuming prediction neural network model.

[0022] Optionally, the feature vector includes the file names of the files to be built in each subtask included in the configuration parameters.

[0023] Optionally, the feature vector includes the subtask name of each subtask included in the configuration parameters.

[0024] Optionally, the feature vector includes a main task name of a main task included in the configuration parameters and corresponding to the corresponding subtask.

[0025] Optionally, the feature vector includes the type of the slave server building the corresponding subtask and the number of parallel processes when the slave server is building the subtask, which are included in the performance parameters.

[0026] Optionally, the preset neural network includes a long short-term memory artificial neural network based on an attention mechanism.

[0027] An embodiment of the present invention also provides a device for training a time-consuming prediction neural network model, comprising: an acquisition module, which is suitable for acquiring the configuration parameters of each subtask in each main task, the performance parameters of the slave server used to construct the corresponding subtask, and a preset neural network, wherein the configuration parameters and the performance parameters both include feature vectors related to the construction time of the corresponding subtask; a construction module, which is suitable for constructing a training sample set based on the configuration parameters and the performance parameters; a training module, which is suitable for using the training sample set to train the preset neural network with the construction time of the corresponding subtask as the target, and obtain a time-consuming prediction neural network model.

[0028] An embodiment of the present invention also provides a terminal for training a time-consuming prediction neural network model, comprising a second memory and a second processor, wherein the second memory stores second computer instructions that can be run on the second processor, and when the second processor runs the second computer instructions, the steps of the method for training a time-consuming prediction neural network model provided in an embodiment of the present invention are executed.

[0029] An embodiment of the present invention further provides a second storage medium on which second computer instructions are stored. When the second computer instructions are executed, the steps of the method for training a time-consuming prediction neural network model provided by an embodiment of the present invention are executed.

[0030] Compared with the existing technology, the technical solution of the embodiment of the present invention has beneficial effects. For example, when slave server resources are limited, the queue order of the subtasks to be built in each master task can be dynamically adjusted to rationally utilize the limited slave server resources and ensure the rapid and timely delivery of each master task. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flowchart of a method for scheduling tasks in an embodiment of the present invention;

[0032] Figure 2 Schematic diagram of the principle of the device for scheduling tasks in an embodiment of the present invention;

[0033] Figure 3 Schematic diagram of the principle of a terminal for scheduling tasks in an embodiment of the present invention;

[0034] Figure 4 1 is a flow chart of a method for training a time-consuming prediction neural network model in an embodiment of the present invention;

[0035] Figure 5 Schematic diagram of the principle of a device for training a time-consuming prediction neural network model in an embodiment of the present invention;

[0036] Figure 6 2 is a schematic diagram of the principle of a terminal for training a time-consuming prediction neural network model in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, features and beneficial effects of the embodiments of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0038] Figure 1 4 is a flowchart of a method for scheduling tasks in an embodiment of the present invention.

[0039] like Figure 1 As shown, the method 100 for scheduling tasks provided by an embodiment of the present invention may include:

[0040] S110, determining whether the queue of the subtasks to be constructed in the main task has changed, if so, executing the following steps;

[0041] S120, obtaining the remaining construction time of each subtask being constructed and each subtask to be constructed in the main task based on the time consumption prediction neural network model, and taking the largest one as the remaining construction time of the main task;

[0042] S130, determining whether the remaining construction time of the main task meets the time requirement, if not, executing step S140;

[0043] S140: Adjust the queue order of the subtasks to be constructed in the main task.

[0044] In the embodiment of the present invention, when a project is constructed, the construction task of each project can be called a main task, each main task includes multiple projects to be constructed, and each project to be constructed can be called a subtask.

[0045] In a specific implementation, a master server and multiple slave servers can be configured, wherein the master server is used to assign subtasks to each slave server, and the slave servers are used to receive and build the subtasks assigned by the master server. The types of slave servers can include physical servers and virtual servers. Physical servers have better subtask building performance than virtual servers. For example, for the same subtask, the time spent on building using a physical server may be less than the time spent on building using a virtual server. When both physical and virtual servers are available, the physical server can be used to build the subtask first.

[0046] In this embodiment of the present invention, the build of multiple main tasks can be triggered simultaneously, and each subtask within the multiple main tasks can be built concurrently. However, since slave server resources cannot be infinitely configured, each slave server can run multiple threads concurrently to build multiple subtasks simultaneously when performing a project build.

[0047] Before building multiple main tasks that are triggered simultaneously, each main task and its subtasks may be queued according to a certain level, and each subtask may be built according to its queueing level.

[0048] Specifically, you can pre-set the queue levels for main tasks and subtasks. For example, you can set the queue level for the main task to VIP level and non-VIP level, and set the queue levels for subtasks from high to low to level 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10. Subtasks with higher levels will start to be built first.

[0049] Before each main task is built, the queue level of each main task and its subtasks can be determined based on the specific construction requirements of each main task and its subtasks. For example, if a main task is more important than other main tasks, then the main task can be assigned a VIP level before construction. The main task assigned a VIP level can be built on a physical server first. For another example, if a main task includes multiple subtasks, and the expected completion time of each subtask is different, then the queue level of each subtask can be determined based on its expected completion time.

[0050] In the specific implementation, each subtask in each main task can be built according to its own queuing level. That is, when multiple main tasks are built at the same time, the subtasks of each main task can be built concurrently without waiting for all subtasks in the main task with a higher level to be built before building the subtasks in other main tasks.

[0051] In the specific implementation of step S110, the construction of the main task has been triggered, and the resources of the slave server suitable for constructing each subtask in the main task are limited. The main task includes one or more subtasks being constructed and one or more subtasks to be constructed. Among them, the subtasks being constructed are subtasks that have been constructed by the slave server, and the subtasks to be constructed are subtasks that have not yet been constructed by the slave server and are queued for construction.

[0052] In the embodiment of the present invention, a plurality may represent greater than or equal to two.

[0053] In the specific implementation of step S110, the changes in the queue of the subtasks to be built in the main task may include the entry and / or dequeue of subtasks in the subtasks to be built in the main task. The entry of a subtask indicates that a new subtask has been added to the main task and is waiting in the queue for construction, and the dequeue of a subtask indicates that a subtask in the subtasks to be built in the main task has been assigned to a corresponding slave server for construction or the construction task of a subtask in the subtasks to be built in the main task has been canceled.

[0054] Specifically, a new subtask can be a newly added subtask that is different from the original subtasks of the main task, or a subtask that was originally in the main task and needs to be re-added due to a build being aborted. The build abortion can be triggered by reasons such as a failure in the slave server or the priority build of another main task with a higher queue level.

[0055] In the specific implementation of step S120, the remaining construction time of each subtask being constructed and each subtask to be constructed in the main task can be obtained based on the time prediction neural network model, and the largest one can be taken as the remaining construction time of the main task.

[0056] In this embodiment of the present invention, the subtasks within the main task can be built concurrently. Therefore, the remaining build time of the main task is determined by the subtask with the largest remaining build time among its subtasks. That is, the remaining build time of the main task is the remaining build time of the subtask with the largest remaining build time among its subtasks currently being built and its subtasks to be built.

[0057] In a specific implementation, obtaining the remaining construction time of each subtask being constructed in the main task based on the time consumption prediction neural network model may include:

[0058] S121, using a time consumption prediction neural network model to predict the construction time consumption of each subtask being constructed in the main task;

[0059] S122, obtaining the construction time of each subtask being constructed in the main task;

[0060] S123 , respectively taking the difference between the predicted construction time of each subtask being constructed in the main task and the construction time of each subtask as the remaining construction time of each subtask being constructed in the main task.

[0061] In the specific implementation of step S121, there may be one or more subtasks under construction within the main task. Each subtask under construction can have its total construction time, i.e., the predicted construction time, predicted by the time consumption prediction neural network model. The implementation method for predicting the predicted construction time of the subtask under construction using the time consumption prediction neural network model will be described in a subsequent section of this article.

[0062] In the specific implementation of step S122 , the construction time consumed by each subtask being constructed in the main task can be obtained by respectively obtaining the difference between the current time and the start construction time of the corresponding subtask being constructed.

[0063] In the specific implementation of step S123, the difference between the predicted construction time of each subtask under construction in the main task (that is, the total time required to construct each subtask under construction) and the respective construction time can be taken as the remaining construction time of each subtask under construction in the main task.

[0064] In the embodiment of the present invention, each subtask being constructed has corresponding predicted construction time, constructed time, and remaining construction time.

[0065] In a specific implementation, obtaining the remaining construction time of each to-be-constructed subtask in the main task based on the time consumption prediction neural network model may include:

[0066] S124, predicting the construction time of each to-be-constructed subtask in the main task using a time-consuming prediction neural network model;

[0067] S125, obtaining the task release time of each subtask being constructed in the slave server suitable for constructing each subtask to be constructed in the main task;

[0068] S126 , taking the sum of the predicted construction time of each to-be-constructed subtask in the main task and the corresponding task release time as the remaining construction time of each to-be-constructed subtask in the main task.

[0069] In the specific implementation of step S124, there may be one or more subtasks to be constructed within the main task. Each subtask to be constructed can have its total construction time, i.e., the predicted construction time, predicted using the time-consuming prediction neural network model. The implementation of predicting the predicted construction time of each subtask to be constructed using the time-consuming prediction neural network model will be described in a subsequent section of this document.

[0070] In the specific implementation of step S125, each subtask being constructed in the slave server may include subtasks in the main task and subtasks in other main tasks, and the task release time of each subtask being constructed in the slave server is the remaining construction time of the corresponding subtask being constructed.

[0071] In the specific implementation of step S126 , the remaining construction time of each subtask to be constructed in the main task is the time from the current time to the completion of the construction of the corresponding subtask to be constructed.

[0072] In an embodiment of the present invention, the subtasks to be built in the main task need to wait for the slave server suitable for building these subtasks to release its subtasks under construction before they can be built (due to limited resources of the slave server, it is necessary to wait for a subtask under construction to complete construction before the next subtask can be built). Therefore, the remaining construction time of each subtask to be built in the main task can be respectively taken as the sum of the predicted construction time of each subtask to be built in the main task (that is, the total time required to build each subtask to be built) and the corresponding task release time as the remaining construction time of each subtask to be built.

[0073] For example, the queue of subtasks to be built of the master task includes the first subtask to be built that is the first in the queue, and the subtasks being built of the slave server suitable for building the first subtask to be built include the first subtask being built that has the shortest remaining construction time. Then, the slave server is suitable for starting to build the first subtask to be built after the first subtask being built completes construction. Thus, the remaining construction time of the first subtask to be built is the sum of the remaining construction time of the first subtask being built and the predicted construction time of the first subtask to be built (that is, the total time required for the first subtask to be built to be built).

[0074] In the embodiment of the present invention, each subtask to be constructed has corresponding predicted construction time, task release time and remaining construction time.

[0075] In a specific implementation of step S130, determining whether the remaining construction time of the main task meets the time requirement may include:

[0076] S131 , determining whether the remaining construction time of the main task is less than or equal to the available construction time of the main task.

[0077] In a specific implementation of step S131 , the available construction time of the main task may be the difference between the expected completion time of the main task and the current time.

[0078] In a specific implementation of step S140, adjusting the queue order of the subtasks to be constructed in the main task may include:

[0079] S141, determining whether the queue level of each subtask to be constructed in the main task has reached the highest level; if not, increasing the queue level of the corresponding subtask to be constructed in the main task, and returning to step S120.

[0080] In the specific implementation of step S141, if the queue level of the subtask to be constructed in the main task has not reached the highest level, the queue level of the corresponding subtask to be constructed is increased. When the queue level of the subtask to be constructed in the main task is increased, the queue of the subtask to be constructed in the main task changes again, and the process can return to step S120.

[0081] In a specific implementation, increasing the queue level of the corresponding subtask to be constructed may include: increasing the queue level of the subtask to be constructed that has the longest remaining construction time among the subtasks to be constructed that have not reached the highest level; and / or increasing the queue level of the subtask to be constructed that has the longest remaining construction time among the subtasks to be constructed that have not reached the highest level; and / or increasing the queue level of the subtask to be constructed that has the longest remaining construction time than the main task's available construction time; and / or increasing the queue level of all subtasks to be constructed that have not reached the highest level. For example, the queue level of the corresponding subtask to be constructed may be increased by one or more levels.

[0082] In the specific implementation of step S141, when the queue level of each subtask to be constructed in the main task reaches the highest level, the following steps may also be included:

[0083] S142: Terminate the subtasks being constructed in the main task of a lower level than the main task.

[0084] In the specific implementation of step S142, although the queue level of each pending subtask in the main task has reached the highest level, the remaining construction time of the main task does not meet the time requirement. In this case, the subtasks currently being built in other main tasks with lower levels than the main task can be suspended to free up slave server resources, so that one or more of the pending subtasks in the main task can begin construction. For example, if the main task is VIP-level, the subtasks currently being built in non-VIP-level main tasks can be suspended.

[0085] In a specific implementation, the subtasks being built in other suspended main tasks can be re-added to the queue of the to-be-built subtasks of the corresponding main task and wait for queue construction.

[0086] An embodiment of the present invention also provides a device for scheduling tasks.

[0087] Figure 2 Schematic diagram of the principle of the device for scheduling tasks in an embodiment of the present invention.

[0088] like Figure 2 As shown, the apparatus 200 for scheduling tasks provided by the embodiment of the present invention may include a judgment module 210 , a calculation module 220 connected to the judgment module 210 , and an adjustment module 230 connected to the calculation module 220 .

[0089] Specifically, the judgment module 210 is suitable for judging whether the queue of subtasks to be constructed in the main task has changed, and whether the remaining construction time of the main task meets the time requirement, wherein the change includes subtasks being queued and / or subtasks being dequeued in the subtasks to be constructed.

[0090] In a specific implementation, the judgment module 210 may include a first judgment unit, a second judgment unit, and a third judgment unit connected to the second judgment unit. Specifically, the first judgment unit is suitable for obtaining the queue of subtasks to be constructed in the main task from the main server and judging whether the queue has changed. The second judgment unit is suitable for judging whether the remaining construction time of the main task is less than or equal to the available construction time of the main task, wherein the available construction time of the main task is the difference between the expected completion time of the main task and the current time. The third judgment unit is suitable for judging whether the queue level of each subtask to be constructed in the main task has reached the highest level when the remaining construction time of the main task is less than or equal to the available construction time of the main task.

[0091] Specifically, the calculation module 220 is suitable for obtaining the remaining construction time of each subtask being constructed and each subtask to be constructed in the main task based on the time consumption prediction neural network model when there is a change in the queue of subtasks to be constructed in the main task, and taking the largest one as the remaining construction time of the main task.

[0092] In a specific implementation, the calculation module 220 may include a prediction unit, an acquisition unit, and a calculation unit connected to the prediction unit and the acquisition unit. Specifically, the prediction unit is adapted to predict the predicted construction time of each subtask under construction and each subtask to be constructed in the main task using a time-consuming prediction neural network model. The acquisition unit is adapted to obtain the completed construction time of each subtask under construction in the main task, as well as the task release time of each subtask under construction in the slave server for constructing each subtask to be constructed in the main task. The calculation unit is adapted to respectively take the difference between the predicted construction time of each subtask under construction in the main task and the completed construction time as the remaining construction time of each subtask under construction in the main task, respectively take the sum of the predicted construction time of each subtask to be constructed in the main task and the corresponding task release time as the remaining construction time of each subtask to be constructed in the main task, and take the remaining construction time of the subtask under construction and the subtask to be constructed in the main task with the largest remaining construction time as the remaining construction time of the main task.

[0093] Specifically, the adjustment module 230 is adapted to adjust the queueing order of the to-be-constructed subtasks in the main task when the remaining construction time of the main task does not meet the time requirement.

[0094] In a specific implementation, the adjustment module 230 is adapted to raise the queue order of a subtask to be constructed within a main task to the highest level when the queue order of the subtask to be constructed has not yet reached the highest level. Specifically, the adjustment module 230 is adapted to raise the queue order of the subtask to be constructed that has not yet reached the highest level and has the longest remaining construction time; and / or raise the queue order of the subtask to be constructed that has not yet reached the highest level and has a remaining construction time greater than the available construction time of the main task; and / or raise the queue order of all subtasks to be constructed that have not yet reached the highest level. For example, the adjustment module 230 may raise the queue order of the corresponding subtask to be constructed by one or more levels.

[0095] In a specific implementation, the adjustment module 230 is further adapted to terminate the subtasks being constructed in other main tasks with a lower queue level than the main task when the queue level of each subtask to be constructed in the main task reaches the highest level.

[0096] In a specific implementation, the device 200 for scheduling tasks may further include a clock connected to the judgment module 210 and the calculation module 220 respectively, which is suitable for collecting the current time during the main task construction process.

[0097] An embodiment of the present invention also provides a terminal for scheduling tasks.

[0098] Figure 3 2 is a schematic diagram of the principle of a terminal for scheduling tasks in an embodiment of the present invention.

[0099] like Figure 3 As shown, the terminal 300 for scheduling tasks provided by the embodiment of the present invention may include a master server 310 and a slave server 320 connected to the master server 310 .

[0100] Specifically, the master server 310 is used to assign the subtasks to be constructed in the master task to the slave server 320. The slave server 320 is used to receive the subtasks to be constructed assigned by the master server 310 and execute construction on them.

[0101] In a specific implementation, the main server 310 may include a first memory and a first processor. The first memory stores a first computer instruction executable on the first processor, and the first processor executes the steps of the method 100 for scheduling tasks provided in an embodiment of the present invention when executing the first computer instruction.

[0102] In a specific implementation, multiple slave servers 320 are provided to concurrently build multiple subtasks in the main task. However, the number of slave servers 320 cannot be unlimited. Typically, the number of slave servers 320 is less than the number of all subtasks that need to be built. This can save resources and reduce operating costs.

[0103] In a specific implementation, the terminal 300 for scheduling tasks may further include a clock, which is suitable for collecting the current time during the main task construction process.

[0104] An embodiment of the present invention further provides a first storage medium on which a first computer instruction is stored. When the first computer instruction is executed, the steps of the method for scheduling tasks provided in the embodiment of the present invention are executed.

[0105] An embodiment of the present invention also provides a method for training a time-consuming prediction neural network model.

[0106] Figure 4 4 is a flow chart of a method for training a time-consuming prediction neural network model in an embodiment of the present invention.

[0107] like Figure 4 As shown, the method 400 for training a time-consuming prediction neural network model provided by an embodiment of the present invention may include:

[0108] S410, obtaining configuration parameters of each subtask in each main task and performance parameters of a slave server used to construct the corresponding subtask, wherein the configuration parameters and the performance parameters both include a feature vector related to the construction time of the corresponding subtask;

[0109] S420, constructing a training sample set based on the configuration parameters and performance parameters;

[0110] S430 , using the training sample set and taking the construction time of the corresponding subtask as a target, the preset neural network is trained to obtain a time consumption prediction neural network model.

[0111] In the embodiment of the present invention, when a project is constructed, the construction task of each project can be called a main task, each main task includes multiple projects to be constructed, and each project to be constructed can be called a subtask.

[0112] In a specific implementation, each subtask can be divided into multiple files that need to be built. For example, under the Android platform, a subtask may include files such as idh, rls_image_idh, ota, fullcode_pac, etc. If the build requirements of these files are all "TRUE", it means that these files need to be built. The construction of each file takes a certain amount of time. Specifically, the build time of each file can at least be related to the file name of the file. The file name of each file determines its type, content, data and other information. Files with different file names have different build times, and files with the same file name in different main tasks also have different build times. This is related to the code compilation time of the file, or the data packaged into the file.

[0113] In the specific implementation of step S410, first, a database containing historical construction data of the main task can be constructed, and then the configuration parameters of each subtask in each main task and the performance parameters of the slave server used to construct the corresponding subtask are obtained from the database.

[0114] Among them, the historical construction data of the main task may include the historical configuration parameters of each subtask in the main task and the historical performance parameters of the slave server used to construct the corresponding subtask, and the historical configuration parameters and historical performance parameters both include feature vectors related to the construction time of the corresponding subtask.

[0115] In a specific implementation, the configuration parameters of each subtask in each main task may include the file names of each file that needs to be constructed in the subtask.

[0116] In a specific implementation, the configuration parameters of each subtask in each main task may further include a subtask name of the subtask. The subtask name of each subtask is at least related to the file names of each file that needs to be built.

[0117] In a specific implementation, the configuration parameters of each subtask in each main task may further include the main task name of the main task corresponding to the subtask. The main task name of each main task is at least related to the subtask names of each subtask included therein.

[0118] In the embodiment of the present invention, as long as the files in each subtask in the main task are built, they will affect the total construction time and the remaining construction time of the main task.

[0119] In a specific implementation of step S410 , the performance parameters of the slave server used to construct the corresponding subtask may include the type of the slave server and the number of parallel processes when the slave server is constructing the subtask.

[0120] Specifically, slave server types can include physical servers and virtual servers. Physical servers have better subtask build performance than virtual servers. For example, for the same subtask, building it on a physical server takes less time than building it on a virtual server. When both physical and virtual servers are available, physical servers can be prioritized for subtask builds.

[0121] When using a slave server to build a subtask, the number of subtasks being built on that slave server, that is, the number of parallel processes on that slave server, also affects the build time of that subtask. For example, when using a slave server to build a subtask, the more parallel processes a slave server has, the longer it will take to build that subtask.

[0122] In the embodiment of the present invention, the configuration parameters of each subtask in each main task and the performance parameters of the slave server used to construct the corresponding subtask all include a feature vector related to the construction time of the corresponding subtask.

[0123] Specifically, the feature vector may include the file names of the files to be built in each subtask included in the configuration parameters, and the types of slave servers building the corresponding subtasks and the number of parallel processes when building the subtasks included in the performance parameters.

[0124] Furthermore, the feature vector may also include the subtask name of each subtask included in the configuration parameters.

[0125] Furthermore, the feature vector may also include a main task name of a main task included in the configuration parameters and corresponding to the corresponding subtask.

[0126] In a specific implementation of step S420 , a training sample set may be constructed based on configuration parameters and performance parameters including feature vectors related to the construction time consumption of corresponding subtasks for training the neural network.

[0127] In the specific implementation of step S430, the above training sample set can be used to train the preset neural network with the construction time of the corresponding subtask (that is, the time required to construct the entire subtask) as the target, and obtain a time-consuming prediction neural network model.

[0128] In a specific implementation, the preset neural network may include a long short-term memory artificial neural network (LSTM) based on an attention mechanism. When training the LSTM artificial neural network, conventional techniques in the art are used.

[0129] In an embodiment of the present invention, when the construction prediction time of a subtask being constructed or a subtask to be constructed is predicted by a time consumption prediction neural network model, it can be achieved by the following steps:

[0130] Obtain the configuration parameters of the subtask being built or the subtask to be built and the performance parameters of the slave server used to build the corresponding subtask;

[0131] Extracting a feature vector related to the construction time of the subtask being constructed or the subtask to be constructed from the above configuration parameters and performance parameters;

[0132] The relevant feature vectors are input into the time consumption prediction neural network model to obtain the construction prediction time consumption of the subtask being constructed or the subtask to be constructed.

[0133] An embodiment of the present invention also provides a device for training a time-consuming prediction neural network model.

[0134] Figure 5 2 is a schematic diagram of the principle of an apparatus for training a time-consuming prediction neural network model in an embodiment of the present invention.

[0135] like Figure 5 As shown, the apparatus 500 for training a time-consuming prediction neural network model provided by an embodiment of the present invention may include an acquisition module 510 , a construction module 520 connected to the acquisition module 510 , and a training module 530 connected to the construction module 520 .

[0136] Specifically, the acquisition module 510 is adapted to acquire the configuration parameters of each subtask within each main task, the performance parameters of the slave server used to construct the corresponding subtask, and a pre-set neural network, wherein the configuration parameters and performance parameters each include a feature vector related to the construction time of the corresponding subtask. The construction module 520 is adapted to construct a training sample set based on the configuration parameters and performance parameters. The training module 530 is adapted to acquire a pre-set neural network and, using the training sample set and targeting the construction time of the corresponding subtask, train the pre-set neural network to obtain a time-consuming prediction neural network model.

[0137] An embodiment of the present invention also provides a terminal for training a time-consuming prediction neural network model.

[0138] Figure 62 is a schematic diagram of the principle of a terminal for training a time-consuming prediction neural network model in an embodiment of the present invention.

[0139] like Figure 6 As shown, the terminal 600 for training a time-consuming prediction neural network model provided in an embodiment of the present invention may include a second memory 610 and a second processor 620 connected to the second memory 610. The second memory 610 stores second computer instructions that can be executed on the second processor 620. When the second processor 620 executes the second computer instructions, the steps of the method for training a time-consuming prediction neural network model provided in an embodiment of the present invention are executed.

[0140] An embodiment of the present invention further provides a second storage medium on which second computer instructions are stored. When the second computer instructions are executed, the steps of the method for training a time-consuming prediction neural network model provided by an embodiment of the present invention are executed.

[0141] By adopting the above-mentioned technical solution provided by the embodiment of the present invention, the queuing order of the subtasks to be constructed in each main task can be dynamically adjusted when the resources of the slave server are limited, so as to reasonably utilize the limited resources of the slave server and ensure that each main task is delivered quickly and on time.

[0142] In specific implementation, the technical solution provided by the embodiment of the present invention can be applied not only to projects built by Jenkins, but also to other construction projects with limited construction resources.

[0143] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even when only a single embodiment is described with respect to specific features. The feature examples provided in the present disclosure are intended to be illustrative, not limiting, unless otherwise stated. In specific implementations, the technical features of one or more dependent claims may be combined with the technical features of the independent claims, depending on actual needs and where technically feasible, and the technical features from the corresponding independent claims may be combined in any appropriate manner rather than solely through the specific combinations listed in the claims.

[0144] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A method (400) for training a time-consuming prediction neural network model, characterized in that: include: S410, obtaining configuration parameters of each subtask in each main task and performance parameters of a slave server used to construct the corresponding subtask, wherein the configuration parameters and the performance parameters both include a feature vector related to the construction time of the corresponding subtask; S420, constructing a training sample set based on the configuration parameters and the performance parameters; S430, using the training sample set and taking the construction time of the corresponding subtask as a target, training a preset neural network and obtaining a time consumption prediction neural network model; Among them, the time consumption prediction neural network model is used to predict the construction predicted time consumption of each subtask under construction and each subtask to be constructed in the main task, so as to obtain the remaining construction time consumption of the main task based on the construction predicted time consumption, and adjust the queuing order of the subtasks to be constructed in the main task.

2. The method (400) according to claim 1, characterized in that The feature vector includes the file names of the files that need to be constructed in each subtask included in the configuration parameters.

3. The method (400) according to claim 2, characterized in that The feature vector includes the subtask name of each subtask included in the configuration parameters.

4. The method (400) according to claim 2 or 3, characterized in that The feature vector includes a main task name of a main task included in the configuration parameters and corresponding to a corresponding subtask.

5. The method (400) according to claim 1, characterized in that The feature vector includes the type of the slave server that constructs the corresponding subtask included in the performance parameter and the number of parallel processes when the slave server is constructing the subtask.

6. The method (400) according to claim 1, characterized in that The preset neural network includes a long short-term memory artificial neural network based on an attention mechanism.

7. A device (500) for training a time-consuming prediction neural network model, characterized in that: include: an acquisition module (510), adapted to acquire configuration parameters of each subtask in each main task, performance parameters of a slave server used to construct the corresponding subtask, and a preset neural network, wherein the configuration parameters and the performance parameters both include a feature vector related to the construction time of the corresponding subtask; A construction module (520), adapted to construct a training sample set based on the configuration parameters and the performance parameters; A training module (530) is adapted to use the training sample set and the construction time of the corresponding subtask as a target to train the preset neural network and obtain a time-consuming prediction neural network model; Among them, the time consumption prediction neural network model is used to predict the construction predicted time consumption of each subtask under construction and each subtask to be constructed in the main task, so as to obtain the remaining construction time consumption of the main task based on the construction predicted time consumption, and adjust the queuing order of the subtasks to be constructed in the main task.

8. A terminal (600) for training a time-consuming prediction neural network model, characterized in that: The invention comprises a second memory (610) and a second processor (620), wherein the second memory (610) stores second computer instructions that can be executed on the second processor (620), and the second processor (620) executes the steps of the method according to any one of claims 1 to 6 when executing the second computer instructions.

9. A second storage medium having a second computer instruction stored thereon, characterized in that: The second computer instructions are executed by a second processor to implement the steps of the method according to any one of claims 1 to 6.

Images