Data system job scheduling arrangement method, device, equipment, medium and product

Abstract

The invention provides a data system job scheduling arrangement method and device, equipment, a storage medium and a program product, and relates to the technical field of table processing, and the method comprises the steps: obtaining at least one to-be-scheduled job and an association relationship between the to-be-scheduled jobs; according to the association relationship between the to-be-scheduled jobs, merging the to-be-scheduled jobs to obtain a merging result; according to a channel load upper limit and the scheduling information of each operation chain in the merging result, determining a scheduling arrangement result of each operation chain in the merging result; inserting each target scheduling job in the merging result into a scheduling arrangement result of each job chain to generate a scheduling arrangement result of the merging result; and determining a scheduling sequence of the to-be-scheduled jobs according to a scheduling arrangement result of the merging result. According to the embodiment of the invention, the execution sequence of the to-be-scheduled jobs can be efficiently and reasonably determined.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, equipment, medium and product for scheduling and arranging operations in a data system. Background Art

[0002] Job scheduling is the basic work of data development. For a data system, a large number of data jobs need to be queued and executed one by one every day. Data jobs refer to data operation commands that need to be run in the data system, where data operation commands are used to complete specific program instructions for different goals.

[0003] In reality, each job is often not an independent individual. Many jobs depend on each other. For example, the completion of job A requires an intermediate calculation result in job B. This makes the job scheduling management system extremely complex. It is necessary to consider the mutual influence between different jobs to avoid the situation where some jobs cannot be executed for a long time, causing other jobs to wait for a long time, or even a complete crash.

[0004] A good job scheduling system can greatly reduce the frequency of system accidents. The current job scheduling system mainly relies on people to arrange and implement. When the number of jobs exceeds a certain limit, it is difficult for manpower to guarantee the uncertainty of system load risk caused by the simultaneous implementation of multiple jobs. Therefore, the simplest and crudest execution method is often adopted. This will greatly increase the time cost of operation and bring unbearable system delay problems to super-large systems and massive data processing systems. Summary of the invention

[0005] The present application provides a data system job scheduling arrangement method, device, equipment, medium and product, which can efficiently and reasonably determine the execution order of the jobs to be scheduled.

[0006] In a first aspect, the present application provides a data system job scheduling method, comprising:

[0007] Obtain at least one job to be scheduled and an association relationship between the jobs to be scheduled;

[0008] Merging the jobs to be scheduled according to the association relationship between the jobs to be scheduled to obtain a merged result, wherein the merged result includes at least one target scheduled job and / or at least one job chain, wherein the job chain includes multiple dependent scheduled jobs;

[0009] Determine the scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result;

[0010] According to the channel load upper limit, the scheduling arrangement results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, inserting each of the target scheduling jobs in the merged result into the scheduling arrangement results of each of the job chains to generate the scheduling arrangement result of the merged result;

[0011] According to the scheduling arrangement result of the merged result, the scheduling order of each of the to-be-scheduled jobs is determined.

[0012] In a second aspect, the present application provides a data system job scheduling and arranging device, comprising:

[0013] A job dependency acquisition module, used to acquire at least one job to be scheduled and the association relationship between the jobs to be scheduled;

[0014] A job merging module, used to merge the jobs to be scheduled according to the association relationship between the jobs to be scheduled, and obtain a merging result, wherein the merging result includes at least one target scheduling job and / or at least one job chain, and the job chain includes multiple dependent scheduling jobs;

[0015] A job chain arrangement module, used to determine the scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result;

[0016] A separate job scheduling module, used to insert each of the target scheduling jobs in the merged result into the scheduling and scheduling results of each of the job chains according to the channel load upper limit, the scheduling and scheduling results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, to generate a scheduling and scheduling result of the merged result;

[0017] The job order determination module is used to determine the scheduling order of each of the jobs to be scheduled according to the scheduling arrangement result of the merged result.

[0018] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0019] The memory stores computer-executable instructions;

[0020] The processor executes the computer-executable instructions stored in the memory to implement the method described in the present application.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the method described in the present application.

[0022] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the present application when executed by a processor.

[0023] The present application provides a data system job scheduling and arrangement method, device, equipment, medium and product. By acquiring the association relationship of the jobs to be scheduled, the jobs to be scheduled are merged to obtain the associated job chain and the independently running target scheduling job, the job chain is scheduled and arranged first, and after the job chain is scheduled and arranged, the target scheduling job is inserted and arranged, and finally the scheduling order of the jobs to be scheduled is obtained, so as to realize the scheduling and arrangement based on the complexity of the jobs, and the order of the jobs can be accurately arranged based on the dependent constraints, while taking into account the optimization goal of minimizing the overall running time of the jobs, so as to achieve a balance between the complexity and accuracy of the scheduling and arrangement, improve the reliability of the scheduling and arrangement, and improve the scheduling and arrangement efficiency of the jobs without constraints by inserting independent jobs into the arrangement order of the job chain, and in the overall scheduling time, the jobs without constraints are executed in parallel as much as possible, so as to improve the effectiveness of the jobs without constraints, and finally optimize the complex job system to efficiently queue, improve the execution efficiency and avoid system crashes. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] Figure 1 A flowchart of a data system job scheduling method provided in an embodiment of the present application;

[0026] Figure 2 A flowchart of a data system job scheduling method provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of a scheduling arrangement result of a job chain provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of a merge chain provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of another merge chain provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of a merged operation chain provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of another merged operation chain provided in an embodiment of the present application;

[0032] Figure 8A flowchart of a data system job scheduling method provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of an insertion operation of a target scheduling job provided in an embodiment of the present application;

[0034] Fig.10 A schematic diagram of a scheduling arrangement result of a target scheduling job provided in an embodiment of the present application;

[0035] Fig.11 A schematic diagram of an insertion scenario of a target scheduling job provided in an embodiment of the present application;

[0036] Fig.12 A flowchart of a data system job scheduling method provided in an embodiment of the present application;

[0037] Fig.13 A schematic diagram of a scheduling arrangement scenario provided in an embodiment of the present application;

[0038] Fig.14 A schematic diagram of a job merging scenario provided in an embodiment of the present application;

[0039] Fig.15 A schematic diagram of the structure of a data system job scheduling and arranging device provided in an embodiment of the present application;

[0040] Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0041] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0042] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0043] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings. It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application, but it does not mean that the applicant has or must use the relevant content of the solution. The acquisition, storage, use, and processing of data in the technical solution of the present application are in compliance with the relevant provisions of national laws and regulations.

[0044] In response to the problem in the prior art that the scheduling and arrangement of jobs is performed by manpower, resulting in inefficient arrangement, high computational cost and low precision, the present application proposes a data system job scheduling and arrangement method, device, equipment, medium and product, which obtains the association relationship of the jobs to be scheduled, merges the jobs to be scheduled, obtains the job chain with the association relationship and the independently running target scheduling job, schedules and arranges the job chain first, and after the job chain is scheduled, inserts and arranges the target scheduling job, and finally obtains the scheduling order of the jobs to be scheduled, which can achieve a more efficient arrangement method that does not rely on manpower, has a lower computational cost and has a higher arrangement precision.

[0045] Figure 1 The process of the data system job scheduling method provided in the embodiment of the present application is applicable to the case where jobs involved in the same data task are scheduled and arranged. The method can be executed by a data system job scheduling device, which can be implemented in the form of hardware and / or software. Figure 1 As shown, the method comprises the following steps:

[0046] S101: Obtain at least one job to be scheduled and an association relationship between the jobs to be scheduled.

[0047] The job to be scheduled may refer to the job that needs to be arranged in order of execution. In the embodiment of the present application, the job is the basic execution unit of the data system operation. Each job represents a complete set of operation instructions, and the operation instruction set is used to perform tasks with specific goals. The task may include at least one of the following: extracting data from the database, transferring and copying data from different databases to each other, and performing data analysis operations. M1, M2, M3, ..., M4, M5, M6, M7, M8, M9, M10, M11, M12, M13, M14, M15, M16, M17, M18, M19, M20, M21, M22, M30, M41, M52, M63, M74, M85, M19, M20, M21, M30, M44, M56, M11, M22, M31, M45, M57, M12 (n-1) 、M n Represents the i-th job, n represents the total number of jobs, and the value of i is an integer ranging from 1 to n.

[0048] Associations are used to describe whether the scheduled jobs have any impact on each other. For example, the completion of some jobs depends on the running results of another job, some jobs must be completed synchronously with other jobs, and some jobs can only be started after all jobs have stopped.

[0049] S102: Merge the jobs to be scheduled according to the association relationship between the jobs to be scheduled to obtain a merge result, wherein the merge result includes at least one target scheduling job and / or at least one job chain, and the job chain includes multiple dependent scheduling jobs.

[0050] The purpose of merging the jobs to be scheduled is to divide the jobs to be scheduled into multiple jobs with dependencies, jobs that are completed synchronously, and jobs that are executed independently. Among them, jobs with dependencies usually refer to jobs that need to be executed in a specified order. Multiple jobs that are completed synchronously can be merged into a new job, replacing multiple jobs for arrangement. Accordingly, before dividing the associated jobs and the independently executed jobs, it is necessary to obtain the jobs that are completed synchronously. After merging the jobs that are completed synchronously, divide the multiple jobs with dependencies and the independently executed jobs into the merged jobs and the remaining unmerged jobs.

[0051] The target scheduling job can refer to an independently executed job to be scheduled, which has no association with other jobs to be scheduled. The target scheduling job can also be understood as a job to be scheduled without constraints. Multiple associated jobs form a job chain. The job chain can refer to multiple jobs that are executed serially and have dependencies. The job chain can be understood as a collection of jobs with constraints. The dependent scheduling job can refer to a job to be scheduled that has dependencies with other jobs to be scheduled.

[0052] S103: Determine a scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result.

[0053] The channel load upper limit may refer to the load upper limit of the jobs that can be run by the system executing the job at the same time. The scheduling information may include the running time and load pressure, etc. Among them, the running time is used as an optimization target, and the purpose of the scheduling arrangement is to minimize the time for all jobs to be executed in the results obtained by the arrangement. The load pressure is used to determine the number of jobs running at the same time. In the scheduling arrangement process of the jobs to be scheduled, in addition to considering the association relationship of the jobs to be scheduled, the running time and load of the jobs to be scheduled also need to be considered. Usually, the running time of different jobs is different, and the system load of different jobs executed at the same time is also different. The scheduling arrangement result of the job chain may include the running start time, running end time and load of the job chain, etc. According to the channel load upper limit and the scheduling information of the job chain, the job chain is scheduled and arranged according to the goal of being as close to the channel load upper limit as possible and minimizing the total running time, and the scheduling arrangement result of the job chain is obtained.

[0054] S104. According to the channel load upper limit, the scheduling and arrangement results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, each of the target scheduling jobs in the merged result is inserted into the scheduling and arrangement results of each of the job chains to generate the scheduling and arrangement results of the merged result.

[0055] When there is at least one job chain and at least one target scheduling job, determine the idle position in the scheduling and arrangement result of the job chain according to the scheduling and arrangement result of each job chain. According to the channel load upper limit, the scheduling information of each target scheduling job and the idle position in the scheduling and arrangement result of the job chain, insert each target scheduling job into the idle position as much as possible, and put the target scheduling jobs that cannot be inserted into the idle position in sequence after the already arranged job chain or target scheduling job. After all the target scheduling jobs are arranged, the scheduling and arrangement result of the merged result is obtained.

[0056] When there is at least one job chain and there is no target scheduling job, the scheduling arrangement results of each job chain are determined as the scheduling arrangement result of the merged result.

[0057] When there is no job chain, the target scheduling job is scheduled and arranged according to the channel load upper limit, in accordance with the goal of being as close to the channel load upper limit as possible and minimizing the total running time, and the scheduling and arrangement result of the target scheduling job is obtained and used as the scheduling and arrangement result of the merged result.

[0058] In an example, the jobs to be scheduled are: M1*, M2*, …, M (k-1) *、M k *. The merged results include two categories. The first category is the sorted chain category, that is, the set M of dependent scheduled jobs: this category is the jobs belonging to a certain job chain, which cannot be scheduled arbitrarily because they are restricted by the order of the job chain.

[0059] The second category is the free category, which is the set M* of target scheduling jobs: this category is the job tasks that do not belong to the job chain. As a free single-point job, it can be scheduled and deployed at any location at will.

[0060] For the set M of dependent scheduled jobs, in order to facilitate processing, each job is identified according to the job chain to which it belongs. Assuming that there are D job chains to which the job M belongs, these job chains are arranged in order of duration and marked as i=1,2……D. Where i1,…,i si is the subscript of the job contained in the i-th job chain Li, and its value range is between 1 and k. si is the number of jobs in the job chain L i The number of jobs included. So the job duration of each job chain is where t j * is each job M in the job chain j *The running time (j=i1,…,i si ), and for each i, T i >T (i+1) At the same time, it is not difficult to see that the dependent scheduling jobs included in this M are

[0061] For the set M* of target scheduling jobs, the target scheduling jobs it contains are

[0062] The job chain contained in M ​​is used as the basic item for scheduling. After all dependent scheduling jobs on the job chain are arranged, the target scheduling job in M*, which is a free point, is inserted into the job chain.

[0063] S105: Determine the scheduling order of each of the jobs to be scheduled according to the scheduling arrangement result of the merged result.

[0064] The merged result is split into jobs to be scheduled, and the scheduling arrangement result of the merged result is split into the scheduling order of the jobs to be scheduled.

[0065] The embodiment of the present application obtains the association relationship of the jobs to be scheduled, merges the jobs to be scheduled, obtains the job chain with the association relationship and the independently running target scheduling job, schedules and arranges the job chain first, and inserts the target scheduling job into the arrangement after the job chain is scheduled, and finally obtains the scheduling order of the jobs to be scheduled, thereby realizing sequential scheduling based on the complexity of the jobs, and can accurately arrange the order of jobs based on dependent constraints, while taking into account the optimization goal of minimizing the overall running time of the jobs, achieving a balance between the complexity and accuracy of scheduling, and improving the reliability of scheduling. In addition, by inserting independent jobs into the arrangement order of the job chain, the scheduling efficiency of jobs without constraints is improved, and in the overall arrangement time, jobs without constraints are executed in parallel as much as possible, thereby improving the effectiveness of jobs without constraints, and finally optimizing the complex job system for efficient queuing, improving execution efficiency, and avoiding system crashes.

[0066] Figure 2 A flowchart of a data system job scheduling method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method comprises the following steps:

[0067] S201: Obtain at least one job to be scheduled and an association relationship between the jobs to be scheduled.

[0068] S202: Merge the jobs to be scheduled according to the association relationship between the jobs to be scheduled to obtain a merge result, wherein the merge result includes at least one target scheduling job and / or at least one job chain, and the job chain includes multiple dependent scheduling jobs.

[0069] S203, selecting each of the operation chains in turn.

[0070] When there is at least one job chain, the job chains are traversed. One job chain is selected each time. The job chains can be selected in descending order based on the running time of the job chains.

[0071] S204: Obtain the arranged job chain.

[0072] The scheduled job chain may refer to a job chain of which the scheduling result is determined. The running start time of the scheduled job chain has been determined.

[0073] S205. When the sum of the loads of the arranged job chains and the load of the selected job chain is less than or equal to the channel load upper limit, determine the scheduling result of the selected job chain, and the operation start time of the selected job chain is the same as that of the corresponding arranged job chain.

[0074] The selected job chain may refer to the job chain currently being traversed. The number of selected job chains is one. The sum of the load of the arranged job chains and the load of the selected job chain is less than or equal to the channel load upper limit, indicating that the selected job chain and the arranged job chain can be executed in parallel, and the running start time of the selected job chain can be configured to be the same as the running start time of the arranged job chain. The time obtained by superimposing the running start time of the selected job chain and the running length of the job chain is used as the running end time of the selected job chain.

[0075] In some embodiments, when the arranged job chain is empty, the start time of the overall execution of the job is used as the start time of the operation of the selected job chain. When the number of arranged job chains is at least one, the sum of the loads of all arranged job chains is obtained, the sum of the selected job chain and the sum of the loads is calculated, and it is detected whether the sum is less than or equal to the channel load upper limit. If so, it indicates that the selected job chain and each arranged job chain can be run at the same time, and accordingly, the selected job chain and each arranged job chain have the same start time of operation. The scheduling and arrangement result of the selected job chain is determined according to the start time of operation of the selected job chain and the load of the selected job chain. If not, it indicates that the load of each arranged job chain has reached the system upper limit, and the selected job chain cannot be run simultaneously with each arranged job chain. Among them, each arranged job chain runs at the same time. The start time of operation of each arranged job chain is the same.

[0076] S206: When the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, determine a merged chain of the selected job chains in each of the arranged job chains.

[0077] The sum of the load of the scheduled job chains and the load of the selected job chain is greater than the channel load upper limit, indicating that the load of each scheduled job chain has reached the system upper limit, and the selected job chain cannot run simultaneously with the scheduled job chains. A merged chain may refer to a job chain that can be merged with a selected job chain. A merged chain may refer to a scheduled job chain that can immediately run the selected job chain after the operation is completed. That is, the end time of the merged chain's operation is the same as the start time of the selected job chain's operation. In fact, a merged chain can be understood as a scheduled job chain that is executed before the selected job chain and adjacent to the selected job chain.

[0078] In some embodiments, after the merged chain runs, the selected job chain is run adjacently, and the selected job chain and the arranged job chains outside the merged chains run simultaneously, that is, the sum of the loads of the selected job chain and the arranged job chains outside the merged chains is less than or equal to the channel load upper limit, and the difference between the channel load upper limit and the sum of the loads of the arranged job chains outside the merged chains is determined as the load margin. In the arranged job chains, the loads of each arranged job chain are compared with the selected job chain separately. When the load of the selected job chain is less than the sum of the load and the load margin of at least one arranged job chain, the job chain with the earliest operation end time can be used as the merged chain of the selected job chain. Among them, the arranged job chains can be taken in order from the earliest to the latest operation end time for load comparison. In this way, as long as the sum of the load and the load margin of the currently compared arranged job chain is greater than the load of the selected job chain, the arranged job chain is the earliest merged chain at the end of the operation. When the load of the selected job chains is greater than the sum of the arranged job chains and the load margin, the arranged job chains are arranged and combined to obtain at least one job chain group, each job chain group includes at least two arranged job chains, and the load of each job chain group is compared with the selected job chain. When the load of the selected job chain is less than the sum of the load and the load margin of at least one job chain group, the job chain group that ends the operation first, including the arranged job chain, can be used as the merged chain of the selected job chain. Similarly, the job chain groups can be taken in order from the earliest to the latest in the order of the end of the operation, and the load comparison can be performed. In this way, as long as the load of the currently compared job chain group is greater than that of the selected job chain, the arranged job chain included in the job chain group is the merged chain that ends the operation first.

[0079] S207: In each of the arranged job chains, merge the selected job chain and the merged chain of the selected job chain, update the corresponding arranged job chain, and update the scheduling and arrangement result of the corresponding arranged job chain.

[0080] The selected job chain is merged with the merged chain to form a new job chain, and the new job chain replaces the merged chain. The running start time of the new job chain is the running start time of the merged chain, and the running end time of the new job chain is the time obtained by adding the running start time of the merged chain to the longest running time in the merged chain and the running time of the selected job chain. The load of the new job chain can be based on the maximum load at the same time in the cooperative chain and the selected job chain. According to the load, running start time and running end time of the new job chain, the scheduling and arrangement result of the new job chain is determined. The scheduling and arrangement result of the new job chain actually includes the scheduling and arrangement result of the selected job chain and the scheduling and arrangement result of at least one merged chain. The running end time of the selected job chain is the running end time of the new job chain, and the running start time of the selected job chain is the latest running end time of each merged chain.

[0081] The selected job chain is inserted after the merge chain so that the selected job chain can be combined with the job that is currently completed the fastest, that is, so that all job chains as a whole can be completed earlier. The specific merging method is also selected to make the two job chains (the selected job chain and the merge chain) form a whole to facilitate further sorting processing later.

[0082] S208. According to the channel load upper limit, the scheduling and arrangement results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, each of the target scheduling jobs in the merged result is inserted into the scheduling and arrangement results of each of the job chains to generate the scheduling and arrangement results of the merged result.

[0083] S209: Determine the scheduling order of each of the jobs to be scheduled according to the scheduling arrangement result of the merged result.

[0084] The embodiment of the present application traverses the job chain and determines whether the selected job chain and the arranged job chain are running simultaneously based on the sum of the load of the arranged job chain and the load of the selected job chain. If they cannot run at the same time, the merged chain of the selected job chain is detected and the selected job chain is placed after the merged chain to run, so as to preferentially arrange the selected job chain to run simultaneously with the arranged job chain. If they cannot run at the same time, the merged chain of the selected job chain is detected and merged with it, and the arranged job chain and the corresponding scheduling and arrangement results are updated, so as to determine the optimal scheduling and arrangement result for the selected job chain on the arranged job chain. By merging the selected job chain and the merged chain, and sequentially using them as the arranged job chain, the subsequent selected job chain is arranged, which simplifies the calculation difficulty of determining the optimal arrangement result and improves the arrangement efficiency. In addition, the selected job chain and the merged chain form a whole, which can facilitate further sorting processing.

[0085] Optionally, in some embodiments, selecting each of the job chains in sequence includes: selecting each of the job chains in sequence in descending order of running time.

[0086] Among the unarranged job chains, the job chains with the longest running time are selected in turn as the selected job chains, that is, the job chains are scheduled and arranged in order from the largest running time to the smallest running time.

[0087] In one example, as in the previous example, for a set M of dependent scheduled jobs, job chains are selected from M one by one. Since the duration of the jobs in M ​​is arranged from large to small, the job chain that is taken out first has a longer running duration. Each job chain is taken out one by one in order from large to small according to this duration and put into the channel for execution.

[0088] In order to prevent the accumulation of operation chains and cause the load to exceed the standard, for each operation chain Pick The load of the job with the largest load is taken as the load of the entire job chain. In an example, if the loads of the jobs in M1*-M2*-M3*-M4* are 100, 120, 90, and 110 respectively, it is not difficult to see that the second job M2* has the largest load, reaching 120, so 120 is the load of this job chain. If the job chain put into the channel for execution is about to exceed the channel load limit B of the channel, stop putting the job chain.

[0089] Exemplarily, the selected job chain L1 is taken out, and its load is less than the channel load upper limit B. The scheduling and arrangement result of the job chain L1 is determined, and the operation start time of the job chain L1 is the start time of the overall job. The selected job chain L2 is taken out, and the sum of the loads of the arranged job chain L1 and the selected job chain L2 is less than the channel load upper limit B. The scheduling and arrangement result of the job chain L2 is determined, wherein the operation start time of the job chain L2 and the job chain L1 is the same. The selected job chain L3 is taken out, and the sum of the loads of the arranged job chain L1 and the job chain L2 and the selected job chain L3 is less than the channel load upper limit B. The scheduling and arrangement result of the job chain L3 is determined, wherein the operation start time of the job chain L3 and the job chain L1 and the job chain L2 is the same. The selected job chain L4 is taken out, and the sum of the loads of the arranged job chain L1, the job chain L2 and the job chain L3 and the selected job chain L4 is less than the channel load upper limit B. At this time, the job chain L4 cannot be directly added. The sum of the loads of job chains L1, L2, and L3 is less than the upper limit of the channel load, while the sum of the loads of job chains L1, L2, L3, and L4 is greater than the upper limit of the channel load. Figure 3 As shown, the scheduling results of job chain L1, job chain L2 and job chain L3.

[0090] Assume that the sum of the loads of job chains L1, L2, and L3, i.e., b1+b2+b3, does not exceed B. However, starting from job chain L4, the load of any job chain added will exceed B. In this case, you should stop adding new job chains. Perform the next step of sorting and judgment, and then continue to add job chains.

[0091] It can be seen that by scheduling and arranging the job chains in order of running time from large to small, the difficulty can be arranged from high to low, the scheduling can be simplified, and the scheduling result can be made more reasonable.

[0092] Optionally, in some embodiments, determining the merged chain of the selected job chain in each of the arranged job chains includes: taking out at least one job chain from each of the arranged job chains in order from the earliest to the latest in terms of the end time of operation, and combining them to obtain at least one job chain group; determining the load of each job chain group based on the load of the job chain included in each job chain group; determining the job chain group corresponding to the selected job chain based on the load of each job chain group, the channel load upper limit and the load of the selected job chain; and determining the merged chain of the selected job chain based on the job chains included in the job chain group corresponding to the selected job chain.

[0093] Among them, the order of the end time of operation from first to last is used to judge the load over-limit with the selected job chain according to the order of the end of operation of the arranged job chains. The load of the job chain group is the sum of the loads of the included job chains. The job chain group corresponding to the selected job chain may refer to the sum of the load and the load margin being greater than the load of the selected job chain. The job chain included in the corresponding job chain group is determined as the merged chain of the selected job chain. Among them, the job chains are taken out in sequence from first to last in the order of the end time of operation for combination, which is equivalent to judging whether the selected job chain can be inserted into the corresponding position according to the order of the start time of operation from first to last, so as to accurately determine the optimal arrangement position of the selected job chain, while taking into account the simplification of the arrangement operation of the job chain and improving the arrangement efficiency of the job chain.

[0094] It can be seen that by generating job chain groups in the order from the earliest to the latest time of the operation end, the number of generated job chain groups can be reduced, the amount of data processed can be reduced, and based on the generated job chain groups, load overlimit judgment is performed with the selected job chains respectively, and it is judged whether the selected job chain can be inserted into the corresponding position in the order from the earliest to the latest time of the operation start time, and the optimal arrangement position of the selected job chain is accurately determined, while taking into account the simplification of the arrangement operation of the job chain and the improvement of the arrangement efficiency of the job chain.

[0095] Optionally, in some embodiments, determining the job chain group corresponding to the selected job chain based on the load of each job chain group, the channel load upper limit and the load of the selected job chain includes: based on the load difference between the sum of the loads of each arranged job chain and the channel load upper limit; calculating the load of each job chain group and the sum of the load differences to obtain the load margin corresponding to each job chain group; comparing the load margin corresponding to each job chain group with the load of the selected job chain to obtain the comparison result of each job chain group; and determining the job chain group corresponding to the selected job chain based on the comparison result of each job chain group.

[0096] Among them, the load margin corresponding to each job chain group is different. The load margin corresponding to the job chain group can be understood as the load upper limit of the job chain that runs simultaneously with the arranged job chains outside the job chain group. According to the load margin of each job chain group and the load of the selected job chain, it is determined that the load of the selected job chain is less than at least one job chain group, and the job chain group with the smallest load is selected from these job chain groups to be determined as the job chain group corresponding to the selected job chain.

[0097] In one example, as in the previous example, the arranged job chains are job chain L1, job chain L2 and job chain L3, the sum of the loads of the three job chains is b1+b2+b3, the channel load upper limit is B, and the load difference is Δ=B-(b1+b2+b3). The generated job chain groups include: job chain group 1: job chain L3, job chain group 2: job chain L3 and job chain L2, job chain group 3: job chain L3, job chain L2 and job chain L1. When judging job chain L4, the load of job chain L4 is b4, the sum of the load of job chain group 1 and the load difference obtains the load margin of job chain group 1: b3+Δ, the sum of the load of job chain group 2 and the load difference obtains the load margin of job chain group 2: b2+b3+Δ, and the sum of the load of job chain group 3 and the load difference obtains the load margin of job chain group 3: b1+b2+b3+Δ.

[0098] It can be seen that by calculating the unused load difference between each arranged job chain and the channel load upper limit, the load margin corresponding to the job chain group is calculated according to the load difference, which is used as the load upper limit of the selected job chain. According to the load margin, the job chain group corresponding to the selected job chain is determined, thereby determining the merged chain. It is possible to accurately determine that the selected job chain follows the running merged chain, thereby improving the detection accuracy of the merged chain.

[0099] Optionally, in some embodiments, at least one job chain is taken out in sequence from each of the arranged job chains in the order of the running end time from the earliest to the latest, and combined to obtain at least one job chain group, including: taking out the first job chain from each of the arranged job chains in the order of the running end time from the earliest to the latest, and generating a job chain group; taking out one job chain in sequence from the job chains other than the first job chain in each of the arranged job chains in the order of the running end time from the earliest to the latest; generating a new job chain group based on the taken out job chain and the job chain group generated last time.

[0100] Among them, a new job chain group is generated according to the extracted job chain and the job chain group generated last time, which can be a combination of the extracted job chain and the job chain group generated last time, and the job chain group generated in the past is also retained, and the new job chain group is updated to the job chain group generated last time. When the new job chain is extracted, the combination of the job chain group generated last time after the update is used as the new job chain group, and so on.

[0101] In some embodiments, the arranged job chains are arranged in the order from the earliest to the latest in terms of the end time of operation: job chain A, job chain B, job chain C, and job chain D. The first job chain A is used to generate job chain group 1. Next time, job chain B is taken out and added to job chain group 1 to obtain a new job chain group 2. Next time, job chain C is taken out and added to job chain group 1 to obtain a new job chain group 3. Next time, job chain D is taken out and added to job chain group 2 to obtain a new job chain group 4. The job chain groups finally generated include job chain group 1, job chain group 2, job chain group 3, and job chain group 4.

[0102] It can be seen that by taking out the first two job chains in the order from first to last of the running end time to generate a job chain group, and continuing to take out the job chains in sequence, and merging them with the previously generated job chain group, a new job chain group is obtained, and the generated job chain group is compared with the selected job chain, which is equivalent to judging whether the selected job chain can follow the operation in the order from first to last of the running end time, and replacing the permutations and combinations of the job chains to generate all the job chain groups, which can avoid the generation and judgment of redundant job chain groups and improve the detection efficiency of the merged chain.

[0103] In one example, as in the previous example, the arranged job chains are job chain L1, job chain L2, and job chain L3. The sum of the loads of the three job chains is b1 + b2 + b3, the upper limit of the channel load is B, and the load difference is Δ = B - (b1 + b2 + b3). The generated job chain groups include: job chain group 1: job chain L3, job chain group 2: job chain L3 and job chain L2, job chain group 3: job chain L3, job chain L2, and job chain L1. When judging job chain L4, where the load of job chain L4 is b4, the load margin of job chain group 1 is: b3 + Δ, the load margin of job chain group 2 is: b2 + b3 + Δ, and the load margin of job chain group 3 is: b1 + b2 + b3 + Δ.

[0104] As Figure 4 shown, according to the job chain group, the merged chain of the selected job chain can be determined. First, it is detected whether the load b4 of job chain L4 satisfies b4 < b3 + Δ. If this inequality is satisfied, the job chain L3 included in job chain group 1 is used as the merged chain of job chain L4, and job chain L4 is arranged to run after the end time T3 of the running of job chain L3, where the time period from 0 to T3 is the running duration of job chain L3.

[0105] As Figure 5 shown, when the load b4 of job chain L4 does not satisfy b4 < b2 + b3 + Δ, it is detected whether the load b4 of job chain L4 satisfies b4 < b2 + b3 + Δ. If this inequality is satisfied, the job chains L3 and L2 included in job chain group 2 are used as the merged chain of job chain L4, and job chain L4 is arranged to run after the end time T2 of the running of job chain L2, where the time period from 0 to T2 is the running duration of job chain L2.

[0106] Finally, when the load b4 of job chain L4 does not satisfy b4 < b1 + b2 + b3 + Δ, it is detected whether the load b4 of job chain L4 satisfies b4 < b1 + b2 + b3 + Δ. If this inequality is satisfied, the job chains L3, L2, and L1 included in job chain group 3 are used as the merged chain of job chain L4, and job chain L4 is arranged to run after the end time T1 of the running of job chain L1, where the time period from 0 to T1 is the running duration of job chain L1.

[0107] Optionally, in some embodiments, in each of the arranged job chains, the selected job chain and the merged chain of the selected job chains are merged to update the corresponding arranged job chain, and the scheduling and arrangement results of the corresponding arranged job chain are updated, including: determining the last running end time among the running end times of the merged chain of the selected job chain, and using it as the running start time of the selected job chain; merging the merged chain of the selected job chain and the selected job chain to obtain a merged job chain; determining the scheduling information of the merged job chain according to the scheduling information of the merged chain of the selected job chain and the scheduling information of the selected job chain; updating the corresponding arranged job chain according to the merged job chain and the scheduling information of the merged job chain, and updating the scheduling and arrangement results of the corresponding arranged job chain.

[0108] Wherein, when the number of merged chains is one, the running end time of the merged chain is taken as the last running end time of the moment; when the number of merged chains is multiple, the running end times of multiple merged chains are compared to obtain the last running end time of the moment. At least one merged chain is merged with the selected job chain to obtain a merged job chain. The scheduling information includes the running start time, the running end time and the load. The running end time of the selected job chain is obtained by superimposing the running time on the running start time. The running start time of the merged job chain is the first running start time of each merged chain, and the running end time of the merged job chain is the running end time of the selected job chain. The maximum value of the load running simultaneously in the merged job chain and the selected job chain is taken as the load of the merged job chain, and the running start time, running end time and load of the merged job chain are determined as the scheduling information of the merged job chain. The merged job chain replaces each merged chain to form an arranged job chain, and at the same time, the scheduling arrangement result of the merged job chain is determined by the scheduling information of the merged job chain, so as to realize adding the selected job chain to the arranged job chain.

[0109] In one example, as in the previous example, the job chain L4 is merged with the merge chain L3, and the merged job chain is as follows: Figure 6 As shown. The operation chain L4 is horizontally merged with the merged chain L3. After the merger, the larger one of b3 and b4 is used as the load of the merged operation chain ( Figure 6 In the example, b4>b3), the choice of the position where the new job is inserted is to allow the new job to be combined with the fastest currently completed job. Figure 6It can be seen that the new job is merged with the shortest job chain block. The purpose of such a merger selection is to enable all job chains to be completed earlier as a whole. The specific merger method selected is also to merge two job chain blocks into a new job chain merger block with the smallest area, so as to facilitate further sorting processing later. Next, the relationship between the duration T3 + T4 of the merged job chain and T1 and T2 can be seen. Here, the principle of "the longer the duration of the job chain, the lower its placement" is adopted to rearrange the job chains. As Figure 6 shown, since T3 + T4 is longer than both T1 and T2, the merged job chain L4 + L3 should be placed at the bottom layer. The purpose of rearranging the job chains is to ensure that all job chains or job chain mergers are always arranged in the form of a pyramid structure, that is, the job chains or job chain mergers at the bottom layer have the longest running duration, and the job chains or job chain mergers higher up have shorter durations. This can ensure that while the automated sorting structure is as efficient and intelligent as possible, the operation logic is simpler. If Figure 6 shown, since T2 < T3 + T4 < T1, the merged job chain L4 + L3 should be placed in the middle, above job chain L1 and below job chain L2.

[0110] As in the previous example, job chain L4 is merged with merged chains L3 and L2. The merged job chain is as Figure 7 shown. Then job chain L3, job chain L2, and job chain L4 are merged into one. The merger principle is to first perform vertical mergers on the job chains that have already been arranged in the channel, and then perform horizontal mergers on the newly added job chains. Specifically, job chain L3 and job chain L2 are first vertically merged, and then horizontally merged with job chain L4. The running duration of the merged job chain is the sum of the maximum running duration in job chain L2 and job chain L3 (i.e., T2) and T4: T2 + T4. The load of the merged job chain is the maximum value among b2 + b3 and b4. According to the size relationship between T2 + T4 and T1, it is determined whether the merged job chain needs to be swapped with job chain L1. As before, this method of merging running durations and merging job chain loads is to integrate several job chain blocks into a new job volume merger block with the smallest area while ensuring that all job chains or job chain mergers are always arranged in the form of a pyramid structure during sorting. When b4 > b2 + b3 and T2 + T4 > T1, the scheduling arrangement result of the merged job chain is as Figure 7 shown.

[0111] It can be seen that by determining the scheduling information of the selected job chain based on the scheduling information of the merged chain, determining the scheduling information of the merged job chain, and updating the arranged job chain and the corresponding scheduling results, the scheduling of the selected job chain can be realized, and the merged chain of the selected job chain's running start time and the final running end time can be run serially, so as to reduce the idle time between the selected job chain and the merged chain, and reduce the overall running time of the job.

[0112] Optionally, in some embodiments, when the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, determining the merged chain of the selected job chains in each of the arranged job chains, including: when the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, when the number of arranged job chains is multiple, determining the merged chain of the selected job chains in each of the arranged job chains; when the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, it also includes: when the number of arranged job chains is one, determining the scheduling and arrangement result of the selected job chain according to the arranged job chains, wherein the running start time of the selected job chain is the same as the last running end time in the arranged job chain.

[0113] Among them, when the sum of the loads of each arranged job chain and the load of the selected job chain is greater than the upper limit of the channel load, if the number of arranged job chains is multiple, determine the merged chain of the selected job chain in each arranged job chain; when the number of arranged job chains is one, place the selected job chain after the arranged job chain and run them in series. At this time, there is no merged chain for the selected job chain, that is, the selected job chain will not be merged with the arranged job chain. When selecting the next job chain, obtain the load of the arranged job chain that runs at the same time at the last time, and judge the comparison result between the sum of the load of the selected job chain and the upper limit of the channel load, and then continue to determine the scheduling and arrangement result of the job chain.

[0114] By judging whether there is a cooperative chain and whether to merge the selected job chain and the merged chain based on the number of arranged job chains when the sum of the load of each arranged job chain and the load of the selected job chain is greater than the channel load upper limit, the scheduling and processing methods of the corresponding job chains can be adaptively executed for different scenarios, enriching the scheduling and processing methods of job chains in various scenarios.

[0115] Figure 8 A flowchart of a data system job scheduling method provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the method comprises the following steps:

[0116] S801: Obtain at least one job to be scheduled and an association relationship between the jobs to be scheduled.

[0117] S802: Merge the jobs to be scheduled according to the association relationship between the jobs to be scheduled to obtain a merge result, wherein the merge result includes at least one target scheduling job and / or at least one job chain, and the job chain includes multiple dependent scheduling jobs.

[0118] S803: Determine a scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result.

[0119] S804: Determine an idle scheduling position of each of the job chains according to the scheduling arrangement result of each of the job chains in the merged result.

[0120] The idle scheduling position may refer to a position where a job can be inserted between job chains and between dependent scheduled jobs in a job chain. The number of the idle scheduling position is at least one.

[0121] S805 : According to the idle scheduling positions of each of the job chains and the scheduling information of each of the target scheduling jobs, query the insertable positions of each of the target scheduling jobs in each of the job chains.

[0122] The insertable position may refer to a position where the target scheduling job can run. The idle time corresponding to the insertable position is greater than or equal to the running time of the target scheduling job, and the load corresponding to the insertable position is greater than or equal to the load of the target scheduling job.

[0123] S806: When there is an insertable position for the target scheduling job in the job chain, determine the scheduling result of the target scheduling job according to the existing insertable position and the scheduling result of the job chain to which it belongs, and update the scheduling result of the corresponding job chain.

[0124] In fact, after the arrangement of the above job chains, the gaps between the job chains are not enough to arrange the operation of the job chains. Therefore, it is only necessary to determine whether the target scheduling job can be inserted according to the idle scheduling position in the job chain.

[0125] S807: When there is no insertable position for the target scheduled job in each of the job chains, determine the scheduled end time according to the scheduling information of the scheduled job chains and the scheduling information of the scheduled target scheduled job.

[0126] There is no insertable position for the target scheduled job, indicating that there are not enough free scheduling positions in any job chain to run the target scheduled job. At this time, the target scheduled job can only be scheduled to run after the running end time of each job chain. The scheduled end time can refer to the last running end time in the currently scheduled job chain.

[0127] S808: Determine the scheduling result of the target scheduling job according to the scheduled end time.

[0128] The scheduled end time can be used as the running start time of the target scheduled job.

[0129] S809: Determine the scheduling order of each of the jobs to be scheduled according to the scheduling arrangement result of the merged result.

[0130] In one example, the set M* of target scheduling jobs consists of scattered individual jobs, and the arrangement is very flexible and free. After the job chain is arranged, the M* concentrated target scheduling jobs are freely arranged in the gaps left by the job chain arrangement. Specifically, the target jobs in M* are inserted into the gaps in the job chain. Fig. 9 As shown in , job chain 1 is: M1*-M2*-M3*-M4*. Now we need to insert the target scheduling job M5* into it. Fig. 9 As shown, the free area just above M2* can accommodate M5*, so M5* can be placed in that free area and completely contained by the dotted square formed by M1*-M2*-M3*-M4*, forming the following Fig.10 The scheduling arrangement result shown here completes the placement operation of the target scheduling job.

[0131] The embodiment of the present application obtains independently running target scheduling jobs and inserts them into the executable idle scheduling positions of the job chain, thereby compactly configuring the running positions of each job and shortening the overall running time of all jobs as much as possible.

[0132] Optionally, in some embodiments, according to the idle scheduling position of each job chain and the scheduling information of each target scheduling job, the insertable position of each target scheduling job in each job chain is queried, including: calculating the product between the running time and the load of each target scheduling job to obtain the job area of ​​each target scheduling job; selecting the target scheduling job in each target scheduling job in order from large to small in job area; according to the job area of ​​the selected target scheduling job and the idle scheduling position of each job chain, querying the insertable position of the selected target scheduling job in each job chain.

[0133] Among them, compared with detecting the insertable position of the target scheduling job in order from small to large or in random order according to the working area, detecting the insertable position of the target scheduling job in order from large to small according to the working area can make the insertable position of the target scheduling job with the largest working area queried first, so that the insertable position of the target scheduling job with a larger working area is occupied by the target scheduling job with a smaller working area, resulting in the final scheduling arrangement result of the target scheduling job not being the optimal scheduling arrangement strategy with the shortest overall running time, resulting in resource waste and increasing the overall job running time.

[0134] In some embodiments, in order to facilitate the arrangement and to minimize the scheduling time of the arranged jobs, which is difficult to achieve the goal of the highest individual efficiency, the target scheduling jobs in M* are reordered. The sorting rule is to arrange the job area of ​​each target scheduling job from large to small, and the job area is from the front Fig. 9 and Fig.10 After arranging the target scheduling jobs from large to small, M* is relabeled as:

[0135]

[0136] When freely arranging the M* target scheduling jobs, we start with the target scheduling jobs with large operation areas, that is, we arrange the M* target scheduling jobs one by one. * f_i This is because the larger the working area, the higher the difficulty of matching.

[0137] In fact, the idle time and idle load corresponding to the insertable position need to be greater than or equal to the running time and load of the target scheduled job, so that the idle time and idle load can be considered comprehensively. The job area represents the fusion result of the two factors of running time and load, and the insertable position screening operation that considers the two factors of idle time and idle load separately can be converted into an insertable position screening operation of one factor, thereby simplifying the screening operation of the insertable position.

[0138] The idle scheduling position can be quickly screened according to the operation area. When the idle area corresponding to the idle scheduling position is larger than the operation area, it is further determined whether the idle duration of the idle scheduling position is larger than the operation duration of the target scheduling operation, and whether the idle load of the idle scheduling position is larger than the load of the target scheduling operation. When the idle duration of the idle scheduling position is larger than or equal to the operation duration of the target scheduling operation, and the idle load of the idle scheduling position is larger than or equal to the load of the target scheduling operation, the idle scheduling position is used as an insertable position for the target scheduling operation. The number of insertable positions is at least one.

[0139] It can be seen that by calculating the job area obtained by the product of the running time and the load, and scheduling and arranging each target scheduling job in order from large to small based on the job area, the target scheduling jobs with larger areas can be arranged in the idle time of the job chain as much as possible, reducing the overall running time of the job and making the overall job scheduling optimal.

[0140] Optionally, in some embodiments, the method of querying the insertable position of the selected target scheduling job in each of the job chains based on the job area of ​​the selected target scheduling job and the idle scheduling position of each of the job chains includes: obtaining the idle time and idle load corresponding to the idle scheduling position of each of the job chains; calculating the insertion probability of the selected target scheduling job and the idle scheduling position of each of the job chains based on the job area of ​​the selected target scheduling job and the idle time and idle load corresponding to the idle scheduling position of each of the job chains; determining the insertable position of the selected target scheduling job in each of the job chains based on the insertion probability of the selected target scheduling job and the idle scheduling position of each of the job chains.

[0141] Among them, the insertion probability may refer to the similarity between the selected target scheduling job and the idle scheduling position. The product of the idle duration and the idle load of the idle scheduling position is calculated to obtain the idle area of ​​the idle scheduling position. The insertion probability may be the ratio between the operating area of ​​the target scheduling job and the idle area. In each idle scheduling position, it is detected whether there is an idle scheduling position whose idle duration is greater than or equal to the operating duration of the target scheduling job and whose idle load is greater than or equal to the load of the target scheduling job. If there is one, the existing idle scheduling position is determined as an insertable position; if there are multiple idle scheduling positions, according to the insertion probabilities of the multiple existing idle scheduling positions, the idle scheduling position with an insertion probability close to 1 (that is, the maximum insertion probability) is selected as the insertable position; if there is no such idle scheduling position, the insertable position is determined to be empty.

[0142] After the scheduling arrangement result of the selected target scheduling job is determined, the target scheduling job with the largest operation area is selected from the remaining unarranged target scheduling jobs, and the insertable position of the selected target scheduling job is continuously detected.

[0143] In addition, after the insertable position is determined, the idle scheduling position may be updated according to the insertable position, for example, a range corresponding to the insertable position may be deleted from the range of the idle scheduling positions.

[0144] In one example, in order to better arrange the job, the load of a job chain is determined by the job with the largest load in the job chain, and the duration is the cumulative duration of all jobs in the job chain. Fig.11As shown, the blocks formed by the dotted lines are the job chain 1, and the solid blocks within the dotted blocks are the jobs included in the job chain 1. The load of each job is b1, ..., b4, among which b1 is the largest. b1 is determined as the load of the job chain 1, and which free areas within the job chain blocks are suitable for placing M5*. Fig.11 There is space above M2* and M4* to place M5*. At this time, you need to choose an idle scheduling position as the insertion position. The calculation formula for the insertion probability is as follows:

[0145]

[0146] b i The load and T of the target scheduling job i Schedule the duration of the job for the target, Fig.11 Medium i For b5, T i For T5. {bi≤Δb} is an indicative function, indicating that when b i When ≤Δb I {b_i≤Δb} =1, when b i >Δb I {b_i≤Δb} =0. {T_i≤ΔT} The calculation formula of the insertion probability is to hope that the idle load and idle time corresponding to the vacant space can be as consistent as possible with the load and running time of the target scheduled job, so as to avoid unnecessary waste caused by inserting a small job into a large idle space. Fig.11 The idle load of the idle area above M2* is Δb=b1-b2, and the idle time is ΔT=T2; the idle load of the idle area above M4* is Δb=b1-b4, and the idle time is ΔT=T4. Obviously, the area of ​​the idle area above M2* is more consistent with the size of M5*, so the matching degree is higher. Therefore, M5* is placed above M2*, and the running start time is aligned with the running start time of M2*. Compare and scan the target scheduling job and each job chain, and select the most suitable size for insertion. If the optimal matching degree values ​​of several job chains are consistent, a job chain is randomly selected for insertion. If there is no suitable free space for the target scheduling job to be inserted in any job chain, the target scheduling job is placed at the end of all job chains and started separately after the job chain is completed.

[0147] It can be seen that the insertion probability of the selected target scheduling job is calculated through the operating area of ​​the target scheduling job, and the idle time and idle load corresponding to the idle scheduling position, and the insertable position is determined based on the insertion probability, so as to detect the idle scheduling position that best matches the target scheduling job, avoid resource waste due to the idle scheduling position still having more idleness after running the target scheduling job, and minimize the overall operation time of the job, so as to optimize the overall scheduling of the job.

[0148] Fig.12 A flowchart of a data system job scheduling method provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the method comprises the following steps:

[0149] S1201: Obtain at least one job to be scheduled and an association relationship between the jobs to be scheduled.

[0150] S1202: When there is a mergeable association relationship among the association relationships between the jobs to be scheduled, merge the jobs to be scheduled with the mergeable association relationship to obtain an association processing result; the mergeable association relationship includes synchronous completion and / or dependent operation.

[0151] A mergeable association relationship may refer to a relationship in which at least two jobs to be scheduled can be considered as a whole for scheduling. The association processing result may include the merged job. All the jobs to be scheduled that are completed synchronously can only run other jobs to be scheduled after all of them have finished running. The jobs to be scheduled that are completed synchronously may refer to jobs that run in parallel. The dependent jobs to be scheduled run serially, and one job to be scheduled runs serially before another job to be scheduled runs.

[0152] S1203: Determine the associated processing result and at least one remaining job to be scheduled in each of the jobs to be scheduled as a merge result, wherein the merge result includes at least one target scheduling job and / or at least one job chain, and the job chain includes multiple dependent scheduling jobs.

[0153] The remaining jobs to be scheduled may refer to the jobs to be scheduled that do not have a mergeable association relationship. The jobs to be scheduled are divided into the jobs to be scheduled that have a mergeable association relationship and the remaining jobs to be scheduled. The associated processing results obtained by merging the jobs to be scheduled that have a mergeable association relationship replace the jobs to be scheduled that have a mergeable association relationship, and are used together with the remaining jobs to be scheduled as the merged results.

[0154] S1204: Determine a scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result.

[0155] S1205. According to the channel load upper limit, the scheduling and arrangement results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, each of the target scheduling jobs in the merged result is inserted into the scheduling and arrangement results of each of the job chains to generate the scheduling and arrangement results of the merged result.

[0156] S1206: Determine the scheduling order of each of the jobs to be scheduled according to the scheduling arrangement result of the merged result.

[0157] The embodiment of the present application can reduce the number of jobs to be scheduled by merging the jobs to be scheduled that are completed synchronously and those that are dependent on each other. At the same time, jobs with a specific running order can be scheduled as a whole, which can reduce errors in scheduling and improve the accuracy and efficiency of scheduling.

[0158] Optionally, in some embodiments, the to-be-scheduled jobs with mergeable association relationships are merged to obtain an association processing result, including: when there is an association relationship of synchronous completion in the association relationship between each of the to-be-scheduled jobs, multiple to-be-scheduled jobs with an association relationship of synchronous completion are merged to obtain an updated scheduled job; when there is an association relationship of dependent operation in the association relationship between each of the to-be-scheduled jobs, at least one job chain is generated based on the multiple to-be-scheduled jobs with an association relationship of dependent operation; each of the updated scheduled jobs and / or each of the job chains is determined as an association processing result.

[0159] Among them, the running process of the update scheduling job is actually the process of running multiple jobs to be scheduled synchronously in parallel. In fact, in order to minimize the running time required for all the multiple jobs to be scheduled to be completed synchronously, multiple jobs to be scheduled can be run in parallel. In this way, the multiple jobs to be scheduled that are completed synchronously can be understood as an update scheduling job including these jobs to be scheduled that is run at the same time. The running process of the job chain is actually the process of running multiple dependent jobs to be scheduled in series. In order to minimize the running time required for all the multiple dependent jobs to be scheduled to be completed, each job to be scheduled can be run serially in sequence according to the dependency relationship without inserting other jobs to be scheduled in the middle. In this way, the multiple dependent jobs to be scheduled can be understood as a job chain including these jobs to be scheduled that is run in series.

[0160] If there are multiple jobs to be scheduled that are completed synchronously and multiple jobs to be scheduled that are dependent on each other, at least one job chain is generated based on the updated scheduling job and the multiple jobs to be scheduled that are associated with the dependent operation. Specifically, the jobs to be scheduled that are completed synchronously among the multiple jobs to be scheduled that are dependent on each other are updated to the updated scheduling job to obtain the updated multiple jobs to be scheduled that are dependent on each other, and the updated multiple jobs to be scheduled that are dependent on each other are combined to form a job chain.

[0161] It can be seen that by merging the scheduled jobs that are completed synchronously to obtain the updated scheduled jobs, and merging the dependent scheduled jobs to obtain the job chain, and determining the updated scheduled job and the job chain as the merged results, the scheduled jobs can be merged from both parallel and serial dimensions, and the jobs that need to be executed in a specific order can be accurately detected. The correct operation of the jobs in the scheduling arrangement can be taken into account, while improving the accuracy and efficiency of the scheduling arrangement.

[0162] Optionally, in some embodiments, the scheduling information includes load and running time; when there is a synchronous completion association relationship among the association relationships between the jobs to be scheduled, after merging multiple jobs to be scheduled with a synchronous completion association relationship to obtain an updated scheduling job, it also includes: determining the sum of the loads of the multiple jobs to be scheduled with a synchronous completion association relationship as the load of the merged updated scheduling job; determining the maximum value of the running time of the multiple jobs to be scheduled with a synchronous completion association relationship as the running time of the merged updated scheduling job; when there is a dependent running association relationship among the association relationships between the jobs to be scheduled, after generating at least one job chain based on the multiple jobs to be scheduled with a dependent running association relationship, it also includes: determining the maximum value of the loads of the multiple jobs to be scheduled with a dependent running association relationship as the load of the merged job chain; determining the sum of the running time of the multiple jobs to be scheduled with a dependent running association relationship as the running time of the merged job chain.

[0163] The running time of the update scheduling job is the maximum running time of multiple scheduled jobs completed synchronously; the load of the update scheduling job is the sum of the loads of multiple scheduled jobs completed synchronously. The running time of the job chain is the sum of the running time of multiple scheduled jobs that are dependent on the running, and the load of the job chain is the maximum load of multiple scheduled jobs that are dependent on the running.

[0164] It can be seen that by determining the load and running time of the updated scheduling job and job chain, the complexity of the detection operation of the scheduling information of the updated scheduling job and job chain can be simplified, and the efficiency of the detection operation of the scheduling information of the updated scheduling job and job chain can be improved.

[0165] In an example, in job scheduling, it is known that there are n jobs to be scheduled M1, M2, M3, ..., M (n-1) 、M n The time taken for each of these jobs is marked as t1, t2, t3, ..., t (n-1) ,t n The interdependencies between the jobs to be scheduled are represented by the matrix X shown in Table 1 below:

[0166] Table 1

[0167] <![CDATA[M1]]> <![CDATA[M2]]> <![CDATA[M3]]> …… <![CDATA[M (n-1) ]]> <![CDATA[M n ]]> <![CDATA[M1]]> <![CDATA[x 11 ]]> <![CDATA[x 12 ]]> x_13 …… <![CDATA[x 1(n-1) ]]> <![CDATA[x 1n ]]> <![CDATA[M2]]> <![CDATA[x 21 ]]> <![CDATA[x 22 ]]> x_23 …… <![CDATA[x 2(n-1) ]]> <![CDATA[x 2n ]]> <![CDATA[M3]]> <![CDATA[x 31 ]]> <![CDATA[x 32 ]]> x_33 …… <![CDATA[x 3(n-1) ]]> <![CDATA[x 3n ]]> …… …… …… …… …… …… …… <![CDATA[M (n-1) ]]> <![CDATA[x (n-1)1 ]]> <![CDATA[x (n-1)2 ]]> <![CDATA[x (n-1)3 ]]> …… <![CDATA[x (n-1)(n-1) ]]> <![CDATA[x (n-1)n ]]> <![CDATA[M n ]]> <![CDATA[x n1 ]]> <![CDATA[x n2 ]]> <![CDATA[x n3 ]]> …… <![CDATA[x n(n-1) ]]> <![CDATA[x nn ]]>

[0168] where x ij Represents the relationship between the i-th job to be scheduled and the j-th job to be scheduled, i = 1, 2, ..., n, j = 1, 2, ..., n. x ij = 0 means that the i-th job to be scheduled does not need to be completed before the j-th job to be scheduled. ij =1 means that the i-th job to be scheduled must be completed before the j-th job to be scheduled, and x ij = 0.5 means that the i-th job to be scheduled must be completed synchronously with the j-th job to be scheduled. It should be noted that there may be more than two jobs to be scheduled that must be completed synchronously. For example, 12 =x 23 =x 34 = 0.5 means that the scheduled jobs 1, 2, and 3 must be performed simultaneously. ii , i = 1, 2, ..., n, which means the relationship between the i-th job to be scheduled and itself. The relationship between itself and itself must be completed synchronously, so x ii is 0.5.

[0169] At the same time, the running of each job to be scheduled will increase the load pressure on the system. The most important type of pressure is memory. Once too many jobs to be scheduled are running at the same time, the memory overflow will cause the system to crash and fall into a complete crash. It is necessary to focus on how to reduce the overall running time of the jobs to be scheduled while taking into account the system load and not causing a crash. The system load of each job to be scheduled is marked as b1, b2, b3, ..., b (n-1) and b n For each system, there is a threshold value B of the upper limit of the load that the performance of its constituent servers can bear, that is, the upper limit of the channel load B. When the instantaneous overall load of the system exceeds B, the system will crash.

[0170] like Fig.13As shown in the figure, the width between the two blue lines is the upper limit B of the channel load of the system. The two blue lines form a channel. Each job is represented by a red long square. The height of the square represents the load of the job on the system, and Fig.13 b1, b2, b3… in the optimization mechanism is simplified as follows: while ensuring that operations M1, M2, M3…, M (n-1) 、M n In the case of a fixed order requirement (determined by the previous interdependence matrix X), the red strips are continuously placed into the channel from left to right, and at the same time, the cumulative height of the strips must not exceed the channel load limit B (that is, the overall load must not exceed the channel load limit). In this case, the overall horizontal length of the strips is the shortest, corresponding to Fig.13 The purpose is to make t1+t6 the shortest.

[0171] The existing conventional algorithm tries all possible combinations of block arrangements and looks for the one with the shortest overall running time as the optimal solution. The biggest disadvantage of this brute force method of finding the optimal solution is that as the number of job tasks gradually increases, the number of all possible combinations increases exponentially. When the number of jobs exceeds 100,000, the total number of combinations is over 10 billion, which has no practical benefits at all. In reality, it is very easy for medium and large enterprises to exceed 100,000 jobs.

[0172] First of all, we need to solve the problem of order, that is, M1, M2, M3..., M (n-1) 、M n The order of the operations is formed to form a job chain that reflects the order, and the operations that need to be completed at the same time are merged. The following loop iteration algorithm can be used:

[0173] For iin[1,2,3,…,n] from

[0174] For j in[1,2,3,…,(i-1),(i+1),…,(n-1),n]

[0175] If x ij =1, M i Place to M j Earlier, we relabeled the system load as b i * = b i , b j * = b j The running time is denoted as t i * = t i , t j * = t j .

[0176] The job is re-labeled as M i *=Mi , M j *=M j

[0177] Else if x ij =0.5, M i With M j The tasks are merged into one whole job and are still marked as M i *, and the sum of the system loads of the two is marked as b i * = b i +b j , the running time is marked as t i *=max(t i ,t j )

[0178] The results of the above double loop are organized into new jobs to be scheduled M1*, M2*, ..., M (k-1) *、M k *, k is the number of jobs remaining after merging. The corresponding loads are b1*, b2*, …, b (k-1) *, b k *.

[0179] For example, there are 6 jobs to be scheduled: M1, M2, M3, ..., M6. The load is b1, b2, b3 ..., b6. The running time is t1, t2, t3 ..., t6. Assume that M2 must be completed before M3 can be completed, and then M4 can be completed. And M1 and M6 must be completed at the same time. The dependency matrix is ​​shown in Table 2:

[0180] Table 2

[0181] <![CDATA[M1]]> <![CDATA[M2]]> <![CDATA[M3]]> <![CDATA[M4]]> <![CDATA[M5]]> <![CDATA[M6 <!-- 17 -->]]> <![CDATA[M1]]> 0.5 0 0 0 0 0.5 <![CDATA[M2]]> 0 0.5 1 1 0 0 <![CDATA[M3]]> 0 0 0.5 1 0 0 <![CDATA[M4]]> 0 0 0 0.5 0 0 <![CDATA[M5]]> 0 0 0 0 0.5 0 <![CDATA[M6]]> 0.5 0 0 0 0 0.5

[0182] First, obtain the scheduled jobs that are completed at the same time, merge them, and get the updated scheduled jobs. Starting from i=1, traverse j from 2 to 6. According to the values ​​of the 2nd to 6th elements in the first row of the above matrix, it can be found that only x 16 =0.5, and the others are all 0, so only M1 and M6 need to be merged, and the merged result is marked as M1*=(M1,M6), t1*=max(t1,t6), b1*=b1+b6. The changed dependency matrix is ​​shown in Table 3:

[0183] Table 3

[0184] <![CDATA[M1*]]> <![CDATA[M2]]> <![CDATA[M3]]> <![CDATA[M4]]> <![CDATA[M5]]> <![CDATA[M1*]]> 0.5 0 0 0 0 <![CDATA[M2]]> 0 0.5 1 1 0 <![CDATA[M3]]> 0 0 0.5 1 0 <![CDATA[M4]]> 0 0 0 0.5 0 <![CDATA[M5]]> 0 0 0 0 0.5

[0185] Then i=2, and we start from 1, skip 2, and then traverse from 3 to 5. According to the values ​​of the first and third to fifth elements in the second row of the above matrix, we can find that x23 =1 and x 24 =1, so M2 is placed in front of M3 and M4, M2 is relabeled as M2*, M3 is relabeled as M3*, and M2 is relabeled as M3*, that is, the sequence chain "M2*-(M3*,M4*)" is obtained. (M3*,M4*) means that the order between job 3 and job 4 has not been determined. The corresponding running time is relabeled as t2*=t2, t3*=t3, t4*=t4, b2*=b2, b3*=b3, b4*=b4. The updated dependency matrix is ​​shown in Table 4:

[0186] Table 4

[0187] <![CDATA[M1*]]> <![CDATA[M2*]]> <![CDATA[M3*]]> <![CDATA[M4*]]> <![CDATA[M5]]> <![CDATA[M1*]]> 0.5 0 0 0 0 <![CDATA[M2*]]> 0 0.5 1 1 0 <![CDATA[M3*]]> 0 0 0.5 1 0 <![CDATA[M4*]]> 0 0 0 0.5 0 <![CDATA[M5]]> 0 0 0 0 0.5

[0188] Then i=3, and traverse j from 1 to 2, skipping 3 and then from 4 to 5. According to the values ​​of the 1st to 2nd and 4th to 5th elements in the third row of the above matrix, find x 34 =1, so M3* is placed in front of M4*, M3* and M4* are still marked as M3* and M4*, and the job chain "M2*-M3*-M4*" is updated. The corresponding running time and load are t3*=t3, t4*=t4, b3*=b3, b4*=b4. The dependency matrix remains unchanged.

[0189] Continue with i=4, and traverse j from 1 to 3, skipping 4 and then to 5. According to the values ​​of the 1st to 3rd and 5th elements in the fourth row of the above matrix, they are all 0, and no operation is required.

[0190] Finally, i=5, and traverse j from 1 to 4. According to the values ​​of the 1st to 4th elements in the fourth row of the above matrix, they are all 0, so no operation is required. At this point, the order arrangement and merging of each operation is completed.

[0191] The final results are M1*, M2*, M3*, M4* and M5, 5 jobs, of which M1* is formed by merging the original M1 and M6, and obtaining the sequential job chain "M2*-M3*-M4**". The corresponding running time is t1*, t2*, t3*, t4*, t5, of which t1* is the maximum value of the original running time t1 and t6. The load is b1*, b2*, b3*, b4*, b5, of which b1*=b1+b6, obtained by adding the load of the original job 1 and the original job 6 in the original job to be scheduled.

[0192] After obtaining the updated scheduled job, check whether there is a scheduled job that is dependent on the running. If there is no scheduled job that is completed at the same time, you can directly check whether there is a scheduled job that is dependent on the running.

[0193] The previous job merging operation and sequential job chain arrangement will cause errors in scheduling optimization, making it impossible to achieve the global optimal solution in scheduling optimization. For example, three jobs need to be completed at the same time and are therefore merged into one job, and the loads and durations of the three jobs are inconsistent. Then the load of the merged new job is the sum of the loads of the three jobs, and the new running time is the longest running time of the three jobs. For example Fig.14 As shown. A is the resource occupancy of each of the three jobs before merging, and the cumulative operation area of ​​the three operation blocks is: b1×t1*+b2×t2*+b3×t3*; B is the resource occupancy of the job after merging, and the block formed by the extension of the dotted line is the area of ​​the merged large block: (b1+b2+b3)×t2*. The difference between the two will cause the optimal solution to be a local optimal solution in the subsequent job arrangement optimization process. The difference between the local optimal solution and the non-global optimal solution comes from the difference in area between A and B. This difference will bring a certain efficiency loss, but it will greatly simplify the calculation difficulty of the optimization problem, and the reduction in calculation cost is enough to make up for the loss of efficiency.

[0194] Optionally, in some embodiments, the data system job scheduling method further includes: adjusting the running time of each of the jobs to be scheduled according to the running delay of each of the jobs to be scheduled; the running delay of the jobs to be scheduled is determined based on the historical delay statistics of the jobs to be scheduled.

[0195] Among them, the running time of the job to be scheduled is not an accurate value. The running delay may refer to the time required for the scheduled job to continue to execute and complete when it has not been completed after the running time. The running delay is used to make up for the inaccurate fault tolerance time of the running time. The historical delay statistical result may refer to the actual time when the scheduled job is actually completed in the case of multiple historical runs of the scheduled job, and the historical delay is determined based on the difference between the actual time and the running time, and the historical delay in multiple historical runs is obtained and statistically analyzed to obtain the historical delay statistical results. The average of multiple historical delays in the historical delay statistical results can be used to obtain the running delay. The running delay can be added to the running time to update the running time.

[0196] After the scheduling results of the jobs to be scheduled are obtained in the embodiment of the present application, they can be put into production practice for operation. The embodiment of the present application can achieve a balance between job scheduling optimization and computing power cost, and can greatly reduce the waste of operating space. However, in some extreme situations, people need manual experience to adjust and repair them. Therefore, users are required to perform regular fine-tuning after they are put into use. Specifically, when scheduling and arranging each job chain and scattered jobs, the previous job is configured to start and run immediately next to the next job, with no time interval in between. This can be dangerous in actual operation. If the previous job ends with a delay of 10 seconds due to an error in the calculation, and the next job is started as planned, the system load will cause memory overflow at a certain moment. Therefore, in actual use, a certain redundancy needs to be left for the execution time of each job. For example, if the theoretical running time is 10 minutes, then a margin needs to be added when setting the job duration during scheduling and arranging, and the job running time is set to Among them, the running delay The operator can set it based on experience. Neither too short nor too long is acceptable. If necessary, all the historically collected operation delay data can be summarized, and the historical delay statistics can be calculated by calculating the data variance and the statistical 3-σ principle to determine the operation delay. The value ensures that in most cases, memory overflow will not occur. The running time of the scheduled job can be adjusted based on the running delay before scheduling, or it can be adjusted based on the running delay after scheduling.

[0197] It can be seen that by adjusting the running delay in the running time, memory overflow can be reduced, the stability of the system running jobs can be improved, and job running errors can be reduced.

[0198] The embodiment of the present application constructs a matrix of interdependence relationships between jobs to analyze whether jobs need to be completed synchronously, and clarifies the sequence relationship between jobs. On this basis, a data system job scheduling and arrangement method that realizes automated job merging and intelligent sorting and scheduling mechanisms is implemented, replacing the inefficient practice of only being able to arrange jobs manually. At the same time, by designing a pyramid structure arrangement and the principle of minimizing the overall duration of jobs, a balance is achieved in the complexity and accuracy of the optimization algorithm, greatly improving the availability of the intelligent job scheduling and arrangement algorithm; at the same time, through the algorithm mechanism of intelligently inserting scattered jobs in an ordered job chain, the problem of the efficiency of arranging jobs without constraints is solved, further improving the effectiveness of the scheduling system.

[0199] Fig.15 This is a schematic diagram of the structure of the data system job scheduling and arranging device provided in the embodiment of the present application. Fig.15 As shown, the data system job scheduling and arranging device includes:

[0200] The job dependency acquisition module 1501 is used to acquire at least one job to be scheduled and the association relationship between the jobs to be scheduled;

[0201] The job merging module 1502 is used to merge the jobs to be scheduled according to the association relationship between the jobs to be scheduled to obtain a merging result, wherein the merging result includes at least one target scheduling job and / or at least one job chain, wherein the job chain includes multiple dependent scheduling jobs;

[0202] The job chain arrangement module 1503 is used to determine the scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result;

[0203] A separate job scheduling module 1504 is used to insert each of the target scheduling jobs in the merged result into the scheduling results of each of the job chains according to the channel load upper limit, the scheduling results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, so as to generate a scheduling result of the merged result;

[0204] The job order determination module 1505 is used to determine the scheduling order of each of the jobs to be scheduled according to the scheduling arrangement result of the merged result.

[0205] Optionally, the job chain arrangement module 1503 includes:

[0206] A job chain traversal unit, used for selecting each of the job chains in turn;

[0207] A scheduled job chain acquisition unit, used to acquire a scheduled job chain;

[0208] A first chain arrangement unit, configured to determine a scheduling arrangement result of the selected job chain when the sum of the loads of the arranged job chains and the load of the selected job chain is less than or equal to the channel load upper limit, and the operation start time of the selected job chain is the same as that of the corresponding arranged job chain;

[0209] A second chain arrangement unit, configured to determine a merged chain of the selected operation chain in each of the arranged operation chains when the sum of the load of each of the arranged operation chains and the load of the selected operation chain is greater than the channel load upper limit;

[0210] The merging and updating unit is used to merge the selected job chain and the merged chain of the selected job chain in each of the arranged job chains, update the corresponding arranged job chain, and update the scheduling and arrangement result of the corresponding arranged job chain.

[0211] Optional, job chain traversal unit, including:

[0212] The duration-decreasing traversal subunit is used to select each of the job chains in order of decreasing running duration.

[0213] Optionally, the second chain arranging unit comprises:

[0214] The job chain combination subunit is used to sequentially take out at least one job chain from each of the arranged job chains in the order of the running end time from the earliest to the latest, and combine them to obtain at least one job chain group;

[0215] A group load determination subunit, used to determine the load of each of the job chain groups according to the load of the job chains included in each of the job chain groups;

[0216] A chain group corresponding subunit, used for determining the job chain group corresponding to the selected job chain according to the load of each of the job chain groups, the channel load upper limit and the load of the selected job chain;

[0217] The merged chain determining subunit is used to determine the merged chain of the selected job chain according to the job chains included in the job chain group corresponding to the selected job chain.

[0218] Optionally, the chain group corresponds to a subunit, specifically used for:

[0219] According to the load difference between the sum of the loads of the arranged operation chains and the upper limit of the channel load;

[0220] Calculating the sum of the load of each of the operation chain groups and the load difference to obtain the load margin corresponding to each of the operation chain groups;

[0221] Comparing the load margin corresponding to each of the operation chain groups with the load of the selected operation chain to obtain a comparison result of each of the operation chain groups;

[0222] According to the comparison results of each of the job chain groups, the job chain group corresponding to the selected job chain is determined.

[0223] Optionally, the job chain combination subunit is used to:

[0224] According to the order of the running end time from the earliest to the latest, the first job chain is taken out from each of the arranged job chains to generate a job chain group;

[0225] From the arranged job chains except the first job chain, one job chain is sequentially taken out in the order of the running end time from earliest to latest;

[0226] Generate a new job chain group based on the retrieved job chain and the previously generated job chain group.

[0227] Optionally, merge update units, including:

[0228] A running time determination subunit, used to determine the last running end time among the running end times of the merged chain of the selected job chain, and use it as the running start time of the selected job chain;

[0229] A job chain merging subunit, used to merge the merged chain of the selected job chain and the selected job chain to obtain a merged job chain;

[0230] A scheduling information updating subunit, used to determine the scheduling information of the merged job chain according to the scheduling information of the merged chain of the selected job chain and the scheduling information of the selected job chain;

[0231] The scheduling and arrangement updating subunit is used to update the corresponding arranged job chain and the scheduling and arrangement result of the corresponding arranged job chain according to the merged job chain and the scheduling information of the merged job chain.

[0232] Optionally, the second chain arrangement unit comprises:

[0233] A first merging determination subunit is used to determine a merged chain of the selected job chain from among the arranged job chains when the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit and when there are multiple arranged job chains;

[0234] The second merging determination subunit is used to determine the scheduling result of the selected job chain according to the arranged job chains when the sum of the loads of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit and when the number of arranged job chains is one, wherein the running start time of the selected job chain is the same as the last running end time of the arranged job chain.

[0235] Optional, separate job scheduling module, including:

[0236] An idle position determining unit, configured to determine an idle scheduling position of each of the job chains according to the scheduling arrangement result of each of the job chains in the merging result;

[0237] An insertion position determination unit, configured to query an insertable position of each target scheduling job in each of the job chains according to an idle scheduling position of each of the job chains and scheduling information of each of the target scheduling jobs;

[0238] A first insertion unit is used to determine the scheduling result of the target scheduling job according to the existing insertable position and the scheduling result of the corresponding job chain when the target scheduling job has an insertable position in the job chain, and update the scheduling result of the corresponding job chain;

[0239] A second insertion unit is used to determine the scheduled end time according to the scheduling information of the scheduled job chain and the scheduling information of the scheduled target scheduled job when there is no insertable position for the target scheduled job in each of the job chains;

[0240] The scheduling update unit is used to determine the scheduling result of the target scheduled job according to the scheduled end time.

[0241] Optionally, the insertion position determination unit comprises:

[0242] An operation area calculation subunit, used to calculate the product between the running time and the load of each target scheduling operation to obtain the operation area of ​​each target scheduling operation;

[0243] The job selection subunit is used to select the target scheduling job in turn from each of the target scheduling jobs in the order of the job area from large to small;

[0244] The insertion position detection unit is used to query the insertable position of the selected target scheduling job in each of the operation chains according to the operation area of ​​the selected target scheduling job and the idle scheduling position of each of the operation chains.

[0245] Optionally, an insertion position detection unit comprises:

[0246] An idle scheduling information acquisition subunit, used to acquire the idle duration and idle load corresponding to the idle scheduling position of each of the job chains;

[0247] An insertion probability calculation subunit, used to calculate the insertion probability of the selected target scheduling job and the idle scheduling position of each of the job chains according to the job area of ​​the selected target scheduling job and the idle duration and idle load corresponding to the idle scheduling position of each of the job chains;

[0248] The insertion position determination subunit is used to determine the insertable position of the selected target scheduling job in each of the job chains according to the insertion probability of the selected target scheduling job and the idle scheduling position of each of the job chains.

[0249] Optional, job merging module, including:

[0250] A relationship detection unit, configured to merge the to-be-scheduled jobs with the mergible association relationship when there is a mergible association relationship among the association relationships between the to-be-scheduled jobs, to obtain an association processing result; the mergible association relationship includes synchronous completion and / or dependent operation;

[0251] The job merging unit is used to determine the associated processing result and at least one remaining job to be scheduled among the jobs to be scheduled as a merging result.

[0252] Optionally, the relationship detection unit includes:

[0253] A synchronous completion merging subunit is used to merge multiple jobs to be scheduled whose association relationship is synchronous completion when there is an association relationship of synchronous completion among the association relationships between the jobs to be scheduled, so as to obtain an updated scheduled job;

[0254] A dependency merging subunit, configured to generate at least one job chain according to a plurality of jobs to be scheduled whose association relationships are dependent on running, when there is an association relationship of dependent running among the association relationships between the jobs to be scheduled;

[0255] The association processing subunit is used to determine each of the update scheduling jobs and / or each of the job chains as an association processing result.

[0256] Optionally, the scheduling information includes load and running time; the data system job scheduling arrangement device further includes:

[0257] A first load updating unit is configured to merge a plurality of to-be-scheduled jobs with a synchronous completion association relationship when there is a synchronous completion association relationship among the association relationships between the to-be-scheduled jobs, and after obtaining an updated scheduling job, determine the sum of the loads of the plurality of to-be-scheduled jobs with a synchronous completion association relationship as the load of the merged updated scheduling job;

[0258] A first duration updating unit is used to determine the maximum value of the running durations of multiple to-be-scheduled jobs that are associated with synchronous completion as the running duration of the merged updated scheduled job;

[0259] A second load updating unit is used for, when there is a dependent running association relationship among the association relationships between the jobs to be scheduled, generating at least one job chain according to the multiple jobs to be scheduled with the dependent running association relationship, and determining the maximum value of the loads of the multiple jobs to be scheduled with the dependent running association relationship as the load of the merged job chain;

[0260] The second duration updating unit is used to determine the sum of the running durations of multiple to-be-scheduled jobs that are associated with a dependent running relationship as the running duration of the merged job chain.

[0261] Optionally, the data system operation scheduling and arranging device further includes:

[0262] The running time adjustment module is used to adjust the running time of each of the jobs to be scheduled according to the running delay of each of the jobs to be scheduled; the running delay of the jobs to be scheduled is determined based on the historical delay statistics of the jobs to be scheduled.

[0263] The data system job scheduling and arranging device provided in the embodiment of the present invention can execute the data system job scheduling and arranging method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0264] Fig.16 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0265] like Fig.16 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0266] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0267] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a data system job scheduling method.

[0268] In some embodiments, the data system job scheduling method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data system job scheduling method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the data system job scheduling method in any other appropriate manner (for example, by means of firmware).

[0269] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0270] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0271] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0272] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0273] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0274] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0275] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the data system job scheduling method provided in any embodiment of the present application.

[0276] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0277] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data system job scheduling method, characterized in that: include: Obtain at least one job to be scheduled and an association relationship between the jobs to be scheduled; Merging the jobs to be scheduled according to the association relationship between the jobs to be scheduled to obtain a merged result, wherein the merged result includes at least one target scheduled job and / or at least one job chain, wherein the job chain includes multiple dependent scheduled jobs; Determine the scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result; According to the channel load upper limit, the scheduling arrangement results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, inserting each of the target scheduling jobs in the merged result into the scheduling arrangement results of each of the job chains to generate the scheduling arrangement result of the merged result; According to the scheduling arrangement result of the merged result, the scheduling order of each of the to-be-scheduled jobs is determined.

2. The method according to claim 1, characterized in that Determining the scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result includes: Selecting each of the operation chains in turn; Get the scheduled job chain; When the sum of the loads of the arranged job chains and the load of the selected job chain is less than or equal to the channel load upper limit, determining the scheduling result of the selected job chain, the operation start time of the selected job chain is the same as that of the corresponding arranged job chain; When the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, determining a merged chain of the selected job chains in each of the arranged job chains; In each of the arranged job chains, the selected job chain and the merged chain of the selected job chains are merged, the corresponding arranged job chain is updated, and the scheduling arrangement result of the corresponding arranged job chain is updated.

3. The method according to claim 2, characterized in that The selecting each of the operation chains in sequence includes: The job chains are selected in descending order of running time.

4. The method according to claim 3, characterized in that The step of determining a merged chain of the selected job chains from among the arranged job chains comprises: In order of the end time of the operation, at least one operation chain is sequentially taken out from each of the arranged operation chains for combination to obtain at least one operation chain group; Determining the load of each of the job chain groups according to the load of the job chains included in each of the job chain groups; Determining the job chain group corresponding to the selected job chain according to the load of each of the job chain groups, the channel load upper limit and the load of the selected job chain; A merge chain of the selected job chain is determined according to the job chains included in the job chain group corresponding to the selected job chain.

5. The method according to claim 4, characterized in that The determining, according to the load of each of the job chain groups, the channel load upper limit and the load of the selected job chain, the job chain group corresponding to the selected job chain includes: According to the load difference between the sum of the loads of the arranged operation chains and the upper limit of the channel load; Calculating the sum of the load of each of the operation chain groups and the load difference to obtain the load margin corresponding to each of the operation chain groups; Comparing the load margin corresponding to each of the operation chain groups with the load of the selected operation chain to obtain a comparison result of each of the operation chain groups; According to the comparison results of each of the job chain groups, the job chain group corresponding to the selected job chain is determined.

6. The method according to claim 4, characterized in that The method of sequentially taking out at least one job chain from each of the arranged job chains in the order of the running end time from earliest to latest to combine them to obtain at least one job chain group includes: According to the order of the running end time from the earliest to the latest, the first job chain is taken out from each of the arranged job chains to generate a job chain group; From the arranged job chains except the first job chain, one job chain is sequentially taken out in the order of the running end time from earliest to latest; Generate a new job chain group based on the retrieved job chain and the previously generated job chain group.

7. The method according to claim 4, characterized in that The step of merging the selected job chain and the merged chain of the selected job chain in each of the arranged job chains, updating the corresponding arranged job chain, and updating the scheduling arrangement result of the corresponding arranged job chain includes: Among the running end times of the merged chain of the selected job chain, determine the last running end time and use it as the running start time of the selected job chain; Merging the merged chain of the selected job chain and the selected job chain to obtain a merged job chain; Determining the scheduling information of the merged job chain according to the scheduling information of the merged chain of the selected job chain and the scheduling information of the selected job chain; According to the merged job chain and the scheduling information of the merged job chain, the corresponding arranged job chain is updated, and the scheduling arrangement result of the corresponding arranged job chain is updated.

8. The method according to claim 5, characterized in that When the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, determining a merged chain of the selected job chains in each of the arranged job chains includes: When the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, when there are multiple arranged job chains, determining a merged chain of the selected job chain in each of the arranged job chains; When the sum of the load of each of the arranged job chains and the load of the selected job chain is greater than the channel load upper limit, the method further includes: When the number of arranged job chains is one, the scheduling result of the selected job chain is determined according to the arranged job chains, wherein the running start time of the selected job chain is the same as the last running end time in the arranged job chain.

9. The method according to claim 1, characterized in that: The step of inserting each of the target scheduling jobs in the merged result into the scheduling and arrangement results of each of the job chains according to the channel load upper limit, the scheduling and arrangement results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, and generating the scheduling and arrangement results of the merged result includes: Determine the idle scheduling position of each of the job chains according to the scheduling arrangement result of each of the job chains in the merged result; According to the idle scheduling position of each of the job chains and the scheduling information of each of the target scheduling jobs, querying the insertable position of each of the target scheduling jobs in each of the job chains; When there is an insertable position for the target scheduling job in the job chain, determine the scheduling result of the target scheduling job according to the existing insertable position and the scheduling result of the corresponding job chain, and update the scheduling result of the corresponding job chain; When there is no insertable position for the target scheduled job in each of the job chains, determining the scheduled end time according to the scheduling information of the scheduled job chains and the scheduling information of the scheduled target scheduled job; According to the scheduled end time, a scheduling result of the target scheduled job is determined.

10. The method according to claim 9, characterized in that The querying of the insertable position of each target scheduling job in each job chain according to the idle scheduling position of each job chain and the scheduling information of each target scheduling job includes: Calculate the product of the running time and the load of each target scheduling operation to obtain the operation area of ​​each target scheduling operation; Selecting the target scheduling operation in turn from the target scheduling operations in the order of the operation area from large to small; According to the operation area of ​​the selected target scheduling operation and the idle scheduling positions of each of the operation chains, the insertable position of the selected target scheduling operation in each of the operation chains is queried.

11. The method according to claim 10, characterized in that The step of querying the insertable position of the selected target scheduling job in each of the job chains according to the job area of ​​the selected target scheduling job and the idle scheduling position of each of the job chains comprises: Obtaining the idle duration and idle load corresponding to the idle scheduling position of each of the job chains; Calculate the insertion probability of the selected target scheduling job and the idle scheduling position of each of the job chains according to the operation area of ​​the selected target scheduling job and the idle duration and idle load corresponding to the idle scheduling position of each of the job chains; According to the insertion probability of the selected target scheduling job and the idle scheduling position of each of the job chains, the insertable position of the selected target scheduling job in each of the job chains is determined.

12. The method according to claim 1, characterized in that The step of merging the jobs to be scheduled according to the association relationship between the jobs to be scheduled to obtain a merging result includes: When there is a mergeable association relationship among the association relationships between the jobs to be scheduled, the jobs to be scheduled with the mergeable association relationship are merged to obtain an association processing result; the mergeable association relationship includes synchronous completion and / or dependent operation; The association processing result and at least one remaining job to be scheduled among the jobs to be scheduled are determined as a merging result.

13. The method according to claim 12, characterized in that The step of merging the to-be-scheduled jobs that have a mergeable association relationship to obtain an association processing result includes: When there is a synchronously completed association relationship among the association relationships between the jobs to be scheduled, merging multiple jobs to be scheduled whose association relationships are synchronously completed to obtain an updated scheduled job; When there is a dependent-running association relationship among the association relationships between the jobs to be scheduled, generating at least one job chain according to the multiple jobs to be scheduled whose association relationships are dependent-running; Each of the update scheduling jobs and / or each of the job chains is determined as an associated processing result.

14. The method according to claim 13, characterized in that The scheduling information includes load and running time; When there is a synchronously completed association relationship among the association relationships between the jobs to be scheduled, after merging the multiple jobs to be scheduled whose association relationships are synchronously completed to obtain an updated scheduled job, the method further includes: The sum of the loads of multiple to-be-scheduled jobs that are associated with synchronous completion is determined as the load of the merged update scheduling job; The maximum value of the running time of multiple to-be-scheduled jobs that are associated with synchronous completion is determined as the running time of the merged update scheduling job; When there is a dependent running association relationship among the association relationships between the jobs to be scheduled, after generating at least one job chain according to the multiple jobs to be scheduled whose association relationships are dependent running, the method further includes: The maximum value among the loads of multiple jobs to be scheduled that are associated with a dependent running relationship is determined as the load of the merged job chain; The sum of the running times of multiple to-be-scheduled jobs that are associated with a dependent running relationship is determined as the running time of the merged job chain.

15. The method according to claim 13, characterized in that Also includes: According to the running delay of each of the jobs to be scheduled, adjusting the running time of each of the jobs to be scheduled; The running delay of the to-be-scheduled job is determined based on the historical delay statistics result of the to-be-scheduled job.

16. A data system job scheduling and arranging device, characterized in that: include: A job dependency acquisition module, used to acquire at least one job to be scheduled and the association relationship between the jobs to be scheduled; A job merging module, used to merge the jobs to be scheduled according to the association relationship between the jobs to be scheduled, and obtain a merging result, wherein the merging result includes at least one target scheduling job and / or at least one job chain, and the job chain includes multiple dependent scheduling jobs; A job chain arrangement module, used to determine the scheduling arrangement result of each of the job chains in the merged result according to the channel load upper limit and the scheduling information of each of the job chains in the merged result; A separate job scheduling module, used to insert each of the target scheduling jobs in the merged result into the scheduling and scheduling results of each of the job chains according to the channel load upper limit, the scheduling and scheduling results of each of the job chains in the merged result, and the scheduling information of each of the target scheduling jobs, to generate a scheduling and scheduling result of the merged result; The job order determination module is used to determine the scheduling order of each of the jobs to be scheduled according to the scheduling arrangement result of the merged result.

17. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the data system job scheduling method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data system job scheduling method according to any one of claims 1 to 15.

19. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the data system job scheduling and arranging method according to any one of claims 1 to 15.

Images