Information processing program, information processing method, and information processing device

Abstract

To easily set a point from which a job flow is to be restarted.SOLUTION: An information processing apparatus 100 stores a job flow 121. The information processing apparatus 100 classifies, based on the content of process of each job in a job group for which an execution order is defined in the job flow 121, the job group into a plurality of groups. The information processing apparatus 100 stores, in association with at least one group of the plurality of groups, information which indicates a beginning job in a group from which the job flow 121 is to be restarted in accordance with an anomaly of a job belonging to the one group. The information processing apparatus 100 detects an anomaly of a first job classified into a first group. The information processing apparatus 100 restarts, when the anomaly of the first job is detected, the job flow 121 from the beginning job in a second group from which the job flow 121 is to be restarted in accordance with the anomaly of the first job, the job being associated with the first group.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing program, an information processing method, and an information processing apparatus.

Background Art

[0002] Conventionally, there is a system that executes a job group according to a job flow. When an abnormality occurs in any one of the jobs in the job group and any one of the jobs ends, the entire job flow also ends once, so it is desirable to restart the job flow.

[0003] As prior art, for example, when a new error of a job is detected and a plurality of recovery candidates are specified at the same time, there is one that executes the recovery process of the job based on the priority order. Also, for example, there is a technique for determining an abnormality of a target job based on a reference time or a reference time changed based on monitoring information obtained by monitoring a monitoring target associated with the target job. Also, for example, there is a technique for re-executing a job specified by the specified information of the job of the notification destination of the event corresponding to the specified information of the job of the notification source of the event. Also, for example, there is a technique for returning the state of the system to the state before the target job and the job having a dependency relationship when the target job is executed are executed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, if an error occurs in any job, one possible approach is to restart the job flow from a pre-configured starting point job, such as the job that experienced the error, the job immediately preceding it, or the job at the beginning of the job flow. However, with the above method, it is difficult to properly set the starting point for restarting the job flow. For instance, as the number of jobs increases, the workload involved in setting or updating the starting point for restarting the job flow increases, making it increasingly difficult to correctly set or update the starting point for restarting the job flow.

[0006] In one aspect, the present invention aims to make it easier to set a starting point for restarting a job flow. [Means for solving the problem]

[0007] According to one embodiment, an information processing program, information processing method, and information processing device are proposed that classify a group of jobs into multiple groups based on the processing content of each job in a group of jobs whose execution order is defined in a job flow, and when an abnormality is detected in a first job classified into the first group of the multiple groups, the device refers to a storage unit that stores information indicating the job that is at the beginning of the execution order in the group of the multiple groups that will be used as the starting point for restarting the job flow, and restarts the job flow starting from the job that is at the beginning of the execution order in the second group, which will be used as the starting point for restarting the job flow, based on the information indicating the job that is at the beginning of the execution order in the second group that will be used as the starting point for restarting the job flow, which is associated with the first group. [Effects of the Invention]

[0008] According to one embodiment, it becomes easier to set a starting point for restarting the job flow. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is an explanatory diagram showing one embodiment of the information processing method according to the embodiment. [Figure 2] Figure 2 is an explanatory diagram showing an example of the information processing system 200. [Figure 3] Figure 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. [Figure 4] Figure 4 is a block diagram showing an example of the functional configuration of the information processing device 100. [Figure 5] Figure 5 is an explanatory diagram showing an example of job flow 500. [Figure 6] Figure 6 is an explanatory diagram showing an example of how to store job flow 500. [Figure 7] Figure 7 is an explanatory diagram showing an example of the contents stored in the attribute information management table 700. [Figure 8] Figure 8 is an explanatory diagram showing an example of the contents stored in the group information management table 800. [Figure 9] Figure 9 is an explanatory diagram showing an example of the contents stored in the leading information management table 900. [Figure 10] Figure 10 is an explanatory diagram showing an example of the contents stored in the restart information management table 1000. [Figure 11] Figure 11 is an explanatory diagram showing an example of the contents stored in the execution history management table 1100. [Figure 12] Figure 12 is an explanatory diagram (part 1) showing a specific example of classifying a group of jobs into multiple groups. [Figure 13] Figure 13 is an explanatory diagram (part 2) showing a specific example of classifying a group of jobs into multiple groups. [Figure 14] Figure 14 is an explanatory diagram (part 3) showing a specific example of classifying a group of jobs into multiple groups. [Figure 15] Figure 15 is an explanatory diagram (part 4) showing a specific example of classifying a group of jobs into multiple groups. [Figure 16] Figure 16 is an explanatory diagram (part 5) showing a specific example of classifying a group of jobs into multiple groups. [Figure 17]FIG. 17 is an explanatory diagram (No. 6) showing a specific example of classifying job groups into a plurality of groups. [Figure 18] FIG. 18 is an explanatory diagram (No. 7) showing a specific example of classifying job groups into a plurality of groups. [Figure 19] FIG. 19 is an explanatory diagram (No. 8) showing a specific example of classifying job groups into a plurality of groups. [Figure 20] FIG. 20 is an explanatory diagram (No. 9) showing a specific example of classifying job groups into a plurality of groups. [Figure 21] FIG. 21 is an explanatory diagram (No. 1) showing a specific example of restarting job flow 500. [Figure 22] FIG. 22 is an explanatory diagram (No. 2) showing a specific example of restarting job flow 500. [Figure 23] FIG. 23 is an explanatory diagram (No. 1) showing a specific example of adding a new job to a job group. [Figure 24] FIG. 24 is an explanatory diagram (No. 2) showing a specific example of adding a new job to a job group. [Figure 25] FIG. 25 is an explanatory diagram (No. 3) showing a specific example of adding a new job to a job group. [Figure 26] FIG. 26 is an explanatory diagram (No. 4) showing a specific example of adding a new job to a job group. [Figure 27] 0]FIG. 27 is a flowchart showing an example of a determination processing procedure. [Figure 28] FIG. 28 is a flowchart showing an example of an overall classification processing procedure. [Figure 29] FIG. 29 is a flowchart showing an example of a first classification processing procedure. [Figure 30] FIG. 30 is a flowchart showing an example of a second classification processing procedure. [Figure 31] FIG. 31 is a flowchart showing an example of a restart processing procedure. [Figure 32] FIG. 32 is a flowchart showing an example of an addition processing procedure. <{

MODE FOR CARRYING OUT THE INVENTION

[0010] Embodiments of the information processing program, information processing method, and information processing apparatus according to the present invention will be described in detail below with reference to the drawings.

[0011] (An embodiment of the information processing method according to the embodiment) Figure 1 is an explanatory diagram showing one embodiment of the information processing method according to the embodiment. The information processing device 100 is a computer for starting a job flow. The information processing device 100 is, for example, a server or a PC (Personal Computer).

[0012] A job flow, for example, represents a group of jobs related to a system that utilizes data. A job flow, for example, defines the execution order of a group of jobs. A job flow, for example, shows the data dependencies of a group of jobs. Any job in a group of jobs may utilize data output by each of the other jobs in the group. Data output by any one job in a group of jobs may be utilized by multiple jobs.

[0013] A job flow can be represented, for example, by a directed graph that includes nodes representing each job in a group of jobs, and directed edges connecting the nodes representing two different jobs within that group. Specifically, a directed edge connects a node representing any of the jobs in the group to a node representing another job that will be executed next and will utilize the data output of that job. Thus, a directed edge specifically indicates data dependencies between jobs.

[0014] In the following explanation, for example, other jobs represented by nodes reachable by traversing directed edges backward from a node representing a certain job will be treated as "preceding jobs" that exist before the job in question. On the other hand, for example, other jobs represented by nodes reachable by traversing directed edges sequentially from a node representing a certain job will be treated as "successor jobs" that exist after the job in question. Furthermore, for example, other jobs represented by nodes that cannot be reached by traversing directed edges sequentially or backward from a node representing a certain job will be treated as "parallel jobs" that run in parallel with the job in question. Also, for example, a certain job and a parallel job that runs in parallel with that job may be executed in parallel.

[0015] Starting a job flow means sequentially starting each job in the group of jobs whose execution order is defined in the job flow, and then starting each job in sequence. Completing all jobs in the group corresponds to completing the job flow. After the group of jobs has started executing according to the job flow, it is possible that an error may occur in one of the jobs in the group, causing that job to terminate.

[0016] An error in a job could be due to, for example, a timeout, an error caused by the environment in which the job is running, an error caused by the processing content of the job, or an error caused by the processing content of another preceding job. If an error occurs in any of the jobs in a group of jobs and that job terminates, the job flow will also terminate temporarily.

[0017] Therefore, it is desirable to restart the job flow. In this case, the group of jobs includes jobs waiting to be executed and jobs that have already been executed. Completed jobs may be jobs that have already been executed but would be preferable to re-execute in order to resolve an error in any of the other jobs. Completed jobs may be jobs that do not need to be re-executed. Completed jobs may be jobs that would cause errors if re-executed and it is preferable not to re-execute them.

[0018] For example, depending on the type of anomaly that occurs in any of the jobs, the job that should preferably be rerun may change. For instance, when restarting a job flow, if a job that should preferably be rerun is not rerun, it may not be possible to resolve an anomaly in any of the jobs. For example, if a job flow is restarted and a job that does not need to be rerun is rerun, it may lead to an increase in the processing load and processing time required to complete the job flow. Therefore, it is desirable to be able to flexibly change the job that serves as the starting point for restarting a job flow.

[0019] In contrast, one possible approach is to restart the job flow from a pre-configured starting job when an error occurs in a particular job. This starting job is set, for example, by the job flow operator. This starting job could be, for example, the job that experienced the error, another preceding job that exists immediately before the job that experienced the error, another job at the beginning of the job flow, or any other job of your choice.

[0020] Specifically, the above method is implemented by utilizing job flow branching technology. Specifically, the above method controls the job flow so that processing branches from a certain job to a pre-configured starting job in response to an error code related to an anomaly that occurred in that job.

[0021] The above method can make it difficult to properly configure the job that serves as the starting point for restarting a job flow. For example, as data utilization systems become larger in scale, the number of jobs included in a group of jobs tends to increase enormously. With the above method, the larger the number of jobs included in a group of jobs, the greater the workload and time required for job flow operators to configure or update the starting point for restarting a job flow for each individual job.

[0022] Furthermore, without understanding the characteristics of each job, the job flow operator cannot determine which other job in the job group is preferable to set as the starting point for restarting the job flow. For this reason, with the above method, the job flow operator may have to refer to the job flow design document or other documents when setting the starting job for restarting the job flow for each job. This leads to an increased workload and time for the job flow operator when setting or updating the starting point for restarting the job flow.

[0023] For example, when a system that utilizes data is updated, new jobs may be added to the job group. With the above method, each time a new job is added to the job group, the job flow operator or other personnel would have to set a starting point for restarting the job flow in association with the new job, which would lead to an increased workload and working time for the job flow operator or other personnel.

[0024] Furthermore, the above method increases the workload and working time for job flow operators, which increases the likelihood of them making mistakes in setting the restart point for the job flow. Thus, the above method tends to make it difficult for job flow operators to correctly set or update the restart point for the job flow.

[0025] Therefore, this embodiment describes an information processing method that makes it easier to set the starting point for restarting a job flow.

[0026] In Figure 1, the information processing device 100 stores a job flow 121. The job flow 121 defines, for example, the execution order of a group of jobs. In the example in Figure 1, the group of jobs specifically includes jobs 1 to 10. The job flow 121 shows, for example, the data dependencies of the group of jobs.

[0027] Job flow 121 is represented by a directed graph containing nodes 101 to 110, each representing a job from 1 to 10. Node 10i, representing job i, and node 10j, representing job j which will be executed after job i and utilizes the data output by job i, are connected by a directed edge. Thus, the directed edge indicates a data dependency.

[0028] The information processing device 100 executes a group of jobs according to the job flow 121. For example, the information processing device 100 executes job 1, job 2, job 3, and job 4 in order. For example, the information processing device 100 executes job 5, job 6, job 7, and job 8 in order. For example, the information processing device 100 may execute jobs 1-4 and jobs 5-8 in parallel. For example, after completing job 4 and job 8, the information processing device 100 executes job 9 and job 10 in order.

[0029] (1-1) The information processing device 100 classifies the job group into multiple groups based on the processing content of each job in the job group whose execution order is defined in the job flow 121.

[0030] The information processing device 100 assigns attributes to each job based on the processing content of each job in the job group. These attributes may include, for example, collection, processing, integration, or storage. Collection, for example, is an attribute indicating that data is collected from a data source. Processing, for example, is an attribute indicating that data obtained from a single preceding job is processed. Integration, for example, is an attribute indicating that data obtained from multiple preceding jobs is processed. Storage, for example, is an attribute indicating that data is stored in a DB (Database).

[0031] Specifically, the information processing device 100 assigns the attribute "collection" to a job if the processing content of that job is to collect data. Specifically, the information processing device 100 assigns the attribute "processing" to a job if the processing content of that job is to process data obtained from a single preceding job.

[0032] Specifically, the information processing device 100 assigns the attribute "integration" to a job if the processing content of that job involves processing data obtained from multiple preceding jobs. Specifically, the information processing device 100 assigns the attribute "processing" to a job if the processing content of that job involves processing data obtained from a single preceding job.

[0033] In the example shown in Figure 1, the information processing device 100 specifically assigns the attribute "collection" to jobs 1, 5, and 6. Specifically, the information processing device 100 assigns the attribute "processing" to jobs 2-4, 7, and 8. Specifically, the information processing device 100 assigns the attribute "integration" to job 9. Specifically, the information processing device 100 assigns the attribute "storage" to job 10. As a result, the information processing device 100 can identify the attribute corresponding to the processing content of each job and obtain guidelines for classifying the group of jobs into multiple groups.

[0034] The information processing device 100 classifies the job group into multiple groups based on attributes assigned to each job. These groups include, for example, a group having the "collection" attribute, which includes a group classifying jobs with the "collection" attribute. Another group includes, for example, a group having the "processing" attribute, which includes a group classifying jobs with the "processing" attribute. Yet another group includes, for example, a group having the "integration" attribute, which includes a group classifying jobs with the "integration" attribute. Finally, another group includes, for example, a group having the "storage" attribute, which includes a group classifying jobs with the "storage" attribute.

[0035] In the example shown in Figure 1, the information processing device 100 specifically classifies jobs 1, 5, and 6 into group 111, which has the attribute "collection". Specifically, the information processing device 100 classifies jobs 2-4, 7, and 8 into group 112, which has the attribute "processing". Specifically, the information processing device 100 classifies job 9 into group 113, which has the attribute "integration". Specifically, the information processing device 100 classifies job 10 into group 114, which has the attribute "storage". This makes it easier for the information processing device 100 to group together and handle one or more jobs with the same attribute from among the job group.

[0036] (1-2) The information processing device 100 has a storage unit 122. The storage unit 122 stores information indicating the first job in a group that will be used as the starting point for restarting the job flow 121 in response to an abnormality in a job belonging to any of the groups, in association with at least one of the groups of a plurality of groups.

[0037] The first job is, for example, a job in a group that has no preceding jobs. The first job is, for example, the job that is first in the execution order within a group. There may be multiple first jobs. Information indicating the first job is set, for example, by the operator of job flow 121.

[0038] The memory unit 122 may store one or more combinations of conditions for restarting the job flow 121 in response to an abnormality in a job belonging to any of the groups, and information indicating the first job in the group from which the restart will begin. The conditions may be, for example, that some kind of abnormality has occurred in a job belonging to any of the groups. The conditions may also be, for example, that a specific type of abnormality has occurred in a job belonging to any of the groups.

[0039] The memory unit 122 may store, for example, one or more combinations of conditions for restarting the job flow 121 in response to an abnormality in a job belonging to the first group, and information indicating the leading job in the second group that will be used as the starting point for the restart, associated with the first group. These combinations are set, for example, by the operator of the job flow 121.

[0040] In the example shown in Figure 1, the memory unit 122 specifically stores, in association with group 113, a combination of the conditions for restarting job flow 121 in response to an abnormality in a job belonging to group 113, and the group 112 that serves as the starting point for restarting job flow 121. Specifically, the memory unit 122 further stores, in association with group 112, information indicating jobs 2 and 7 as the leading jobs in group 112.

[0041] (1-3) The information processing device 100 detects an anomaly in the first job classified into the first group. The information processing device 100 may also identify the type of anomaly in the first job that occurred. Here, if the first job fails to execute due to an anomaly in the first job of the job group, the information processing device 100 will terminate without completing the job flow 121.

[0042] Therefore, if the information processing device 100 fails to execute the first job in the job group due to an abnormality in the first job, other jobs in the job group that are currently running may also fail to execute. Thus, an abnormality in the first job can cause abnormalities in other jobs. An abnormality in another job might be, for example, a forced execution failure caused by the abnormality in the first job.

[0043] In the example shown in Figure 1, the information processing device 100 specifically detects an anomaly in job 9 belonging to group 113. The information processing device 100 may also specifically identify the type of anomaly in job 9. Here, the information processing device 100 assumes that when an anomaly occurs in job 9 of the job group, there are no other jobs running in the job group, and no anomalies occur in other jobs. As a result, the information processing device 100 can determine that it is preferable to restart job flow 121, and obtain a trigger to restart job flow 121.

[0044] (1-4) When the information processing device 100 detects an abnormality in the first job, it refers to the storage unit 122 and obtains information indicating the first job in the second group, which is associated with the first group and will serve as the starting point for restarting the job flow 121 in response to the abnormality in the first job. The information processing device 100 also refers to the storage unit 122 and obtains any conditions associated with the first group for restarting the job flow 121 in response to the abnormality in the first job.

[0045] If there are no conditions, the information processing device 100 restarts the job flow 121 starting from the first job in the second group based on the acquired information. If there are conditions, the information processing device 100 determines whether or not the conditions are met, and if it determines that the conditions are met, it restarts the job flow 121 starting from the first job in the second group based on the acquired information.

[0046] In the example shown in Figure 1, the information processing device 100 specifically refers to the storage unit 122 and obtains information indicating jobs 2 and 7 as the leading jobs in group 112, which are associated with group 113 and serve as the starting point for restarting job flow 121 in response to an abnormality in job 9. Furthermore, the information processing device 100 specifically refers to the storage unit 122 and obtains the conditions for restarting job flow 121 in response to an abnormality in job 9, which are associated with group 113.

[0047] Specifically, the information processing device 100 determines whether the acquired conditions are met based on the type of anomaly in job 9. Here, it is assumed that the information processing device 100 specifically determines that the acquired conditions are met. Specifically, because the information processing device 100 has determined that the conditions are met, it restarts job flow 121, starting from jobs 2 and 7, based on the acquired information.

[0048] Specifically, when the information processing device 100 restarts the job flow 121, it executes jobs 2 and 7 as the starting point, and then sequentially executes other subsequent jobs that follow jobs 2 and 7, according to the job flow 121. Specifically, when the information processing device 100 restarts the job flow 121, it is preferable that any other parallel jobs that run in parallel with the starting job 2 and 7, and have not yet been executed, are executed sequentially.

[0049] More specifically, when the information processing device 100 restarts the job flow 121, it executes jobs 2 to 4 in order, and then jobs 7 and 8 in order. More specifically, after completing jobs 4 and 8, the information processing device 100 executes jobs 9 and 10 in order.

[0050] This allows the information processing device 100 to appropriately control the group that serves as the starting point for restarting the job flow 121. Even if the number of jobs included in the job group becomes enormous, the information processing device 100 allows the operator of the job flow 121 to simply set the restart starting point on a group basis. Furthermore, the information processing device 100 can classify the job group into multiple groups based on the processing content of each job, so that the operator of the job flow 121 does not need to understand the properties of each job. When restarting the job flow 121, the information processing device 100 can avoid re-executing the entire job group, thereby reducing the processing burden and processing time required when restarting the job flow 121.

[0051] As described above, the information processing device 100 can reduce the workload and working time for operators of the job flow 121. In order to reduce the workload and working time for operators of the job flow 121, the information processing device 100 can reduce the probability that operators of the job flow 121 will make a mistake in setting the starting point for restarting the job flow 121.

[0052] Furthermore, the information processing device 100 can, for example, switch the group that will be the starting point for restarting the job flow 121 depending on the type of abnormality that occurs in any of the jobs. As a result, the information processing device 100 can efficiently restart the job flow 121, thereby reducing the processing burden and processing time required to complete all jobs in the job group.

[0053] This section describes a case where the information processing device 100 detects a job anomaly and restarts the job flow 121 without waiting for input from the device's user, but it is not limited to this case. The device's user is, for example, the operator of the job flow 121.

[0054] For example, after the information processing device 100 detects a job anomaly, it may wait for a specific input from a user of its device before restarting the job flow 121. The specific input could be, for example, an input indicating permission to restart the job flow 121. This allows the information processing device 100 to make it easier for the operator of the job flow 121 to deal with any anomalies that have occurred in any job before restarting the job flow 121.

[0055] This section describes the case where the information processing device 100 operates independently, but it is not limited to this. For example, the information processing device 100 may collaborate with other computers. Specifically, the information processing device 100 may not execute a group of jobs on its own, but rather control other computers that execute the group of jobs. For example, multiple computers may collaborate to realize the functions of the information processing device 100. Specifically, the functions of the information processing device 100 may be realized on the cloud. An example of the information processing device 100 collaborating with other computers will be described later using Figure 2.

[0056] (An example of information processing system 200) Next, using Figure 2, we will describe an example of an information processing system 200 to which the information processing device 100 shown in Figure 1 is applied.

[0057] Figure 2 is an explanatory diagram showing an example of an information processing system 200. In Figure 2, the information processing system 200 includes an information processing device 100, one or more job execution devices 201, and a client device 202.

[0058] In the information processing system 200, the information processing device 100 and the job execution device 201 are connected via a wired or wireless network 210. The network 210 can be, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the Internet. Also in the information processing system 200, the information processing device 100 and the client device 202 are connected via a wired or wireless network 210.

[0059] The information processing device 100 is a computer that starts a job flow. The information processing device 100 stores the job flow. The information processing device 100 classifies the job group into multiple groups based on the processing content of each job in the job group whose execution order is defined in the job flow.

[0060] The information processing device 100 receives a start trigger to begin executing the job group. For example, the information processing device 100 receives a request from the client device 202 to execute the job group, which serves as the start trigger to begin executing the job group.

[0061] The information processing device 100 controls one or more job execution devices 201. For example, the information processing device 100 distributes a group of jobs to one or more job execution devices 201. For example, the information processing device 100 controls one or more job execution devices 201 to execute the jobs distributed to each job execution device 201 according to the job flow.

[0062] Specifically, the information processing device 100, upon receiving a start trigger, controls one or more job execution devices 201 to execute the jobs assigned to each job execution device 201, thereby starting a job flow.

[0063] The information processing device 100 has a storage unit. The storage unit stores one or more combinations of conditions for restarting a job flow in response to an abnormality in a job belonging to any group, and information indicating the first job in the group that will be used as the starting point for the restart.

[0064] The information processing device 100 detects an anomaly in a first job belonging to the first group, which is executed by one of the job execution devices 201 within the job group. The information processing device 100 detects the anomaly in the first job, for example, by receiving a notification from one of the job execution devices 201 indicating the anomaly in the first job. The notification may include, for example, information indicating the type of anomaly in the first job. The information processing device 100 identifies the type of anomaly in the first job that occurred, for example, based on the notification. When the information processing device 100 detects an anomaly in the first job, for example, it controls one or more job execution devices 201 to terminate abnormally without completing the job flow.

[0065] The information processing device 100 refers to the storage unit to determine whether the detected abnormality in the first job satisfies the conditions for restarting the job flow associated with the first group to which the first job belongs. If the information processing device 100 determines that the conditions are met, it refers to the storage unit to identify one or more leading jobs based on information indicating one or more leading jobs in the group that will serve as the starting point for restarting the job flow associated with the first group to which the first job belongs.

[0066] The information processing device 100 restarts the job flow starting from the first job of each of the one or more identified first jobs. The information processing device 100 controls one or more job execution devices 201 to execute all or part of the job group, starting from the first job of each of the one or more identified first jobs. When the information processing device 100 completes all jobs in the job group and completes the job flow, it sends a notification to the client device 202 indicating that all jobs in the job group have been completed and the job flow has been completed. The information processing device 100 is, for example, a server or a PC.

[0067] The job execution device 201 is a computer that executes one of the jobs in the job group. The job execution device 201 executes the job assigned to it according to the control of the information processing device 100. If the job to be executed after the job assigned to the job execution device 201 is assigned to another job execution device 201, the job execution device 201 transmits the data output by the job assigned to it to the other job execution device 201 to which the next job to be executed is assigned.

[0068] If an abnormality occurs in a job assigned to the job execution device 201, it generates a notification indicating the abnormality of the assigned job and sends it to the information processing device 100. The job execution device 201 may, for example, identify the type of abnormality in the assigned job, generate a notification indicating the abnormality of the assigned job that includes information indicating the type of abnormality of the identified job, and send it to the information processing device 100. The job execution device 201 is, for example, a server or a PC.

[0069] The client device 202 is a computer used by users of the information processing system 200. Users of the information processing system 200 are, for example, job flow operators. The client device 202 generates a request to execute a group of jobs based on the operation input of the user of the information processing system 200, and sends it to the information processing device 100.

[0070] The client device 202 receives, for example, a notification from the information processing device 100 indicating that it has completed all jobs in the job group and completed the job flow. The client device 202 then outputs, for example, a notification indicating that it has completed all jobs in the job group and completed the job flow, so that it can be viewed by users of the information processing system 200. The information processing device 100 is, for example, a PC, a tablet terminal, or a smartphone.

[0071] This explanation describes a case where the information processing device 100 and the job execution device 201 are different devices, but this is not the only case. For example, the information processing device 100 may have the functionality of a job execution device 201 and may operate as a job execution device 201.

[0072] This explanation describes a case where the information processing device 100 and the client device 202 are different devices, but it is not limited to this case. For example, the information processing device 100 may have the functionality of a client device 202 and may operate as a client device 202.

[0073] (Example of hardware configuration of information processing device 100) Next, an example of the hardware configuration of the information processing device 100 will be described using Figure 3.

[0074] Figure 3 is a block diagram showing an example of the hardware configuration of the information processing device 100. In Figure 3, the information processing device 100 includes a CPU (Central Processing Unit) 301, memory 302, network interface 303, recording medium interface 304, and recording medium 305. Each component is connected by a bus 300.

[0075] Here, the CPU 301 is responsible for the overall control of the information processing device 100. The memory 302 includes, for example, ROM (Read Only Memory), RAM (Random Access Memory), and flash ROM. Specifically, for example, flash ROM and ROM store various programs, and RAM is used as the work area for the CPU 301. Programs stored in memory 302 are loaded into the CPU 301, causing the CPU 301 to execute the coded processes.

[0076] The network interface 303 is connected to network 210 via a communication line, and then connects to other computers via network 210. The network interface 303 manages the internal interface with network 210 and controls the input and output of data from other computers. The network interface 303 is, for example, a modem or a LAN adapter.

[0077] The recording medium interface (I / F) 304 controls the reading and writing of data to the recording medium 305 according to the control of the CPU 301. The recording medium interface (I / F) 304 is, for example, a disk drive, an SSD (Solid State Drive), or a USB (Universal Serial Bus) port. The recording medium 305 is a non-volatile memory that stores the data written under the control of the recording medium interface (I / F) 304. The recording medium 305 is, for example, a disk, semiconductor memory, or USB memory. The recording medium 305 may be detachable from the information processing device 100.

[0078] In addition to the components described above, the information processing device 100 may also have, for example, a keyboard, mouse, display, printer, scanner, microphone, speaker, etc. Furthermore, the information processing device 100 may have multiple recording medium interfaces 304 and recording mediums 305. Alternatively, the information processing device 100 may not have recording medium interfaces 304 and recording mediums 305.

[0079] (Example hardware configuration of job execution device 201) The hardware configuration example of the job execution device 201 is specifically the same as the hardware configuration example of the information processing device 100 shown in Figure 3, so a detailed explanation is omitted.

[0080] (Example hardware configuration for client device 202) The hardware configuration example for client device 202 is specifically the same as the hardware configuration example for information processing device 100 shown in Figure 3, so a detailed explanation is omitted.

[0081] The following explanation primarily describes the case where the information processing device 100 operates independently and executes a group of jobs.

[0082] (Example of the functional configuration of the information processing device 100) Next, an example of the functional configuration of the information processing device 100 will be described using Figure 4.

[0083] Figure 4 is a block diagram showing an example of the functional configuration of the information processing device 100. The information processing device 100 includes a storage unit 400, an acquisition unit 401, a classification unit 402, an execution unit 403, a detection unit 404, an activation unit 405, and an output unit 406.

[0084] The storage unit 400 is implemented by a storage area such as the memory 302 or recording medium 305 shown in Figure 3. The following description will focus on the case where the storage unit 400 is included in the information processing device 100, but is not limited to this case. For example, the storage unit 400 may be included in a device different from the information processing device 100, and the contents of the storage unit 400 may be accessible from the information processing device 100.

[0085] The acquisition unit 401 to the output unit 406 function as an example of a control unit. Specifically, the acquisition unit 401 to the output unit 406 realize their functions, for example, by having the CPU 301 execute a program stored in a storage area such as the memory 302 or recording medium 305 shown in Figure 3, or by using the network I / F 303. The processing results of each functional unit are stored in a storage area such as the memory 302 or recording medium 305 shown in Figure 3.

[0086] The storage unit 400 stores various information that is referenced or updated in the processing of each functional unit. The storage unit 400 stores, for example, a job flow. A job flow represents, for example, a group of jobs. A job flow defines, for example, the execution order of a group of jobs. A job flow shows, for example, the data dependencies of a group of jobs. Any job in a group of jobs may be a job that utilizes data output by each of the other jobs in the group. Multiple jobs may utilize data output by any job in a group of jobs.

[0087] A job flow can be represented, for example, by a directed graph that includes nodes representing each job in a group of jobs and directed edges connecting the nodes representing two different jobs within the group. Specifically, a directed edge connects a node representing any job in the group to a node representing another job that is one execution step later than that job and utilizes the data output by that job. The storage unit 400 specifically stores the job flow using a job flow management table 600. An example of the job flow management table 600 will be described later with reference to Figure 6. The job flow is acquired, for example, by the acquisition unit 401.

[0088] The memory unit 400 stores, for example, the attributes of each job in the job group. These attributes may include, for example, collection, processing, integration, or storage. Collection, for example, is an attribute indicating that data is collected from a data source. The data source may be, for example, a file. The data source may also be, for example, a database. The data source may also be the source of data collected by an ETL (Extract, Transform, Load) tool.

[0089] Processing is an attribute that indicates, for example, processing data obtained from a single preceding job. Integration is an attribute that indicates, for example, processing data obtained from multiple preceding jobs. Storage is an attribute that indicates, for example, storing data in a database. Specifically, the storage unit 400 stores the attributes of each job in the job group using the attribute information management table 700. An example of the attribute information management table 700 will be described later with reference to Figure 7. The attributes of each job are identified, for example, by the classification unit 402.

[0090] The storage unit 400 stores, for example, the results of classifying each job in the job group into multiple groups. These groups include, for example, collection groups having the attribute of collection. A collection group is, for example, a group that classifies collection jobs having the attribute of collection. Another group includes, for example, processing groups having the attribute of processing. A processing group is, for example, a group that classifies processing jobs having the attribute of processing. Another group includes, for example, integration groups having the attribute of integration. An integration group is, for example, a group that classifies integration jobs having the attribute of integration. Finally, another group includes, for example, storage groups having the attribute of storage. A storage group is, for example, a group that classifies storage jobs having the attribute of storage. Specifically, the storage unit 400 stores the results of classifying each job in the job group into multiple groups using a group information management table 800. An example of the group information management table 800 will be described later with reference to Figure 8. Job classification is performed, for example, by a classification unit 402.

[0091] The storage unit 400 stores information indicating the first job in a group that will be used as the starting point for restarting the job flow in response to an abnormality in a job belonging to that group, for example, by associating it with at least one group. Specifically, the storage unit 400 uses the restart information management table 1000 to store information indicating the group that will be used as the starting point for restarting the job flow in response to an abnormality in a job belonging to that group, by associating it with one of the groups. An example of the restart information management table 1000 will be described later with reference to Figure 10. Specifically, the storage unit 400 uses the starting information management table 900 to store information indicating the first job in a group, by associating it with each group. An example of the starting information management table 900 will be described later with reference to Figure 9.

[0092] The storage unit 400 may store, for example, one or more combinations of a condition for restarting a job flow in response to an abnormality in a job belonging to at least one group, and information indicating the first job in the group from which the restart will be initiated, associated with at least one group. The condition may be, for example, that some kind of abnormality has occurred in a job belonging to any group. The condition may also be, for example, that a specific type of abnormality has occurred in a job belonging to any group. Specifically, the storage unit 400 uses the restart information management table 1000 to store, for example, one or more combinations of a condition for restarting a job flow and information indicating the first job in the group from which the restart will be initiated, associated with any group. An example of the restart information management table 1000 will be described in detail later with reference to Figure 10. Specifically, the storage unit 400 uses the first information management table 900 to store information indicating the first job in each group, associated with each group. An example of the first information management table 900 will be described in detail later with reference to Figure 9.

[0093] The storage unit 400 stores information representing the completion status of each job in the job group at the time the job flow terminates abnormally. Abnormal termination of a job flow means that the job flow has ended without completing. The completion status indicates whether the job is waiting to be executed, has completed normally, or has terminated abnormally. Abnormal termination of a job means that the job has ended without completing. Specifically, the storage unit 400 stores an execution history management table 1100 that manages the completion status of each job in the job group. An example of the execution history management table 1100 will be described later using Figure 11. Information representing the completion status of each job in the job group is generated, for example, by the execution unit 403.

[0094] For example, if a job flow terminates abnormally, the storage unit 400 may store information indicating the type of abnormality in any of the jobs that have terminated abnormally. Examples of job abnormalities include errors due to job timeouts, errors caused by the environment in which the job is executed, errors caused by the processing content of a job, or errors caused by the processing content of other preceding jobs. Specifically, the storage unit 400 stores information indicating the type of job abnormality in the execution history management table 1100. An example of the execution history management table 1100 will be described later using Figure 11. Information indicating the type of job abnormality is generated, for example, by the execution unit 403.

[0095] The acquisition unit 401 acquires various types of information used in the processing of each functional unit. The acquisition unit 401 stores the acquired information in the storage unit 400 or outputs it to each functional unit. The acquisition unit 401 may also output the information stored in the storage unit 400 to each functional unit. The acquisition unit 401 acquires various types of information, for example, based on user input. The acquisition unit 401 may also receive various types of information from a device other than the information processing device 100, for example.

[0096] The acquisition unit 401 acquires the job flow. Specifically, the acquisition unit 401 acquires the job flow by receiving the job flow input based on the operation input of the user of its own device. Specifically, the acquisition unit 401 may acquire the job flow by receiving it from another computer. The other computer is, for example, a client device.

[0097] The acquisition unit 401 acquires, for example, an additional request that requests to add a preceding job to a job group that has been classified into one of several groups. Specifically, the acquisition unit 401 acquires the additional request by accepting input of the additional request based on operation input by a user of its own device. Specifically, the acquisition unit 401 may acquire the additional request by receiving it from another computer. The other computer is, for example, a client device.

[0098] The acquisition unit 401 acquires, for example, a deletion request that requests the removal of a job classified into one of several groups from a group of jobs. Specifically, the acquisition unit 401 acquires deletion requests by receiving input of deletion requests based on operation input by a user of its own device. Specifically, the acquisition unit 401 may acquire deletion requests by receiving them from another computer. The other computer may be, for example, a client device.

[0099] The acquisition unit 401 acquires, for example, execution requests that request the execution of a group of jobs. Specifically, the acquisition unit 401 acquires execution requests by receiving input of execution requests based on operation input from a user of its own device. Specifically, the acquisition unit 401 may acquire execution requests by receiving them from another computer.

[0100] The acquisition unit 401 acquires information for detecting, for example, that a job flow has terminated abnormally. The acquisition unit 401 acquires, for example, a notification indicating that a job flow has terminated abnormally. The notification is issued, for example, by the information processing device 100 when a job flow terminates abnormally. The notification may also be issued, for example, by the execution unit 403.

[0101] The acquisition unit 401 acquires information to detect an anomaly in one of the jobs in the job group, for example. Specifically, the acquisition unit 401 acquires information representing the completion status of each job in the job group at the time the job flow terminates abnormally. The completion status indicates whether the job is waiting to be executed, whether the job has completed normally, or whether the job has terminated abnormally. More specifically, the acquisition unit 401 acquires the execution history management table 1100.

[0102] The acquisition unit 401 may receive a start trigger to initiate processing in any of the functional units. A start trigger may be, for example, a predetermined operation input by a user. A start trigger may also be, for example, the receipt of predetermined information from another computer. A start trigger may also be, for example, the output of predetermined information by any of the functional units.

[0103] The acquisition unit 401 may, for example, accept the acquisition of a job flow as a start trigger to start processing in the classification unit 402. The acquisition unit 401 may, for example, accept the acquisition of an add request or a delete request as a start trigger to start processing in the classification unit 402. The acquisition unit 401 may, for example, accept the acquisition of an execution request as a start trigger to start processing in the execution unit 403. The acquisition unit 401 may, for example, accept the acquisition of information for detecting that a job flow has terminated abnormally as a start trigger to start processing in the detection unit 404. The acquisition unit 401 may, for example, accept the acquisition of information for detecting an abnormality in any of the jobs in the job group as a start trigger to start processing in the detection unit 404.

[0104] The classification unit 402 classifies the job group into multiple groups based on the processing content of each job in the job group indicated by the job flow. For example, if the classification unit 402 determines that any of the jobs is a collection job based on the processing content of any of the jobs, it classifies that job into the collection group. Specifically, if the processing content of any of the jobs is to collect data from a data source, the classification unit 402 determines that any of the jobs is a collection job and classifies that job into the collection group. In this way, the classification unit 402 can classify jobs into groups and make it possible to handle jobs on a group basis.

[0105] The classification unit 402, for example, if it determines that any of the jobs is a processing job based on the processing content of any of the jobs, it classifies that job into a processing group. Specifically, if the processing content of any of the jobs involves processing data obtained from a single preceding job, the classification unit 402 determines that any of the jobs is a processing job and classifies that job into a processing group. In this way, the classification unit 402 can classify jobs into groups and make it possible to handle jobs on a group basis.

[0106] The classification unit 402, for example, if it determines that any of the jobs is an integrated job based on the processing content of any of the jobs, it classifies that job into an integrated group. Specifically, if the processing content of any of the jobs involves integrating data obtained from multiple preceding jobs, the classification unit 402 determines that any of the jobs is an integrated job and classifies that job into an integrated group. In this way, the classification unit 402 can classify jobs into groups and make it possible to handle jobs on a group basis.

[0107] The classification unit 402, for example, if it determines that any of the jobs is an accumulation job based on the processing content of any of the jobs, it classifies that job into an accumulation group. Specifically, if the processing content of any of the jobs involves storing data in the DB, the classification unit 402 determines that any of the jobs is an accumulation job and classifies that job into an accumulation group. In this way, the classification unit 402 can classify jobs into groups and make it possible to handle jobs on a group basis.

[0108] When the Classification Unit 402 receives an additional request from the Acquisition Unit 401, it updates the job flow to add the requested job to the job group. Based on the processing content of the job requested to be added to the job group, the Classification Unit 402 classifies the job into one of several groups. For example, as described above, the Classification Unit 402 determines whether the job requested to be added to the job group is a collection job, a processing job, an integration job, or an accumulation job, and classifies the job into one of the groups. This allows the Classification Unit 402 to add a new job to the job group.

[0109] When the classification unit 402 receives an additional request from the acquisition unit 401, if the job requested to be added to the job group becomes the new leading job in the group into which the job was classified, the classification unit 402 updates the information indicating the leading job in that group, which is stored in the storage unit 400. This allows the classification unit 402 to appropriately reflect in the storage unit 400 that a new job has been added to the job group.

[0110] When the classification unit 402 receives a deletion request from the acquisition unit 401, it updates the job flow to delete the job that has been requested to be deleted from the job group. This allows the classification unit 402 to delete existing jobs from the job group.

[0111] When the classification unit 402 receives a deletion request from the acquisition unit 401, if the job requested to be deleted from the job group is the first job in the group into which the job was classified, the classification unit 402 updates the information stored in the storage unit 400 that indicates the first job in the group. For example, the classification unit 402 updates the information stored in the storage unit 400 that indicates the first job in the group so that the subsequent job that is one position after the job requested to be deleted from the job group becomes the new first job in the group into which the job was classified. This allows the classification unit 402 to appropriately reflect in the storage unit 400 that an existing job has been deleted from the job group.

[0112] The execution unit 403 executes the job group sequentially according to the job flow. For example, if the execution unit 403 receives an execution request from the acquisition unit 401, it executes the job group sequentially according to the job flow. In this way, the execution unit 403 can execute the job group and complete the job flow.

[0113] For example, if the job flow is restarted by the startup unit 405, the execution unit 403 executes all or part of the job group sequentially, starting with the first job in each group that serves as the starting point for restarting the job flow, according to the job flow. Specifically, the execution unit 403 executes the first job in each starting group and one or more subsequent jobs that exist after each first job in each starting group in order.

[0114] In this case, the execution unit 403 does not necessarily have to re-execute other jobs that have already been executed, which are running in parallel with the leading job in each of the starting groups. Specifically, the execution unit 403 executes other jobs that have not yet been executed, which are running in parallel with the leading job in each of the starting groups, according to the job flow. As a result, the execution unit 403 can appropriately re-execute all or part of the job group and restart the job flow to complete it.

[0115] If any job terminates abnormally due to an error in any of the jobs, the execution unit 403 will terminate abnormally without completing the job flow. When the execution unit 403 terminates abnormally without completing the job flow, if there are any jobs currently running, it will terminate those jobs abnormally as well. When the execution unit 403 terminates abnormally without completing the job flow, it will reflect the termination status of each job in the execution history management table 1100. If any job terminates abnormally, the execution unit 403 may also reflect information indicating the type of error in that job in the execution history management table 1100.

[0116] As a result, if the execution unit 403 terminates abnormally without completing the job flow, it can retain the termination status of each job, which is useful when restarting the job flow, making it easier to determine how to restart the job flow. If an abnormality occurs in any of the jobs, the execution unit 403 can retain information indicating the type of abnormality in that job, making it easier to determine how to restart the job flow.

[0117] The detection unit 404 detects abnormalities in the first job classified into the first group. For example, the detection unit 404 detects abnormalities in the first job based on information for detecting abnormalities in the first job classified into the first group, acquired by the acquisition unit 401. Specifically, the detection unit 404 determines that an abnormality has occurred in the first job if the first job classified into the first group has terminated abnormally, based on information representing the completion status of each job acquired by the acquisition unit 401, and detects the abnormality in the first job. This allows the detection unit 404 to obtain guidance for restarting the job flow.

[0118] If the detection unit 404 detects an abnormality in the first job, the startup unit 405 restarts the job flow. The startup unit 405, for example, refers to the storage unit 400 to obtain information indicating the leading job in the second group, which will be used as the starting point for restarting the job flow associated with the first group to which the abnormal first job belongs. Based on the obtained information, the startup unit 405 restarts the job flow, starting from the leading job in each of the second groups.

[0119] Specifically, the startup unit 405 controls the execution unit 403 to execute the first job in each of the second groups, starting with the first job in each group, according to the job flow, along with the subsequent jobs for that first job and other unexecuted jobs running in parallel with that first job. More specifically, the startup unit 405 configures the execution unit 403 to start with the first job in each of the second groups.

[0120] As a result, the startup unit 405 can appropriately set the group that will serve as the starting point for restarting the job flow, complete the job group appropriately, and complete the job flow appropriately. In addition, the startup unit 405 can make it easier for the execution unit 403 to complete the job flow. For example, the startup unit 405 can avoid having the execution unit 403 execute all of the job group, thereby reducing the processing load and processing time.

[0121] When the detection unit 404 detects an abnormality in the first job, the startup unit 405 refers to the storage unit 400 and determines whether there is one or more conditions associated with the first group to which the abnormal first job belongs that cause the job flow to be restarted in response to the abnormality in the first job.

[0122] If the startup unit 405 determines that one or more conditions do not exist, it restarts the job flow starting from one of the pre-configured jobs. One of the pre-configured jobs is, for example, the first job, the preceding job of the first job, or the first job in the first group to which the first job belongs.

[0123] Specifically, the startup unit 405 identifies one of the pre-configured jobs as the starting point, along with any subsequent jobs that run in parallel with that job, in accordance with the job flow. Specifically, the startup unit 405 controls the execution unit 403 to execute the pre-configured job, any subsequent jobs that run in parallel with the identified job, and any other unexecuted jobs that run in parallel with that job, in accordance with the job flow.

[0124] As a result, the startup unit 405 can restart a job flow, complete the group of jobs, and complete the job flow even if a group that serves as the starting point for restarting the job flow has not been set. Furthermore, the startup unit 405 can make it easier for the execution unit 403 to complete the job flow. For example, the startup unit 405 can avoid having the execution unit 403 execute all of the job group, thereby reducing the processing load and processing time.

[0125] Here, if the startup unit 405 determines that one or more conditions exist, it refers to the storage unit 400 to determine whether the abnormality of the first job detected by the detection unit 404 satisfies any of the one or more conditions. For example, the startup unit 405 refers to the storage unit 400 to obtain one or more conditions for restarting the job flow, which are associated with the first group to which the first job belongs. For example, if the first condition among the one or more conditions obtained indicates that some kind of abnormality has occurred in the first job, the startup unit 405 determines that the abnormality of the first job detected by the detection unit 404 satisfies the first condition.

[0126] The startup unit 405, for example, if the first condition among the one or more acquired conditions indicates that a specific type of abnormality has occurred in the first job, refers to the storage unit 400 to identify the type of abnormality in the first job. The startup unit 405 determines that the first condition is met if the identified type matches the specific type indicated by the first condition. The startup unit 405 determines that the first condition is not met if the identified type does not match the specific type indicated by the first condition.

[0127] This allows the startup unit 405 to decide whether or not to restart the job flow. The startup unit 405 can flexibly change the second group from which the job flow restart will begin, based on whether or not one or more conditions for restarting the job flow, which are associated with the first group to which the first job belongs, are met.

[0128] Here, the startup unit 405 restarts the job flow if it determines that the abnormality of the first job detected by the detection unit 404 satisfies one of the conditions for restarting the job flow associated with the first group to which the first job belongs. The startup unit 405, for example, refers to the storage unit 400 to obtain information indicating the first job in the second group, which is associated with the first group and will serve as the starting point for restarting the job flow. Based on the obtained information, the startup unit 405 restarts the job flow, starting from each of the first jobs in the second group.

[0129] Specifically, the startup unit 405 controls the execution unit 403 to execute the first job, the subsequent jobs that follow the first job, and any other unexecuted jobs running in parallel with the first job, in accordance with the job flow. More specifically, the startup unit 405 configures the execution unit 403 to start with the first job.

[0130] As a result, the startup unit 405 can appropriately set the group that will serve as the starting point for restarting the job flow, complete the job group appropriately, and complete the job flow appropriately. In addition, the startup unit 405 can make it easier for the execution unit 403 to complete the job flow. For example, the startup unit 405 can avoid having the execution unit 403 execute all of the job group, thereby reducing the processing load and processing time.

[0131] If the startup unit 405 determines that the abnormality detected in the first job by the detection unit 404 does not satisfy any of the conditions associated with the first group to which the first job belongs, it restarts the job flow starting from one of the pre-configured jobs. One of the pre-configured jobs is, for example, the first job, the preceding job immediately before the first job, or the first job in the first group to which the first job belongs.

[0132] Specifically, the startup unit 405 identifies one of the pre-configured jobs as the starting point, along with any subsequent jobs that run in parallel with that job, in accordance with the job flow. Specifically, the startup unit 405 controls the execution unit 403 to execute the pre-configured job, any subsequent jobs that run in parallel with the identified job, and any other unexecuted jobs that run in parallel with that job, in accordance with the job flow.

[0133] As a result, the startup unit 405 can restart the job flow even if the abnormality of the first job detected by the detection unit 404 does not satisfy any of the conditions associated with the first group to which the first job belongs. The startup unit 405 can complete the job group and complete the job flow. Furthermore, the startup unit 405 can make it easier for the execution unit 403 to complete the job flow. For example, the startup unit 405 can avoid having the execution unit 403 execute all of the job group, thereby reducing the processing load and processing time.

[0134] The output unit 406 outputs the processing result of at least one of the functional units. The output format can be, for example, display on a screen, print to a printer, transmit to an external device via the network interface 303, or store in a storage area such as the memory 302 or recording medium 305. This allows the output unit 406 to notify the user of the processing result of at least one of the functional units, thereby improving the usability of the information processing device 100.

[0135] The output unit 406 outputs, for example, a notification indicating that the job flow has been completed, which can be viewed by the user of the device. This allows the user of the device to be aware that the job flow has been completed.

[0136] (Example of operation of the information processing device 100) Next, we will explain an example of the operation of the information processing device 100 using Figures 5 to 26. First, using Figures 5 and 6, we will explain an example of how the information processing device 100 stores the job flow 500 using the job flow management table 600.

[0137] Figure 5 is an explanatory diagram showing an example of job flow 500. As shown in Figure 5, job flow 500 defines the execution order of a group of jobs. In the example in Figure 5, the group of jobs specifically includes jobs 1 to 10. Job flow 500 shows the data dependencies of the job group.

[0138] Specifically, the job flow 500 is represented by a directed graph containing nodes 501 to 510, each representing a job from 1 to 10. Node 50i, representing job i, and node 50j, representing job j, which will be executed after job i and utilizes the data output by job i, are connected by a directed edge. i and j are, for example, integers greater than or equal to 1. Thus, the directed edge indicates a data dependency. Next, we will move on to the explanation of Figure 6.

[0139] Figure 6 is an explanatory diagram showing an example of how a job flow 500 is stored. In Figure 6, the information processing device 100 stores the job flow 500 using a job flow management table 600. The job flow management table 600 is implemented, for example, by a storage area such as the memory 302 or recording medium 305 of the information processing device 100 shown in Figure 3.

[0140] As shown in Figure 6, the job flow management table 600 has fields for job flow name, job ID, job name, preceding job ID, and succeeding job ID. By setting information in each field for each job, the job flow management table 600 stores job-related information as record 600-a, where a is an arbitrary integer.

[0141] The Job Flow Name field is set to the job flow name assigned to the job flow that defines the execution order of the job group. For example, the Job Flow Name field is set to "Job Flow A" as the job flow name assigned to job flow 500.

[0142] The Job ID field is set to the Job ID that identifies one of the jobs in the above job group. For example, the Job ID that identifies job i in the job group whose execution order is defined in Job Flow 500 is i. The Job Name field is set to the Job Name assigned to one of the jobs identified by the above Job ID. For example, the Job Name assigned to job i is job i.

[0143] The Preceding Job ID field is set to the Preceding Job ID, which identifies another preceding job that will be executed one step before any of the jobs identified by the above Job ID. The Successor Job ID field is set to the Successor Job ID, which identifies another subsequent job that will be executed one step after any of the jobs identified by the above Job ID.

[0144] Next, using Figure 7, an example of the contents of the attribute information management table 700, in which the information processing device 100 stores attribute information representing the attributes of each job, will be explained. The attribute information management table 700 is implemented, for example, by a storage area such as the memory 302 or recording medium 305 of the information processing device 100 shown in Figure 3. How the information processing device 100 generates, updates, and uses the attribute information management table 700 will be described later, for example, using Figures 12 to 26.

[0145] Figure 7 is an explanatory diagram showing an example of the contents stored in the attribute information management table 700. As shown in Figure 7, the attribute information management table 700 has fields for job flow name, job ID, job name, and type. By setting information in each field for each job, attribute information is stored in the attribute information management table 700 as record 700-b, where b is an arbitrary integer.

[0146] The Job Flow Name field is set to the job flow name assigned to the job flow that defines the execution order of the job group. For example, the Job Flow Name field is set to "Job Flow A" as the job flow name assigned to job flow 500.

[0147] The Job ID field is set to a Job ID that identifies one of the jobs in the above job group. For example, the Job ID that identifies job i in the job group whose execution order is defined in Job Flow 500 is i. The Job Name field is set to a Job Name assigned to one of the jobs identified by the above Job ID. For example, the Job Name assigned to job i is Job i(attribute k). Attribute k is an alias assigned by the Information Processing Device 100, for example, according to the attributes of one of the above jobs. Attribute k may be collection k, processing k, integration k, or storage k. k is, for example, an integer of 1 or more.

[0148] The Type field is set with an attribute that represents the type of job identified by the Job ID mentioned above. The attribute may be, for example, Collection, Processing, Integration, or Storage. The attribute may also be, for example, Not for Use. Collection is an attribute that indicates, for example, that data is collected from a data source. Processing is an attribute that indicates, for example, that data obtained from a single preceding job is processed. Integration is an attribute that indicates, for example, that data obtained from multiple preceding jobs is processed. Storage is an attribute that indicates, for example, that data is stored in a database. Not for Use is an attribute that indicates that the job does not fall under Collection, Processing, Integration, or Storage.

[0149] Next, using Figure 8, an example of the contents of the group information management table 800, in which the information processing device 100 stores group information representing the groups into which each job is classified, will be explained. The group information management table 800 is implemented using, for example, the memory 302 and recording medium 305 of the information processing device 100 shown in Figure 3. How the information processing device 100 generates, updates, and uses the group information management table 800 will be described later using, for example, Figures 12 to 26.

[0150] Figure 8 is an explanatory diagram showing an example of the contents stored in the group information management table 800. As shown in Figure 8, the group information management table 800 has fields for job flow name, job ID, job name, type, and group ID. By setting information in each field for each job, the group information is stored as record 1100-c in the group information management table 800. c is an arbitrary integer.

[0151] The Job Flow Name field is set to the job flow name assigned to the job flow that defines the execution order of the job group. For example, the Job Flow Name field is set to "Job Flow A" as the job flow name assigned to job flow 500.

[0152] The Job ID field is set to the Job ID that identifies one of the jobs in the above job group. For example, the Job ID that identifies job i in the job group whose execution order is defined in Job Flow 500 is i. The Job Name field is set to the Job Name assigned to one of the jobs identified by the above Job ID. For example, the Job Name assigned to job i is job i.

[0153] The "Type" field is set with an attribute of any of the jobs identified by the above job ID, representing the type of job. The "Group ID" field is set with a group ID that identifies the group into which any of the jobs identified by the above job ID are classified. The group ID may be, for example, Group 1, Group 2, Group 3, or Group 4.

[0154] Here, Group 1 is a group ID that identifies the group that classifies jobs having the attribute of collection. Group 2 is a group ID that identifies the group that classifies jobs having the attribute of processing. Group 3 is a group ID that identifies the group that classifies jobs having the attribute of integration. Group 4 is a group ID that identifies the group that classifies jobs having the attribute of storage.

[0155] Next, using Figure 9, an example of the contents of the lead information management table 900, in which the information processing device 100 stores lead information representing the lead job of each group, will be explained. The lead information management table 900 is implemented by storage areas such as the memory 302 and recording medium 305 of the information processing device 100 shown in Figure 3. How the information processing device 100 generates, updates, and uses the lead information management table 900 will be described later, for example, using Figures 12 to 26.

[0156] Figure 9 is an explanatory diagram showing an example of the contents stored in the lead information management table 900. As shown in Figure 9, the lead information management table 900 has fields for group ID, type, job ID, and lead job ID. The lead information management table 900 stores lead information as record 900-d by setting information in each field for each group. d is an arbitrary integer.

[0157] The Group ID field is set to a Group ID that identifies one of several groups. The Type field is set to an attribute that one of the above groups possesses, representing the type of one of the above groups identified by the Group ID. The attribute that a group possesses corresponds, for example, to the attribute that a job classified into the group possesses. Specifically, the attribute that a group possesses may be collection, processing, integration, or storage.

[0158] The Job ID field contains a list of Job IDs that identify each of the one or more jobs belonging to any of the groups identified by the Group ID. The First Job ID field contains a list of First Job IDs that identify each of the one or more first jobs in any of the groups identified by the Group ID.

[0159] Next, using Figure 10, an example of the contents of the restart information management table 1000, which is stored by the information processing device 100 and used when restarting the job flow 500, will be explained. The restart information management table 1000 is implemented by storage areas such as the memory 302 and recording medium 305 of the information processing device 100 shown in Figure 3. How the information processing device 100 uses the restart information management table 100 will be described later using, for example, Figures 12 to 26.

[0160] Figure 10 is an explanatory diagram showing an example of the contents stored in the restart information management table 1000. As shown in Figure 10, the restart information management table 1000 has fields for group ID, restart condition, and restart point. The restart information management table 1000 stores restart information as record 1000-e by setting information in each field for each group. e is an arbitrary integer. Also, for example, there may be multiple restart information entries for a single job.

[0161] The Group ID field is set to a Group ID that identifies one of several groups. A condition is set to restart Job Flow 500 in response to an abnormality in any job belonging to any of the groups identified by the above Group ID. The Restart Condition field is set to "n / a" if the condition for restarting Job Flow 500 is that some kind of abnormality has occurred, and the type of abnormality is not specified. The Restart Condition field is set to "n / a" if there is no condition for restarting Job Flow 500.

[0162] The restart condition field may, for example, contain an exit code representing the type of anomaly that occurs in any job belonging to any of the groups identified by the group ID, as a condition for restarting the job flow. The set exit code indicates that the job flow 500 will be restarted if the type of anomaly that occurs in any job belonging to any of the groups identified by the group ID matches the specific type represented by the exit code.

[0163] The restart point field contains information indicating the group from which job flow 500 will be restarted. The restart point field contains a group ID that identifies the group, indicating the group from which job flow 500 will be restarted. If no group exists to which job flow 500 will be restarted, the restart point field will be set to "n / a".

[0164] Here, if anything other than "n / a" is set in the restart point field, "n / a" set in the restart condition field indicates that any abnormality has occurred in the job, without specifying the type of abnormality, and serves as the condition for restarting job flow 500. Also, if "n / a" is set in the restart point field, "n / a" set in the restart condition field indicates that there is no condition for restarting job flow 500.

[0165] Next, using Figure 11, an example of the contents of the execution history management table 1100, which stores execution history information representing the completion status of each job in the job group when the information processing device 100 abnormally terminates the job flow 500, will be explained. The execution history management table 1100 is implemented by storage areas such as the memory 302 and recording medium 305 of the information processing device 100 shown in Figure 3. How the information processing device 100 generates, updates, and uses the execution history management table 1100 will be described later using, for example, Figures 12 to 26.

[0166] Figure 11 is an explanatory diagram showing an example of the contents stored in the execution history management table 1100. As shown in Figure 11, the execution history management table 1100 has fields for job flow name, job ID, job name, job status, and job exit code. The execution history management table 1100 stores execution history information as record 1100-f by setting information in each field for each job. f is an arbitrary integer.

[0167] The Job Flow Name field is set to the job flow name assigned to the job flow that defines the execution order of the job group. For example, the Job Flow Name field is set to "Job Flow A" as the job flow name assigned to job flow 500.

[0168] The Job ID field is set to the Job ID that identifies one of the jobs in the above job group. For example, the Job ID that identifies job i in the job group whose execution order is defined in Job Flow 500 is i. The Job Name field is set to the Job Name assigned to one of the jobs identified by the above Job ID. For example, the Job Name assigned to job i is job i.

[0169] The job status field is set to a status that indicates the completion status of one of the jobs identified by the above job ID. The status indicates whether one of the jobs identified by the above job ID terminated abnormally, one of the jobs identified by the above job ID terminated normally, or one of the jobs identified by the above job ID is waiting to run.

[0170] The job exit code field is set to the exit code of one of the jobs identified by the above job ID. For example, if the value of the exit code is 0, it indicates that no abnormalities occurred in one of the jobs identified by the above job ID and that one of the jobs identified by the above job ID completed successfully.

[0171] The exit code, for example, if its value is 1 or greater, indicates that an error occurred in one of the jobs identified by the above job ID, and that one of the jobs identified by the above job ID terminated abnormally. The exit code, for example, if its value is 1 or greater, represents the type of error corresponding to that value. The exit code, for example, is "n / a" if one of the jobs identified by the above job ID is waiting to be executed.

[0172] Next, using Figures 12 to 20, we will explain a specific example of how the information processing device 100 classifies a group of jobs into multiple groups.

[0173] Figures 12 to 20 are explanatory diagrams illustrating specific examples of classifying a group of jobs into multiple groups. In Figure 12, the information processing device 100 acquires a job flow 500 and stores the acquired job flow 500 using a job flow management table 600. Here, the job flow 500 does not represent the attributes of each job in the group of jobs. Therefore, in order to classify the group of jobs into multiple groups, the information processing device 100 will identify the attributes of each job in the group of jobs.

[0174] The attribute information management table 700 is empty in its initial state. The information processing device 100 stores records in the attribute information management table 700 where the type field is empty, corresponding to each job in the job group. The information processing device 100 retrieves the processing details for each job in the job group.

[0175] The information processing device 100 assigns the "collection" attribute to any job if the processing content of that job involves collecting data from a data source. For example, the information processing device 100 determines that a job involves collecting data from a data source if the processing content of that job involves acquiring data using an ETL tool. For example, the information processing device 100 determines that a job involves collecting data from a data source if the processing content of that job involves issuing a data collection request to the DB using SQL (Structured Query Language).

[0176] The information processing device 100 determines, for example, that if the processing content of any job generates an intermediate file to be used in the next job to be executed, then the processing content is to collect data from a data source. In this case, the information processing device 100 updates the attribute information management table 700 by setting the collection attribute to the record in the attribute information management table 700 that corresponds to the job in question.

[0177] In the example shown in Figure 12, the information processing device 100 specifically sets the collection attribute in the records corresponding to jobs 1, 7, and 8 in the attribute information management table 700. The information processing device 100 assigns the alias "Collection 1" to job 1. The information processing device 100 assigns the alias "Collection 2" to job 7. The information processing device 100 assigns the alias "Collection 3" to job 8.

[0178] If the processing content of any job involves processing data obtained from a single preceding job, the information processing device 100 assigns the processing attribute to that job. In this case, the information processing device 100 updates the attribute information management table 700 by setting the processing attribute to the record in the attribute information management table 700 that corresponds to that job.

[0179] In the example shown in Figure 12, the information processing device 100 specifically sets the processing attribute in the records in the attribute information management table 700 that correspond to jobs 2-4, 9, and 10. The information processing device 100 assigns the alias "Processing 1" to job 2. The information processing device 100 assigns the alias "Processing 2" to job 3. The information processing device 100 assigns the alias "Processing 3" to job 4. The information processing device 100 assigns the alias "Processing 4" to job 9. The information processing device 100 assigns the alias "Processing 5" to job 10.

[0180] If the processing content of any job involves integrating data obtained from multiple preceding jobs, the information processing device 100 assigns the integration attribute to that job. In this case, the information processing device 100 updates the attribute information management table 700 by setting the integration attribute to the record in the attribute information management table 700 that corresponds to that job.

[0181] In the example shown in Figure 12, the information processing device 100 specifically sets the integration attribute in the record corresponding to job 5 in the attribute information management table 700. The information processing device 100 assigns the alias "Integration 1" to job 5.

[0182] If the processing content of any job involves storing data in a database, the information processing device 100 assigns the "storage" attribute to that job. In this case, the information processing device 100 updates the attribute information management table 700 by setting the "storage" attribute to the record in the attribute information management table 700 that corresponds to that job.

[0183] In the example shown in Figure 12, the information processing device 100 specifically sets the storage attribute in the record corresponding to job 6 in the attribute information management table 700. The information processing device 100 assigns the alias storage 1 to job 6. This allows the information processing device 100 to obtain guidance for classifying the group of jobs into multiple groups. Next, we will move on to the explanation of Figure 13.

[0184] In Figure 13, the initial group information management table 800 is empty. The information processing device 100 stores in the group information management table 800 a record in which the group ID field is empty, corresponding to each job in the job group.

[0185] First, the information processing device 100 prepares one group for each job that has the integration attribute, and classifies the job that has the integration attribute into the group prepared for that job that has the integration attribute. The information processing device 100 updates the group information management table 800 by setting a group ID that identifies the group into which the job was classified in the record corresponding to the job that has the integration attribute in the group information management table 800.

[0186] In the example shown in Figure 13, the information processing device 100 specifically prepares a group 513 that has the attribute of integration, and classifies integration 1 into group 513 that has the attribute of integration. Specifically, the information processing device 100 sets the group ID = group 3, which identifies group 513, in the record corresponding to integration 1 in the group information management table 800.

[0187] This allows the information processing device 100 to classify jobs with the integration attribute into groups. Therefore, the information processing device 100 can avoid classifying two or more jobs that do not share common processing content and are not desirable to be treated together as a starting point for restarting the job flow 500 into a single group. For example, the information processing device 100 can avoid classifying jobs with the integration attribute and jobs with non-integration attributes into a single group.

[0188] Furthermore, the information processing device 100 considers that jobs with the integration attribute tend to have processing content independent of other jobs with the integration attribute. Therefore, the information processing device 100 can classify jobs with the integration attribute into different groups, and can handle jobs with the integration attribute in a unit that is preferable as a starting point for restarting the job flow 500. When restarting the job flow 500, the information processing device 100 can handle jobs with the integration attribute in a group unit. Next, we will move on to the explanation of Figure 14.

[0189] In Figure 14, the information processing device 100 identifies Integration 1 as the leading job in group 513, which has the attribute of integration, in accordance with its classification of Integration 1 into group 513, which has the attribute of integration. The information processing device 100 stores a record in the leading information management table 900 that associates group 513, which has the attribute of integration, one or more jobs belonging to group 513, and the identified leading job, Integration 1.

[0190] Specifically, the information processing device 100 generates a record 900-3 that associates a group ID that identifies group 513, a list of job IDs that identify each job belonging to group 513, and a job ID that identifies the first job. The information processing device 100 stores the generated record 900-3 in the first information management table 900.

[0191] This allows the information processing device 100 to identify which job belonging to group 513 should be used as the starting point for restarting the job flow 500, when group 513 is to be used as the starting point for restarting the job flow 500. Next, we will move on to the explanation of Figure 15.

[0192] In Figure 15, the information processing device 100 searches for preceding jobs that have the integration attribute for each job that has the integration attribute, finds one or more jobs that have the processing attribute, and classifies them into one group. For example, the information processing device 100 starts with one job that has the integration attribute and classifies one or more preceding jobs that have the processing attribute and exist consecutively before that job into one group of jobs with the processing attribute. The information processing device 100 updates the group information management table 800 by setting a group ID that identifies the group into which the job was classified in the record corresponding to the job that has the processing attribute in the group information management table 800.

[0193] In the example shown in Figure 15, the information processing device 100 specifically classifies the processes 1 to 3 that precede integration 1 consecutively, and the processes 4 and 5 that precede integration 1 consecutively, into a group 512 having processing attributes. Specifically, the information processing device 100 sets the group ID = group 2 to identify group 512 in the records corresponding to processes 1 to 5 in the group information management table 800.

[0194] This allows the information processing device 100 to classify jobs with processing attributes into groups. Here, since two or more jobs with common processing content are considered to have similar properties as starting points for restarting the job flow 500, it is considered unlikely that problems will occur even if they are treated together as starting points for restarting the job flow 500.

[0195] In contrast, the information processing device 100 can classify two or more jobs that have common processing content and are preferable to be treated together as a starting point for restarting the job flow 500 into a single group. For example, the information processing device 100 can classify two or more jobs having the same processing attributes into a single group.

[0196] Furthermore, the information processing device 100 can avoid classifying two or more jobs into a single group that do not share common processing content and are therefore undesirable to treat together as a starting point for restarting the job flow 500. For example, the information processing device 100 can avoid classifying jobs with processing attributes and jobs with non-processing attributes into a single group. When restarting the job flow 500, the information processing device 100 can handle jobs with processing attributes on a group basis. Next, we will move on to the explanation of Figure 16.

[0197] In Figure 16, the information processing device 100 classifies processing 1 to 5 into a group 512 having the attribute of processing, and identifies processing 1 and 4 as the first jobs in the group 512 having the attribute of processing. The information processing device 100 stores in the first information management table 900 a record that associates the group 512 having the attribute of processing, one or more jobs belonging to the group 512, and the identified first jobs, processing 1 and 4.

[0198] Specifically, the information processing device 100 generates a record 900-2 that associates a group ID identifying group 512 with a list of job IDs identifying each job belonging to group 512 with a job ID identifying the first job. The information processing device 100 stores the generated record 900-2 in the first information management table 900.

[0199] This allows the information processing device 100 to identify which job belonging to group 512 should be used as the starting point for restarting the job flow 500, when group 512 is to be used as the starting point for restarting the job flow 500. Next, we will move on to the explanation of Figure 17.

[0200] In Figure 17, the information processing device 100 searches for preceding jobs that exist before each leading job in each group having the processing attribute, finds one or more jobs that have the collection attribute, and classifies them into a single group. For example, the information processing device 100 starts from each leading job in a group having the processing attribute and classifies one or more preceding jobs that have the collection attribute and exist consecutively before that leading job into a single group having the collection attribute. The information processing device 100 updates the group information management table 800 by setting a group ID that identifies the group into which the job was classified in the record corresponding to the job having the collection attribute in the group information management table 800.

[0201] In the example shown in Figure 17, the information processing device 100 specifically refers to the leading information management table 900 to identify processing 1 and 4, which are the leading jobs in group 512. Specifically, the information processing device 100 classifies collection 1, which exists before the identified processing 1, and collections 2 and 3, which exist consecutively before the identified processing 4, into group 511, which has the attribute of collection. Specifically, the information processing device 100 sets the group ID = group1 to identify group 511 in the records in the group information management table 800 corresponding to collections 1 to 3.

[0202] This allows the information processing device 100 to classify jobs with the attribute of collection into groups. Here, since two or more jobs with common processing content are considered to have similar properties as starting points for restarting the job flow 500, it is considered unlikely that problems will occur even if they are treated together as starting points for restarting the job flow 500.

[0203] In contrast, the information processing device 100 can classify two or more jobs that have common processing content and are preferable to be treated together as a starting point for restarting the job flow 500 into a single group. For example, the information processing device 100 can classify two or more jobs having the same collection attributes into a single group.

[0204] Furthermore, the information processing device 100 can avoid classifying two or more jobs into a single group that do not share common processing content and are therefore undesirable to treat together as a starting point for restarting the job flow 500. For example, the information processing device 100 can avoid classifying jobs with collection attributes and jobs with non-collection attributes into a single group. When restarting the job flow 500, the information processing device 100 can handle jobs with collection attributes on a group basis. Next, we will move on to the explanation of Figure 18.

[0205] In Figure 18, the information processing device 100 classifies collections 1 to 3 into a group 511 having the attribute of collection, and identifies collections 1 and 2 as the leading jobs in group 511 having the attribute of collection. The information processing device 100 stores in the leading information management table 900 a record that associates group 511 having the attribute of collection, one or more jobs belonging to group 511, and the identified leading jobs, collections 1 and 2.

[0206] Specifically, the information processing device 100 generates a record 900-1 that associates a group ID that identifies group 511, a list of job IDs that identify each job belonging to group 511, and a job ID that identifies the first job. The information processing device 100 stores the generated record 900-1 in the first information management table 900.

[0207] This allows the information processing device 100 to identify which job belonging to group 511 should be used as the starting point for restarting the job flow 500, when group 511 is to be used as the starting point for restarting the job flow 500. Next, we will move on to the explanation of Figure 19.

[0208] In Figure 19, the information processing device 100 searches for subsequent jobs that have the integration attribute for each job that has the integration attribute, finds one or more jobs that have the storage attribute, and classifies them into one group. For example, the information processing device 100 starts with one job that has the integration attribute and classifies one or more subsequent jobs that have the storage attribute that are consecutively present after that job into one group of jobs with the storage attribute. The information processing device 100 updates the group information management table 800 by setting a group ID that identifies the group into which the job was classified in the record corresponding to the job that has the storage attribute in the group information management table 800.

[0209] In the example shown in Figure 19, the information processing device 100 specifically classifies the storage 1 that exists before integration 1 into group 514, which has the attributes of storage. Specifically, the information processing device 100 sets the group ID = group 4, which identifies group 514, in the record corresponding to storage 1 in the group information management table 800.

[0210] This allows the information processing device 100 to classify jobs with the attribute of storage into groups. Here, since two or more jobs with common processing content are considered to have similar properties as starting points for restarting the job flow 500, it is considered unlikely that problems will occur even if they are treated together as starting points for restarting the job flow 500.

[0211] In contrast, the information processing device 100 can classify two or more jobs that have common processing content and are preferable to be treated together as a starting point for restarting the job flow 500 into a single group. For example, the information processing device 100 can classify two or more jobs that have the same storage attribute into a single group.

[0212] Furthermore, the information processing device 100 can avoid classifying two or more jobs into a single group that do not share common processing content and are therefore undesirable to treat together as a starting point for restarting the job flow 500. For example, the information processing device 100 can avoid classifying jobs with the storage attribute and jobs with attributes other than storage into a single group. When restarting the job flow 500, the information processing device 100 can handle jobs with the storage attribute on a group basis. Next, we will move on to the explanation of Figure 20.

[0213] In Figure 20, the information processing device 100 identifies Storage 1 as the first job in Group 514, which has the attribute of storage, in accordance with its classification of Storage 1 into Group 514, which has the attribute of storage. The information processing device 100 stores a record in the first information management table 900 that associates Group 514, which has the attribute of storage, one or more jobs belonging to Group 514, and Storage 1, which is the identified first job.

[0214] Specifically, the information processing device 100 generates a record 900-4 that associates a group ID identifying group 514 with a list of job IDs identifying each job belonging to group 514 and a job ID identifying the first job. The information processing device 100 stores the generated record 900-4 in the first information management table 900. This allows the information processing device 100 to identify which job belonging to group 514 should be used as the starting point for restarting the job flow 500, if group 514 is to be used as the starting point for restarting the job flow 500. In this way, the information processing device 100 can carry out preparations to make the job flow 500 restartable.

[0215] Here, the operator of the job flow 500 may use the information processing device 100 to generate the restart information management table 1000. The operator of the job flow 500 may, for example, refer to the group information management table 800 and determine which group to use as the starting point for restarting the job flow 500, depending on any abnormalities in the jobs belonging to any of the groups. Then, the operator of the job flow 500 may use the information processing device 100 to generate the restart information management table 1000 based on the result of that determination. The information processing device 100 generates the restart information management table 1000, for example, based on the operation input of the operator of the job flow 500.

[0216] The information processing device 100 may, in advance, acquire a rule based on the operator's input of the job flow 500, which designates a group having the "processing" attribute as the starting point for restarting the job flow 500 in response to an abnormality in a job belonging to a group having the "integration" attribute. For example, the information processing device 100 may store information indicating the group having the "processing" attribute into which the jobs were actually classified, as the starting point for restarting the job flow 500, in association with the group having the "integration" attribute into which the jobs were actually classified, according to the rule.

[0217] Next, using Figures 21 and 22, we will describe a specific example of how the information processing device 100 restarts the job flow 500.

[0218] Figures 21 and 22 are explanatory diagrams illustrating a specific example of restarting the job flow 500. In Figure 21, it is assumed that the information processing device 100 has started executing each job in the job group in order according to the job flow 500. It is assumed that the information processing device 100 has abnormally terminated the job that was running because an abnormality occurred in one of the jobs in the job group, and has therefore abnormally terminated the job flow 500.

[0219] Assume that the information processing device 100 has stored in the execution history management table 1100 execution history information representing the completion status of each job in the job group and the completion code of each job in the job group at the time the job flow 500 terminated abnormally. In this case, the information processing device 100 will restart the job flow 500 in response to the abnormal termination of the job flow 500.

[0220] (21-1) The information processing device 100, for example, in response to the abnormal termination of the job flow 500, refers to the execution history management table 1100 to identify one or more jobs from the group of jobs that terminated abnormally. The information processing device 100 also refers to the execution history management table 1100 to identify the exit code of each of the identified one or more jobs that terminated abnormally.

[0221] In the example shown in Figure 21, the information processing device 100 specifically refers to the "Job Status" field in each record 1100-i of the execution history management table 1100 and extracts record 1100-5 where the "Job Status" is "Abnormal Termination". Specifically, the information processing device 100 refers to the "Job ID" field in the extracted record 1100-5 to identify the abnormally terminated job 5 = Integration 1. The information processing device 100 refers to the "Job Exit Code" field in the extracted record 1100-5 to identify the exit code "1" corresponding to the abnormally terminated job 5.

[0222] (21-2) Before restarting the job flow 500, the information processing device 100 may output information indicating one or more jobs that terminated abnormally and the exit code of each job that terminated abnormally, so that the operator of the job flow 500 can refer to it. The information processing device 100 may restart the job flow 500 after receiving a specific operation input from the operator of the job flow 500. This allows the information processing device 100 to leave room for the operator of the job flow 500 to deal with one or more jobs that terminated abnormally. Here, it is assumed that the information processing device 100 has received a specific operation input from the operator of the job flow 500.

[0223] (21-3) The information processing device 100 receives a specific operation input from the operator of the job flow 500 and refers to the group information management table 800 to identify the group to which the abnormally terminated job belongs. In the example in Figure 21, the information processing device 100 refers to record 800-5 in the group information management table 800 that corresponds to the abnormally terminated job 5 and identifies group 513, which has the group ID "Group 3" and to which job 5 belongs. Next, we will move on to the explanation of Figure 22.

[0224] In Figure 22, the information processing device 100 restarts the job flow 500 as shown in (22-1) to (22-5) below.

[0225] (22-1) The information processing device 100 refers to the "restart condition" field in the record 1000-i of the restart information management table 1000 that corresponds to the group to which each abnormally terminated job belongs, and identifies one or more conditions corresponding to that group.

[0226] In the example shown in Figure 22, the information processing device 100 specifically reads record 1000-5 from the restart information management table 1000, which corresponds to group 513 to which the abnormally terminated job 5 belongs. Specifically, the information processing device 100 refers to the "restart condition" field in the read record 1000-5 to identify the condition corresponding to group 513 to which the abnormally terminated job 5 belongs. More specifically, the information processing device 100 identifies the condition "n / a" to which group 513 to which the abnormally terminated job 5 belongs.

[0227] (22-2) The information processing device 100 determines whether the exit code identified for each abnormally terminated job satisfies at least one of the one or more conditions corresponding to the group to which the job belongs. The information processing device 100 refers to the restart information management table 1000 and identifies a group to which the job flow 500 will be restarted, in combination with the conditions corresponding to the group to which the abnormally terminated job belongs that the device determined to satisfy. The information processing device 100 registers the combination of the group to which each abnormally terminated job belongs and the group to which the job flow 500 will be restarted, as identified for that group, in a list.

[0228] In the example shown in Figure 22, the information processing device 100 specifically determines that the exit code "1" identified for the abnormally terminated job 5 satisfies the condition "n / a" corresponding to the group 513 to which the identified job 5 belongs. Specifically, the information processing device 100 identifies a group 512 that will serve as the starting point for restarting the job flow 500, combined with the condition "Code≦1" that it determined to satisfy for the group 513 to which the abnormally terminated job 5 belongs. Specifically, the information processing device 100 registers the combination of the group 513 to which the abnormally terminated job 5 belongs and the group 512 identified for the group 513 to which the abnormally terminated job 5 belongs as the starting point for restarting the job flow 500 in a list.

[0229] (22-3) It is possible that there is no one or more conditions corresponding to the group to which any job that terminated abnormally belongs. In this case, the information processing device 100 registers in a list a combination of a group to which there is no one or more conditions and a predetermined job to which the restart of the job flow 500 will be the starting point. The predetermined job is, for example, a job that terminated abnormally belonging to a group to which there is no one or more conditions, a preceding job that exists immediately before the job in question, or the first job in each of the groups to which there is no one or more conditions.

[0230] (22-4) It is possible that none of the conditions corresponding to one or more conditions for the group to which any abnormally terminated job belongs are met. In this case, the information processing device 100 registers in a list a combination of a group to which none of the conditions are met and a predetermined job to which the restart of the job flow 500 will be the starting point for restarting the job flow 500. The predetermined job may be, for example, an abnormally terminated job belonging to a group to which none of the conditions are met, a preceding job that exists immediately before the job in question, or the first job in each of the groups to which none of the conditions are met.

[0231] (22-5) The information processing device 100 restarts the job flow 500 starting from the first job in each group registered in the list, which serves as the starting point for restarting the job flow 500, in order to prevent duplicate execution of the same job. In the example in Figure 22, the information processing device 100 specifically restarts the job flow 500 starting from processing 1 and 4, which are the first jobs in each group 512 registered in the list, which serve as the starting point for restarting the job flow 500.

[0232] This allows the information processing device 100 to appropriately control the starting point for restarting the job flow 500 on a group basis. For example, if any job belonging to any group terminates abnormally, the information processing device 100 can switch the group that will be the starting point for restarting the job flow 500, depending on the type of abnormality.

[0233] The information processing device 100 eliminates the need for the operator of the job flow 500 to set a starting job for restarting the job flow 500 in response to the abnormal termination of a job. Therefore, the information processing device 100 can reduce the workload and working time for the operator of the job flow 500.

[0234] The information processing device 100 can restart the job flow 500 after waiting for a specific operation input from a user of its device. Therefore, before restarting the job flow 500, the information processing device 100 makes it easier for the operator of the job flow 500 to deal with any abnormalities that occur in any of the jobs.

[0235] This description explains the case in which the information processing device 100 restarts the job flow 500 after receiving specific operational input from the operator of the job flow 500, but it is not limited to this case. For example, the information processing device 100 may restart the job flow 500 without receiving specific operational input from the operator of the job flow 500. In this case, the information processing device 100 does not need to output information indicating one or more abnormally terminated jobs and the exit code of each abnormally terminated job in a way that is accessible to the operator of the job flow 500.

[0236] Next, using Figures 23 to 26, we will explain a specific example of how the information processing device 100 adds a new job to the job group.

[0237] Figures 23 to 26 are explanatory diagrams illustrating specific examples of adding new jobs to a job group. In Figure 23, it is assumed that the information processing device 100 has received an additional request to add new jobs to the job group. Specifically, in the example in Figure 23, it is assumed that the information processing device 100 has received an additional request to add jobs 11 to 13 to the job group in response to the addition of a data source for collecting data. Next, we will move on to the explanation of Figure 24.

[0238] In Figure 24, the information processing device 100 stores records in the attribute information management table 700 in which the type field is empty, corresponding to each additional job. Specifically, in the example in Figure 24, the information processing device 100 stores records 700-11 to 700-13 in the attribute information management table 700 in which the type field is empty, corresponding to jobs 11 to 13.

[0239] The information processing device 100, similar to Figure 12, assigns the attributes of collection, processing, integration, or storage to each of the one or more additional jobs based on the processing content of each additional job, and updates the attribute information management table 700.

[0240] In the example in Figure 24, the information processing device 100 specifically sets the "collection" attribute in record 700-11, which corresponds to job 11, in the attribute information management table 700. The information processing device 100 assigns the alias "collection 4" to job 11. The information processing device 100 also specifically sets the "processing" attribute in records, which correspond to jobs 12 and 13, in the attribute information management table 700. The information processing device 100 assigns the alias "processing 6" to job 12. The information processing device 100 assigns the alias "processing 7" to job 13. Next, we will move on to the explanation of Figure 25.

[0241] In Figure 25, the information processing device 100 identifies existing jobs with integration attributes that exist after the additional jobs. The information processing device 100 searches for preceding jobs that exist before the identified existing jobs, discovers one or more additional jobs with processing attributes, and classifies them into an existing group with processing attributes.

[0242] The information processing device 100 may, for example, classify one or more additional jobs having processing attributes that are consecutively preceding a identified existing job into an existing group having processing attributes. The information processing device 100 may also, for example, classify one or more additional jobs having processing attributes that are consecutively preceding a identified existing job via one or more existing jobs having processing attributes into an existing group having processing attributes.

[0243] The information processing device 100 updates the group information management table 800 by setting a group ID that identifies the group that classified the additional job in the record corresponding to the additional job having the processing attribute in the group information management table 800.

[0244] In the example in Figure 25, the information processing device 100 specifically identifies integration 1, which is located after the additional jobs of collection 4 and processing 6 and 7. Specifically, the information processing device 100 classifies processing 6 and 7, which are located consecutively before integration 1, into a group 512 that has the attributes of processing. Specifically, the information processing device 100 sets the group ID = group 2 to identify group 512 in records 800-12 and 13 of the group information management table 800, which correspond to processing 6 and 7.

[0245] This allows the information processing device 100 to classify additional jobs with processing attributes into groups. When restarting the job flow 500, the information processing device 100 can handle the additional jobs with processing attributes together with existing jobs with processing attributes in a group. Next, we will move on to the explanation of Figure 26.

[0246] In Figure 26, the information processing device 100 classifies processes 6 and 7 into group 512 which has the attribute of processing, and identifies process 6 as the new leading job in group 512 which has the attribute of processing. The information processing device 100 sets the identified process 6 in record 900-2 of the leading information management table 900 which corresponds to group 512 which has the attribute of processing.

[0247] This allows the information processing device 100 to identify which job belonging to group 512 should be used as the starting point for restarting the job flow 500, when group 512 is to be used as the starting point for restarting the job flow 500. Now, let's return to the explanation of Figure 25.

[0248] In Figure 25, the information processing device 100 identifies a group of jobs that have the attribute of processing and that exist after the identified existing job. For each identified group, the information processing device 100 searches for preceding jobs that exist before the first job in that group, finds one or more additional jobs that have the attribute of collection, and classifies them into the existing group that has the attribute of collection.

[0249] The information processing device 100 identifies, for example, the first job in a group having the attribute of processing. The information processing device 100 classifies, for example, one or more additional jobs having the attribute of collection that are consecutively preceding each identified first job into an existing group having the attribute of collection. The information processing device 100 may also classify, for example, one or more additional jobs having the attribute of collection that are consecutively preceding each identified first job via one or more existing jobs having the attribute of collection into an existing group having the attribute of collection.

[0250] The information processing device 100 updates the group information management table 800 by setting a group ID that identifies the group that classified the additional job in the record corresponding to the additional job having the attribute of collection.

[0251] In the example shown in Figure 25, the information processing device 100 specifically refers to the leading information management table 900 to identify processing 1, 4, and 6, which are the leading jobs in group 512. Specifically, the information processing device 100 classifies collection 4, which exists before the identified processing 6, into group 511, which has the attribute of collection. Specifically, the information processing device 100 sets the group ID = group1 to identify group 511 in record 800-11 of the group information management table 800, which corresponds to collection 1.

[0252] This allows the information processing device 100 to classify additional jobs with the collection attribute into groups. When restarting the job flow 500, the information processing device 100 can handle the additional jobs with the collection attribute together with the existing jobs with the collection attribute in a group-based manner. Next, we will move on to the explanation of Figure 26.

[0253] In Figure 26, the information processing device 100 identifies collection 4 as the new leading job in group 511, in accordance with its classification of collection 4 into group 511, which has the attribute of collection. The information processing device 100 sets the identified collection 4 in record 900-1 of the leading information management table 900, which corresponds to group 511, which has the attribute of collection.

[0254] This allows the information processing device 100 to identify which job belonging to group 511 should be used as the starting point for restarting the job flow 500, when group 511 is to be used as the starting point for restarting the job flow 500. In this way, the information processing device 100 can appropriately reflect information about additional jobs in various tables. Because the information processing device 100 can reflect information about additional jobs in various tables, it is possible to reduce the workload and working time for operators of the job flow 500.

[0255] As shown in the example above, when the information processing device 100 acquires a job flow 500, it prepares various tables to enable the restart of the job flow 500, thereby reducing the workload and time required for the operator of the job flow 500. These tables include, for example, a job flow management table 600, an attribute information management table 700, a group information management table 800, and a first-person information management table 900. Furthermore, when the information processing device 100 adds a new job to the job group, it updates the various tables to enable the job flow 500 to be restarted appropriately, thereby reducing the workload and time required for the operator of the job flow 500.

[0256] For example, in the past, the attributes of each job in a job group within a job flow 500 tended not to be specified. Also, for example, it was difficult to assign the "collection" attribute to a job that collects data from sources other than input files. Furthermore, for example, it was difficult to assign attributes to jobs that integrate multiple data sets, or jobs that store data in a database. For this reason, it was difficult to handle jobs in groups according to the attributes they possessed.

[0257] In response to this, the information processing device 100 can automatically identify the attributes of each job based on the processing content of each job. The information processing device 100 can then group together one or more jobs that have the same attributes and that it has determined can be treated together as a starting point for restarting the job flow 500. As a result, the information processing device 100 enables the operator of the job flow 500 to set a starting point for restarting the job flow 500 on a group basis, thereby reducing the workload and working time for the operator of the job flow 500.

[0258] (Examples of applications of the information processing device 100) Next, we will describe an example of the application of the information processing device 100. The information processing device 100 can be applied, for example, to the job flow of a large-scale system in the field of data utilization, such as collecting, processing, and integrating data from dozens or more business systems.

[0259] This allows the information processing device 100 to flexibly restart the job flow of the large-scale system. As a result, the information processing device 100 can improve the recovery speed in the event of a failure in the large-scale system, improve the stability of the large-scale system, or reduce the operating costs of the large-scale system.

[0260] Furthermore, the information processing device 100 enables job flow operators and others who manage large-scale systems to handle jobs in groups. Therefore, the information processing device 100 eliminates the need for job flow operators and others to set a starting job for restarting job flows for all jobs that make up a large-scale system. As a result, the information processing device 100 can reduce the workload and working time for job flow operators. In addition, the information processing device 100 can make it easier to prevent errors made by job flow operators and others when managing large-scale systems.

[0261] Furthermore, when a new job is added to the job group due to a large-scale system modification, the information processing device 100 can appropriately update various tables with respect to that job. These tables include, for example, the job flow management table 600, the attribute information management table 700, the group information management table 800, and the lead information management table 900. Similarly, when an existing job is deleted from the job group, the information processing device 100 can appropriately update various tables.

[0262] As a result, when the large-scale system is modified, the information processing apparatus 100 enables an operator of a job flow or the like to avoid updating various tables, and can reduce the work load and work time required for an operator of a job flow or the like. For this reason, the information processing apparatus 100 can easily prevent work mistakes or the like when an operator of a job flow or the like manages various tables.

[0263] (Determination processing procedure) Next, an example of a determination processing procedure executed by the information processing apparatus 100 will be described with reference to FIG. 27. The determination processing is realized, for example, by the CPU 301 shown in FIG. 3, a storage area such as the memory 302 and the recording medium 305, and the network I / F 303.

[0264] FIG. 27 is a flowchart showing an example of the determination processing procedure. In FIG. 27, the information processing apparatus 100 sequentially selects jobs included in a job group (step S2701).

[0265] Next, the information processing apparatus 100 determines whether the selected job is a job for collecting data from a data source based on the processing content of the selected job (step S2702). Here, when the job is not a job for collecting data (step S2702: No), the information processing apparatus 100 proceeds to the processing of step S2704. On the other hand, when the job is a job for collecting data (step S2702: Yes), the information processing apparatus 100 proceeds to the processing of step S2703.

[0266] In step S2703, the information processing apparatus 100 assigns a collection attribute to the selected job and updates the attribute information management table 700 (step S2703). Then, the information processing apparatus 100 returns to the processing of step S2701.

[0267] In step S2704, the information processing apparatus 100 determines whether all jobs have been selected (step S2704). Here, if there is a job that has not been selected (step S2704: No), the information processing apparatus 100 returns to the process of step S2701. On the other hand, if all jobs have been selected (step S2704: Yes), the information processing apparatus 100 proceeds to the process of step S2705.

[0268] In step S2705, the information processing apparatus 100 sequentially selects jobs included in the job group that do not have the attribute of collection (step S2705).

[0269] Next, the information processing apparatus 100 determines whether there is a job having the attribute of collection that precedes the selected job (step S2706). Here, if there is no job having the attribute of collection (step S2706: No), the information processing apparatus 100 proceeds to the process of step S2707. On the other hand, if there is a job having the attribute of collection (step S2706: Yes), the information processing apparatus 100 proceeds to the process of step S2708.

[0270] In step S2707, the information processing apparatus 100 assigns an out-of-use attribute to the selected job and updates the attribute information management table 700 (step S2707). Then, the information processing apparatus 100 proceeds to the process of step S2713.

[0271] In step S2708, the information processing apparatus 100 determines whether the selected job is a job that accesses the DB and stores data (step S2708). Here, if it is a job that stores data (step S2708: Yes), the information processing apparatus 100 proceeds to the process of step S2709. On the other hand, if it is not a job that stores data (step S2708: No), the information processing apparatus 100 proceeds to the process of step S2710.

[0272] In step S2709, the information processing device 100 assigns the storage attribute to the selected job and updates the attribute information management table 700 (step S2709). Then, the information processing device 100 proceeds to the processing in step S2713.

[0273] In step S2710, the information processing device 100 determines whether there is a single job with the attribute of collection that precedes the selected job (step S2710). If there is a single job (step S2710: Yes), the information processing device 100 proceeds to the process in step S2711. On the other hand, if there are multiple jobs instead of a single one (step S2710: No), the information processing device 100 proceeds to the process in step S2712.

[0274] In step S2711, the information processing device 100 assigns processing attributes to the selected job and updates the attribute information management table 700 (step S2711). Then, the information processing device 100 proceeds to the process in step S2713.

[0275] In step S2712, the information processing device 100 assigns the integration attribute to the selected job and updates the attribute information management table 700 (step S2712). Then, the information processing device 100 proceeds to the processing in step S2713.

[0276] In step S2713, the information processing device 100 determines whether it has selected all jobs included in the job group that do not have the attribute of collection (step S2713). If there are jobs that have not been selected (step S2713: No), the information processing device 100 returns to the process in step S2705. On the other hand, if all jobs have been selected (step S2713: Yes), the information processing device 100 terminates the determination process.

[0277] (Overall classification process procedure) Next, an example of the overall classification processing procedure performed by the information processing device 100 will be described using Figure 28. The overall classification processing is realized, for example, by the CPU 301 shown in Figure 3, storage areas such as memory 302 and recording medium 305, and network I / F 303.

[0278] Figure 28 is a flowchart showing an example of the overall classification processing procedure. In Figure 28, the information processing device 100 performs the determination process shown in Figure 27 (step S2801). Then, the information processing device 100 extracts the integrated jobs that have the integrated attribute from the job group (step S2802).

[0279] Next, the information processing device 100 determines whether or not an integrated job exists (step S2803). If no integrated job exists (step S2803: No), the information processing device 100 proceeds to the process in step S2806. On the other hand, if an integrated job exists (step S2803: Yes), the information processing device 100 proceeds to the process in step S2804.

[0280] In step S2804, the information processing device 100 classifies each extracted integrated job into a single group having the attribute of integration, and updates the group information management table 800 (step S2804).

[0281] Next, the information processing device 100 performs the first classification process, which will be described later in Figure 29, classifying the preceding jobs that exist before the integrated job into one or more groups and updating the group information management table 800 (step S2805). Then, the information processing device 100 proceeds to the process in step S2806.

[0282] In step S2806, the information processing device 100 extracts the last job from the job group that has no subsequent jobs (step S2806).

[0283] Next, the information processing device 100 selects the extracted end jobs in order (step S2807). Then, the information processing device 100 classifies the selected end jobs into groups having one of the attributes of collection, processing, or storage, and updates the group information management table 800 (step S2808).

[0284] Next, the information processing device 100 determines whether or not there is a preceding job for the classified last job (step S2809). If there is no preceding job (step S2809: No), the information processing device 100 proceeds to the process in step S2812. On the other hand, if there is a preceding job (step S2809: Yes), the information processing device 100 proceeds to the process in step S2810.

[0285] In step S2810, the information processing device 100 determines whether the preceding job to the classified trailing job is an integrated job (step S2810). If it is an integrated job (step S2810: Yes), the information processing device 100 proceeds to the process in step S2812. On the other hand, if it is not an integrated job (step S2810: No), the information processing device 100 proceeds to the process in step S2811.

[0286] In step S2811, the information processing device 100 performs the second classification process, which will be described later in Figure 30, starting from the classified last job (step S2811). By performing the second classification process, which will be described later in Figure 30, the information processing device 100 classifies the preceding jobs that exist before the classified last job into one or more groups and updates the group information management table 800. Then, the information processing device 100 returns to the process of step S2810.

[0287] In step S2812, the information processing apparatus 100 determines whether all the classified end jobs have been selected (step S2812). Here, if there is an end job that has not been selected (step S2812: No), the information processing apparatus 100 returns to the process of step S2807. On the other hand, if all the end jobs have been selected (step S2812: Yes), the information processing apparatus 100 ends the overall classification process.

[0288] (First Classification Processing Procedure) Next, an example of the first classification processing procedure executed by the information processing apparatus 100 will be described with reference to FIG. 29. The first classification processing is realized by, for example, the CPU 301 shown in FIG. 3, storage areas such as the memory 302 and the recording medium 305, and the network I / F 303.

[0289] FIG. 29 is a flowchart showing an example of the first classification processing procedure. In FIG. 29, the information processing apparatus 100 sequentially selects the extracted integrated jobs (step S2901).

[0290] Next, the information processing apparatus 100 executes the second classification processing described later in FIG. 30 starting from the selected integrated job, classifies the preceding jobs existing before the selected integrated job into one or more groups, and updates the group information management table 800 (step S2902).

[0291] Then, the information processing apparatus 100 sequentially selects one or more classified groups (step S2903).

[0292] Next, the information processing apparatus 100 determines whether there is a preceding job for the head job in the selected group (step S2904). Here, if there is no preceding job (step S2904: No), the information processing apparatus 100 proceeds to the process of step S2907. On the other hand, if there is a preceding job (step S2904: Yes), the information processing apparatus 100 proceeds to the process of step S2905.

[0293] In step S2905, the information processing device 100 determines whether the preceding job to the first job in the selected group is an integrated job (step S2905). If it is an integrated job (step S2905: Yes), the information processing device 100 proceeds to step S2907. On the other hand, if it is not an integrated job (step S2905: No), the information processing device 100 proceeds to step S2906.

[0294] In step S2906, the information processing device 100 performs the second classification process described later in Figure 30, starting with the preceding job for the first job in the selected group (step S2906). By performing the second classification process described later in Figure 30, the information processing device 100 classifies the preceding jobs that exist before the preceding job into one or more groups and updates the group information management table 800. Then, the information processing device 100 proceeds to the process in step S2907.

[0295] In step S2907, the information processing device 100 determines whether it has selected all of the one or more classified groups (step S2907). If there are any groups that have not been selected (step S2907: No), the information processing device 100 returns to the process in step S2903. On the other hand, if it has been selected (step S2907: Yes), the information processing device 100 proceeds to the process in step S2908.

[0296] In step S2908, the information processing device 100 determines whether all extracted integrated jobs have been selected (step S2908). If there are integrated jobs that have not been selected (step S2908: No), the information processing device 100 returns to the process in step S2901. On the other hand, if they have been selected (step S2908: Yes), the information processing device 100 terminates the first classification process.

[0297] (Second Classification Processing Procedure) Next, an example of a second classification processing procedure executed by the information processing device 100 will be described using Figure 30. The second classification processing is realized, for example, by the CPU 301 shown in Figure 3, storage areas such as memory 302 and recording media 305, and network I / F 303.

[0298] Figure 30 is a flowchart showing an example of the second classification processing procedure. In Figure 30, the information processing device 100 sequentially selects one or more preceding jobs that precede the starting job (step S3001). Then, the information processing device 100 classifies the selected preceding jobs into groups having one of the attributes of collection, processing, or storage, and updates the group information management table 800 (step S3002).

[0299] Next, the information processing device 100 determines whether the attributes of the preceding job classified immediately before match those of the preceding job that preceded the classified preceding job (step S3003). If they match (step S3003: Yes), the information processing device 100 proceeds to step S3004. On the other hand, if they do not match (step S3003: No), the information processing device 100 proceeds to step S3005.

[0300] In step S3004, the information processing device 100 classifies the preceding job that is immediately preceding the preceding job that was classified just before into the same group and updates the group information management table 800 (step S3004). Then, the information processing device 100 returns to the process in step S3003.

[0301] In step S3005, the information processing device 100 sets the first job in the group into which the selected preceding jobs have been classified and updates the first information management table 900 (step S3005). For example, the information processing device 100 sets the jobs in the group into which the selected preceding jobs have been classified, jobs whose attributes do not match the preceding job that is immediately preceding them, and jobs that do not have a preceding job, as the first jobs in that group.

[0302] Next, the information processing device 100 determines whether it has selected all of the one or more preceding jobs that preceded the starting job (step S3006). If there are any preceding jobs that have not been selected (step S3006: No), the information processing device 100 returns to the process in step S3001. On the other hand, if it has selected jobs (step S3006: Yes), the information processing device 100 terminates the second classification process.

[0303] (Restart procedure) Next, an example of a restart procedure performed by the information processing device 100 will be described using Figure 31. The restart process is implemented, for example, by the CPU 301 shown in Figure 3, storage areas such as memory 302 and recording medium 305, and network I / F 303.

[0304] Figure 31 is a flowchart showing an example of a restart procedure. In Figure 31, the information processing device 100 refers to the execution history management table 1100 to identify one or more jobs that have terminated abnormally (step S3101).

[0305] Next, the information processing device 100 refers to the group information management table 800 to identify the group to which each of the one or more abnormally terminated jobs belongs (step S3102). Then, the information processing device 100 refers to the restart information management table 1000 to identify the restart point corresponding to each of the identified groups (step S3103).

[0306] Next, the information processing device 100 determines whether or not a restart point exists (step S3104). If no restart point exists (step S3104: No), the information processing device 100 proceeds to step S3107. On the other hand, if a restart point exists (step S3104: Yes), the information processing device 100 proceeds to step S3105.

[0307] In step S3105, the information processing device 100 refers to the group information management table 800 to identify one or more leading jobs in each of the identified groups (step S3105).

[0308] Next, the information processing device 100 restarts the first job of each of the one or more identified first jobs, restarting the entire job flow (step S3106). Then, the information processing device 100 terminates the restart process.

[0309] In step S3107, the information processing device 100 restarts a predetermined job corresponding to one or more jobs that terminated abnormally, and restarts the entire job flow (step S3107). Then, the information processing device 100 finishes the restart process.

[0310] (Additional processing steps) Next, an example of an additional processing procedure executed by the information processing device 100 will be described using Figure 32. The additional processing is realized, for example, by the CPU 301 shown in Figure 3, storage areas such as memory 302 and recording medium 305, and network I / F 303.

[0311] Figure 32 is a flowchart showing an example of an additional processing procedure. In Figure 32, the information processing device 100 performs the determination process shown in Figure 17 on the additional job group (step S3201). By performing the determination process shown in Figure 17, the information processing device 100 assigns one of the following attributes to each additional job in the additional job group: collection, processing, integration, storage, or non-utilization.

[0312] Next, the information processing device 100 extracts additional integration jobs that have the integration attribute from the additional job group (step S3202).

[0313] The information processing device 100 then determines whether or not there are additional integration jobs among the additional job group (step S3203). If there are no additional integration jobs (step S3203: No), the information processing device 100 proceeds to step S3206. On the other hand, if there are additional integration jobs (step S3203: Yes), the information processing device 100 proceeds to step S3204.

[0314] In step S3204, the information processing device 100 classifies each extracted additional integration job into a single group having the attribute of integration, and updates the group information management table 800 (step S3204).

[0315] Next, the information processing device 100 executes the first classification process shown in Figure 19, classifying the additional preceding jobs that exist before the extracted additional integrated jobs into one or more groups from the additional job group, and updates the group information management table 800 (step S3205). Then, the information processing device 100 proceeds to the process in step S3206.

[0316] In step S3206, the information processing device 100 identifies an existing integrated job that is located after at least one additional trailing job in the additional job group (step S3206).

[0317] Next, the information processing device 100 sequentially selects the identified existing integration jobs (step S3207). Then, the information processing device 100 sequentially selects the additional end jobs that exist before the selected existing integration jobs from among the one or more additional end jobs in the additional job group (step S3208).

[0318] Next, the information processing device 100 determines whether the attributes of the selected additional trailing job match those of the existing job that is one position after the selected additional trailing job (step S3209). If they do not match (step S3209: No), the information processing device 100 proceeds to step S3215. On the other hand, if they do match (step S3209: Yes), the information processing device 100 proceeds to step S3210.

[0319] In step S3210, the information processing device 100 classifies the selected additional trailing job into an existing group to which the existing job belongs (step S3210).

[0320] Next, the information processing device 100 further classifies any additional jobs that have the same attributes as the additional job that was just classified into an existing group into an existing group if such jobs exist immediately before the additional job that was just classified into an existing group (step S3211).

[0321] Then, the information processing device 100 sets the first job in the existing group and updates the first information management table 900 (step S3212). For example, the information processing device 100 sets a job in an existing group whose attributes do not match those of the preceding job that exists immediately before it, and a job for which there is no preceding job, as the first job in that group.

[0322] Next, the information processing device 100 determines whether there is an additional preceding job among the additional job group that has different attributes from the preceding job, after the additional integration job and before the designated leading job (step S3213). If no additional preceding jobs exist (step S3213: No), the information processing device 100 terminates the additional processing. On the other hand, if additional preceding jobs exist (step S3213: Yes), the information processing device 100 proceeds to the processing in step S3214.

[0323] The information processing device 100 then determines whether there is an additional preceding job with the same attributes as the designated leading job, located after the additional integrated job and immediately before the designated leading job (step S3214). If no additional preceding jobs exist (step S3214: No), the information processing device 100 proceeds to step S3215. On the other hand, if additional preceding jobs exist (step S3214: Yes), the information processing device 100 returns to step S3211.

[0324] In step S3215, the information processing device 100 executes the second classification process shown in Figure 20, starting from the preceding job that exists immediately before each of the first jobs set immediately before (step S3215). By executing the second classification process shown in Figure 20, the information processing device 100 classifies the preceding jobs that exist before the first job into one or more groups and updates the group information management table 800.

[0325] In step S3216, the information processing device 100 determines whether there are any additional preceding jobs before the first job in any of the groups classified immediately before (step S3216). If there are no additional preceding jobs (step S3216: No), the information processing device 100 terminates the additional processing. On the other hand, if there are additional preceding jobs (step S3216: Yes), the information processing device 100 proceeds to the processing in step S3217.

[0326] In step S3217, the information processing device 100 determines whether the attribute of the additional preceding job before the first job in any of the groups classified immediately before is that of integration (step S3217). If it is not integration (step S3217: No), the information processing device 100 returns to the process in step S3215. On the other hand, if it is integration (step S3217: Yes), the information processing device 100 terminates the additional processing.

[0327] Here, the information processing device 100 may execute some steps in the flowcharts of Figures 27 to 32 in a different order. Also, the information processing device 100 may omit some steps in the flowcharts of Figures 27 to 32.

[0328] As explained above, the information processing device 100 can classify a group of jobs into multiple groups based on the processing content of each job in the job flow whose execution order is defined in the job flow. The information processing device 100 can store information in the storage unit 400 that indicates the first job in the group that will be used as the starting point for restarting the job flow, associating it with at least one of the multiple groups. If the information processing device 100 detects an abnormality in the first job classified into the first group, it can refer to the storage unit 400 to identify the first job in the second group that will be used as the starting point for restarting the job flow associated with the first group. The information processing device 100 can restart the job flow starting from the identified first job. This allows the information processing device 100 to handle the group of jobs in groups when restarting the job flow. Therefore, the information processing device 100 can make it easier to set the starting point for restarting the job flow, and can reduce the workload and time required for the job flow operator when setting the starting point for restarting the job flow.

[0329] According to the information processing device 100, upon receiving a request to add a third job that precedes a second job classified into the second group of multiple groups to a job group, the third job can be added to the job group. According to the information processing device 100, the third job can be classified into the third group of multiple groups based on the processing content of the third job. According to the information processing device 100, if the third job becomes the first job in the third group, the information indicating the first job in the third group stored in the storage unit 400 can be updated. As a result, the information processing device 100 can add a new job to the job group and appropriately reflect the addition of a new job to the job group in the storage unit 400. The information processing device 100 can reduce the workload and working time for job flow operators and others when adding a new job to a job group.

[0330] According to the information processing device 100, upon receiving a request to remove a fourth job, which has been classified into the fourth group of multiple groups, from the job group, the fourth job can be removed from the job group. According to the information processing device 100, if the fourth job is the first job in the fourth group, the information indicating the first job in the fourth group stored in the storage unit 400 can be updated. As a result, the information processing device 100 can remove an existing job from the job group and appropriately reflect the removal of the existing job from the job group in the storage unit 400. The information processing device 100 can reduce the workload and time required for job flow operators and others when removing an existing job from a job group.

[0331] According to the information processing device 100, the storage unit 400 can store in association with at least one of a plurality of groups the conditions for restarting a job flow and information indicating the first job in the group that will be used as the starting point for restarting the job flow. According to the information processing device 100, it can determine whether the detected abnormality in the first job satisfies the conditions for restarting the job flow associated with the first group. According to the information processing device 100, if it determines that the conditions are met, it can refer to the storage unit 400 to identify the first job in the second group that will be used as the starting point for restarting the job flow associated with the first group. According to the information processing device 100, it can restart the job flow starting from the identified first job. This makes it easier for the information processing device 100 to switch the group that will be used as the starting point for restarting the job flow depending on the conditions.

[0332] According to the information processing device 100, if it determines that the fifth job included in the job group is a collection job based on its processing content, it can classify the fifth job into a collection group. This allows the information processing device 100 to appropriately classify collection jobs into groups.

[0333] According to the information processing device 100, if it determines that the sixth job included in the job group is a processing job based on its processing content, it can classify the sixth job into a processing group. This allows the information processing device 100 to appropriately classify processing jobs into groups.

[0334] According to the information processing device 100, if it determines that the seventh job included in the job group is an integrated job based on its processing content, it can classify the seventh job into an integrated group. This allows the information processing device 100 to appropriately classify integrated jobs into groups.

[0335] According to the information processing device 100, if it determines that the eighth job included in the job group is a storage job based on its processing content, it can classify the eighth job into a storage group. This allows the information processing device 100 to appropriately classify storage jobs into groups.

[0336] The information processing method described in this embodiment can be implemented by executing a pre-prepared program on a computer such as a PC or workstation. The information processing program described in this embodiment is recorded on a computer-readable recording medium and executed by being read from the recording medium by the computer. The recording medium can be a hard disk, flexible disk, CD (Compact Disc)-ROM, MO (Magneto Optical Disc), DVD (Digital Versatile Disc), etc. Furthermore, the information processing program described in this embodiment may be distributed via a network such as the Internet.

[0337] With regard to the embodiments described above, the following additional information is disclosed.

[0338] (Note 1) Based on the processing content of each job in a group of jobs whose execution order is defined in the job flow, the group of jobs is classified into multiple groups. If an anomaly is detected in the first job classified into the first group of the plurality of groups, the system refers to a storage unit that stores information indicating the job that is first in the execution order in the group that will be used as the starting point for restarting the job flow, and based on the information indicating the job that is first in the execution order in the second group that will be used as the starting point for restarting the job flow, the system restarts the job flow starting with the job that is first in the second group, which is associated with the first group. An information processing program characterized by having a computer perform the processing.

[0339] (Note 2) In response to receiving a request to add a third job preceding a second job classified into the second group of the aforementioned multiple groups to the job group, the third job is added to the job group. Based on the processing content of the third job, the third job is classified into the third group of the plurality of groups, If the third job is the first job in the execution order of the third group, the information stored in the memory unit indicating the first job in the execution order of the third group is updated. The information processing program described in Appendix 1, characterized in that it causes the computer to perform the processing.

[0340] (Note 3) In response to receiving a request to remove the fourth job, which has been classified into the fourth group of the multiple groups, from the job group, the fourth job is removed from the job group. If the fourth job is the first job in the execution order of the fourth group, the information stored in the memory unit indicating the first job in the execution order of the fourth group is updated. An information processing program according to Appendix 1 or 2, characterized in that it causes the computer to perform the processing.

[0341] (Note 4) The storage unit stores, in association with at least one of the plurality of groups, the conditions for restarting the job flow and information indicating the job that is first in the execution order in the group of the plurality of groups that is the starting point for restarting the job flow. The system determines whether the detected anomaly in the first job satisfies the conditions for restarting the job flow associated with the first group. The computer is made to perform the process, The aforementioned restart process is: The information processing program according to Appendix 1 or 2, characterized in that, if it is determined that the detected abnormality in the first job satisfies the conditions for restarting the job flow associated with the first group, it refers to the storage unit and restarts the job flow starting from the job that is at the beginning of the execution order in the second group, based on information indicating the job that is at the beginning of the execution order in the second group which serves as the starting point for restarting the job flow associated with the first group.

[0342] (Note 5) The aforementioned groups include collection groups that classify collection jobs that collect data from data sources, The aforementioned classification process is, The information processing program according to Appendix 1, characterized in that, if it is determined that the fifth job is a collection job based on the processing content of the fifth job included in the job group, the fifth job is classified into the collection group.

[0343] (Note 6) The aforementioned multiple groups include processing groups that classify processing jobs that obtain and process data indicating processing results from a single job, The aforementioned classification process is, The information processing program according to Appendix 5, characterized in that, if it is determined that the sixth job is a processing job based on the processing content of the sixth job included in the job group, the sixth job is classified into the processing group.

[0344] (Note 7) The aforementioned multiple groups include an integration group that classifies an integration job that acquires and integrates data showing processing results from multiple jobs, The aforementioned classification process is, The information processing program according to Appendix 6, characterized in that, if it is determined that the seventh job is an integrated job based on the processing content of the seventh job included in the job group, the seventh job is classified into the integrated group.

[0345] (Note 8) The aforementioned multiple groups include storage groups that classify storage jobs that acquire data and store it in a database. The aforementioned classification process is, The information processing program according to Appendix 7, characterized in that, if it is determined that the eighth job is a storage job based on the processing content of the eighth job included in the job group, the eighth job is classified into the storage group.

[0346] (Note 9) Based on the processing content of each job in the job group for which the execution order is defined in the job flow, the job group is classified into multiple groups. If an anomaly is detected in the first job classified into the first group of the plurality of groups, the system refers to a storage unit that stores information indicating the job that is first in the execution order in the group that will be used as the starting point for restarting the job flow, and based on the information indicating the job that is first in the execution order in the second group that will be used as the starting point for restarting the job flow, the system restarts the job flow starting with the job that is first in the second group, which is associated with the first group. An information processing method characterized in that the processing is performed by a computer.

[0347] (Note 10) Based on the processing content of each job in the job group for which the execution order is defined in the job flow, the job group is classified into multiple groups. If an anomaly is detected in the first job classified into the first group of the plurality of groups, the system refers to a storage unit that stores information indicating the job that is first in the execution order in the group that will be used as the starting point for restarting the job flow, and based on the information indicating the job that is first in the execution order in the second group that will be used as the starting point for restarting the job flow, the system restarts the job flow starting with the job that is first in the second group, which is associated with the first group. An information processing device characterized by having a control unit. [Explanation of symbols]

[0348] Nodes 101-110, 501-510 100 Information Processing Devices Groups 111-114, 511-514 121,500 job flows 122,400 storage unit 200 Information Processing Systems 201 Job Execution Unit 202 Client Devices 210 Network 300 bus 301 CPU 302 memory 303 Network I / F 304 Recording medium interface 305 Recording media 401 Acquisition Department 402 Classification Department 403 Execution Unit 404 Detection Unit 405 Starting part 406 Output section 600 Job Flow Management Table 700 Attribute Information Management Table 800 Group Information Management Table 900 First Information Management Table 1000 Restart Information Management Table 1100 Execution History Management Table

Claims

1. Based on the processing content of each job in a group of jobs whose execution order is defined in the job flow, the group of jobs is classified into multiple groups. If an anomaly is detected in the first job classified into the first group of the plurality of groups, the system refers to a storage unit that stores information indicating the job that is first in the execution order in the group that will be used as the starting point for restarting the job flow, and based on the information indicating the job that is first in the execution order in the second group that will be used as the starting point for restarting the job flow, the system restarts the job flow starting with the job that is first in the second group. An information processing program characterized by having a computer perform the processing.

2. In response to receiving a request to add a third job preceding a second job classified into the second group of the aforementioned multiple groups to the job group, the third job is added to the job group. Based on the processing content of the third job, the third job is classified into the third group of the plurality of groups, If the third job is the first job in the execution order of the third group, the information stored in the storage unit indicating the first job in the execution order of the third group is updated. The information processing program according to claim 1, characterized in that it causes the computer to perform the processing.

3. The storage unit stores, in association with at least one of the plurality of groups, the conditions for restarting the job flow, and information indicating the job that is at the top of the execution order in the group that is the starting point for restarting the job flow among the plurality of groups. The system determines whether the detected abnormality in the first job satisfies the conditions for restarting the job flow associated with the first group. The computer is made to perform the process, The aforementioned restart process is: The information processing program according to claim 1 or 2, characterized in that, if it is determined that the detected abnormality in the first job satisfies the conditions for restarting the job flow associated with the first group, it refers to the storage unit and restarts the job flow starting from the job that is at the beginning of the execution order in the second group, based on information indicating the job that is at the beginning of the execution order in the second group which is the starting point for restarting the job flow associated with the first group.

4. The aforementioned groups include collection groups that classify collection jobs that collect data from data sources, The aforementioned classification process is, The information processing program according to claim 1, characterized in that, if it is determined that the fifth job is a collection job based on the processing content of the fifth job included in the group of jobs, the fifth job is classified into the collection group.

5. The aforementioned multiple groups include processing groups that classify processing jobs that obtain and process data indicating processing results from a single job, The aforementioned classification process is, The information processing program according to claim 4, characterized in that, if it is determined that the sixth job is a processing job based on the processing content of the sixth job included in the job group, the sixth job is classified into the processing group.

6. The aforementioned multiple groups include an integration group that classifies an integration job that acquires and integrates data showing processing results from multiple jobs, The aforementioned classification process is, The information processing program according to claim 5, characterized in that, if it is determined that the seventh job is an integrated job based on the processing content of the seventh job included in the group of jobs, the seventh job is classified into the integrated group.

7. The aforementioned multiple groups include storage groups that classify storage jobs that acquire data and store it in a database, The aforementioned classification process is, The information processing program according to claim 6, characterized in that, if it is determined that the eighth job is a storage job based on the processing content of the eighth job included in the job group, the eighth job is classified into the storage group.

8. Based on the processing content of each job in a group of jobs whose execution order is defined in the job flow, the group of jobs is classified into multiple groups. If an anomaly is detected in the first job classified into the first group of the plurality of groups, the system refers to a storage unit that stores information indicating the job that is first in the execution order in the group that will be used as the starting point for restarting the job flow, and based on the information indicating the job that is first in the execution order in the second group that will be used as the starting point for restarting the job flow, the system restarts the job flow starting with the job that is first in the second group. An information processing method characterized in that the processing is performed by a computer.

9. Based on the processing content of each job in a group of jobs whose execution order is defined in the job flow, the group of jobs is classified into multiple groups. If an anomaly is detected in the first job classified into the first group of the plurality of groups, the system refers to a storage unit that stores information indicating the job that is first in the execution order in the group that will be used as the starting point for restarting the job flow, and based on the information indicating the job that is first in the execution order in the second group that will be used as the starting point for restarting the job flow, the system restarts the job flow starting with the job that is first in the second group. An information processing device characterized by having a control unit.

Images