Source-driven job relevance assessment

Abstract

Described herein is a system and method for ranking jobs for a shared computing cluster based on predicted user impact and / or taking action regarding the execution of the jobs. For example, information regarding previous executions of multiple jobs is obtained from job execution logs. Data dependencies of the multiple jobs are determined. A job impact of each of the multiple jobs is calculated based on the determined data dependencies. A user impact of each of the multiple jobs is calculated based on the determined data dependencies, the calculated job impact, and time. The multiple jobs are ranked based on the calculated user impact. An action is taken based on the ranking of the multiple jobs. The action may include automatic scheduling of the jobs and / or providing information about the ranking to a user.

Description

Background Art

[0001] Large companies use increasingly complex infrastructure to collect, store, and analyze large amounts of data. For example, a particular infrastructure may include numerous very large clusters (e.g., up to 50,000 nodes per cluster) that serve thousands of consumers (e.g., data scientists), run hundreds of thousands of jobs per day, and access billions of files. Summary of the Invention

[0002] A system for ranking jobs for a shared computing cluster based on predicted user impact and / or taking action regarding the execution of the jobs is described herein, the system comprising a processing system including one or more processors and one or more memories having computer-executable instructions stored thereon, the computer-executable instructions, when executed by the one or more processors, causing the processing system to: obtain information regarding previous executions of a plurality of jobs; determine data dependencies of the plurality of jobs; calculate a job impact of each of the plurality of jobs based on the determined data dependencies; calculate a user impact of each of the plurality of jobs based on the determined data dependencies, the calculated job impact, and time; rank the plurality of jobs based on the calculated user impact; and take action based on the ranking of the plurality of jobs.

[0003] This Summary is provided to introduce some concepts in a simplified form that are further described in the Detailed Description below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 is a functional block diagram illustrating a system for ranking jobs for a shared computing cluster based on predicted user impact and / or taking action regarding the execution of the jobs.

[0005] Figure 2 It is a graphical user interface.

[0006] Figure 3 is a flow chart illustrating a method for ranking jobs for a shared compute cluster and / or taking action regarding the execution of the jobs based on predicted user impact.

[0007] Figure 4 is a flow chart illustrating a method of ranking tasks and / or taking action regarding performance of the tasks based on predicted user impact.

[0008] Figure 5 is a functional block diagram illustrating an exemplary computing system. DETAILED DESCRIPTION

[0009] Various techniques related to ranking jobs of a shared computing cluster and / or taking action regarding the execution of the job based on predicted user impact will now be described with reference to the accompanying drawings, wherein the same reference numerals are used throughout to represent the same elements. In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding of one or more aspects. However, it is clear that such (multiple) aspects can be practiced without these specific details. In other examples, well-known structures and devices are shown in block diagram form to facilitate description of one or more aspects. In addition, it should be understood that the functions described as being performed by certain system components can be performed by multiple components. Similarly, for example, a component can be configured to perform the functions described as being performed by multiple components.

[0010] The present disclosure supports various products and processes that perform or are configured to perform various actions related to ranking jobs for a shared computing cluster based on predicted user impact and optionally prioritizing the execution of the jobs. The following are one or more exemplary systems and methods.

[0011] Aspects of the present disclosure relate to the following technical problem: prioritizing jobs to be executed based on job impact while taking into account job utility as a function of time. Technical features associated with solving this problem involve obtaining information about previous executions of multiple jobs (e.g., from (multiple) job execution logs); determining data dependencies between the multiple jobs; calculating the job impact of each of the multiple jobs based on the determined data dependencies; calculating the user impact of each of the multiple jobs based on the determined data dependencies, job impact, and time; ranking the multiple jobs based on the calculated user impact; and taking action based on the ranking of the multiple jobs. Therefore, aspects of these technical features exhibit the technical effect of more efficiently and effectively scheduling jobs, for example, reducing resource consumption and / or resource waste.

[0012] Furthermore, the term "or" is intended to mean an inclusive or rather than an exclusive or. That is, unless specified otherwise or clear from the context, the phrase "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, the phrase "X employs A or B" is satisfied by any of the following instances: X employs A; X employs B; or X employs both A and B. Furthermore, the articles "a" and "an" as used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from the context to be directed to the singular.

[0013] As used herein, the terms "component" and "system" and their various forms (e.g., component, system, subsystem, etc.) are intended to refer to a computer-related entity, which is either hardware or a combination of hardware and software, software, or software execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an instance, an executable file, an execution thread, a program, and / or a computer. As an illustration, both an application running on a computer and a computer can be a component. One or more components can reside in an execution process and / or thread, and a component can be located on one computer and / or distributed between two or more computers. In addition, as used herein, the term "exemplary" is intended to mean an illustration or example of something and is not intended to indicate a preference.

[0014] Large companies use increasingly complex infrastructure to collect, store, and analyze vast amounts of data. For example, a particular infrastructure can include numerous very large clusters (e.g., up to 50,000 nodes each) that serve thousands of consumers (e.g., data scientists), run hundreds of thousands of jobs daily, and access billions of files. Determining the scheduling of these jobs based on available resources, consumer expectations, contractual obligations, and more can be a complex task.

[0015] Described herein is a system and method for ranking jobs for a shared compute cluster and / or taking action regarding the execution of the job based on the predicted job impact. The present invention addresses the problem of understanding the importance of a job (e.g., in an absolute sense compared to any other job), and doing so "over time," i.e., how the timeliness of delivering the job's output affects the importance of the job. This can be relevant to both customers (e.g., assessing whether they are dedicating resources to the right jobs) and cluster operators (e.g., selecting jobs to migrate first to a new version of software, or which jobs to run in the event of a capacity impairment).

[0016] Previous manual solutions relied on imprecise and partial user understanding of job importance. In some cases, it was inherently difficult for users to quantify the importance of their jobs relative to others' jobs (e.g., due to lack of visibility). For example, some systems allow users to provide information about "queue size," "queue priority," and / or "job parallelism allocation." Often, users were unable to assign these variables to successfully deliver jobs in a timely manner without wasting resources.

[0017] The systems and methods described herein leverage "provenance" (e.g., lineage of data dependencies between jobs) and their telemetry (e.g., statistics about job executions, such as job start / finish time, CPU hours spent by the job, total read / write data) to automatically derive a notion of "relevance / importance" of a job and correlate it with job execution time ("job impact").

[0018] refer to Figure 1 , shows a system for ranking jobs for a shared computing cluster and / or taking action regarding the execution of the job based on predicted user impact 100. The concept of a "job" is used in the context of data processing system(s) ("DPS"), such as databases, streaming systems, big data clusters, etc. A "job" comprises a finite amount of computation performed by a potentially large number of tasks to transform a given "input" into a given "output." A job is typically the unit of scheduling (e.g., how many resources are allocated to a particular job and when the resources are allocated) and is used by customers to typically accomplish mission-critical goals.

[0019] As DPSs grow larger and more complex, it can become difficult to track and understand which jobs are receiving resources, when the jobs will complete, etc. In some cases, timely execution of such jobs can be critical, as jobs completing too late can violate service level objectives (SLOs) at the application level, which can have potentially devastating consequences.

[0020] Typically, quantifying the importance of a job and its timely execution is left to the user and is a labor-intensive and error-prone task. Jobs with execution latency goals are typically run recurringly on new versions of the data derived at regular intervals (such as hourly, daily, etc.).

[0021] The system 100 can leverage this observation to open up opportunities for leveraging observations from past runs to make decisions for future runs. The system 100 provides a mechanism that leverages historical traces of job executions and tracks their provenance (i.e., data dependency lineage between jobs) and their telemetry (e.g., statistics about job executions, such as job start / finish time, CPU hours spent by jobs, total read / write data) to automatically derive a notion of "relevance / importance" of jobs and correlates this with job execution time, thereby deriving (multiple) utility functions based on job impact.

[0022] The system 100 includes a job history component 110 that obtains information about previous executions of multiple jobs, for example, from one or more job execution logs 120. The system 100 also includes a data dependency component 130 that determines data dependencies between the multiple jobs. In some embodiments, the data dependencies can be represented by a DAG.

[0023] The system 100 also includes a job impact calculation component 140 that calculates an impact of each of the plurality of jobs based on the determined data dependencies.

[0024] The utility function based on job impact is based on expressing the job utility in terms of the overall job impact on downstream jobs, where the impact can be measured based on metrics of the target job and its downstream jobs (e.g., all of them). In some embodiments, the job impact can be expressed as how much downstream work can be unblocked once the job completes, or how much work cannot be performed due to job failure or delay. By way of illustration and not limitation, the job metrics can include the total job CPU hours and / or the total number of "downloads" of output files generated by the target job and its downstream jobs (e.g., how many times any direct or indirect byproducts of the job are externalized from the DPS).

[0025] In some embodiments, the utility of an unfinished job inevitably decays over time (e.g., data becomes stale), and downstream jobs with hard dependencies cannot start. Therefore, to model job utility as a function of time in the context of job impact, the target job's "deadline" (e.g., the time when utility "declines") is set to the time to submission of the hard-dependent child job (e.g., relative to the target job's submission time).

[0026] In some embodiments, unlike the shape of most job utility functions (in which a job has only a single deadline at which its utility decreases rapidly), the amount and magnitude of the utility decrease in system 100 can depend on the sub-jobs of the job.

[0027] Constructing a utility function based on job impact can provide insight into the workload of a particular cluster. In some embodiments, problematic issues and / or properties of interest can be identified. For purposes of explanation and not limitation, these can include (1) a large number of abandoned scheduled jobs, (2) a large number of cross-organizational job dependencies, (3) ridiculous priorities, and / or (4) long job dependency chains that include up to hundreds of jobs spanning multiple weeks.

[0028] In some embodiments, using a scheduler with a utility function based on job impact can achieve parallelism (e.g., maximum) in a cluster by unblocking jobs in an efficient manner. However, the scheduler may focus on jobs that are closer to the "root" of a directed acyclic graph (DAG) of job dependencies and ignore jobs deeper into the DAG, which in some embodiments may have significant value to the user.

[0029] The system 100 also includes a user impact calculation component 150 that calculates a user impact of each of the plurality of jobs based on the determined data dependencies, the calculated job impact, and time.

[0030] In some embodiments, the job utility function can be refined by considering not only the impact of the job on downstream jobs, but also the actual value of the job to the submitting user by looking at the workflow description and alerts based on job wait times. That is, by incorporating user influence into the utility function based on realistic job impact, the system 100 can take action(s) (e.g., scheduling) on ​​the job more efficiently and effectively.

[0031] Many user business tasks cannot be accomplished by running a single job. Instead, users often define repetitive job pipelines or workflows that consist of a set of interdependent jobs and data operations. In some embodiments, the jobs with the greatest user impact (e.g., those that contribute the most "value" to the user or cause incidents to operators or users) are typically placed at the end of the workflow because there's no need to submit jobs that consume the output of intermediate jobs if they've already produced the output the user cares about.

[0032] In some embodiments, automatic event alerts related to delayed outputs of jobs in a workflow can be examined. These automatic event alerts can indicate situations where one or more jobs are waiting for the output(s) of other jobs. If most alerts are polling the outputs of jobs near the end of the workflow, these "leaf" jobs are likely the ones contributing the most "value" to the user. In some embodiments, while using a utility function based on job impact can likely unblock the most downstream jobs for execution, using only job impact-based utility may conflict with the goal of optimizing user value, as earlier jobs in the workflow will inevitably have higher job impact-based utility than later jobs in the workflow. System 100 can employ a "realistic" job utility function that uses features and signals from user-submitted workflows to account for user impact-based utility, ultimately resulting in the generation of a realistic utility function that balances job impact-based utility with user impact-based utility.

[0033] The system 100 also includes a job ranking component 160 that ranks the plurality of jobs based on the calculated user impact. The system 100 also includes a job action component 170 that takes an action based on the ranking of the plurality of jobs. In some embodiments, the action may include automatically scheduling the job based on the ranking of the plurality of jobs. In some embodiments, the action may include providing information about the ranking of the plurality of jobs to the user(s) (e.g., via a graphical user interface) to allow the user(s) to accept, reject, and / or modify the ranking of the plurality of jobs. In some embodiments, the provided information may include information utilized by the system 100 in deriving the ranking, such as information about previous executions of the plurality of jobs, determined data dependencies, calculated job impacts, and / or calculated user impacts.

[0034] In some embodiments, the system 100 may further include a feedback component 180 that provides feedback (e.g., results) regarding the actions taken to the job impact calculation component 140 and / or the user impact calculation component 150. In this manner, the job impact utility function utilized by the job impact calculation component 140 and / or the user impact utility function utilized by the user impact calculation component 150 may be adaptively updated based on the results taken / obtained using the actions taken.

[0035] In some embodiments, when the action taken is to automatically schedule a job based on the ranking of multiple jobs, the feedback may include any anomalies received, such as(s) automatic event alert(s),(s) user action(s), information about the execution of the job, etc. In some embodiments, when the action taken is to provide(s) information about the ranking of multiple jobs to user(s), the feedback may include(s) action(s) taken by the user(s) regarding the scheduling of the multiple jobs (e.g., modification(s) of the ranking) and / or information about the execution of the job(s).

[0036] Traditionally, in some cases, customers pay for the hardware resources used to produce a dataset. In some embodiments, the system 100 may include "pricing" based on data consumption and / or data production. The system 100 may provide a form of pricing in which consumers of a particular dataset share (e.g., all or some) of the cost of producing the particular dataset. For example, a particular consumer may be responsible for part of the computational costs of an upstream job. In some embodiments, a particular consumer may be compensated by (multiple) other downstream consumers for the cost of running the particular consumer's (multiple) jobs. The system 100 may thus provide arbitrage, for example, particularly where a job consumes (multiple) "public" datasets and produces a "unique" or very desirable dataset, it may bill for the difference in value added by that computation.

[0037] In some embodiments, a user (e.g., a consumer) may specify the importance of a dataset(s) downloaded from a computing cluster to the system 100. The system 100 may back-propagate the specified importance to one or more jobs involved in producing the downloaded dataset(s). For example, the specified importance may be back-propagated to each job involved in producing the downloaded dataset(s).

[0038] For purposes of explanation and not limitation, data dependencies have been used to quantify job impact / importance. In some embodiments, other factors (or factors) may be utilized to quantify job impact / importance instead of or in addition to data dependencies. Information about the importance of the user submitting a particular job (e.g., role in the corporate / government hierarchy, specified in contractual obligations associated with the computing cluster), the size of a particular job, the number of downloads of its output, etc. may be used to quantify job impact / importance.

[0039] In some embodiments, the system 100 may also add different objectives when scheduling jobs based on their impact / ranking. For example, the system 100 may attempt to minimize service level agreement (SLA) violations and / or monetary losses resulting from not executing jobs on time.

[0040] In some embodiments, the system 100 may utilize (multiple) machine learning models in which accuracy may be traded for time and / or resources, resulting in a utility function U(t, a) that depends on time t and accuracy a.

[0041] Although the disclosed subject matter has been discussed in the context of scheduling jobs for a data processing system, in some embodiments, the system 100 can be used to schedule tasks (e.g., manually and / or automatically) such as workflows. For example, using historical data, dependencies between tasks can be determined. A task impact of each of a plurality of tasks can be calculated based on the determined dependencies. Thereafter, a user impact of each of the plurality of tasks can be calculated based on the determined dependencies, the calculated task impact, and time. The tasks can then be ranked based on the calculated user impact. Task scheduling can then be performed based on the ranking of the plurality of tasks.

[0042] Go to Figure 2In some embodiments, information regarding the impact of jobs calculated by system 100 can be made available to user(s) in a graphical user interface(s), allowing the user to analyze the impact of their jobs on downstream jobs and operations. Quantifying the "utility" or value of recurring jobs by analyzing job history and telemetry can be part of a broader effort. The information provided can be consistent with information to be data-driven, thereby supporting the "value" of jobs determined by the system with specific historical data.

[0043] The graphical user interface(s) may provide job volume insights for the job(s). Figure 2 2 is a graphical user interface 200 that includes a graph 210 showing the utility of job A as a function of time, and a dependency graph 250. As described above, the utility function indicates the value of a job as a function of time. In some embodiments, the "value" of a job is based on the total downstream dependencies. As the deadline for a job and / or operation passes, the utility of the job decreases.

[0044] With respect to graph 200 and dependency graph 250, the utility of job A decreases as dependencies B, C, and E are submitted. The utility decreases more for C than for B because C has a downstream dependency and B does not. The utility decreases more for E than for C because E has two downstream dependencies.

[0045] (Multiple) graphical user interfaces that allow users to analyze the impact of their jobs on downstream jobs and operations may include search capabilities that allow (multiple) users to search for (multiple) job instances and / or (multiple) duplicate jobs using different parameters (e.g., using a search box).

[0046] Searching for jobs may result in a list of jobs matching the search criteria being presented.The user may click on a repeating job link for a job instance, or click on a details link for more instance details about the job.

[0047] Clicking the details link takes the user to the job details page. The job details page may display a graph of the utility function of the job instance, the utility functions of other job instances that replicate the job, and the average utility function of the replicated job.

[0048] In some embodiments, the job details page also shows a dependency graph of the job's utility based on the utility contributed by downstream jobs / operations. Additionally, in some embodiments, the user can use a drop-down list to switch the type of utility function / dependency graph displayed (e.g., by PnHours and / or by Downloads).

[0049] Figure 3Exemplary methods are shown for ranking jobs for a shared computing cluster based on predicted user impact and / or taking action regarding the execution of the jobs. Figure 4 An exemplary method is shown for ranking tasks based on predicted user impact and / or taking actions regarding the performance of the tasks. Although the method is shown and described as a series of actions performed in sequence, it should be understood and appreciated that the method is not limited by sequence. For example, certain actions may occur in a different order than described herein. In addition, an action may occur simultaneously with another action. Furthermore, in some cases, not all actions may be required to implement the method described herein.

[0050] Furthermore, the actions described herein may be computer-executable instructions that can be implemented by one or more processors and / or stored on one or more computer-readable media. The computer-executable instructions may include routines, subroutines, programs, execution threads, etc. Furthermore, the results of the actions of the method may be stored on a computer-readable medium, displayed on a display device, etc.

[0051] refer to Figure 3 , a method 300 is shown for ranking jobs for a shared computing cluster and / or taking action regarding the execution of the jobs based on predicted user impact. In some embodiments, the method 300 is performed by the system 100.

[0052] At 310, information about previous execution(s) of a plurality of jobs is obtained. At 320, data dependencies (if any) between the jobs are determined.

[0053] At 330 , a job impact of each of the plurality of jobs is calculated based on the determined data dependencies. At 340 , a user impact of each of the plurality of jobs is calculated based on the determined data dependencies, the calculated job impact, and time.

[0054] At 350, the plurality of jobs are ranked according to the calculated user impact. At 360, an action is taken according to the ranking of the plurality of jobs.

[0055] refer to Figure 4 , a method 400 is shown for ranking tasks and / or taking action regarding the performance of the tasks based on predicted user impact. In some embodiments, the method 400 is performed by the system 100.

[0056] At 410, information about previous executions of a plurality of tasks is obtained. At 420, data dependencies (if any) between the tasks are determined.

[0057] At 430 , a task impact of each of the plurality of tasks is calculated based on the determined data dependencies. At 440 , a user impact of each of the plurality of tasks is calculated based on the determined data dependencies, the calculated task impact, and time.

[0058] At 450, the plurality of tasks are ranked according to the calculated user impact. At 460, actions are taken according to the ranking of the plurality of tasks. For example, tasks may be scheduled according to the ranking of the plurality of tasks.

[0059] Described herein is a system for ranking jobs for a shared computing cluster and / or taking actions regarding the execution of jobs based on predicted user impact. The system may include a processing system comprising one or more processors and one or more memories having computer-executable instructions stored thereon, the computer-executable instructions, when executed by the processors, causing the processing system to: obtain information regarding previous executions of a plurality of jobs; determine data dependencies of the plurality of jobs; calculate a job impact of each of the plurality of jobs based on the determined data dependencies; calculate a user impact of each of the plurality of jobs based on the determined data dependencies, the calculated job impact, and time; rank the plurality of jobs based on the calculated user impact; and take actions based on the ranking of the plurality of jobs.

[0060] The system may also include: wherein information about previous executions of the plurality of jobs is obtained from one or more job execution logs. The system may also include: wherein the action taken includes automatically scheduling the plurality of jobs based on a ranking of the plurality of jobs and a time-based attribute of the job utilities. The system may also include: wherein the action taken includes providing information about the ranking of the plurality of jobs to a user via a graphical user interface.

[0061] The system may include one or more memories having additional computer-executable instructions stored thereon, which, when executed by one or more processors, cause the processing system to: receive input from a user for at least one of: accepting, rejecting, or modifying a ranking of the plurality of jobs. The system may also include: wherein the information provided to the user includes information utilized by the system in deriving the ranking. The system may also include: wherein the information provided to the user includes at least one of information about previous executions of the plurality of jobs, determined data dependencies, calculated job impacts, or calculated user impacts.

[0062] The system may include: one or more memories having additional computer-executable instructions stored thereon, the additional computer-executable instructions, when executed by one or more processors, causing the processing system to: receive feedback about actions taken; adaptively update a job impact utility function for calculating job impact; and adaptively update a user impact utility function for calculating user impact.

[0063] Described herein is a method for ranking jobs for a shared computing cluster and / or taking action regarding the execution of jobs based on predicted user impact, the method comprising: obtaining information about previous executions of multiple jobs; determining data dependencies of the multiple jobs; calculating a job impact of each of the multiple jobs based on the determined data dependencies; calculating a user impact of each of the multiple jobs based on the determined data dependencies, the calculated job impact, and time; ranking the multiple jobs based on the calculated user impact; and taking action based on the ranking of the multiple jobs.

[0064] The method may further include: wherein information about previous executions of the plurality of jobs is obtained from one or more job execution logs. The method may further include: wherein the action taken includes automatically scheduling the plurality of jobs based on a ranking of the plurality of jobs. The method may further include: wherein the action taken includes providing information about the ranking of the plurality of jobs to a user via a graphical user interface.

[0065] The method may further include receiving input from a user for at least one of accepting, rejecting, or modifying the ranking of the plurality of jobs. The method may further include wherein the information provided to the user includes information utilized by the system in deriving the ranking. The method may further include wherein the information provided to the user includes at least one of information about previous executions of the plurality of jobs, determined data dependencies, calculated job impacts, or calculated user impacts.

[0066] The method may further include receiving feedback regarding the action taken; adaptively updating a job impact utility function for calculating job impact; and adaptively updating a user impact utility function for calculating user impact.

[0067] Described herein is a computer storage medium storing computer-readable instructions that, when executed, cause a computing device to: obtain information about previous executions of a plurality of jobs; determine data dependencies of the plurality of jobs; calculate a job impact of each of the plurality of jobs based on the determined data dependencies; calculate a user impact of each of the plurality of jobs based on the determined data dependencies, the calculated job impact, and time; rank the plurality of jobs based on the calculated user impact; and take an action based on the ranking of the plurality of jobs.

[0068] The computer storage medium may further include: wherein the action taken includes automatically scheduling the plurality of jobs based on the ranking of the plurality of jobs. The computer storage medium may further include: wherein the action taken includes providing information about the ranking of the plurality of jobs to a user via a graphical user interface. The computer storage medium may store additional computer-readable instructions that, when executed, cause the computing device to: receive feedback about the action taken; adaptively update a job impact utility function for calculating job impact; and adaptively update a user impact utility function for calculating user impact.

[0069] refer to Figure 5 , an example general-purpose processing system, computer, or computing device 502 (e.g., a mobile phone, desktop, laptop, tablet, watch, server, handheld, programmable consumer or industrial electronics, set-top box, gaming system, computing node, etc.) is shown. For example, computing device 502 can be used in system 100 for ranking jobs for a shared computing cluster based on predicted user impact and / or taking actions regarding the execution of the jobs.

[0070] The computer 502 includes one or more processors 520, a memory 530, a system bus 540, (a plurality of) mass storage devices 550, and one or more interface components 570. The system bus 540 communicatively couples at least the aforementioned system components. However, it should be understood that the computer 502, in its simplest form, may include one or more processors 520 coupled to the memory 530 to execute various computer-executable actions, instructions, and / or components stored in the memory 530. Such instructions may be, for example, instructions for implementing the functions described as being performed by one or more of the aforementioned components or instructions for implementing one or more of the aforementioned methods.

[0071] The processor(s) 520 may be implemented using a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. A general purpose processor may be a microprocessor, but alternatively, the processor may be any processor, controller, microcontroller, or state machine. The processor(s) 520 may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, a multi-core processor, one or more microprocessors in combination with a DSP core, or any other such configuration. In one embodiment, the processor(s) 520 may be a graphics processor.

[0072] Computer 502 may include or otherwise interact with various computer-readable media to facilitate controlling computer 502 to implement one or more aspects of the claimed subject matter. Computer-readable media can be any available media that can be accessed by computer 502 and includes both volatile and nonvolatile media, and removable and non-removable media. Computer-readable media can include two distinct and mutually exclusive types: computer storage media and communication media.

[0073] Computer storage media includes volatile and nonvolatile removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes storage devices such as memory devices (e.g., random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), etc.), magnetic storage devices (e.g., hard disks, floppy disks, cassettes, tapes, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), and solid-state devices (e.g., solid-state drives (SSDs), flash drives (e.g., cards, sticks, key drives), etc.), or any other similar media that store (rather than transport or communicate) desired information that can be accessed by the computer 502. Therefore, computer storage media does not include modulated data signals and signals described with respect to communication media.

[0074] Communication media embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and includes any information delivery media. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0075] Memory 530 and mass storage device(s) 550 are examples of computer-readable storage media. Depending on the exact configuration and type of computing device, memory 530 may be volatile (e.g., RAM), non-volatile (e.g., ROM, flash memory, etc.), or some combination of the two. As an example, a basic input / output system (BIOS), including basic routines for transferring information between elements within computer 502, such as during startup, may be stored in non-volatile memory, while volatile memory may act as an external cache to facilitate processing by processor(s) 520, etc.

[0076] Mass storage device(s) 550 include removable / non-removable volatile / non-volatile computer storage media for storing large amounts of data relative to memory 530. For example, mass storage device(s) 550 include, but are not limited to, one or more devices such as magnetic or optical disk drives, floppy disk drives, flash memory, solid state drives, or memory sticks.

[0077] The memory 530 and the mass storage device(s) 550 may include or have stored therein an operating system 560, one or more applications 562, one or more program modules 564, and data 566. The operating system 560 is used to control and allocate resources of the computer 502. The applications 562 include one or both of system and application software and can utilize the management of resources by the operating system 560 to perform one or more operations through the program modules 564 and data 566 stored in the memory 530 and / or the mass storage device(s) 550. Thus, the applications 562 can transform the general-purpose computer 502 into a special-purpose machine based on the logic provided by the applications 562.

[0078] All or part of the claimed subject matter can be implemented using standard programming and / or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer to implement the disclosed functionality. By way of example and not limitation, the system 100 or a portion thereof can be or form part of an application 562 and include one or more modules 564 and data 566 stored in memory and / or mass storage device(s) 550, the functionality of which can be implemented when executed by one or more processors 520.

[0079] According to a specific embodiment, (multiple) processors 520 may correspond to a system on a chip (SOC) or similar architecture, including or in other words integrating both hardware and software on a single integrated circuit substrate. Here, (multiple) processors 520 may include one or more processors, and memories at least similar to (multiple) processors 520 and memory 530, etc. Conventional processors include a minimum amount of hardware and software and rely extensively on external hardware and software. In contrast, the SOC implementation of a processor is more powerful because it has hardware and software embedded therein, thereby enabling specific functions to be implemented with minimal or no reliance on external hardware and software. For example, system 100 and / or associated functions may be embedded in hardware in a SOC architecture.

[0080] The computer 502 also includes one or more interface components 570 that are communicatively coupled to the system bus 540 and facilitate interaction with the computer 502. By way of example, the interface component 570 can be a port (e.g., serial, parallel, PCMCIA, USB, FireWire, etc.) or an interface card (e.g., sound, video, etc.). In one example implementation, the interface component 570 can be embodied as a user input / output interface to enable a user to enter commands and information into the computer 502 through one or more input devices (e.g., a pointing device such as a mouse, trackball, stylus, touchpad, keyboard, microphone, joystick, game pad, satellite dish, scanner, camera, other computer, etc.), for example, through one or more gestures or voice input. In another example implementation, the interface component 570 can be embodied as an output peripheral interface for providing output to a display (e.g., LCD, LED, plasma, etc.), speakers, printer, and / or other computers. Further, the interface component 570 can be embodied as a network interface to enable communication with other computing devices (not shown), such as through a wired or wireless communication link.

[0081] What has been described above includes examples of various aspects of the claimed subject matter. It is, of course, not possible to describe every possible combination of components or methods for the purposes of describing the claimed subject matter, but one of ordinary skill in the art will recognize that many other combinations and permutations of the disclosed subject matter are possible. Accordingly, the disclosed subject matter is intended to encompass all such changes, modifications, and variations that come within the spirit and scope of the appended claims. Furthermore, if the term "includes" is used in the detailed description or the claims, such term is intended to be inclusive, in a manner similar to the interpretation of the term "comprising" when used as a transitional word in a claim.

Claims

1. A job scheduling system comprising: one or more processors; and one or more memories having computer-executable instructions stored thereon, which, when executed by the one or more processors, cause the system to: Get information about previous executions of multiple jobs; determining data dependencies of the plurality of jobs based at least on the information regarding the previous execution; A job impact of an individual job is determined based on the data dependencies, wherein the respective job impact for the individual job decreases over time as deadlines of other jobs that depend on the individual job pass, and wherein the job impact indicates at least one of: How many downstream jobs the job can unblock once it is completed, and How much work could not be performed because said work failed or was delayed; determining a user impact of the individual job based on the data dependency and the job impact; determining a ranking of the individual jobs based on the user impact; as well as At least one job scheduling action is taken based on the ranking of the individual jobs. 2 . The system of claim 1 , wherein the information about the previous execution is obtained from one or more job execution logs. 3 . The system of claim 1 , wherein the at least one job scheduling action involves automatically scheduling the plurality of jobs based on the rankings of the individual jobs and a time-based nature of job utility.

4. The system of claim 1 , the one or more memories having stored thereon additional computer-executable instructions that, when executed by the one or more processors, cause the system to: Information regarding the ranking of the individual jobs is output via a graphical user interface.

5. The system of claim 4, the one or more memories having stored thereon further computer-executable instructions that, when executed by the one or more processors, cause the system to: Input is received via the graphical user interface for at least one of: accepting, rejecting, and modifying the ranking of the individual job.

6. The system of claim 4, wherein the information output via the graphical user interface includes information utilized by the system in deriving the ranking.

7. The system of claim 6, wherein the information output via the graphical user interface includes at least one of the following: the information about the previous execution, information about the data dependency, information about the job impact, and information about the user impact.

8. The system of claim 1 , the one or more memories having stored thereon additional computer-executable instructions that, when executed by the one or more processors, cause the system to: receiving feedback regarding the at least one job scheduling action; Adaptively updating an activity impact utility function used to determine the activity impact based at least on the feedback; and A user influence utility function for determining the user influence is adaptively updated based at least on the feedback.

9. A job scheduling method, comprising: Determine data dependencies among multiple jobs; A job impact of an individual job is determined based on the data dependencies, wherein the respective job impact for the individual job decreases over time as deadlines of other jobs that depend on the individual job pass, and wherein the job impact indicates at least one of the following: How many downstream jobs the job can unblock once it is completed, and How much work could not be performed because said work failed or was delayed; determining a user impact of the individual job based on the data dependency and the job impact; determining a ranking of the individual jobs based on the user impact; as well as At least one job scheduling action is taken based on the ranking of the individual jobs.

10. The method according to claim 9, further comprising: The first job impact on the first job decreases at least two-fold over time as deadlines of at least two other jobs that depend on the first job pass.

11. The method according to claim 9, further comprising: Information regarding the ranking is provided via a graphical user interface.

12. The method according to claim 11, further comprising: Input is received to at least one of: accept, reject, and modify the ranking.

13. The method of claim 11 , wherein the information provided via the graphical user interface comprises: The information used to derive the ranking.

14. The method of claim 13, wherein the information provided via the graphical user interface comprises at least one of: information about previous executions of the plurality of jobs, information about the data dependencies, and information about the job impacts.

15. The method according to claim 9, further comprising: receiving feedback regarding the at least one job scheduling action; as well as An activity impact utility function is adaptively updated, the activity impact utility function being used to determine the activity impact based at least on the feedback.

16. The method of claim 15, wherein the feedback indicates that an automatic event alert related to delayed job output has occurred.

17. A computer storage medium storing computer-readable instructions that, when executed, cause a computing device to: Get information about previous executions of multiple jobs; determining data dependencies of the plurality of jobs from the information regarding the previous execution; A job impact of an individual job is determined based on the data dependencies, wherein the respective job impact for the individual job decreases over time as deadlines of other jobs that depend on the individual job pass, and wherein the job impact indicates at least one of: How many downstream jobs the job can unblock once it is completed, and How much work could not be performed because said work failed or was delayed; determining a user impact of the individual job based on the data dependency and the job impact; determining a ranking of the individual job based at least on the user influence; as well as At least one job scheduling action is taken based on the ranking of the individual jobs.

18. The computer storage medium of claim 17, wherein the computer readable instructions, when executed, cause the computing device to: Information regarding the ranking is output via a graphical user interface.

19. The computer storage medium of claim 17, wherein the computer readable instructions, when executed, cause the computing device to: receiving feedback regarding the at least one job scheduling action; and At least based on the feedback: Adaptively updating an activity impact utility function used to determine the activity impact; and A user influence utility function for determining the user influence is adaptively updated.

20. The computer storage medium of claim 19, wherein the feedback indicates that an automated event alert has occurred.

Images