82 results about "Job queue" patented technology

In some embodiments, an automation optimizer is configured to determine whether a provisioning condition is satisfied, for instance according to a current length of a job queue, or according to a current workload of a selected RPA host platform executing a plurality of software robots. When to the provisioning condition is satisfied, some embodiments automatically provision additional VMs onto the respective RPA host platform, and automatically remove VMs when automation demand is low. Exemplary RPA hosts include cloud computing platforms and on-premises servers, among others.

The invention discloses a multi-machine resource configuration management method and device, equipment, a medium and a product, and relates to the technical field of multi-machine resource configuration.The method comprises the steps that queue allocation is conducted on satellites according to processing priorities; distributing machine nodes for each job queue according to machine node resources required by the job queues to obtain a plurality of job queues and a plurality of corresponding machine node groups; acquiring state information of each host node in the machine node group, host resource load information and resource information required by each job in the job queue in real time, and matching each job in the job queue to the machine node in the corresponding machine node group according to the state information and the host resource load information of each host node in the machine node group; and activating the jobs meeting the conditions one by one according to the priority of each job in the job queue and executing the jobs in parallel. The whole computer network system is changed into a relatively single virtual host or super computer, so that the effect of providing a relatively single service operation platform for upper-layer conventional service scheduling is realized.

A machine learning service implements programmatic interfaces for a variety of operations on several entity types, such as data sources, statistics, feature processing recipes, models, and aliases. A first request to perform an operation on an instance of a particular entity type is received, and a first job corresponding to the requested operation is inserted in a job queue. Prior to the completion of the first job, a second request to perform another operation is received, where the second operation depends on a result of the operation represented by the first job. A second job, indicating a dependency on the first job, is stored in the job queue. The second job is initiated when the first job completes.

Techniques are described for enabling replication-aware resource management and task management in a cloud infrastructure for cross-region replication. In some embodiments, each replication job is associated with a set of replication-related information. In certain embodiments, the replication-aware resource management allocates resources, using a combination of various resource allocation schemes, to a fleet of replicators to allow the fleet to select replication jobs in a job queue, and perform resource scaling based on monitored performance metrics reported by the fleet. In some embodiments, the replication-aware task management enables replication job selection based on the set of replication-related information to optimize the performance of all cross-region replications in the region.

The invention discloses a quay crane operation queue intelligent scheduling data processing method, and belongs to the technical field of intelligent scheduling, and the method comprises the steps: 1, preparing data, processing the data including ship information, equipment information, queue information and rule configuration, and then setting constraint conditions including a preset limiting condition and a target constraint condition; 2, building a CWP algorithm framework; 3, initializing parameters, carrying out data preprocessing, initializing parameters and variables required by an algorithm, and executing scheduling; step 4, constructing a neural network, establishing a neural network model of reinforcement learning, and realizing data progressive extraction from an input layer, a processing layer, a feature fusion layer, a decision layer to an output layer; and 5, training the network and updating parameters. The defects of a traditional scheduling algorithm are overcome, scheduling mode diversification is achieved, global optimization of intelligent scheduling is achieved, and a more flexible, efficient and intelligent job scheduling solution is provided.

The invention provides a job instance scheduling method and device, equipment, a medium and a product. Relates to the technical field of big data processing. The method comprises the following steps: acquiring at least one to-be-scheduled first job instance and related information of the first job instance; in response to a second job instance which is determined according to each piece of related information and meets a ready state, writing the second job instance into a ready job queue under the corresponding tenant; for a current second job instance in the ready job queue under each tenant, determining the type of the current second job instance, and obtaining a target available resource detection strategy matched with the type of the current second job instance; detecting available resources corresponding to the current second job instance based on the target available resource detection strategy; in response to the fact that the available resources meet the execution requirement of the current second job instance, submitting the current second job instance to a distributed queue; and scheduling the second job instance in the distributed queue to the corresponding physical resource group. According to the method provided by the invention, a job instance scheduling process is provided, the dependence of job scheduling on a bottom layer resource management framework is reduced, and the job instance scheduling efficiency is improved.

The invention discloses a GPU sharing scheduling method and device supporting heterogeneous job scale, and the method comprises the steps: carrying out the interference portrait analysis of a newly submitted job, obtaining the interference feature representation of the job, and adding the interference feature representation into a job queue; collecting cluster operation information and job information; determining a job scheduling interval; on the basis of the job runtime information, the job priority is updated, and the jobs are sorted through a maximum sharing efficiency priority method; traversing jobs in the job queue, and determining jobs to be run and jobs to be preempted; calling an optimal packaging strategy and job co-located interference prediction model to determine a placement position of a to-be-operated job; placing jobs and updating a cluster state; and monitoring a new job submission condition and updating a scheduling interval, if a new job is submitted, skipping to carry out interference portrait analysis on the newly submitted job, otherwise, skipping to the step of determining the job scheduling interval. According to the method, the job completion time and the queuing time can be effectively reduced, and the resource scheduling efficiency of the GPU cluster is remarkably improved.

Techniques are disclosed for executing a data processing pipeline. The techniques may include receiving a job at a data pipeline queue, setting up one or more distributed processing environments, and allocating the job to one of the distributed processing environments. The techniques may further include receiving the allocated job at a job queue within the distributed processing environment, increasing a priority level of the job, and executing the job within the distributed processing environment. The techniques can further include providing a retry pipeline at the data processing pipeline, and re-executing the job at a stage following a failure of at least one of its components. The techniques may decrement the retry budget as the job is re-executed.

A printing system includes a printing device that receives print jobs. The print jobs are placed in a job queue within the digital front end (DFE) of the printing device. The printing device also includes a user interface to display information related to processing the print jobs. Methods executed at the DFE include defining a time period to process the print jobs at the printing device. A production time is estimated for each print job within the job queue related to parameters of the respective print jobs. A magnification level for each print job is generated based on the respective production time. The magnification level determines a visual display within the user interface. As a current job indicator moves over the slots related to the print jobs, the visual display is adjusted according to the magnification level.

Technologies are provided for a multi-cloud bursting service. An example method can include receiving, via a cloud bursting service associated with different clouds, a cloud bursting configuration enabling the cloud bursting service for a local compute environment; based on the cloud bursting configuration, determining a number of jobs in a jobs queue associated with one or more cloud environments from the different clouds; determining a number of nodes available to process the number of jobs in the jobs queue; based on the number of jobs in the jobs queue and number of nodes available, determining whether to spin up a new node, take offline an existing node, or shutdown the existing node to yield a determination; and based on the determination and cloud bursting configuration, performing a cloud bursting action including spinning up the new node, taking offline the existing node, or shutting down the existing node.

Methods for managing printing operations include querying a job queue at a digital front end (DFE) of a printing device to determine a number of print jobs in the job queue. A production time is estimated for each print job within the job queue. The production time relates to a processing time to print the respective print job at the printing device. Each print job is scheduled within a printing shift in a defined printing schedule according to the production times. Further, the order of the print jobs within the printing schedule is modified according to a condition, policy, or time usage optimization. The order of the print jobs within the job queue are reordered to reflect the modified order in the printing schedule.

A memory system includes at least one memory device that includes memory areas. The memory system includes a device allocation manager and security erase circuitry. The device allocation manager determines which of the memory areas is allocated or released based on a request input from at least one host. The security erase circuitry stores a security erase task for a first memory area, which is associated with a release determined by the device allocation manager, in an erase job queue, and removes the security erase task from the erase job queue when the first memory area is reallocated to a first host to which the first memory area was allocated before the release.

Methods for managing printing operations include querying a job queue at a digital front end (DFE) of a printing device to determine a number of print jobs in the job queue. A production time is estimated for each print job within the job queue. The production time relates to a processing time to print the respective print job at the printing device. Each print job is scheduled within a printing shift in a defined printing schedule according to the production times. Further, the order of the print jobs within the printing schedule is modified according to a condition, policy, or time usage optimization. The order of the print jobs within the job queue are reordered to reflect the modified order in the printing schedule.

To support the reproduction of research data in research using an HPC system.SOLUTION: A history management system includes: a first collection unit for collecting job management number information representing a management number of a job and job script identification information identifying a job script from a job queue which sequentially assigns an input job to one of a plurality of server computers, and generates first job information; a second collection unit for collecting job management number information and program identification information identifying a program corresponding to a process from a server computer which executes the process based on the job, and generates second job information; and a storage unit for generating and storing first history information by using the first job information acquired from the first collection unit and the second job information acquired from the second collection unit.SELECTED DRAWING: Figure 3

Novel tools and techniques are provided for implementing intelligent alert automation ("IAA"). In various embodiments, IAA receives alert / event feeds from several different alerting and ticketing systems via input Redis queues, and uses a triage system to determine whether to process the alert / event or disregard it. If so, IAA may create a flow instance, assign a unique instance ID, and place the flow instance in one of a plurality of jobs queues based on alert / event type and / or or source. An abattoir system retrieves a flow instance from one of the jobs queues (in order of the queue's priority), and processes the next node or step in the flow instance. The flow instance is placed back into the jobs queue for subsequent processing by the same or different abattoir system until no additional nodes or steps remain in the flow, at which point the flow instance is considered complete.

The present invention proposes a method and system for online analysis of massive data based on multiple computing engines. The method comprises: responding to job requests initiated by third-party applications and submitting the job requests to a web-based online engine job queue; an engine job submission executor (EJSR) polling a session computing engine job queue, extracting job information (JCI), and submitting it to a session computing engine (WebActor); starting the session computing engine (WebActor) and, upon receiving the job information, publishing job execution conditions (JEC) to a remote message queue (MQT); automatically starting the session computing engine (Actor) after the web service is started and subscribing to the remote message queue topic (MQT); and simultaneously starting a job executor (JER), which continuously polls the job queue and calls a specific engine for processing. Calling a multi-computing engine-based online analysis interface for massive data not only improves data processing efficiency and computing reliability, but also enhances system flexibility and scalability, and helps reduce system maintenance costs.

The present application relates to a method, device, computer equipment, storage medium and computer program product for job scheduling. The method comprises: obtaining a scheduling request for a job, determining the number of threads corresponding to the job in response to the scheduling request, and starting the threads required for processing the job according to the number of threads; processing the job through the thread, and obtaining the job status of the job in the process of processing the job; when there is a timeout job in the job whose job status is a timeout state, releasing the thread matching the timeout job from the thread according to the timeout job, and adding the timeout job back to the job queue; when the processing time of the timeout job arrives, repeating the above steps to process the timeout job until the job is processed. The method can dynamically adjust the thread according to the number of threads required for the job in the process of processing the job, release the thread of the timeout job, improve the thread execution efficiency, and thus improve the efficiency of the enterprise system in processing jobs.

Methods and systems for processing requests with load-dependent throttling. The system compares a count of active job requests being currently processed for a user associated with a new job request with an active job cap number for that user. When the count of active job requests being currently processed for that user does not exceed the active job cap number specific to that user, the job request is added to an active job queue for processing. However, when the count of active job requests being currently processed for that user exceeds the active job cap number, the job request is placed on a throttled queue to await later processing when an updated count of active job requests being currently processed for that user is below the active job cap number. Once the count is below the cap, the throttle request is moved to the active job queue for processing.

The invention discloses a parallel greedy genetic algorithm for cluster job scheduling, which comprises the following steps that: a main scheduler firstly receives jobs and initially arranges the jobs by using a backfill strategy, then encodes the jobs into DAG chromosomes, generates a plurality of initial individuals according to a greedy heuristic form, performs selection, crossover, variation and elitist retention on populations by the genetic algorithm, and iteratively optimizes a scheduling scheme; under the master-slave parallel architecture, a master process is responsible for job preprocessing and population generation, and slave processes complete genetic computation in parallel and update a job queue in real time. The method considers completion time, load balance and resource utilization rate, and can significantly improve cluster scheduling efficiency.

The invention discloses a job scheduling method and system in the technical field of high-performance computing cluster resource management, and aims to solve the problems that in the prior art, if multiple different granularity scheduling requirements need to be met, multiple sets of clusters need to be deployed, each cluster selects one scheduling strategy for deployment, different cluster resources cannot communicate with one another, and the scheduling efficiency is high. And independent maintenance is needed. The method comprises the following steps: acquiring a resource use state of each node, and constructing a global resource view according to the resource use state of each node; obtaining a job queue state, and determining a working strategy of each node according to the global resource view and the job queue state; obtaining an operation state and a current strategy of each node; according to the method, the same cluster can support two resource allocation granularities at the same time, meanwhile, the working strategy of each node can be determined and switched according to the global resource view and the job queue state, and allocation of node resources is optimized.

The invention discloses a multi-queue job processing method and system and a nonvolatile storage medium. The method comprises the steps that queue state information of all sub-jobs located in multiple job queues is obtained, the sub-jobs located in the different job queues belong to a complete job to be executed, each sub-job comprises one or more tasks to be executed, and the queue state information comprises an initial queuing sequence and resource information; based on the queue state information, respectively determining a target queuing sequence of each sub-job in the plurality of job queues; based on the target queuing sequence, a sorting instruction is issued to the controllers corresponding to the multiple job queues, and the sorting instruction is used for sorting the multiple sub-jobs according to the target queuing sequence. According to the method and the device, the technical problem of resource interlocking among multiple queues caused by the fact that the sub-jobs of the multiple queues cannot be queued cooperatively is solved.

Systems, methods, and non-transitory computer readable media are provided for managing assignment of modules. A job from a client may be received. The job may be inserted into a job queue. The job within the job queue may be compared with a set of cooldown modules to determine whether a compatible cooldown module is available. The job may be assigned to the compatible cooldown module responsive to the compatible cooldown module being available.

This invention discloses a method and apparatus for managing job groups in a print queue. Step S1: Job selection – Select one or more jobs from the existing print queue as members of a group; Step S2: Create a job group – Based on the selection in Step S1, create a new job group from these jobs; simultaneously create job group information; Step S3: Group editing – Edit the job group, modifying its information; Step S4: Group printing – Select the job group for printing. This invention can create separate job groups for some jobs and arrange these job groups within the entire job queue for normal printing. Furthermore, within each job group, parameters can be individually modified, improving the convenience of job adjustments.

The present application proposes a method and device for processing small data volume jobs based on a big data engine, which method includes the following steps: S1, obtaining a data file to be processed, and determining whether the data file to be processed is a small data volume job. If so, executing step S2; S2, determining whether the resource pool has an idle runtime environment. If so, the business code is uniformly encapsulated, and the runtime environment interprets and executes the encapsulated code. If not, executing step S3; S3, determining whether the resource pool has reached the resource capacity upper limit. If so, the data file is processed and enters the job queue to wait. If not, a runtime environment is created and put into the resource pool, and the business code is uniformly encapsulated, and the runtime environment interprets and executes the encapsulated code. S4, detecting whether the idle time of the runtime environment in the resource pool exceeds a preset threshold. If so, the runtime resources are released.

The invention relates to the technical field of artificial intelligence, and discloses a distributed training cluster job scheduling system which comprises a queue management service, a job queue, a job deployment management service, a job analysis service, a job termination management service and a fault-tolerant service. The number of the job queues is multiple. According to the method, a plurality of queues are managed for a large-scale distributed AI training scene, the training jobs are analyzed, corresponding processing is adaptively executed, the training jobs are reasonably scheduled, it is ensured that the training jobs have high availability and reliability, and the training efficiency and the training effect can also be ensured when a large number of training jobs exist.

The invention relates to the technical field of cluster testing, and discloses a test platform of a distributed training cluster, which comprises a test case service, a test plan service, a test plan instance service, a test job service, a job queue service, a test execution service and a test result service. According to the method, the target test plan containing the corresponding test case is constructed based on the test requirement of the user, multiple rounds of instantiation are carried out on the target test plan, and the corresponding test plan instance is executed; moreover, corresponding target test jobs are generated for the test plan instances, and the test jobs are subsequently executed in units, so that the test jobs can be isolated while a large number of parallel test plans are dealt with, and the possibility of passing the test jobs can be improved; the whole test process is divided into a plurality of services, the services can be matched with one another, it is guaranteed that test work can be accurately and orderly executed, and finally all test plans are completed.

A system and method for capturing heat energy are disclosed. The system comprises a central processing platform, at least one heating system comprising at least one computing unit, one or more Application Programming Interfaces (API) components on the central processing platform, and a job queue. The at least one computing unit has one or more computing nodes communicatively connected over a wired and / or wireless network and to the central processing platform. The at least one heating system is installed at a premise. The at least one heating system is configured to output heat generated from the respective at least one computing unit by the one or more computing nodes executing a processing job.

Technologies are provided for a multi-cloud bursting service. An example method can include receiving, via a cloud bursting service associated with different clouds, a cloud bursting configuration enabling the cloud bursting service for a local compute environment; based on the cloud bursting configuration, determining a number of jobs in a jobs queue associated with one or more cloud environments from the different clouds; determining a number of nodes available to process the number of jobs in the jobs queue; based on the number of jobs in the jobs queue and number of nodes available, determining whether to spin up a new node, take offline an existing node, or shutdown the existing node to yield a determination; and based on the determination and cloud bursting configuration, performing a cloud bursting action including spinning up the new node, taking offline the existing node, or shutting down the existing node.

The invention discloses a resource allocation method and device for a heterogeneous cluster, equipment and a storage medium. The method comprises the following steps: receiving resource request jobs sent by at least one system, and pre-queuing a plurality of resource request jobs to obtain a first job queue; wherein the resource request job comprises a GPU resource request job and / or a CPU resource request job; respectively determining a value index corresponding to each resource request job in the first job queue based on an analytic hierarchy process, and adjusting the first job queue based on the plurality of value indexes to obtain a second job queue; and determining allocable resources of the cluster, and performing resource allocation on the second job queue based on the allocable resources. And carrying out resource allocation by adopting a strategy of combining priority scheduling with resource matching. And the resource allocation efficiency and the cluster resource utilization rate are improved.