Job scheduling method, device, computer equipment, and storage medium

Abstract

The present application relates to a method, device, computer equipment, storage medium and computer program product for job scheduling. The method comprises: obtaining a scheduling request for a job, determining the number of threads corresponding to the job in response to the scheduling request, and starting the threads required for processing the job according to the number of threads; processing the job through the thread, and obtaining the job status of the job in the process of processing the job; when there is a timeout job in the job whose job status is a timeout state, releasing the thread matching the timeout job from the thread according to the timeout job, and adding the timeout job back to the job queue; when the processing time of the timeout job arrives, repeating the above steps to process the timeout job until the job is processed. The method can dynamically adjust the thread according to the number of threads required for the job in the process of processing the job, release the thread of the timeout job, improve the thread execution efficiency, and thus improve the efficiency of the enterprise system in processing jobs.

Description

Technical Field

[0001] The present application relates to the technical field of big data resource scheduling, and in particular to a job scheduling method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] As small or start-up businesses grow, the number of batch jobs processed by enterprise systems increases, and the cost of operating and maintaining enterprise systems increases. Therefore, in order to save costs, enterprises generally develop their own scheduling platforms in their enterprise systems to process batch jobs.

[0003] In traditional technology, the scheduling platform of an enterprise system usually writes a simple scheduled job processing business logic on each functional branch, and performs job scheduling and processing on the jobs on each functional branch through multiple scheduled job processing business logics. However, too many scheduled jobs on the scheduling platform can easily cause insufficient server memory, making it impossible for the server to create new threads, resulting in low efficiency in job processing by the enterprise system. Summary of the invention

[0004] Based on this, it is necessary to provide a job scheduling method, apparatus, computer equipment, computer-readable storage medium and computer program product that can improve system processing efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a job scheduling method. The method comprises:

[0006] Obtaining a scheduling request for a job, determining the number of threads corresponding to the job in response to the scheduling request, and starting threads required to process the job according to the number of threads;

[0007] Processing the job through the thread, and obtaining the job status of the job during the processing of the job;

[0008] When there is a timeout job in the job state, releasing a thread matching the timeout job from the thread according to the timeout job, and adding the timeout job back to the job queue;

[0009] When the processing time of the timed-out job arrives, the above steps are repeated to process the timed-out job until the job is processed.

[0010] In one embodiment, obtaining the job status of the job includes:

[0011] In the process of processing the job, generating a current state of the job according to the current time and the start time of processing the job, the current state including a new, processing, and timeout state;

[0012] A timeout scanning thread is called, and the current state of the job is obtained by scanning through the timeout scanning thread.

[0013] In one embodiment, the processing of the job by the thread and obtaining the job status of the job during the processing of the job include:

[0014] The job is scheduled from the job scheduling platform to the job system through the thread, and during the scheduling process, a scheduling start time for the thread to schedule the job and a scheduling completion time when the job system receives the job are determined;

[0015] The job status of the job is determined according to the scheduled start time and the scheduled completion time, and the job status includes a timeout status and a non-timeout status.

[0016] In one embodiment, obtaining a scheduling request for a job includes:

[0017] Obtaining a scheduling request for the job that is periodically triggered by the job scheduling platform;

[0018] or,

[0019] In response to a scheduling operation on the job triggered by a user through a business platform, a scheduling request for the job is obtained.

[0020] In one of the embodiments, after releasing the thread matching the timeout job from the thread according to the timeout job, the method further includes:

[0021] A job record status of the job is generated, where the job record status is timeout.

[0022] In one embodiment, after the processing of the job is completed, the method further includes:

[0023] The current job status of the job is obtained, and when the current job status of the job is not in a timeout state, a thread matching the job is released from the thread, and the job record status of the job is updated from timeout to success.

[0024] In one embodiment, the method further comprises:

[0025] Obtaining job configuration information of the job, wherein the job configuration information includes an initial number of threads;

[0026] When starting threads required for processing the job according to the number of threads, increasing the number of threads on the initial number of threads;

[0027] When the thread matching the timed-out job is released from the threads, the number of threads matching the timed-out job is reduced based on the current number of threads.

[0028] In a second aspect, the present application also provides a job scheduling device. The device comprises:

[0029] A scheduling request response module, used to obtain a scheduling request for a job, determine the number of threads corresponding to the job in response to the scheduling request, and start the threads required to process the job according to the number of threads;

[0030] A job status acquisition module, used to process the job through the thread, and acquire the job status of the job during the process of processing the job;

[0031] The timeout job processing module is used to release the thread matching the timeout job from the thread when there is a timeout job in the job, and add the timeout job back to the job queue; when the processing time of the timeout job arrives, repeat the above steps to process the timeout job until the job is processed.

[0032] In one embodiment, the job status acquisition module includes:

[0033] A job status generating unit, used to generate the current status of the job according to the current time and the start time of processing the job during the processing of the job, wherein the current status includes a new, processing, and timeout status;

[0034] The timeout thread scanning unit is used to call the timeout scanning thread and obtain the current state of the job through scanning by the timeout scanning thread.

[0035] In one embodiment, the job status acquisition module includes:

[0036] A scheduling time determination unit, used to schedule the job from the job scheduling platform to the job system through the thread, and during the scheduling process, determine the scheduling start time for the thread to schedule the job and the scheduling completion time when the job system receives the job;

[0037] The job status determination unit is used to determine the job status of the job according to the scheduled start time and the scheduled completion time, and the job status includes a timeout state and a non-timeout state.

[0038] In one embodiment, the scheduling request response module includes:

[0039] A platform triggering unit, used for obtaining a scheduling request for the job triggered regularly by the job scheduling platform;

[0040] The user triggering unit is used to obtain a scheduling request for the job in response to a scheduling operation on the job triggered by a user through a business platform.

[0041] In one embodiment, the timeout job processing module further includes:

[0042] The job record status generating unit is used to generate the job record status of the job after releasing the thread matching the timed-out job from the thread according to the timed-out job, wherein the job record status is timed-out.

[0043] In one embodiment, the device further comprises:

[0044] The job record status update module is used to obtain the current job status of the job after the job is processed. When the current job status of the job is not in the timeout state, the thread matching the job is released from the thread, and the job record status of the job is updated from timeout to success.

[0045] In one embodiment, the device further comprises:

[0046] A job configuration information acquisition module, used to acquire the job configuration information of the job, wherein the job configuration information includes an initial number of threads;

[0047] An initial thread number changing module, used for increasing the number of threads on the initial thread number when starting threads required for processing the job according to the thread number;

[0048] When the thread matching the timed-out job is released from the threads, the number of threads matching the timed-out job is reduced based on the current number of threads.

[0049] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method for scheduling jobs described in any one of the embodiments of the first aspect is implemented.

[0050] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the job scheduling method described in any one of the embodiments of the first aspect is implemented.

[0051] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the job scheduling method described in any one of the embodiments of the first aspect is implemented.

[0052] The above-mentioned job scheduling method, device, computer equipment, storage medium and computer program product obtain a scheduling request for a job, determine the number of threads to start processing the job according to the number of threads corresponding to the job, process the job through the thread, obtain the job status of the job, and when the job status is a timeout state, release the thread matching the timeout job, add the timeout job back to the job queue, and when the processing time of the timeout job arrives, repeat the above steps to process the timeout job until the job is processed. In the process of processing the job, the thread can be dynamically adjusted according to the number of threads required for the job, the thread of the timeout job can be released, and the thread execution efficiency can be improved, thereby improving the efficiency of the enterprise system in processing jobs. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of a process flow of a job scheduling method in one embodiment;

[0054] Figure 2 A schematic diagram of a process for obtaining a job status step in an embodiment;

[0055] Figure 3 A schematic diagram of a flow chart of a step of obtaining a job status in another embodiment;

[0056] Figure 4 A schematic diagram of a flow chart of a step of updating the number of threads in one embodiment;

[0057] Figure 5a A schematic diagram of a flow chart of a job scheduling method in another embodiment;

[0058] Figure 5b A schematic diagram of a flow chart of a job scheduling method in another embodiment;

[0059] Figure 6 is a structural block diagram of a job scheduling device in one embodiment;

[0060] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] In one embodiment, Figure 1 As shown, a job scheduling method is provided. This embodiment uses the method applied to a server as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the above-mentioned terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart TVs, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0063] In this embodiment, the method includes the following steps:

[0064] Step S102, obtaining a scheduling request for the job, determining the number of threads corresponding to the job in response to the scheduling request, and starting threads required for processing the job according to the number of threads.

[0065] The scheduling request may be used to represent a request for scheduling a job to be processed. The number of threads may be used to represent the number of threads required to process the job. The number of threads may include a maximum number of threads and a current number of threads. The maximum number of threads may be used to represent the upper limit of threads set by the system. When the number of threads in the system exceeds the maximum number of threads, no new threads can be created. The current number of threads may be used to represent the number of threads existing in the system at the current moment.

[0066] Specifically, the server periodically obtains a scheduling request for a job (e.g., every twelve hours), and determines the number of threads required to process the job in response to the scheduling request. The server obtains the current number of threads, compares the current number of threads with the maximum number of threads, and when the current number of threads is less than the maximum number of threads, the server starts a corresponding number of threads to process the job according to the number of threads required to process the job. The server updates the current number of threads and the maximum number of threads according to the number of threads required to process the job. The scheduling request for the job can be manually triggered by the user on demand, such as the user manually clicking on the application interface to trigger the scheduling request; it can also be automatically triggered by the server, such as the server periodically obtaining multiple jobs and automatically triggering the scheduling request for the job.

[0067] Step S104, processing the job through the thread, and obtaining the job status of the job during the process of processing the job.

[0068] The job status can be generated according to the status of the thread processing the job, and the thread status can include but is not limited to at least one of newly created, pending, processing, ended, timed out, and not timed out. For example, when the thread status of the job processing is newly created, processing, or timed out, the job status of the generated job is timed out; when the thread status of the job processing is newly created, processing, or not timed out, the job status of the generated job is not timed out.

[0069] Specifically, the server schedules the jobs to be processed through threads. During the process of scheduling the jobs, the server obtains the job status of the jobs in real time.

[0070] Step S106: when there is a timeout job in the job state, a thread matching the timeout job is released from the thread according to the timeout job, and the timeout job is added to the job queue again.

[0071] The job queue can be used to sort and store multiple jobs to be processed, and the server can schedule and process the jobs according to the order of the jobs in the job queue.

[0072] Specifically, when the server detects that there is a timeout job with a job status of timeout, the thread processing the timeout job is released from the thread, and the timeout job is added to the job queue again. The server updates the current number of threads and the maximum number of threads according to the number of threads processing the timeout job.

[0073] Step S108, when the processing time of the timed-out job is reached, repeat the above steps to process the timed-out job until the job processing is completed.

[0074] Specifically, when the processing time of the timed-out job arrives (for example, the timed-out job is obtained from the job queue), the server processes the timed-out job according to the contents described in steps S102 to S106 until the job processing is completed.

[0075] In the above-mentioned job scheduling method, by obtaining a scheduling request for a job, the number of threads to start processing the job is determined according to the number of threads corresponding to the job, the job is processed by the thread, and the job status of the job is obtained. When the job status is a timeout state, the thread matching the timeout job is released, and the timeout job is added back to the job queue. When the processing time of the timeout job arrives, the above-mentioned steps are repeated to process the timeout job until the job is processed. In the process of processing the job, the thread can be dynamically adjusted according to the number of threads required for the job, the thread of the timeout job can be released, and the thread execution efficiency can be improved, thereby improving the efficiency of the enterprise system in processing jobs.

[0076] In one embodiment, Figure 2As shown, step S104, processing the job through the thread, in the process of processing the job, obtaining the job status of the job, including:

[0077] Step S202, during the process of processing the job, the current job status of the job is generated according to the current time and the start time of processing the job.

[0078] Step S204, calling the timeout scanning thread, and obtaining the current job status of the job through scanning by the timeout scanning thread.

[0079] The current job status may include but is not limited to one of a timeout status and a non-timeout status.

[0080] Specifically, in the process of processing the job, the server obtains the current time and the processing start time of the server processing the job, determines the time difference between the current time and the processing start time, compares the time difference with the preset timeout threshold, and when the time difference is greater than or equal to the timeout threshold, the current job state of the generated job is the timeout state; when the time difference is less than the timeout threshold, the current job state of the generated job is the non-timeout state. The server periodically calls the timeout scanning thread, scans the job through the timeout scanning thread, and obtains the current job state of the job.

[0081] In this embodiment, by calling the timeout scanning thread to scan the job and obtain the current status of the job, it is possible to implement regular monitoring of the current job status of the job being processed, promptly discover the timed jobs, and then promptly release the threads occupied by the timed jobs, thereby releasing system resources.

[0082] In one embodiment, Figure 3 As shown, step S104, processing the job through the thread, obtaining the job status of the job during the processing of the job, also includes:

[0083] Step S302, schedule the job from the job scheduling platform to the job system through the thread. During the scheduling process, determine the scheduling start time for the thread to schedule the job and the scheduling completion time when the job is received by the job system.

[0084] Step S304, determining the job status of the job according to the scheduled start time and the scheduled completion time.

[0085] The job scheduling platform can be used to schedule jobs, and the job system can be used to execute jobs.

[0086] Specifically, the server schedules the job from the job scheduling platform to the job system through a thread. During the scheduling process, the scheduling start time of the thread scheduling the job and the scheduling completion time when the job system receives the job are determined. The server determines the scheduling time difference based on the scheduling start time and the scheduling completion time, and compares the scheduling time difference with the timeout threshold. When the scheduling time difference is less than the timeout threshold, the job status of the job is determined to be in the non-timeout state; when the scheduling time difference is greater than or equal to the timeout threshold, the job status of the job is determined to be in the timeout state.

[0087] In this embodiment, by obtaining the scheduling start time and the scheduling completion time, the current status of the job is determined, so that the job status of the scheduled job can be monitored; by scheduling the job to the job system, asynchronous processing of the job can be achieved, and the processing efficiency of the job can also be accelerated.

[0088] In one embodiment, obtaining a scheduling request for a job may be implemented in any of the following ways:

[0089] (1) Obtain the scheduling request for the job triggered by the job scheduling platform at a fixed time.

[0090] Specifically, the scheduling request for the job obtained by the server may be actively triggered by the job scheduling platform at a scheduled time, that is, when the job scheduling platform detects that the current time reaches a pre-configured scheduled time, a scheduling request is automatically generated.

[0091] (2) In response to a job scheduling operation triggered by a user through a business platform, a job scheduling request is obtained.

[0092] The business platform may be a platform corresponding to each business scenario, for example, a comprehensive business platform. Specifically, when the business platform detects a scheduling operation for a job triggered by a user, for example, detecting that a user clicks a job processing button on the business platform, in response to the scheduling operation, a scheduling request for the job is sent to the server.

[0093] In this embodiment, the scheduling request for the job is triggered by two triggering methods: the job scheduling platform actively triggers the job scheduling request regularly and the user triggers the job through the business platform. This can improve the availability of the scheduling system and the flexibility of the corresponding business system in processing business.

[0094] In one embodiment, in step S106, when there is a timeout job in the job whose job status is timeout, a thread matching the timeout job is released from the thread according to the timeout job, and the timeout job is added back to the job queue. After that, it also includes: generating a job record status of the job, and the job record status is timeout.

[0095] The job record status may include but is not limited to timeout or success.

[0096] Specifically, after releasing the thread matching the timed-out job from the thread according to the timed-out job, the server generates a job record status of the job, and the job record status is timed-out. The job record status is stored in the job record table.

[0097] In one embodiment, in step S108, when the processing time of the timed-out job is reached, the above steps are repeated to process the timed-out job until the job is processed, and the process also includes: obtaining the current job status of the job, and when the current job status of the job is not in the timed-out state, releasing the thread matching the job from the thread, and updating the job record status of the job from timed-out to successful.

[0098] Specifically, after the job is processed, the server obtains the data returned by the asynchronous callback of the job system. The server queries the job record status of the job in the job record table according to the job identifier corresponding to the job. The server obtains the current job status of the job. When the current job status of the job is not in the timeout state, the server releases the thread that processes the job from the thread, and updates the job record status of the job from timeout to success. The server updates the current number of threads and the maximum number of threads according to the number of released threads; when the current job status of the job is in the timeout state, the server updates the job record status of the job to timeout, and adds the timed-out job to the job queue to wait for the next processing.

[0099] In this embodiment, by acquiring the current job status of the job after the job processing is completed and updating the job record status of the job according to the current job status, the execution efficiency of the thread can be improved.

[0100] In one embodiment, Figure 4 As shown, the method also includes:

[0101] Step S402, obtaining job configuration information of the job.

[0102] The job configuration information includes a job identifier (including but not limited to a job sequence identifier, an organization number identifier, a processing time identifier, and a processing identifier), and an initial number of threads. The initial number of threads may include but is not limited to an initial maximum number of threads and an initial current number of threads. The following table exemplifies the configuration information of the job identifier:

[0103] XH NUMBER(2) Job sequence number identification BRAND VARCHAR2(3) Institution ID DATATIME VARCHAR2(8) Processing time stamp ISPROC VARCHAR2(1) Whether to process the mark

[0104] Specifically, the server obtains multiple pending jobs, obtains job configuration information such as a job identifier, an initial maximum number of threads, and an initial current number of threads configured by a business person for each job, and groups the multiple pending jobs.

[0105] In one example, the server can determine the job that needs to be scheduled based on the job identifier of the job, obtain the file path, scheduling file date, incremental identification unit, partition identifier and other information corresponding to the job, and the server updates the next processing time of the job based on the above information.

[0106] Step S404, when starting threads required for processing the job according to the number of threads, the number of threads is increased on the basis of the initial number of threads.

[0107] Specifically, when the server starts threads according to the number of threads required to process a job, the server updates the maximum number of threads and the current number of threads, adds a number equal to the number of threads required to process the job to the initial maximum number of threads to obtain the maximum number of threads, and adds a number equal to the number of threads required to process the job to the initial current number of threads to obtain the current number of threads.

[0108] Step S406: when the thread matching the timed-out job is released from the threads, the number of threads matching the timed-out job is reduced from the current number of threads.

[0109] Specifically, when the server releases a thread that matches the timed-out job from the thread, the server updates the maximum number of threads and the current number of threads, reduces the number of threads that match the timed-out job from the current maximum number of threads to obtain the maximum number of threads, and reduces the number of threads that match the timed-out job from the current number of threads to obtain the current number of threads.

[0110] In this embodiment, by configuring the initial number of threads, the number of threads is updated in real time according to the threads required for processing the job, thereby improving the execution efficiency of the threads.

[0111] In one embodiment, Figure 5a As shown, a job scheduling method is provided, comprising:

[0112] Step S502: Obtain job configuration information of the job.

[0113] Specifically, the server obtains multiple pending jobs, obtains job configuration information such as a job identifier, an initial maximum number of threads, and an initial current number of threads configured by a business person for each job, and groups the multiple pending jobs.

[0114] Step S504, obtaining a scheduling request for the job, determining the number of threads corresponding to the job in response to the scheduling request, starting threads required for processing the job according to the number of threads, and updating the number of threads.

[0115] Specifically, the server obtains a scheduling request for a job, and in response to the scheduling request, determines the number of threads required to process the job. The server obtains the current number of threads, compares the current number of threads with the initial maximum number of threads, and when the current number of threads is less than the initial maximum number of threads, the server starts the corresponding number of threads required to process the job according to the number of threads required to process the job to process the job. The server updates the current number of threads and the maximum number of threads according to the number of threads required to process the job. Among them, the scheduling request for the job can be actively triggered by the job scheduling platform at a fixed time; or it can be triggered by the user clicking the corresponding job processing button on the business platform when he finds that there is a job that needs to be done. The specific operation of updating the number of threads can be implemented with reference to the method provided in the above embodiment, and will not be elaborated here.

[0116] Step S506, processing the job through the thread, and obtaining the job status of the job during the process of processing the job.

[0117] Specifically, the server schedules the job to the job system for processing through a thread, and obtains the job status of the job during the job processing. The job status can be obtained by calling a timeout scanning thread, or can be determined based on the scheduling start time and the scheduling completion time. The specific job status acquisition operation can be implemented with reference to the method in the above embodiment, and will not be elaborated here.

[0118] Step S508, when there is a timeout job in the job, release the thread from the thread according to the timeout job, add the timeout job back to the job queue, update the number of threads, and generate the job record status of the job.

[0119] Step S510, when the processing time of the timed-out job is reached, repeat the above steps to process the timed-out job until the job is processed.

[0120] Specifically, when the server determines that there is a job with a timeout status, it releases the thread from the thread according to the timeout job, adds the timeout job back to the job queue, updates the number of threads, and generates a job record status of the job as timeout. When the processing time of the timeout job arrives, the server processes the timeout job according to the contents described in steps S502 to S508 until all jobs are processed. The specific thread release operation and thread number update operation can be implemented by referring to the method provided in the above embodiment, and will not be elaborated here.

[0121] Step S512, obtaining the current job status of the job, and updating the job record status of the job.

[0122] Specifically, when the job is processed, the job system asynchronously calls back to return data to the job scheduling system. The server queries the job record status of the job in the job record table according to the sequence number corresponding to the job. The server obtains the current job status of the job and updates the job record status of the job according to the current job status. The specific job record status update operation can be implemented by referring to the method provided in the above embodiment, which will not be elaborated here.

[0123] In one example, a batch processing scheduling framework (Spring-batch) is combined with Java (computer programming language) technology to generate a job scheduling platform, and an open source unified scheduling framework (quartz) is used for scheduled job scheduling.

[0124] In an example, Figure 5b As shown, an interaction mode among a job scheduling platform, a job system and a business platform is provided.

[0125] In this embodiment, by obtaining the job configuration information of the job, obtaining the scheduling request for the job, determining the number of threads to start processing the job according to the number of threads corresponding to the job, processing the job through the thread, obtaining the job status of the job, and when the job status is a timeout state, releasing the thread matching the timeout job, adding the timeout job back to the job queue, and when the processing time of the timeout job arrives, repeating the above steps to process the timeout job until the job processing is completed, the job can be uniformly scheduled and managed through parameterized job configuration information, greatly improving the job executableness and processing efficiency; in the process of processing the job, by obtaining the job status of the job, dynamically adjusting the number of threads according to the number of threads required for the job, releasing the thread of the timeout job, and improving the thread execution efficiency; by providing a user-triggerable job re-do operation, the flexibility of the corresponding business system in processing business can be improved; by adopting the scheduling framework quartz to implement timed scheduling, the availability of the scheduling system can be improved; thereby improving the efficiency of the enterprise system in processing jobs.

[0126] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0127] Based on the same inventive concept, the embodiment of the present application also provides a job scheduling device for implementing the job scheduling method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more job scheduling device embodiments provided below can refer to the limitations on the job scheduling method above, and will not be repeated here.

[0128] In one embodiment, Figure 6 As shown, a job scheduling device 600 is provided, including: a scheduling request response module 602, a job status acquisition module 604 and a timeout job processing module 606, wherein:

[0129] The scheduling request response module 602 is used to obtain a scheduling request for a job, determine the number of threads corresponding to the job in response to the scheduling request, and start threads required to process the job according to the number of threads.

[0130] The job status acquisition module 604 is used to process the job through a thread, and acquire the job status of the job during the process of processing the job.

[0131] The timeout job processing module 606 is used to release a thread matching the timeout job from the thread when there is a timeout job in the job, and add the timeout job back to the job queue; when the processing time of the timeout job arrives, repeat the above steps to process the timeout job until the job processing is completed.

[0132] In one embodiment, the job status acquisition module 604 includes: a job status generation unit, which is used to generate the current job status of the job according to the current time and the start time of processing the job during the process of processing the job, and the current job status includes a timeout status and a non-timeout status; a timeout thread scanning unit, which is used to call the timeout scanning thread and obtain the current status of the job through the timeout scanning thread scanning.

[0133] In one embodiment, the job status acquisition module 604 also includes: a scheduling time determination unit, which is used to schedule the job from the job scheduling platform to the job system through a thread, and during the scheduling process, determines the scheduling start time for the thread to schedule the job, and the scheduling completion time when the job system receives the job; a job status determination unit, which is used to determine the job status of the job based on the scheduling start time and the scheduling completion time, and the job status includes a timeout status and a non-timeout status.

[0134] In one embodiment, the scheduling request response module 602 includes: a platform trigger unit, which is used to obtain a scheduling request for a job triggered periodically by a job scheduling platform; and a user trigger unit, which is used to obtain a scheduling request for a job in response to a scheduling operation for a job triggered by a user through a business platform.

[0135] In one embodiment, the timeout job processing module 606 includes: a job record status generating unit, which is used to generate a job record status of the job after releasing the thread matching the timeout job from the thread according to the timeout job, and the job record status is timeout.

[0136] In one embodiment, the job scheduling device 600 also includes: a job record status update module, which is used to obtain the current job status of the job after the job is processed. When the current job status of the job is a success status, the thread matching the job is released from the thread, and the job record status of the job is updated from timeout to success.

[0137] In one embodiment, the job scheduling device 600 also includes: a job configuration information acquisition module, which is used to obtain the job configuration information of the job, the job configuration information includes the initial number of threads; an initial thread number change module, which is used to increase the number of threads on the initial number of threads when starting the threads required to process the job according to the number of threads; when releasing the thread matching the timed-out job from the thread, reducing the number of threads matching the timed-out job on the current number of threads.

[0138] Each module in the above-mentioned job scheduling device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0139] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a job scheduling method is implemented.

[0140] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0141] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0142] In one embodiment, a computer-readable storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0143] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A job scheduling method, characterized in that: The method comprises: Obtaining a scheduling request for a job, determining the number of threads corresponding to the job in response to the scheduling request, and starting threads required to process the job according to the number of threads; Processing the job through the thread, and obtaining the job status of the job during the processing of the job; When there is a timeout job in the job state, releasing a thread matching the timeout job from the thread according to the timeout job, and adding the timeout job back to the job queue; When the processing time of the timed-out job arrives, the above steps are repeated to process the timed-out job until the job is processed.

2. The method according to claim 1, characterized in that The obtaining the job status of the job includes: In the process of processing the job, generating a current job state of the job according to the current time and the processing start time of the job, wherein the current job state includes a timeout state and a non-timeout state; A timeout scanning thread is called, and the current job status of the job is obtained by scanning through the timeout scanning thread.

3. The method according to claim 1, characterized in that The processing of the job by the thread and obtaining the job status of the job during the processing of the job includes: The job is scheduled from the job scheduling platform to the job system through the thread, and during the scheduling process, a scheduling start time for the thread to schedule the job and a scheduling completion time when the job system receives the job are determined; The job status of the job is determined according to the scheduled start time and the scheduled completion time, and the job status includes a timeout status and a non-timeout status.

4. The method according to claim 3, characterized in that: The obtaining of the scheduling request for the job includes: Obtaining a scheduling request for the job that is periodically triggered by the job scheduling platform; or, In response to a scheduling operation on the job triggered by a user through a business platform, a scheduling request for the job is obtained.

5. The method according to claim 1, characterized in that After releasing the thread matching the timed-out job from the thread according to the timed-out job, the method further includes: A job record status of the job is generated, where the job record status is timeout.

6. The method according to claim 5, characterized in that After the operation is processed, the method further includes: The current job status of the job is obtained, and when the current job status of the job is not in a timeout state, a thread matching the job is released from the thread, and the job record status of the job is updated from timeout to success.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtaining job configuration information of the job, wherein the job configuration information includes an initial number of threads; When starting threads required for processing the job according to the number of threads, increasing the number of threads on the initial number of threads; When the thread matching the timed-out job is released from the threads, the number of threads matching the timed-out job is reduced based on the current number of threads.

8. A job scheduling device, characterized in that: The device comprises: A scheduling request response module, used to obtain a scheduling request for a job, determine the number of threads corresponding to the job in response to the scheduling request, and start the threads required to process the job according to the number of threads; A job status acquisition module, used to process the job through the thread, and acquire the job status of the job during the process of processing the job; The timeout job processing module is used to release the thread matching the timeout job from the thread when there is a timeout job in the job, and add the timeout job back to the job queue; when the processing time of the timeout job arrives, repeat the above steps to process the timeout job until the job is processed.

9. The device according to claim 8, characterized in that The job status acquisition module includes: A job status generating unit, used to generate a current job status of the job according to a current time and a processing start time of the job during processing of the job, wherein the current job status includes a timeout status and a non-timeout status; The timeout thread scanning unit is used to call the timeout scanning thread, and obtain the current job status of the job through the timeout scanning thread.

10. The device according to claim 8, characterized in that The job status acquisition module includes: A scheduling time determination unit, used to schedule the job from the job scheduling platform to the job system through the thread, and during the scheduling process, determine the scheduling start time for the thread to schedule the job and the scheduling completion time when the job system receives the job; The job status determination unit is used to determine the job status of the job according to the scheduled start time and the scheduled completion time, and the job status includes a timeout state and a non-timeout state.

11. The device according to claim 10, characterized in that The scheduling request response module includes: A platform triggering unit, used for obtaining a scheduling request for the job triggered regularly by the job scheduling platform; The user triggering unit is used to obtain a scheduling request for the job in response to a scheduling operation on the job triggered by a user through a business platform.

12. The device according to claim 8, characterized in that The timeout job processing module also includes: The job record status generating unit is used to generate the job record status of the job after releasing the thread matching the timed-out job from the thread according to the timed-out job, wherein the job record status is timed-out.

13. The device according to claim 12, characterized in that The device also includes: The job record status update module is used to obtain the current job status of the job after the job is processed. When the current job status of the job is a success status, the thread matching the job is released from the thread, and the job record status of the job is updated from timeout to success.

14. The device according to any one of claims 8 to 13, characterized in that: The device also includes: A job configuration information acquisition module, used to acquire the job configuration information of the job, wherein the job configuration information includes an initial number of threads; An initial thread number changing module, used for increasing the number of threads on the initial thread number when starting threads required for processing the job according to the thread number; When the thread matching the timed-out job is released from the threads, the number of threads matching the timed-out job is reduced based on the current number of threads.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images