Scheduling job grading method and device, equipment, medium and program product

Abstract

The embodiment of the invention discloses a scheduling job grading method and device, equipment, a medium and a program product, and relates to the field of big data, and the method comprises the steps: inputting workload data, resource data and target operation data corresponding to a target server into a pre-constructed job grading model for processing, obtaining a grading result of each target scheduling job corresponding to the target server; executing each target scheduling job according to the grading result to obtain a job execution result, and updating a pre-constructed job grading model according to the job execution result to obtain an updated job grading model; when a to-be-executed job is detected, determining a grading result of the to-be-executed job through the updated job grading model; and after the to-be-executed job is taken as the target scheduling job and the updated job grading model is taken as the pre-constructed job grading model, returning to execute the operation of executing each target scheduling job according to the grading result until the to-be-executed job is not detected, so that the execution efficiency of the scheduling job is improved.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a scheduling job grading method, device, equipment, medium and program product. Background Art

[0002] At present, with the gradual increase in business scenarios, the number of jobs requesting servers is growing rapidly. Therefore, how to reasonably schedule jobs has become an urgent problem to be solved.

[0003] In the prior art, a hierarchical strategy for scheduling jobs is usually formulated according to fixed rules, and the hierarchical strategy for scheduling jobs is always used to allocate and run each scheduling job.

[0004] However, the existing technology cannot dynamically adjust the grading strategy of the scheduling job, resulting in poor flexibility of the grading strategy of the scheduling job and low execution efficiency of the scheduling job. Summary of the invention

[0005] The present invention provides a scheduling job grading method, device, equipment, medium and program product, which can realize a scheduling job grading method that dynamically updates the scheduling job, improves the practicality of the scheduling job grading results, and improves the execution efficiency of each scheduling job.

[0006] In a first aspect, an embodiment of the present invention provides a method for scheduling job classification, including:

[0007] Acquire workload data, resource data, and job running data corresponding to the target server, wherein the job running data is target running data of each target scheduling job corresponding to the target server;

[0008] Input the workload data, resource data and target operation data into the pre-built job classification model for processing to obtain the classification results of each target scheduling job;

[0009] Execute each target scheduling job according to the classification result, obtain the job execution result corresponding to each target scheduling job, and input the job execution result into the pre-built job classification model for processing to obtain an updated job classification model;

[0010] If it is detected that there are pending jobs in each target scheduled job, the current running data corresponding to the pending job, as well as the updated workload data and updated resource data corresponding to the target server are obtained, and the current running data, the updated workload data and the updated resource data are input into the updated job classification model for processing to obtain the classification result corresponding to the pending job;

[0011] After taking the to-be-executed job as the target scheduling job and taking the updated job classification model as the pre-built job classification model, the operation of executing each target scheduling job according to the classification result is returned until no to-be-executed job is detected in each target scheduling job.

[0012] Optionally, execute each target scheduling job according to the grading results to obtain job execution results corresponding to each target scheduling job, including: obtaining the first level in the grading results as the target level, and obtaining the target scheduling job belonging to the target level according to the grading results; after executing the target scheduling job belonging to the target level and obtaining the job execution results corresponding to the target scheduling job belonging to the target level, take the next level of the target level as the new target level; return to execute the operation of obtaining the target scheduling job belonging to the target level according to the grading results until the conditions for stopping the job execution are met.

[0013] Optionally, after obtaining the target scheduling job belonging to the target level according to the grading results, it also includes: expanding or reducing the capacity of the target server according to the workload data of the target server and the number of target scheduling jobs belonging to the target level; wherein the workload data includes at least the number of concurrent jobs, and the number of concurrent jobs is used to reflect the number of jobs that the target server can process simultaneously.

[0014] Optionally, the target server is expanded or reduced in capacity based on the workload data of the target server and the target number of scheduled jobs belonging to the target level, including: if the concurrent number of jobs of the target server is less than or equal to the target number of scheduled jobs belonging to the target level, the target server is expanded in capacity; if the concurrent number of jobs of the target server is greater than the target number of scheduled jobs belonging to the target level, the target server is reduced in capacity.

[0015] Optionally, workload data, resource data and job running data corresponding to the target server are obtained, including: after determining the target process corresponding to the target server, collecting the workload data corresponding to the target server through the first thread of the target process, collecting the resource data corresponding to the target server through the second thread of the target process, and collecting the job running data corresponding to the target server through the third thread of the target process.

[0016] Optionally, the workload data, resource data and target operation data are input into a pre-built job grading model for processing to obtain the grading results of each target scheduling job, including: inputting the workload data, resource data, target operation data and a preset number of levels into a pre-built job grading model for processing to obtain the grading results of each target scheduling job.

[0017] In a second aspect, an embodiment of the present invention further provides a scheduling job classification device, including:

[0018] A resource acquisition module, used to acquire workload data, resource data and job running data corresponding to the target server, wherein the job running data is target running data of each target scheduling job corresponding to the target server;

[0019] A first job classification module is used to input workload data, resource data and target operation data into a pre-built job classification model for processing to obtain a classification result of each target scheduling job;

[0020] A hierarchical model updating module is used to execute each target scheduling job according to the hierarchical results, obtain the job execution results corresponding to each target scheduling job, and input the job execution results into a pre-built job hierarchical model for processing to obtain an updated job hierarchical model;

[0021] The second job classification module is used for obtaining the current operation data corresponding to the job to be executed, and the updated workload data and the updated resource data corresponding to the target server if it is detected that there is a job to be executed in each target scheduled job, and inputting the current operation data, the updated workload data and the updated resource data into the updated job classification model for processing to obtain the classification result corresponding to the job to be executed;

[0022] The loop processing module is used to return to execute the operation of executing each target scheduling job according to the classification result after taking the to-be-executed job as the target scheduling job and the updated job classification model as the pre-built job classification model, until no to-be-executed job is detected in each target scheduling job.

[0023] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0024] at least one processor; and

[0025] a memory communicatively connected to at least one processor; wherein,

[0026] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the scheduling job grading method provided by any embodiment of the present invention.

[0027] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a processor to implement the scheduling job grading method provided by any embodiment of the present invention when executed.

[0028] The technical solution of the embodiment of the present invention can realize a method for dynamically updating the grading of scheduled jobs by inputting the job execution results into a pre-built job grading model for processing, obtaining an updated job grading model, and grading the to-be-executed jobs in the target scheduled job according to the updated job grading model, thereby avoiding the situation in which the prior art formulates the grading strategy of the scheduled job according to fixed rules, resulting in the inability to dynamically adjust the grading strategy of the scheduled job, resulting in low execution efficiency of the scheduled job, improving the practicability of the grading results of the scheduled job and improving the execution efficiency of each scheduled job.

[0029] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 is a flowchart of a scheduling job classification method provided according to Embodiment 1 of the present invention;

[0032] Figure 2 is a flowchart of another scheduling job classification method provided according to Embodiment 2 of the present invention;

[0033] Figure 3 is a schematic diagram of the structure of a scheduling job classification device provided according to Embodiment 3 of the present invention;

[0034] Figure 4 It is a schematic diagram of the structure of an electronic device provided by Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Embodiment 1

[0038] Figure 1 This is a flowchart of a scheduling job grading method provided according to Example 1 of the present invention. This embodiment is applicable to the situation where scheduling jobs are graded. The method can be executed by a scheduling job grading device. The scheduling job grading device can be implemented in the form of hardware and / or software. The scheduling job grading device can be configured in an electronic device such as a computer.

[0039] like Figure 1 As shown, a scheduling job classification method disclosed in this embodiment includes:

[0040] S110. Obtain workload data, resource data, and job running data corresponding to the target server, wherein the job running data is target running data of each target scheduling job corresponding to the target server.

[0041] In this embodiment, the workload data may include the number of concurrent jobs, the number of queued jobs, and the number of executable jobs. The resource data may be used to reflect the occupancy of the central processing unit, memory, and input / output devices on the target server, and the input / output devices may include disk input / output devices and network input / output devices. The target operation data includes, but is not limited to, the type, priority, and number of each target scheduling job.

[0042] In this step, specifically, multiple target processes may be used to concurrently collect workload data, resource data, and job running data corresponding to each target server, wherein the number of target processes is the same as the number of target servers.

[0043] S120: Input the workload data, resource data and target operation data into a pre-built job classification model for processing to obtain a classification result for each target scheduling job.

[0044] In this step, specifically, the pre-built job classification model can be used to classify each target scheduling job corresponding to the target server from top to bottom according to the workload data, resource data and target operation data, so as to obtain the classification results of each target scheduling job. The job classification model can be obtained by training a preset neural network model using the historical workload data, historical resource data and historical job operation data corresponding to the target server as training samples.

[0045] S130, executing each target scheduling job according to the classification result, obtaining the job execution result corresponding to each target scheduling job, and inputting the job execution result into the pre-built job classification model for processing to obtain an updated job classification model.

[0046] In this embodiment, the job execution result can be used to reflect whether the target scheduled job has been executed. If the target scheduled job has been executed, its job execution result also includes the job running effect, execution time, and the amount of resources used. The job running effect can include success and failure.

[0047] In this step, specifically, the target scheduling jobs under each level can be executed in sequence according to the classification results to obtain the job execution results corresponding to each target scheduling job. Among them, the execution order of each target scheduling job is related to the level to which it belongs. For example, the lower the level to which the target scheduling job belongs, the higher the execution order of the target scheduling job. Then, the pre-built job classification model can be reinforced learning trained according to the job execution results to obtain an updated job classification model.

[0048] Exemplarily, assuming that the levels to which each target scheduling job belongs include the first level, the second level and the third level, the target scheduling job belonging to the first level can be executed first, then the target scheduling job belonging to the second level can be executed, and finally the target scheduling job belonging to the third level can be executed.

[0049] S140. If it is detected that there are pending jobs in each target scheduling job, the current running data corresponding to the pending job and the updated workload data and updated resource data corresponding to the target server are obtained, and the current running data, updated workload data and updated resource data are input into the updated job grading model for processing to obtain the grading result corresponding to the pending job.

[0050] In this embodiment, the jobs to be executed may include target scheduled jobs that have failed to execute and target scheduled jobs that have not been executed. The current running data includes but is not limited to the type, priority and quantity of each job to be executed.

[0051] In this step, specifically, the updated job classification model can be used to classify each pending job corresponding to the target server from top to bottom according to the current running data, updated workload data and updated resource data to obtain the classification results of each pending job.

[0052] S150. After taking the to-be-executed job as the target scheduling job and taking the updated job classification model as the pre-built job classification model, return to execute S130 until no to-be-executed job is detected in each target scheduling job.

[0053] In this step, specifically, after taking the to-be-executed job as the target scheduling job and the updated job grading model as the pre-built job grading model, return to execute the operation of executing each target scheduling job according to the grading result until there is no to-be-executed job in each target scheduling job.

[0054] The technical solution of this embodiment is to obtain workload data, resource data and job running data corresponding to the target server, wherein the job running data is the target running data of each target scheduling job corresponding to the target server; input the workload data, resource data and target running data into a pre-built job grading model for processing to obtain a grading result of each target scheduling job; execute each target scheduling job according to the grading result to obtain a job execution result corresponding to each target scheduling job, and input the job execution result into a pre-built job grading model for processing to obtain an updated job grading model; if it is detected that there is a job to be executed in each target scheduling job, obtain the current running data corresponding to the job to be executed and the updated workload data corresponding to the target server. and update resource data, and input the current running data, updated workload data and updated resource data into the updated job grading model for processing, and obtain the grading result corresponding to the job to be executed; after taking the job to be executed as the target scheduling job and taking the updated job grading model as the pre-built job grading model, return to execute the operation of executing each target scheduling job according to the grading result until the technical means of not detecting the existence of the job to be executed in each target scheduling job solves the problem that the prior art formulates the grading strategy of scheduling jobs according to fixed rules, resulting in the inability to dynamically adjust the grading strategy of scheduling jobs, resulting in low execution efficiency of scheduling jobs, and can realize the dynamic update of the grading method of scheduling jobs, improve the practicality of the grading results of scheduling jobs, and improve the execution efficiency of each scheduling job.

[0055] Embodiment 2

[0056] Figure 2It is a flowchart of another scheduling job classification method provided according to the second embodiment of the present invention. This embodiment is based on the further optimization and expansion of the above embodiments and can be combined with various optional technical solutions in the above implementation methods.

[0057] like Figure 2 As shown, a scheduling job classification method disclosed in this embodiment includes:

[0058] S210. Obtain workload data, resource data, and job running data corresponding to the target server, wherein the job running data is target running data of each target scheduling job corresponding to the target server.

[0059] Optionally, workload data, resource data and job running data corresponding to the target server are obtained, including: after determining the target process corresponding to the target server, collecting the workload data corresponding to the target server through the first thread of the target process, collecting the resource data corresponding to the target server through the second thread of the target process, and collecting the job running data corresponding to the target server through the third thread of the target process.

[0060] Specifically, the workload data collected by the first thread, the resource data collected by the second thread, and the job running data collected by the third thread can be cached in a pre-built resource pool. The first thread, the second thread, and the third thread can concurrently execute data collection operations.

[0061] The advantage of this setting is that after determining the target processes corresponding to each target server, multiple threads under the same target process are used to collect different data corresponding to the same target server. This not only realizes the concurrent collection of data from different servers, but also realizes the concurrent collection of data corresponding to the same server, greatly improving the data collection efficiency and providing a basis for timely determining and updating the level to which each target scheduling job belongs.

[0062] S220: Input the workload data, resource data and target operation data into a pre-built job classification model for processing to obtain classification results for each target scheduling job.

[0063] Optionally, the workload data, resource data, and target operation data are input into a pre-built job classification model for processing to obtain the classification results of each target scheduling job, including: inputting the workload data, resource data, target operation data, and a preset number of levels into a pre-built job classification model for processing to obtain the classification results of each target scheduling job. The preset level data is used to reflect the number of levels into which each target scheduling job can be divided.

[0064] The advantage of this setting is that by inputting workload data, resource data, target operation data and preset number of levels into a pre-built job classification model for processing, the classification results of each target scheduling job are obtained, avoiding dividing each target scheduling job into too many or too few levels, and improving the practicality of the scheduling job classification results.

[0065] S230: Obtain the first level in the classification result as the target level.

[0066] S240. Obtain target scheduling tasks belonging to the target level according to the classification result.

[0067] S250: Expand or reduce the capacity of the target server according to the workload data of the target server and the number of target scheduling jobs belonging to the target level.

[0068] In this embodiment, the workload data includes at least the number of concurrent jobs, and the number of concurrent jobs is used to reflect the number of jobs that can be processed simultaneously by the target server.

[0069] In this step, specifically, the target server may be expanded or reduced in capacity according to a comparison result between the number of concurrent jobs and the number of target scheduled jobs belonging to the target level.

[0070] The advantage of this setting is that by expanding or reducing the capacity of the target server before executing the target scheduling job at each level, the execution efficiency of the scheduling job can be improved while avoiding resource waste.

[0071] Optionally, the target server is expanded or reduced in capacity based on the workload data of the target server and the target number of scheduled jobs belonging to the target level, including: if the concurrent number of jobs of the target server is less than or equal to the target number of scheduled jobs belonging to the target level, the target server is expanded in capacity; if the concurrent number of jobs of the target server is greater than the target number of scheduled jobs belonging to the target level, the target server is reduced in capacity.

[0072] The advantage of this setting is that by reducing the capacity of the target server when the number of target scheduling jobs under the target level is small, it is possible to avoid invalid occupation of resources and improve resource utilization. Secondly, by expanding the capacity of the target server when the number of target scheduling jobs under the target level is large, it is possible to avoid the situation where the target scheduling jobs cannot be executed in time and improve the execution efficiency of the scheduling jobs.

[0073] S260. After executing the target scheduling job belonging to the target level, obtaining the job execution results corresponding to the target scheduling job belonging to the target level, and taking the next level of the target level as the new target level, return to execute S240 until the condition for stopping the job execution is met.

[0074] In this step, specifically, after executing the target scheduling job belonging to the target level, obtaining the job execution results respectively corresponding to the target scheduling job belonging to the target level, and taking the next level of the target level as the new target level, return to execute the operation of obtaining the target scheduling job belonging to the target level according to the grading result, until the condition for stopping the job execution is met. Among them, the condition for stopping the job execution may be that no unexecuted target scheduling job is detected, or force majeure factors such as hardware failure are detected. It should be noted that since the execution operation of each target scheduling job will be exited when force majeure factors such as hardware failure are detected, there may be unexecuted target scheduling jobs.

[0075] The advantage of this setting is that by executing each target scheduling job step by step, it can achieve rapid response to important jobs and effective allocation of resources, thereby improving user satisfaction.

[0076] S270: Input the job execution results into a pre-built job classification model for processing to obtain an updated job classification model.

[0077] S280. If it is detected that there are pending jobs in each target scheduling job, the current running data corresponding to the pending job and the updated workload data and updated resource data corresponding to the target server are obtained, and the current running data, updated workload data and updated resource data are input into the updated job grading model for processing to obtain the grading result corresponding to the pending job.

[0078] S290, after taking the to-be-executed job as the target scheduling job and taking the updated job classification model as the pre-built job classification model, return to execute S230 until no to-be-executed job is detected in each target scheduling job.

[0079] In this step, specifically, after taking the job to be executed as the target scheduling job and the updated job grading model as the pre-built job grading model, return to execute the operation of obtaining the first level in the grading result as the target level until no job to be executed is detected in each target scheduling job.

[0080] The technical solution of this embodiment obtains the classification results of each target scheduling job by inputting workload data, resource data, target operation data and preset number of levels into a pre-built job classification model for processing, thereby avoiding dividing each target scheduling job into too many or too few levels, and improving the practicality of the scheduling job classification results. Secondly, by expanding or shrinking the target server before executing the target scheduling job at each level, the execution efficiency of the scheduling job can be improved while avoiding resource waste.

[0081] In a preferred embodiment, a scheduling job classification system is provided that can implement the scheduling job classification method of any embodiment of the present invention. Specifically, the scheduling job classification system includes a real-time data acquisition subsystem, a job classification subsystem, a real-time optimization subsystem, a scheduling engine and a dynamic update subsystem. Among them, the real-time data acquisition subsystem can be used to collect workload data, resource data and job operation data corresponding to the target server. More specifically, after determining the target processes corresponding to each target server, multiple threads under the same target process can be used to collect different data corresponding to the same target server, and the data collected by the threads can be buffered into a pre-constructed data pool. Optionally, the real-time data acquisition subsystem can be horizontally expanded according to the workload data, resource data and job operation data corresponding to each target server to enhance the data acquisition capability of the real-time data acquisition subsystem.

[0082] The job classification subsystem includes a job classification model. The job classification model can be used to classify each target scheduling job corresponding to the target server according to the workload data, resource data, target operation data and preset number of levels corresponding to the target server, and obtain the level to which each target scheduling job belongs. Among them, the preset number of levels can be dynamically updated according to the job execution results, and the parameters of the job classification model can also be dynamically updated according to the job execution results.

[0083] The real-time optimization subsystem can be used to formulate the expansion and contraction strategy of the target server, as well as the allocation and operation strategy of the job, so as to execute the expansion and contraction strategy of the target server, as well as the allocation and operation strategy of the job through the cloud computing platform and the scheduling engine. The specific process of formulating the expansion and contraction strategy of the target server is as follows: according to the workload data of the target server and the number of target scheduling jobs belonging to the target level, the target server is expanded or reduced. Optionally, the real-time optimization subsystem can also be used to feed back the job execution results to the job classification model as samples for reinforcement learning training.

[0084] The scheduling engine can be used to parse the expansion and contraction strategies of the target servers formulated by the real-time optimization subsystem, and expand or contract the target servers according to the expansion and contraction strategies of the target servers after the analysis. The scheduling engine can also be used to parse the job allocation and operation strategies formulated by the real-time optimization subsystem, and execute each target scheduling job according to the analyzed job allocation and operation strategies. Specifically, according to the analyzed job allocation and operation strategies, the target scheduling jobs corresponding to each target server can be queued, and the target servers can be woken up by multiple processes concurrently to run the jobs. Optionally, when an idle server with no scheduled jobs running is detected, the idle server can be controlled to enter a sleep state before shrinking. The dynamic update subsystem can be used to obtain the job execution results fed back by the real-time optimization subsystem through subscription, and update the job grading model according to the job execution results.

[0085] The advantage of this setting is that the job classification model can be trained through reinforcement learning based on the job execution results, which can continuously generate job classification results that meet the actual situation to adapt to the changing workload and server status, improve the practicality of the job classification results, and improve the execution efficiency of the scheduled jobs. Secondly, by executing the expansion and contraction strategy of the target server and the allocation and operation strategy of the job through the cloud computing platform and the scheduling engine, the execution efficiency of the scheduled jobs can be further improved while improving resource utilization.

[0086] Embodiment 3

[0087] Figure 3 It is a structural diagram of a scheduling job grading device provided according to Example 3 of the present invention. This embodiment can be applied to the situation where scheduling jobs are graded and executed. The scheduling job grading device can be implemented in the form of hardware and / or software and can be configured in an electronic device such as a computer.

[0088] like Figure 3 As shown, the scheduling job classification device disclosed in this embodiment includes: a resource acquisition module 31, a first job classification module 32, a classification model update module 33, a second job classification module 34 and a loop processing module 35, wherein:

[0089] The resource acquisition module 31 is used to acquire workload data, resource data and job running data corresponding to the target server, wherein the job running data is target running data of each target scheduling job corresponding to the target server;

[0090] A first job classification module 32 is used to input workload data, resource data and target operation data into a pre-built job classification model for processing, and obtain classification results of each target scheduling job;

[0091] The hierarchical model updating module 33 is used to execute each target scheduling job according to the hierarchical result, obtain the job execution result corresponding to each target scheduling job, and input the job execution result into the pre-built job hierarchical model for processing to obtain an updated job hierarchical model;

[0092] The second job classification module 34 is used for obtaining the current operation data corresponding to the job to be executed, and the updated workload data and updated resource data corresponding to the target server if it is detected that there is a job to be executed in each target scheduled job, and inputting the current operation data, the updated workload data and the updated resource data into the updated job classification model for processing to obtain the classification result corresponding to the job to be executed;

[0093] The loop processing module 35 is used to return to execute the operation of executing each target scheduling job according to the grading result after taking the to-be-executed job as the target scheduling job and taking the updated job grading model as the pre-built job grading model, until no to-be-executed job is detected in each target scheduling job.

[0094] The technical solution in this embodiment, through the mutual cooperation of the resource acquisition module, the first job grading module, the grading model updating module, the second job grading module and the loop processing module, solves the problem that the prior art formulates the grading strategy of the scheduling job according to fixed rules, resulting in the inability to dynamically adjust the scheduling job grading strategy, resulting in low execution efficiency of the scheduling job. It can realize the dynamic update of the grading method of the scheduling job, improve the practicability of the scheduling job grading results, and improve the execution efficiency of each scheduling job.

[0095] Optionally, the hierarchical model updating module 33 is specifically used to: obtain the first level in the grading result as the target level, and obtain the target scheduling job belonging to the target level according to the grading result; after executing the target scheduling job belonging to the target level and obtaining the job execution results corresponding to the target scheduling job belonging to the target level, use the next level of the target level as the new target level; return to execute the operation of obtaining the target scheduling job belonging to the target level according to the grading result until the conditions for stopping the job execution are met.

[0096] Optionally, the device also includes a server management module, which is used to expand or reduce the capacity of the target server according to the workload data of the target server and the number of target scheduling jobs belonging to the target level; wherein the workload data includes at least the number of concurrent jobs, and the number of concurrent jobs is used to reflect the number of jobs that the target server can process simultaneously.

[0097] Optionally, the server management module is specifically used to: if the concurrent number of jobs of the target server is less than or equal to the target number of scheduled jobs belonging to the target level, then expand the capacity of the target server; if the concurrent number of jobs of the target server is greater than the target number of scheduled jobs belonging to the target level, then shrink the capacity of the target server.

[0098] Optionally, the resource acquisition module 31 is specifically used to: after determining the target process corresponding to the target server, collect the workload data corresponding to the target server through the first thread of the target process, collect the resource data corresponding to the target server through the second thread of the target process, and collect the job running data corresponding to the target server through the third thread of the target process.

[0099] Optionally, the first job classification module 32 is specifically used to: input workload data, resource data, target operation data and preset number of levels into a pre-built job classification model for processing to obtain a classification result for each target scheduling job.

[0100] The scheduling job classification device provided in the embodiment of the present invention can execute the scheduling job classification method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For the contents not described in detail in this embodiment, reference can be made to the description in any method embodiment of the present application.

[0101] Embodiment 4

[0102] Figure 4 FIG. 1 is a schematic diagram showing the structure of an electronic device 10 that can be used to implement an embodiment of the present invention. Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0103] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0104] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a scheduling job grading method.

[0105] In some embodiments, the scheduling job grading method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the scheduling job grading method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the scheduling job grading method in any other appropriate manner (e.g., by means of firmware).

[0106] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0107] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0108] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0109] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0110] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0111] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0113] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A scheduling job classification method, characterized in that: The method comprises: Acquire workload data, resource data, and job running data corresponding to the target server, wherein the job running data is target running data of each target scheduling job corresponding to the target server; Inputting the workload data, resource data and target operation data into a pre-built job classification model for processing to obtain a classification result for each target scheduling job; Execute each target scheduling job according to the classification result, obtain the job execution result corresponding to each target scheduling job, and input the job execution result into the pre-built job classification model for processing to obtain an updated job classification model; If it is detected that there are pending jobs in each target scheduled job, the current running data corresponding to the pending job, as well as the updated workload data and updated resource data corresponding to the target server are obtained, and the current running data, the updated workload data and the updated resource data are input into the updated job classification model for processing to obtain the classification result corresponding to the pending job; After taking the to-be-executed job as the target scheduling job and taking the updated job classification model as the pre-built job classification model, the operation of executing each target scheduling job according to the classification result is returned until no to-be-executed job is detected in each target scheduling job.

2. The method according to claim 1, characterized in that Execute each target scheduling job according to the classification result to obtain job execution results corresponding to each target scheduling job, including: Obtaining the first level in the classification result as the target level, and obtaining the target scheduling job belonging to the target level according to the classification result; After executing the target scheduling jobs belonging to the target level and obtaining the job execution results respectively corresponding to the target scheduling jobs belonging to the target level, the next level of the target level is used as a new target level; Return to execute the operation of obtaining the target scheduling job belonging to the target level according to the classification result until the condition for stopping the execution of the job is met.

3. The method according to claim 2, characterized in that After obtaining the target scheduling job belonging to the target level according to the classification result, the method further includes: Expand or reduce the capacity of the target server according to the workload data of the target server and the number of target scheduling jobs belonging to the target level; The workload data includes at least the number of concurrent jobs, which is used to reflect the number of jobs that can be processed simultaneously by the target server.

4. The method according to claim 3, characterized in that Based on the workload data of the target server and the number of target scheduling jobs belonging to the target level, the target server is expanded or reduced in capacity, including: If the number of concurrent jobs of the target server is less than or equal to the number of target scheduled jobs belonging to the target level, the target server is expanded; If the number of concurrent jobs of the target server is greater than the number of target scheduled jobs belonging to the target level, the target server is scaled down.

5. The method according to any one of claims 1 to 4, characterized in that: Obtain workload data, resource data, and job running data corresponding to the target server, including: After determining the target process corresponding to the target server, the workload data corresponding to the target server is collected through the first thread of the target process, the resource data corresponding to the target server is collected through the second thread of the target process, and the job running data corresponding to the target server is collected through the third thread of the target process.

6. The method according to any one of claims 1 to 4, characterized in that: The workload data, resource data and target operation data are input into a pre-built job classification model for processing to obtain classification results of each target scheduling job, including: The workload data, resource data, target operation data and preset number of levels are input into a pre-built job classification model for processing to obtain classification results for each target scheduling job.

7. A scheduling operation classification device, characterized in that: The device comprises: A resource acquisition module, used to acquire workload data, resource data and job running data corresponding to the target server, wherein the job running data is target running data of each target scheduling job corresponding to the target server; A first job classification module is used to input the workload data, resource data and target operation data into a pre-built job classification model for processing to obtain a classification result of each target scheduling job; A hierarchical model updating module is used to execute each target scheduling job according to the hierarchical result, obtain the job execution result corresponding to each target scheduling job, and input the job execution result into the pre-built job hierarchical model for processing to obtain an updated job hierarchical model; The second job classification module is used for obtaining the current operation data corresponding to the job to be executed, and the updated workload data and the updated resource data corresponding to the target server if it is detected that there is a job to be executed in each target scheduled job, and inputting the current operation data, the updated workload data and the updated resource data into the updated job classification model for processing to obtain the classification result corresponding to the job to be executed; The loop processing module is used to return to execute the operation of executing each target scheduling job according to the classification result after taking the to-be-executed job as the target scheduling job and taking the updated job classification model as the pre-built job classification model, until no to-be-executed job is detected in each target scheduling job.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the scheduling job classification method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the scheduling job classification method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the scheduling job classification method according to any one of claims 1 to 6.

Images