Job classification method and apparatus, computer device, and storage medium

Abstract

The application relates to a job classification method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of big data. The method comprises the following steps: determining a center point corresponding to each job type based on a standard starting time and a standard ending time of each job type; calculating the similarity between each job contained in a to-be-classified job and the center point of each job type based on each center point and a target classification function; and updating the center point of each job type based on each similarity to obtain a job classification result of each job contained in the to-be-classified job. By using the method, the standard starting time and the standard ending time of each job type can be obtained in advance, the initial center point of each job type is accurate, the classification of jobs based on job operation conditions is effectively improved, and the accuracy of the job classification result is improved. Meanwhile, the initial center point of each job type obtained by statistics can guarantee the integrity and comprehensiveness.

Description

Technical Field

[0001] This application relates to the field of big data technology, and in particular to a job classification method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the continuous development and application of computer technology, job scheduling has become an important component of enterprise information management. In the computer field, a job is a task executed by a computer system. Users can determine whether a job is running normally by monitoring the start and end times of jobs executed each day, and based on the results of whether jobs are running normally, jobs can be categorized into multiple types.

[0003] In related technologies, the mean transfer algorithm can be used to classify multiple jobs running each day. Specifically, an initial point is randomly selected, and the average vector distance between this initial point and the surrounding points within a radius R is calculated. This average distance is used as the transfer vector to update the transfer position of the initial point. The initial point is then not updated further, and the points within the radius surrounding the initial point are grouped into a cluster. However, the accuracy of the job classification results can be low because noise points within the jobs can affect the classification. Summary of the Invention

[0004] Therefore, it is necessary to provide a job classification method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the accuracy of job classification results in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for classifying tasks. The method includes:

[0006] Obtain the standard start time and standard end time for each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time.

[0007] Based on the centroids and the target classification function, the similarity between each job included in the job to be classified and the centroids of each job type is calculated.

[0008] Based on the aforementioned similarity, the center points of each job type are updated to obtain the updated center points of each job type.

[0009] If the updated center points of each job type meet the preset convergence conditions, the job classification results of each job included in the job to be classified are obtained based on the updated center points of each job type and the preset similarity threshold.

[0010] In one embodiment, the method further includes:

[0011] If the updated center points of each job type do not meet the preset convergence conditions, the process returns to the step of calculating the similarity between each job included in the job to be classified and the center points of each job type based on each center point and the target classification function, until the updated center points of each job type meet the preset convergence conditions.

[0012] In one embodiment, obtaining the standard start time and standard end time for each job type includes:

[0013] Obtain standard operation data, which includes the start time and end time of standard operations for each type of operation that are predetermined;

[0014] For each job type, calculate the first average start time and the second average end time of the standard job for that job type.

[0015] The first average time is determined to be the standard start time, and the second average time is determined to be the standard end time.

[0016] In one embodiment, calculating the similarity between each job included in the job to be classified and the centroids of each job type, based on each centroid and the target classification function, includes:

[0017] For each job type, the target range corresponding to the job type is determined based on a preset similarity threshold and the start and end times of the center point corresponding to the job type.

[0018] Based on the target range, each job included in the job to be classified is filtered to obtain multiple target jobs;

[0019] The distance between each target task and the center point is calculated using the target classification function, and the distance is determined as the similarity between the target task and the center point.

[0020] In one embodiment, updating the center points of each job type based on the similarity to obtain updated center points for each job type includes:

[0021] For each job type, a weighted average is calculated on the similarity between each target job and the center point corresponding to the job type, and the resulting average is determined as the center point offset vector of the center point.

[0022] Based on the center point and the center point offset vector corresponding to the center point, the updated center point of the job type is obtained.

[0023] In one embodiment, the method further includes:

[0024] When the center point of each updated job type reaches the preset number of iterations, it is determined that the center point of each updated job type satisfies the preset convergence condition.

[0025] Alternatively, if the center point is consistent with the center point of each updated job type, the center point of each updated job type is determined to satisfy a preset convergence condition.

[0026] In one embodiment, the method further includes:

[0027] Among the jobs included in the job classification, those jobs that do not belong to the job classification results are identified as abnormal jobs.

[0028] Secondly, this application also provides a job sorting device. The device includes:

[0029] The first acquisition module is used to acquire the standard start time and standard end time of each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time of each job type.

[0030] The first calculation module is used to calculate the similarity between each job in the data job to be classified and the center points of each job type, based on each of the center points and the target classification function.

[0031] An update module is used to update the center points of each job type based on the aforementioned similarity, thereby obtaining the updated center points of each job type.

[0032] The classification module is used to obtain the job classification results of each job included in the job to be classified data based on the updated center points of each job type and a preset similarity threshold, provided that the center points of each job type meet the preset convergence conditions.

[0033] In one embodiment, the device further includes:

[0034] The second calculation module is used to return to the step of calculating the similarity between each job included in the job to be classified and the center point of each job type based on each center point and the target classification function when the updated center points of each job type do not meet the preset convergence conditions, until the updated center points of each job type meet the preset convergence conditions.

[0035] In one embodiment, the first acquisition module is specifically used for:

[0036] Obtain standard operation data, which includes the start time and end time of standard operations for each type of operation that are predetermined; for each type of operation, calculate the first average start time and the second average end time of the standard operation for that type of operation; determine the first average time as the standard start time and the second average time as the standard end time.

[0037] In one embodiment, the first computing module is specifically used for:

[0038] For each job type, the target range corresponding to the job type is determined based on a preset similarity threshold and the start and end times of the center point corresponding to the job type.

[0039] Based on the target range, each job included in the job to be classified is filtered to obtain multiple target jobs;

[0040] The distance between each target task and the center point is calculated using the target classification function, and the distance is determined as the similarity between the target task and the center point.

[0041] In one embodiment, the update module is specifically used for:

[0042] For each job type, a weighted average is calculated on the similarity between each target job and the center point corresponding to the job type, and the resulting average is determined as the center point offset vector of the center point.

[0043] Based on the center point and the center point offset vector corresponding to the center point, the updated center point of the job type is obtained.

[0044] In one embodiment, the device further includes:

[0045] The first determining module is used to determine that the center points of the updated job types satisfy a preset convergence condition when the center points of the updated job types have reached a preset number of iterations.

[0046] The second determining module is used to determine, either, that the center point of each updated job type satisfies a preset convergence condition when the center point is consistent with the center point of each updated job type.

[0047] In one embodiment, the device further includes:

[0048] The third determining module is used to determine, among the jobs included in the job to be classified, jobs that do not belong to the job classification results as abnormal jobs.

[0049] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0050] Obtain the standard start time and standard end time for each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time.

[0051] Based on the centroids and the target classification function, the similarity between each job included in the job to be classified and the centroids of each job type is calculated.

[0052] Based on the aforementioned similarity, the center points of each job type are updated to obtain the updated center points of each job type.

[0053] If the updated center points of each job type meet the preset convergence conditions, the job classification results of each job included in the job to be classified are obtained based on the updated center points of each job type and the preset similarity threshold.

[0054] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0055] Obtain the standard start time and standard end time for each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time.

[0056] Based on the centroids and the target classification function, the similarity between each job included in the job to be classified and the centroids of each job type is calculated.

[0057] Based on the aforementioned similarity, the center points of each job type are updated to obtain the updated center points of each job type.

[0058] If the updated center points of each job type meet the preset convergence conditions, the job classification results of each job included in the job to be classified are obtained based on the updated center points of each job type and the preset similarity threshold.

[0059] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0060] Obtain the standard start time and standard end time for each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time.

[0061] Based on the centroids and the target classification function, the similarity between each job included in the job to be classified and the centroids of each job type is calculated.

[0062] Based on the aforementioned similarity, the center points of each job type are updated to obtain the updated center points of each job type.

[0063] If the updated center points of each job type meet the preset convergence conditions, the job classification results of each job included in the job to be classified are obtained based on the updated center points of each job type and the preset similarity threshold.

[0064] The aforementioned job classification method, apparatus, computer equipment, storage medium, and computer program product include the following steps: acquiring the standard start time and standard end time of each job type, and determining the center point corresponding to each job type based on the standard start time and standard end time; calculating the similarity between each job included in the job to be classified and the center point of each job type based on each center point and a target classification function; and updating the center points of each job type based on the similarity to obtain the job classification result for each job included in the job to be classified. By employing this method, the standard start time and standard end time of each job type can be obtained in advance, resulting in accurate initial center points for each job type, effectively improving the accuracy of job classification based on job operation status and obtaining the job classification result. Furthermore, the statistically obtained initial center points for each job type ensure completeness and comprehensiveness. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating a job classification method in one embodiment;

[0066] Figure 2 This is a flowchart illustrating the steps for determining the standard start time and standard end time in one embodiment;

[0067] Figure 3 This is a flowchart illustrating the steps for calculating similarity in one embodiment;

[0068] Figure 4 This is a flowchart illustrating the step of updating the center point in one embodiment;

[0069] Figure 5 This is a flowchart illustrating the steps for determining whether a preset convergence condition is met in one embodiment.

[0070] Figure 6 This is a schematic diagram of the job classification results in one embodiment;

[0071] Figure 7 This is a structural block diagram of a job sorting device in one embodiment;

[0072] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0074] In one embodiment, such as Figure 1 As shown, a job classification method is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and is implemented through the interaction between the terminal and the server. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc., and the server can be a standalone server or a server cluster composed of multiple servers. In this embodiment, the job classification method includes the following steps:

[0075] Step 102: Obtain the standard start time and standard end time for each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time.

[0076] Among them, the job type is a classification of different jobs. Specifically, it can be a classification of jobs based on their operation status, which can include the job's start time and end time. For example, the job type can include normal type, late start time type, late end time type, and late start time and past end time type. The standard start time can be calculated based on the start times of multiple standard jobs, and the standard end time can be calculated based on the end times of multiple standard jobs. The standard job can be a classification of jobs within a historical time period. For example, the standard jobs corresponding to the normal type can be multiple normal type jobs determined based on historical jobs. The center point of each job type can be the cluster center of each job type. For example, the center point of the normal type can be the cluster center of the normal type job cluster.

[0077] In implementation, jobs can be collected according to different time periods. The terminal can pre-acquire standard jobs corresponding to each job type in the historical time period closest to the current time period, and extract the start time and end time of each standard job corresponding to each job type. In this way, for each job type, the terminal can calculate the average of the start time and end time of multiple standard jobs corresponding to that job type, and use the calculated average as the standard start time and standard end time for that job type.

[0078] Step 104: Based on each centroid and the target classification function, calculate the similarity between each job included in the job to be classified and the centroid of each job type.

[0079] The target classification function can be a function used to classify multiple tasks by task type. For example, it can be a kernel function in machine learning. For instance, a kernel function that matches the actual application scenario can be determined based on the needs of the actual application scenario. Each centroid can be the centroid of each task type. The task to be classified can be multiple tasks that need to be classified under the current situation. For instance, it can be multiple unclassified tasks running in the system on the current day. The similarity can be the distance between the task calculated by the kernel function and the centroid of the corresponding task type. The similarity of the task can also be described as the similarity between the task and the task represented by the centroid of the task type.

[0080] In practice, the terminal determines the target classification function based on the actual application scenario, and calculates the similarity between each job included in the job to be classified and the centroid of the job type corresponding to each job through the target classification function.

[0081] Step 106: Based on the similarity scores, update the center points of each task type to obtain the updated center points of each task type.

[0082] In practice, the terminal can update the center point corresponding to each job type based on the calculated similarity, and obtain the updated center point.

[0083] Step 108: If the updated center points of each job type meet the preset convergence conditions, the job classification results of each job included in the job to be classified are obtained based on the updated center points of each job type and the preset similarity threshold.

[0084] The preset convergence condition is used to determine whether the center point corresponding to each job type is the most suitable center point. The preset convergence condition can be that the number of updates to the center point corresponding to each job type reaches a preset number of iterations, or that the updated center point is consistent with the unupdated center point, etc. The preset similarity threshold can be a threshold determined based on the actual application scenario. This disclosure does not specifically limit the specific value of the preset similarity threshold; the preset similarity threshold is a value used to classify jobs based on similarity. The job classification result of each job included in the job to be classified can be the job type of each job, or it can be an abnormal job that has not been classified into any job type, etc., which will be described in detail in subsequent embodiments.

[0085] In implementation, after obtaining the updated center points of each job type, the terminal can determine whether the updated center points of each job type meet the preset convergence conditions. If the terminal determines that the updated center points of each job type meet the preset convergence conditions, the terminal can classify each job contained in the job to be classified based on the center points of each job type that meet the preset convergence conditions and a pre-determined preset similarity threshold. That is, the terminal determines the job type of each job contained in the job to be classified, and obtains the job classification result of each job contained in the job to be classified.

[0086] In the above-described job classification method, the standard start time and standard end time of each job type are obtained, and the center point corresponding to each job type is determined based on the standard start time and standard end time. Based on each center point and the target classification function, the similarity between each job included in the job to be classified and the center point of each job type is calculated. Based on each similarity, the center points of each job type are updated to obtain the job classification results for each job included in the job to be classified. By adopting this method, the standard start time and standard end time of each job type can be obtained in advance, resulting in accurate initial center points for each job type, effectively improving the accuracy of job classification based on job operation status and obtaining job classification results. At the same time, the initial center points of each job type obtained through statistics can ensure completeness and comprehensiveness.

[0087] In one embodiment, the job classification method further includes:

[0088] If the updated centroids of each job type do not meet the preset convergence conditions, return to the step of calculating the similarity between each job included in the job to be classified and the centroids of each job type based on each centroid and the target classification function, until the updated centroids of each job type meet the preset convergence conditions.

[0089] In implementation, after obtaining the updated center points for each job type, the terminal can determine whether the updated center points for each job type meet the preset convergence conditions. If the terminal determines that the updated center points for each job type do not meet the preset convergence conditions, the terminal can return to the step of calculating the similarity between the center points of each job included in the job to be classified and the center points of each job type based on each center point and the target classification function. In other words, the terminal can use the updated center points of each job type as center points and calculate the similarity between the center points of each job included in the job to be classified and the center points of each job type through the target classification function until the updated center points of each job type meet the preset convergence conditions.

[0090] In this embodiment, by determining whether the updated center point meets the preset convergence condition, the center points corresponding to each job type are updated and iterated multiple times to ensure the accuracy of the center points of each job type, thereby improving the accuracy of the job classification results.

[0091] In one embodiment, such as Figure 2 As shown, the specific processing steps for the step "obtaining the standard start time and standard end time for each job type" include:

[0092] Step 202: Obtain standard operating procedure data.

[0093] In implementation, standard operation data includes the pre-determined start and end times of standard operations for each job type. That is, the job to be classified can be multiple jobs running within the current time period, and the corresponding standard operation can be multiple jobs running in the previous time period adjacent to the current time period, with each of these jobs having a defined job type. The standard operation data corresponding to the job to be classified is the job execution status of the standard operation corresponding to each job type, which may include, for example, the start and end times of each job. Based on this, the terminal can obtain the standard operation data corresponding to the job to be classified.

[0094] Step 204: For each job type, calculate the first average start time and the second average end time of the standard job for that job type.

[0095] The first time mean represents the average start time of each standard operation; the second time mean represents the average end time of each standard operation.

[0096] In implementation, based on the operation status of the job, the terminal can predetermine multiple job types. For each job type, the terminal needs to obtain the start time and end time of each standard job corresponding to that job type. The terminal performs average processing on the start time of multiple standard jobs to calculate the first time average. The terminal performs average processing on the end time of multiple standard jobs to calculate the second time average.

[0097] Step 206: Determine the first average time as the standard start time and the second average time as the standard end time.

[0098] In practice, for each of the multiple job types, the terminal can use the calculated first average time as the standard start time for that job type and the calculated second average time as the standard end time for that job type.

[0099] In one example, the job type can be a normal type, i.e., a normal operation type. Based on this, the terminal can obtain multiple jobs that are determined to be normal types in the previous time period of the current time period as standard jobs. In this way, the terminal can perform average processing on the start time of multiple standard jobs that are normal types in the previous time period, and use the calculated first average time as the standard start time of the normal type. Similarly, the terminal can perform average processing on the end time of multiple standard jobs that are normal types in the previous time period, and use the calculated second average time as the standard end time of the normal type.

[0100] In this embodiment, the standard start time and standard end time are obtained by averaging the start time and end time of the standard jobs for each job type, which can predetermine the initial center point of each job type, reduce the probability of using noise points as the initial center point, and improve the efficiency of job classification.

[0101] In one embodiment, such as Figure 3 As shown, the specific processing steps of the step "calculating the similarity between each job included in the job to be classified and the centroids of each job type based on each centroid and the target classification function" include:

[0102] Step 302: For each job type, determine the target range corresponding to the job type based on the preset similarity threshold and the start time and end time of the center point corresponding to the job type.

[0103] Among them, the center point corresponding to the job type is the center point of that job type, that is, the center point determined by the standard start time and standard end time of that job type, or it can be the center point obtained after updating, etc.; the target range corresponding to the job type can be the filtering criteria for obtaining jobs belonging to that job type from the jobs to be classified.

[0104] In implementation, for each of the multiple job types, the terminal can determine the area covered by the job type, i.e., the target range, based on the start time, end time of the center point corresponding to the job type and the preset similarity threshold. In other words, the terminal can determine the target area based on the start time, end time of the center point corresponding to the job type and the preset similarity threshold. The target area is the area centered on the start time and end time of the center point corresponding to the job type and with the preset similarity threshold as the radius.

[0105] Step 304: Based on the target range, filter the various jobs included in the job to be classified to obtain multiple target jobs.

[0106] In practice, the terminal can filter multiple jobs included in the job to be classified based on the target range. In other words, the terminal can identify the job that matches the target range among the multiple jobs included in the job to be classified as the target job. That is, the terminal can identify the job that is within the target range (target area) among the multiple jobs included in the job to be classified as the target job corresponding to that job type.

[0107] Step 306: Calculate the distance between each target task and the center point using the target classification function, and determine the distance as the similarity between the target task and the center point.

[0108] In practice, the terminal can use the target classification function to calculate the similarity distance between the centroids of the target job and the job type corresponding to the target job, and determine the calculated similarity distance as the similarity between the target job and the centroid.

[0109] Optionally, the terminal can use the target classification function in the job classification algorithm, that is, the kernel function in the job classification algorithm, to calculate the similarity distance (i.e., similarity vector) between each target job in the job type and the center point of the job type, and use the calculated similarity distance as the similarity between the target job and the center point of the job type. The job classification algorithm can be the Meanshift algorithm.

[0110] In this embodiment, the similarity between the target job corresponding to each job type and the corresponding center point is calculated by using the target classification function, which can ensure the accuracy of the calculated similarity.

[0111] In one embodiment, such as Figure 4 As shown, the specific processing steps for the step "updating the center points of each job type based on their similarity scores to obtain the updated center points of each job type" include:

[0112] Step 402: For each job type, perform weighted average processing on the similarity between each target job and the center point corresponding to the job type, and determine the obtained average as the center point offset vector of the center point.

[0113] In implementation, for all target jobs in each job type, the terminal can perform weighted averaging of the similarity (similarity vector) between each target job and the center point of that job type to obtain a vector mean. The terminal can then use this vector mean as the center point offset vector for that job type. The center point for each job type can be the center point calculated based on the above embodiment, or it can be the initial center point for that job type calculated based on historical time period data, etc. The center point offset vector can be a Meanshift vector calculated based on the Meanshift algorithm.

[0114] Step 404: Based on the center point and the center point offset vector corresponding to the center point, obtain the updated center point of the job type.

[0115] In practice, the terminal can update the center point of the job type based on the calculated center point offset vector. In other words, the center point can be moved by the center point offset vector to obtain the updated center point of the job type.

[0116] In one example, for each job type, the terminal determines the target jobs corresponding to that job type based on the center point of that job type and a preset similarity threshold. It then calculates the similarity between each target job and the center point using a target classification function. The terminal can then normalize the calculated similarities of multiple target jobs corresponding to that job type and perform a weighted average of the normalized similarities to obtain the offset vector of the center point for that job type. Based on this, the terminal can move the center point of that job type according to this offset vector to obtain an updated center point.

[0117] In this embodiment, the accuracy of the task classification results can be guaranteed by continuously updating the center point of the task type through the task classification algorithm.

[0118] In one embodiment, such as Figure 5 As shown, this job classification method also includes:

[0119] Step 502: When the number of updates to the center points of each updated job type reaches the preset number of iterations, it is determined that the center points of each updated job type meet the preset convergence condition.

[0120] The preset number of iterations can be the number of times the center point of each job type is updated. The specific value of the preset number of iterations is not limited in this disclosure, but can be determined by those skilled in the art based on the actual application scenario.

[0121] In implementation, after obtaining the updated center points for each job type, the terminal can obtain the number of times the center points for each job type have been updated under the current circumstances, and compare the obtained update count with the preset iteration count. If the terminal determines that the number of updates for the center points for each job type is equal to or greater than the preset iteration count, the terminal can determine that the current center points for each job type satisfy the preset convergence condition.

[0122] Step 504, or, if the center point is consistent with the center point of each updated job type, determine that the center point of each updated job type satisfies the preset convergence condition.

[0123] In practice, after obtaining the updated center points for each job type, the terminal can compare the updated center point (hereinafter referred to as the first center point) with the center point before the update (hereinafter referred to as the second center point). If it is determined that the first center point and the second center point are at the same location, it can be determined that the updated center point of the job type meets the preset convergence condition. Alternatively, if the terminal determines that the calculated center point offset vector of the job type is 0, the terminal can also determine that the center points corresponding to the current job types meet the preset convergence condition.

[0124] In this embodiment, by making timely judgments on the updated center point, both the efficiency and accuracy of task classification can be taken into account.

[0125] In one embodiment, the job classification method further includes:

[0126] Among the tasks included in the task to be classified, those tasks that do not belong to the task classification results are identified as abnormal tasks. Each abnormal task is then attributed to a cause to obtain the cause of the abnormality. Based on the cause of the abnormality, an anomaly adjustment strategy is determined for the abnormal tasks, and the abnormal tasks are adjusted according to the anomaly adjustment strategy.

[0127] In implementation, the process of attributing abnormal jobs by the terminal can be as follows: The terminal determines whether the abnormal job is one with a significant delay. If it is, the terminal checks whether the upstream job or the job's dependent job has been completed. If the significant delay (i.e., failure to start on time) is caused by the incomplete completion of the dependent job or upstream job, the terminal can determine that the cause of the abnormality is the incomplete completion of the dependent job or upstream job. In this case, the terminal can determine the abnormality adjustment strategy to adjust the on-time execution of the job's dependent job or upstream job. If the dependent job or upstream job has been completed, the terminal can determine that the cause of the abnormality may be due to insufficient queue resources or low job priority.

[0128] When queue resources are scarce, the terminal can determine that the corresponding anomaly adjustment strategy could be to increase the computing resources allocated to job execution, or to adjust the deployment of job execution time. When queue execution priority is low, the terminal can determine that the corresponding anomaly adjustment strategy could be to adjust job priority, allowing the job to be executed first in the queue. The anomaly adjustment strategy determined based on the cause of the anomaly can be pre-determined based on data from historical time periods.

[0129] The reason for the late end time is usually due to queue resource preemption. In this case, the terminal can determine that the abnormal adjustment strategy can be to adjust the deployment of job execution time to avoid a large number of jobs running at the same time.

[0130] In this embodiment, by attributing abnormal operations, timely adjustments to operations can be made to avoid their occurrence, ensuring the reliability of system operation. It also enables monitoring of the system's operating status, timely detection of problems in the system's operation, timely adjustments, and evaluation of the system's operational quality and efficiency, thereby improving the system's stability and reliability.

[0131] The following describes the specific execution process of the above-described job classification method with reference to a detailed embodiment:

[0132] With the continuous development and application of computer technology, job scheduling has become an important component of enterprise information management and plays a crucial role in scenarios such as large data centers. In the computer field, a daily job refers to a task executed by a computer system within a specific time frame. Terminals can monitor the operation of daily jobs, judging whether the jobs are running normally by analyzing their start time (i.e., startup time) and end time, and identifying necessary adjustments to improve system efficiency.

[0133] In the job classification method provided in this disclosure, the terminal can pre-determine multiple job types and the corresponding center point (i.e., cluster center) of each job type by statistically analyzing data within a historical time period. The terminal can classify jobs based on their operation status, for example, into four job types: normal operation, late start time, late end time, and both late start and end time. The terminal can determine the cluster center of the job cluster corresponding to the normal job type by pre-calculating the average of the start and end times of multiple normal jobs. Other categories can also determine their cluster centers using the method described in the above embodiments, thereby effectively improving the accuracy of job classification, i.e., accurately determining the operation status of jobs.

[0134] Specifically, the terminal can use a job classification algorithm to perform cluster analysis on the jobs to be classified, and obtain the job type of each job.

[0135] The job classification algorithm can be the Meanshift algorithm, a nonparametric density estimation algorithm based on a probability density function. The terminal can classify jobs into four types: normal operation, late start time, late end time, and both late start and end times. The terminal can determine a kernel function that matches the job classification scenario, which can be represented by the following formula:

[0136] K(x)=(‖x‖ 2 )

[0137] Where, ‖x‖ 2 =x T Let x represent the modulus of x, and let x represent the job set, which includes d jobs, denoted as x = {x1, x2, x3, ..., x...}. d}

[0138] The terminal can pre-calculate multiple standard jobs corresponding to each job type, as well as the start time and end time of the standard jobs, calculate the average start time, and calculate the average end time to obtain the cluster center of the job type. For example, the terminal can pre-obtain the average start time and end time of multiple normal jobs, and determine the cluster center (referred to as the initial point) of the normal job cluster based on the average value. Other categories also calculate their own cluster centers in turn, which can reduce the probability of selecting outliers for the cluster center and improve the running efficiency. In other words, this embodiment can divide the clusters by determining the size of the radius, i.e., the threshold of the start time and end time, and fluctuate it by a certain value to avoid the situation where abnormal jobs are classified as normal jobs.

[0139] Thus, for an initial point, the weights of other points within a region centered on that initial point and with a radius (a preset similarity threshold) are calculated. This involves using a kernel function to estimate the similarity of all points to the center point, normalizing the weights, and calculating the weighted average of all points to obtain the center point offset vector. The updated center point is then obtained based on this offset vector. For example, the center point offset vector M can be calculated using the following formula. h (x):

[0140]

[0141] M h (x)= h If (x), then the updated center point can be calculated using the following formula:

[0142] M h (x)= h (x)+

[0143] Where x is the center point before the update, and M h (x) is the center point offset vector, M h (x) is the updated center point.

[0144] If the terminal determines that the updated centroids do not meet the preset convergence criteria, it can re-execute the step of calculating the similarity between the centroids of each job included in the job to be classified and each job type, based on each centroid and the target classification function, until the updated centroids of each job type meet the preset convergence criteria. The preset convergence criteria may be that the centroids no longer change or a certain number of iterations are reached. Optionally, when the centroids no longer change or the number of iterations is reached, all data points that converge to the same centroid are assigned to the same category, which is a clustering result. Data points that have not yet been assigned to a cluster will be listed as outliers and their causes will be analyzed.

[0145] Alternatively, the convergence process can be expressed using the following formula:

[0146] Calculate m h (x), let x = m h (x), if ||m h If (x)-||<ε, terminate the loop; otherwise, determine that the currently updated center point does not meet the preset convergence condition.

[0147] Finally, as Figure 6As shown, the terminal can determine that a region (i.e., the multiple jobs included in the job to be classified) will be divided into four regions, corresponding to four types: normal operation, late start time, late end time, and both late start and end times. Each job is classified using this method. Analyzing the daily job performance based on start and end times helps operations personnel monitor the system's operational status, identify potential problems, evaluate the system's quality and efficiency, and improve system stability and reliability. In future work, continued analysis of this indicator will provide a better understanding of the daily job performance, thereby optimizing the system.

[0148] Terminals can determine whether the job queue is saturated and congested within a certain period by analyzing the differences in end times. A large number of jobs running simultaneously leads to queue strain, causing jobs to wait for extended periods. Terminals can identify periods of resource scarcity and normal operation based on these end-time differences, and adjust job execution times accordingly to effectively manage job deployment and prevent a large number of jobs from running concurrently. From the perspective of job end times, terminals can determine whether jobs can complete on time. If a job's execution time exceeds the planned execution time (the cluster core of a normal job cluster), system adjustments are necessary. For example, while adjusting job priorities, the overall job execution time can be reduced by dividing the job into smaller subtasks. This process can also be used for monitoring internal and external server faults; errors in jobs that have been running for an extended period (e.g., over 24 hours) indicate a potential hardware failure on that node.

[0149] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0150] Based on the same inventive concept, this application also provides a job classification device for implementing the job classification method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more job classification device embodiments provided below can be found in the limitations of the job classification method described above, and will not be repeated here.

[0151] In one embodiment, such as Figure 7 As shown, a job classification device is provided, including: a first acquisition module 702, a first calculation module 704, an update module 706, and a classification module 708, wherein:

[0152] The first acquisition module 702 is used to acquire the standard start time and standard end time of each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time of each job type.

[0153] The first calculation module 704 is used to calculate the similarity between each job in the data job to be classified and the centroid of each job type based on each centroid and the target classification function.

[0154] The update module 706 is used to update the center points of each job type based on the similarity, so as to obtain the updated center points of each job type.

[0155] The classification module 708 is used to obtain the job classification results of each job included in the job to be classified based on the updated center points of each job type and the preset similarity threshold, provided that the updated center points of each job type meet the preset convergence conditions.

[0156] In one embodiment, the device further includes:

[0157] The second calculation module is used to return to the step of calculating the similarity between the center points of each job included in the job to be classified and the center points of each job type, based on each center point and the target classification function, if the updated center points of each job type do not meet the preset convergence conditions, until the updated center points of each job type meet the preset convergence conditions.

[0158] In one embodiment, the first acquisition module is specifically used for:

[0159] Obtain standard operation data, which includes the start time and end time of standard operations for each pre-determined operation type; for each operation type, calculate the first average start time and the second average end time of the standard operation for that operation type; determine the first average time as the standard start time and the second average time as the standard end time.

[0160] In one embodiment, the first computing module is specifically used for:

[0161] For each job type, the target range corresponding to the job type is determined based on a preset similarity threshold and the start and end times of the center point corresponding to the job type.

[0162] Based on the target range, each task included in the task to be classified is filtered to obtain multiple target tasks;

[0163] The distance between each target task and the center point is calculated using the target classification function, and the distance is determined as the similarity between the target task and the center point.

[0164] In one embodiment, the update module is specifically used for:

[0165] For each job type, the similarity between each target job and the center point corresponding to the job type is weighted and averaged, and the resulting average is determined as the center point offset vector of the center point.

[0166] Based on the center point and its corresponding center point offset vector, the updated center point of the job type is obtained.

[0167] In one embodiment, the device further includes:

[0168] The first determining module is used to determine that the center points of each updated job type meet the preset convergence condition when the center points of each updated job type have reached the preset number of iterations.

[0169] The second determining module is used to determine, either, that the center point of each updated job type satisfies the preset convergence condition, provided that the center point is consistent with the center point of each updated job type.

[0170] In one embodiment, the device further includes:

[0171] The third determination module is used to identify jobs that do not belong to the job classification results as abnormal jobs among the jobs included in the job to be classified.

[0172] Each module in the aforementioned job classification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0173] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores job data. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a job classification method.

[0174] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0175] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0176] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0177] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0178] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0179] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0180] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0181] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for classifying assignments, characterized in that, The method includes: Obtain the standard start time and standard end time for each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time. For each job type, the target range corresponding to the job type is determined based on a preset similarity threshold and the start and end times of the center point corresponding to the job type. Based on the target range, each job included in the job to be classified is filtered to obtain multiple target jobs; The distance between each target task and the center point is calculated using a target classification function, and the distance is determined as the similarity between the target task and the center point. For each job type, a weighted average is calculated on the similarity between each target job and the center point corresponding to the job type, and the resulting average is determined as the center point offset vector of the center point. Based on the center point and the center point offset vector corresponding to the center point, the updated center point of the job type is obtained; If the updated center points of each job type meet the preset convergence conditions, the job classification results of each job included in the job to be classified are obtained based on the updated center points of each job type and the preset similarity threshold.

2. The method according to claim 1, characterized in that, The method further includes: If the updated center points of each job type do not meet the preset convergence conditions, the process returns to the step of calculating the similarity between each job included in the job to be classified and the center points of each job type based on each center point and the target classification function, until the updated center points of each job type meet the preset convergence conditions.

3. The method according to claim 1, characterized in that, The acquisition of the standard start time and standard end time for each job type includes: Obtain standard operation data, which includes the start time and end time of each type of standard operation as predetermined; For each job type, calculate the first average start time and the second average end time of the standard job for that job type. The first average time is determined to be the standard start time, and the second average time is determined to be the standard end time.

4. The method according to claim 1, characterized in that, The method further includes: When the center point of each updated job type reaches the preset number of iterations, it is determined that the center point of each updated job type satisfies the preset convergence condition. Alternatively, if the center point is consistent with the center point of each updated job type, the center point of each updated job type is determined to satisfy a preset convergence condition.

5. The method according to claim 1, characterized in that, The method further includes: Among the jobs included in the job classification, those jobs that do not belong to the job classification results are identified as abnormal jobs.

6. A job sorting device, characterized in that, The device includes: The first acquisition module is used to acquire the standard start time and standard end time of each job type, and determine the center point corresponding to each job type based on the standard start time and standard end time of each job type. The first calculation module is used to determine the target range corresponding to each job type based on a preset similarity threshold and the start time and end time of the center point corresponding to the job type; to filter each job included in the job to be classified based on the target range to obtain multiple target jobs; and to calculate the distance between each target job and the center point through a target classification function, and to determine the distance as the similarity between the target job and the center point. The update module is used to perform weighted average processing on the similarity between each target job and the center point corresponding to the job type for each job type, and determine the obtained average as the center point offset vector of the center point; based on the center point and the center point offset vector corresponding to the center point, the updated center point of the job type is obtained; The classification module is used to obtain the job classification result of each job included in the job to be classified based on the updated center points of each job type and a preset similarity threshold, provided that the center points of each updated job type meet the preset convergence conditions.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images