Model task time limit optimization method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of data processing, in particular to a model task time limit optimization method and device, electronic equipment and a storage medium, metadata of a model task is acquired; an important task meeting a preset rule is acquired from the model task according to the metadata; a substandard task meeting a screening rule is acquired from the important task according to the metadata; a substandard reason of the substandard task is determined according to the metadata, and an optimization strategy of the substandard task is generated according to the substandard reason; in the above manner, the important task is acquired first, then the substandard task is acquired from the important task, then the substandard reason of the substandard task is determined, and finally the optimization strategy of the substandard task is generated according to the substandard reason, so that the existing model task time limit is managed in a targeted manner, the model task time limit problem can be handled in advance, the business demand is met, the business satisfaction is improved, and the model task time limit is managed effectively in a long term.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a model task timeliness optimization method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the advent of a digital society, data as an asset of an enterprise is increasingly important for the survival and development of the enterprise and has become a consensus. In this case, digital transformation is an essential step for the development of modern enterprises. Under the background of enterprise digital transformation, the timeliness requirement of data users for data is also increasingly high. Timely data processing can help enterprises quickly develop business strategies, seize market opportunities, help governments provide data support for social emergency management strategies, and reduce the loss of slow decision-making.

[0003] The existing problem is that the timeliness of a large number of model tasks on the existing platform cannot be managed, and business personnel often feedback that the data timeliness of some applications does not meet the business scenario.

[0004] To solve the above problems, the existing solution is to arrange personnel to optimize the timeliness of the model tasks relied on by the slow timeliness applications feedback by business personnel. The deficiencies and shortcomings of this solution are as follows: 1. After-treatment, only after the feedback of business personnel, the processing is affected, the business personnel carry out business, and the business satisfaction is poor; 2. Temporary, there is no long-term timeliness optimization plan, the task timeliness is not continuous, and the timeliness does not meet the business situation frequently occurs; 3. Not systematic, task timeliness optimization can only rely on the experience of developers to process, no experience documents are formed, and there is no way to evaluate whether the developers have maximized the task timeliness optimization.

[0005] The existing problem of poor model task timeliness causes some applications to be unable to better support front-line business development, unable to help administrators make business decisions, resulting in the data value of these model tasks being low, the planning data transformation effect being poor, and the enterprise digital transformation being easily abandoned halfway. SUMMARY

[0006] The purpose of the present application is to provide a model task timeliness optimization method, device, electronic equipment and storage medium to solve the technical problem of slow model task timeliness in the prior art.

[0007] The technical solution of the present application is as follows: a model task timeliness optimization method is provided, comprising:

[0008] Obtaining metadata of a model task;

[0009] According to the metadata, an important task meeting a preset rule is obtained from the model task, the preset rule comprising at least one preset condition, the important task being a model task meeting at least one preset condition;

[0010] According to the metadata, an under-achievement task meeting a screening rule is obtained from the important task, the screening rule comprising at least one screening condition, the under-achievement task being the important task meeting at least one screening condition;

[0011] According to the metadata, an under-achievement reason of the under-achievement task is determined, and an optimization strategy of the under-achievement task is generated according to the under-achievement reason.

[0012] As an optional implementation, the metadata comprises a start time of an initial task on which the model task depends, a start time and an end time of an upstream model task on which the model task depends, a number of downstream tasks depending on the model task, an application level of a downstream application, and a task start time, a task end time, a task running time and a processing link level of the model task.

[0013] As an optional implementation, the preset rule comprises a first preset condition, the first preset condition being that an application downstream of a processing link where the model task is located is an important application.

[0014] The important task meeting the preset rule is obtained from the model task according to the metadata, comprising:

[0015] According to the metadata, it is determined whether an application downstream of a processing link where the model task is located is an important application.

[0016] If the determination result is yes, the model task is taken as the important task.

[0017] As an optional implementation, the preset rule comprises a second preset condition, the second preset condition being that a number of downstream tasks depending on the model task is greater than a first quantity threshold.

[0018] The important task meeting the preset rule is obtained from the model task according to the metadata, comprising:

[0019] According to the metadata, it is determined whether a number of downstream tasks depending on the model task is greater than a first quantity threshold.

[0020] If the determination result is yes, the model task is taken as the important task.

[0021] As an optional implementation, the screening rule comprises a first screening condition, the first screening condition being that an achievement rate of the important task is less than an achievement rate threshold.

[0022] the metadata, the method comprises:

[0023] determining, according to the metadata, whether the achievement rate of the important task is less than an achievement rate threshold value;

[0024] if the determination result is yes, the important task is taken as the under-achievement task.

[0025] As an optional implementation, the method further comprises:

[0026] acquiring, according to the metadata, a time difference between a start time of the under-achievement task and an end time of an upstream model task of the under-achievement task, and determining that a reason for under-achievement of the under-achievement task is a scheduling time difference if the time difference is greater than a first time threshold value;

[0027] acquiring, according to the metadata, a start time of an initial task on which the under-achievement task depends, and determining that the reason for under-achievement of the under-achievement task is that the initial task starts late if the start time is later than a preset time point;

[0028] acquiring, according to the metadata, a running duration of the under-achievement task, and determining that the reason for under-achievement of the under-achievement task is that the task runs for a long time if the running duration is greater than a second time threshold value;

[0029] acquiring, according to the metadata, a processing link level of the under-achievement task, and determining that the reason for under-achievement of the under-achievement task is that the task has a high processing link level if the processing link level is greater than a second quantity threshold value;

[0030] As an optional implementation, the method further comprises:

[0031] if the reason for under-achievement of the under-achievement task is determined to be the scheduling time difference, the optimization strategy is to advance the start time of the under-achievement task;

[0032] if the reason for under-achievement of the under-achievement task is determined to be that the initial task starts late, the optimization strategy is to advance the start time of the initial task on which the under-achievement task depends;

[0033] if the reason for under-achievement of the under-achievement task is determined to be that the task runs for a long time, the optimization strategy is to determine a reason why the under-achievement task runs for a long time and generate a corresponding optimization strategy;

[0034] If the reason for the substandard task is determined to be that the task processing link level is high, the optimization strategy is to decouple the task.

[0035] Another technical solution of the present application is as follows: a model task timeliness optimization device is provided, comprising:

[0036] A metadata acquisition module is configured to acquire metadata of a model task.

[0037] An important task acquisition module is configured to acquire an important task meeting a preset rule from the model task according to the metadata, wherein the preset rule comprises at least one preset condition, and the important task is a model task meeting at least one preset condition.

[0038] A substandard task screening module is configured to acquire a substandard task meeting a screening rule from the important task according to the metadata, wherein the screening rule comprises at least one screening condition, and the substandard task is the important task meeting at least one screening condition.

[0039] An optimization strategy generation module is configured to determine a reason for the substandard task according to the metadata, and generate an optimization strategy for the substandard task according to the reason.

[0040] Another technical solution of the present application is as follows: an electronic device is provided, comprising a processor and a memory coupled with the processor, wherein the memory stores program instructions executable by the processor; and the processor implements the model task timeliness optimization method described above when executing the program instructions stored in the memory.

[0041] Another technical solution of the present application is as follows: a storage medium is provided, wherein the storage medium stores program instructions, and the program instructions are executed by a processor to implement the model task timeliness optimization method described above.

[0042] The model task timeliness optimization method, device, electronic equipment and storage medium provided by the application obtain metadata of a model task; important tasks meeting preset rules are obtained from the model task according to the metadata, the preset rules include at least one preset condition, and the important tasks are model tasks meeting at least one preset condition; substandard tasks meeting screening rules are obtained from the important tasks according to the metadata, the screening rules include at least one screening condition, and the substandard tasks are important tasks meeting at least one screening condition; substandard reasons of the substandard tasks are determined according to the metadata, and optimization strategies of the substandard tasks are generated according to the substandard reasons. In the foregoing manner, important tasks meeting preset rules are obtained first, then substandard tasks meeting screening rules are obtained from the important tasks, then substandard reasons of the substandard tasks are determined according to the metadata, and finally optimization strategies of the substandard tasks are generated according to the substandard reasons. The existing model task timeliness can be managed in a targeted manner, the model task timeliness problem can be handled in advance, the business demand can be met, the business satisfaction can be improved, and the model task timeliness can be managed effectively in the long term. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart of a model task timeliness optimization method of a first embodiment of the application is shown in the figure.

[0044] Figure 2 A structural diagram of a model task timeliness optimization device of a second embodiment of the application is shown in the figure.

[0045] Figure 3 A structural diagram of an electronic equipment of a third embodiment of the application is shown in the figure.

[0046] Figure 4 A structural diagram of a storage medium of a fourth embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0048] The terms "first", "second", "third" in the present application are only for descriptive purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second", "third" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise explicitly and specifically limited. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present application are only used to explain the relative position relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications also change accordingly. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0049] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment can be included in at least one embodiment of the application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor does it necessarily refer to a separate or alternative embodiment, in isolation or in combination with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0050] Figure 1 is a flowchart of a model task timeliness optimization method of a first embodiment of the present application. It should be noted that the model task timeliness optimization method of the present application is not limited to the flow order shown in Figure 1 , as shown in Figure 1 , the model task timeliness optimization method includes the following steps:

[0051] S101, obtaining metadata of a model task.

[0052] The metadata includes a start time of an initial task on which the model task depends, a start time of an upstream model task on which the model task depends, an end time of the upstream model task on which the model task depends, a number of downstream tasks that depend on the model task, a downstream application level, a task start time of the model task, a task end time of the model task, a task runtime of the model task, and a processing link level of the model task.

[0053] The model is a data table. The task is a computing task based on the data table.

[0054] The task processing link comprises at least one initial task, a plurality of model tasks executed in sequence and dependent on the initial task, and at least one downstream application, that is, the initial task is the starting point of the task processing link, and the downstream application is the end point of the task processing link.

[0055] The upstream model task is a preceding task of the downstream model task, and the downstream model task is a subsequent task of the upstream model task.

[0056] The processing link level of the model task refers to the layer of the task processing link where the model task is located.

[0057] S102, obtaining an important task from the model tasks according to the metadata and a preset rule, the preset rule comprising at least one preset condition, the important task being a model task satisfying at least one preset condition.

[0058] Since the number of model tasks is large, but the computing resources such as CPU are limited, the model tasks need to be selectively optimized when optimizing the model tasks, otherwise, not only the time and effort are wasted, the optimization effect is poor, and the normal operation cannot be guaranteed. For example, a server needs to process 50,000 model tasks every day, of which 25,000 model tasks have timeliness problems. Of the 25,000 model tasks with timeliness problems, only 2,000 model tasks cannot be completed in the morning, which will have an adverse effect. Whether the remaining 20,000 model tasks with timeliness problems can be completed in the morning has no effect, as long as they are completed on the same day. If the computing resources are evenly distributed, the 25,000 model tasks with timeliness problems are optimized without distinction, not only the workload is huge, but also it will lead to insufficient resources to optimize the above-mentioned 2,000 model tasks, thereby causing the downstream tasks dependent on the above-mentioned 2,000 model tasks to be unable to be completed on time, and the downstream application dependent on the above-mentioned 2,000 model tasks to be unable to be normally used. Therefore, in order to guarantee the overall timeliness, important tasks need to be selected from the model tasks according to the metadata, and selective optimization is performed.

[0059] As an optional implementation, the preset rule comprises a first preset condition, and the first preset condition is that the downstream application of the processing link where the model task is located is an important application.

[0060] Correspondingly, the important task is obtained from the model tasks according to the metadata and the preset rule, comprising:

[0061] determining whether the downstream application of the processing link where the model task is located is an important application according to the metadata;

[0062] if the determination result is yes, the model task is regarded as the important task.

[0063] The downstream application level is artificially set, specifically according to the importance of the application. The downstream application level has three levels in total, namely, low, medium and high. When the level of a downstream application is high, the downstream application is an important application. All model tasks on the model task processing link corresponding to the important application are important tasks.

[0064] Correspondingly, the determining whether the downstream application of the model task processing link is an important application according to the metadata specifically includes:

[0065] S201, obtaining all downstream applications of the model task processing link according to the metadata;

[0066] S202, obtaining the level of each downstream application according to the metadata;

[0067] S203, determining whether there is an important application in the all downstream applications according to the level of the downstream application;

[0068] S204, if there is at least one important application in the all downstream applications, determining that the downstream application of the model task processing link is an important application.

[0069] As an optional embodiment, the preset rule includes a second preset condition, and the second preset condition is that the number of downstream tasks dependent on the model task is greater than a first number threshold.

[0070] Correspondingly, the obtaining important task from the model task according to the metadata includes:

[0071] determining whether the number of downstream tasks dependent on the model task is greater than a first number threshold according to the metadata;

[0072] if the determination result is yes, regarding the model task as the important task.

[0073] For example, the first number threshold is 10, and the number of downstream tasks dependent on a model task is 15, then the model task is regarded as an important task.

[0074] In a most preferred embodiment, the preset rule includes a first preset condition and a second preset condition, the first preset condition is that the downstream application of the model task is an important application, and the second preset condition is that the number of downstream tasks dependent on the model task is greater than a first number threshold. When the model task meets at least one of the first preset condition and the second preset condition, the model task is regarded as the important task.

[0075] Accordingly, the important task is obtained from the model task according to the metadata, and specifically includes the following steps:

[0076] S301, determining whether the downstream application of the model task is an important application according to the metadata;

[0077] If the result of the determination is yes, the model task is regarded as the important task.

[0078] If the result of the determination is no, the next step is continued.

[0079] S302, determining whether the number of downstream tasks depending on the model task is greater than a first quantity threshold according to the metadata;

[0080] If the result of the determination is yes, the model task is regarded as the important task.

[0081] If the result of the determination is no, the model task is not regarded as the important task.

[0082] S103, obtaining a substandard task meeting a screening rule from the important task according to the metadata, the screening rule including at least one screening condition, and the substandard task being at least one important task meeting the screening condition.

[0083] In some preferred embodiments, the screening rule includes a first screening condition, and the first screening condition is that the achievement rate of the important task is less than an achievement rate threshold.

[0084] Accordingly, the important task is obtained from the model task according to the metadata, and specifically includes the following steps:

[0085] Determining whether the achievement rate of the important task is less than an achievement rate threshold according to the metadata;

[0086] If the result of the determination is yes, the important task is regarded as the substandard task.

[0087] Specifically, if an important task is completed before a specified time, it is considered that the important task is achieved on that day. The achievement rate of the important task refers to the ratio of the number of days that the important task is actually achieved to the number of days in a certain period. For example, if an important task is completed before 8 a.m. on a day, it is considered that the important task is achieved on that day, and the achievement rate of the important task in 30 days is equal to the number of days that the important task is achieved in 30 days / 30. If the achievement rate threshold of an important task is 90%, but the number of days that the important task is achieved in 30 days is 21 days, that is, the achievement rate of the important task is 70%, then the important task meets the first screening condition, that is, the important task is a substandard task.

[0088] S104, determine the substandard reason of the substandard task according to the metadata, and generate an optimization strategy of the substandard task according to the substandard reason.

[0089] The substandard reason of the substandard task includes a scheduling time difference between the substandard task and a dependent upstream model task, late start of an initial task relied on by the substandard task, long running time of the substandard task itself, and multiple processing link levels of the substandard task.

[0090] Correspondingly, the step of determining the substandard reason of the substandard task according to the metadata includes the steps of:

[0091] S401, obtaining a time difference between a start time of the substandard task and an end time of an upstream model task of the substandard task according to the metadata, and determining that the substandard reason of the substandard task is a scheduling time difference if the time difference is greater than a first time threshold.

[0092] Specifically, the step S401 includes obtaining the start time of the substandard task according to the metadata, and obtaining the end time of the upstream model task relied on by the substandard task according to the metadata, and determining that the substandard reason of the substandard task at least includes the scheduling time difference if the time difference between the start time of the substandard task and the end time of the upstream model task of the substandard task is greater than the first time threshold.

[0093] For example, the first time threshold is 3 minutes, the start time of a certain substandard task is 11:00 am, the end time of the upstream model task relied on by the substandard task is 10:30 am, and the time difference between the start time of the substandard task and the end time of the upstream model task of the substandard task is 30 minutes. Therefore, the substandard reason of the substandard task at least includes the scheduling time difference.

[0094] S402, obtaining a start time of an initial task relied on by the substandard task according to the metadata, and determining that the substandard reason of the substandard task is late start of the initial task if the start time is later than a preset time point.

[0095] Specifically, if the substandard reason of the substandard task is determined to be late start of the initial task, the substandard reason of the substandard task at least includes late start of the initial task.

[0096] For example, the preset time point is 1:00 am, and the start time of the initial task relied on by a certain substandard task is 8:00 am. Therefore, the substandard reason of the substandard task at least includes late start of the initial task.

[0097] S403, obtaining a running duration of the substandard task according to the metadata, and determining that a substandard reason of the substandard task is long running duration if the running duration of the substandard task is greater than a second time threshold.

[0098] Specifically, if the substandard reason of the substandard task is determined to be long running duration, the substandard reason of the substandard task at least includes long running duration.

[0099] For example, the second time threshold is 2 hours, and if the running duration of a substandard task is 3 hours, the substandard reason of the substandard task at least includes long running duration.

[0100] S404, obtaining a processing link level of the substandard task according to the metadata, and determining that a substandard reason of the substandard task is high processing link level if the processing link level is greater than a second number threshold.

[0101] Specifically, if the substandard reason of the substandard task is determined to be high processing link level, the substandard reason of the substandard task at least includes high processing link level.

[0102] For example, the second number threshold is the 30th layer, and if a substandard task is at the 42nd layer of a processing link, the substandard reason of the substandard task at least includes high processing link level.

[0103] As an optional implementation, the generating the optimization strategy of the substandard task according to the substandard reason comprises:

[0104] If the substandard reason of the substandard task is determined to be scheduling time difference, the optimization strategy is to advance the start time of the substandard task.

[0105] For example, the first time threshold is 3 minutes, the time difference between the start time of a substandard task and the end time of an upstream model task of the substandard task is 30 minutes, and the optimization strategy is to advance the start time of the substandard task by 30 minutes.

[0106] If the substandard reason of the substandard task is determined to be late initial task start, the optimization strategy is to advance the start time of an initial task on which the substandard task depends.

[0107] For example, the preset time point is 1:00 a.m., the start time of an initial task on which a substandard task depends is 8:00 a.m., and the optimization strategy is to advance the start time of the initial task to no later than 1:00 a.m.

[0108] If the reason for the substandard task is determined to be long running time, the optimization strategy is to determine the reason for the long running time of the substandard task and generate a corresponding optimization strategy.

[0109] In this embodiment, the data table exists in the server in the form of slices, and updating the data table generates a new file, and each file internally stores the data updated on the same day. Correspondingly, the determination of the reason for the long running time of the substandard task includes: determining whether there is data skew; determining whether there is a large HDFS file read; and determining whether there is too much HDFS file read.

[0110] If the running time of a single computing node in a substandard task exceeds 30 minutes, or the running time of a single computing node exceeds 5 times the average computing node running time of the substandard task, it is determined that there is data skew, i.e., the reason for the long running time of the substandard task at least includes data skew. The corresponding optimization strategy for data skew is: 1. In some embodiments, data skew is caused by the association of a large table with a small table, and map join processing can be used; 2. In some embodiments, data skew is caused by a large number of null values in the associated field, and the null values can be deleted before association; 3. In some embodiments, data skew is caused by inconsistent data types in the associated field, and the string type can be uniformly processed for conversion; 4. In some embodiments, data skew is caused by excessive values in the associated field, and the data of a certain value can be separately associated and processed, and then combined with other data for processing; 5. In some embodiments, system parameters can be set to enable load balancing and set map size; 6. In some embodiments, data skew is caused by group by, and a random value can be concatenated to the group by aggregation field for local aggregation, and then the random value is removed from the local aggregation result for aggregation again.

[0111] The rule for determining whether there is a large HDFS file read is: if a substandard task needs to read a file larger than 2T, it is determined that there is a large HDFS file read, i.e., the reason for the long running time of the substandard task at least includes a large HDFS file read. The corresponding optimization strategy for a large HDFS file read is to filter data and / or trim data. Filtering data means that the data that must be read when executing the task is filtered according to the filtering conditions, such as filtering the customers in Guangdong province from the customer information table in the whole country to read only the relevant data of the customers in Guangdong province. Trimming data means reading only useful fields and not reading unnecessary fields, for example, only reading the ID number field when only the ID information is needed.

[0112] The rule for determining whether there is excessive reading of HDFS files is that if a certain substandard task needs to read more than 1000 files, it is determined that there is excessive reading of HDFS files, that is, the long running time of the substandard task is at least caused by excessive reading of HDFS files. The optimization strategy corresponding to excessive reading of HDFS files is to merge files. For example, the prior art stores only the data updated on the same day for each file, and 365 files will be generated in a year. If a model task needs to read the data of the last year, 365 files need to be read. The corresponding optimization strategy is to write all the files of the last year to the file of the last day of the last year, that is, to the last generated file of the last year.

[0113] If the substandard cause of the substandard task is determined to be that the task processing link level is high, the optimization strategy is to decouple the task.

[0114] Specifically, the substandard task is split into multiple new tasks according to a certain rule, and then the data reading path of each new task is re-planned.

[0115] For example, a substandard task is located at the 49th layer of the processing link, and the task needs to read the customer's ID number, home address and phone number from the task at the 48th layer. The corresponding optimization strategy is to split the substandard task into three tasks: the first task needs to read the customer's ID number, which can read the customer's ID number field from the task at the 8th layer; the first task needs to read the customer's home address, which can read the customer's home address field from the task at the 12th layer; and the third task needs to read the customer's phone number, which can read the customer's phone number field from the task at the 9th layer.

[0116] In some preferred embodiments, after the optimization strategy of the substandard task is generated according to the substandard cause, the method further comprises:

[0117] S105, optimizing the substandard task according to the optimization strategy of the substandard task.

[0118] Since some defects are easy to optimize and some defects are difficult to optimize, when the substandard task is optimized, the optimization needs to be performed in the order from easy to difficult. Among the four defects of scheduling time difference, initial task starting late, long task running time and high processing link level, the two defects of scheduling time difference and initial task starting late are the easiest to optimize, the defect of long task running time is the second easiest to optimize, and the defect of high processing link level is the most difficult to optimize. Therefore, the optimization order is: first, optimize the two defects of scheduling time difference and initial task starting late, then optimize the defect of long task running time, and finally optimize the defect of high processing link level.

[0119] For example, the substandard reasons of a certain substandard task include the existence of scheduling time difference, late initial task start, long task running time and multiple processing link levels. When optimizing, the defects of the existence of scheduling time difference and late initial task start are first optimized. The end time of the task is obtained the next day after optimization; if the end time of the task is not later than the specified time, that is, the task is completed on the same day, the optimization is completed; if the end time of the task is later than the specified time, that is, the task is not completed on the same day, the defect of long task running time needs to be continuously optimized. The end time of the task is obtained the next day after optimization; if the end time of the task is not later than the specified time, that is, the task is completed on the same day, the optimization is completed; if the end time of the task is later than the specified time, that is, the task is not completed on the same day, the defect of optimizing the processing link level needs to be continuously optimized.

[0120] In the embodiment of the application, the important task meeting the preset rule is first obtained, then the substandard task meeting the screening rule is obtained from the important task, the substandard reason of the substandard task is determined according to the metadata, and finally the optimization strategy of the substandard task is generated according to the substandard reason. Not only can the existing model task time be managed with a clear purpose, avoiding the calculation resource shortage caused by indiscriminately optimizing the task time and the inability to timely optimize the time of important tasks. But also can deal with model task time problems in advance, meet business needs, improve business satisfaction, and effectively manage model task time in the long term.

[0121] Figure 2 is a structural schematic diagram of a model task time optimization device of a second embodiment of the application. As shown in Figure 2 the model task time optimization device 20 includes a metadata acquisition module 21, an important task acquisition module 22, a substandard task screening module 23 and an optimization strategy generation module 24.

[0122] The metadata acquisition module 21 is configured to acquire metadata of a model task. The important task acquisition module 22 is configured to acquire an important task meeting a preset rule from the model task according to the metadata, the preset rule including at least one preset condition, and the important task being a model task meeting at least one preset condition. The substandard task screening module 23 is configured to acquire a substandard task meeting a screening rule from the important task according to the metadata, the screening rule including at least one screening condition, and the substandard task being the important task meeting at least one screening condition. The optimization strategy generation module 24 is configured to determine a substandard reason of the substandard task according to the metadata, and generate an optimization strategy of the substandard task according to the substandard reason.

[0123] Further, the metadata comprises a start time of an initial task on which the model task depends, a start time and an end time of an upstream model task on which the model task depends, a number of downstream tasks depending on the model task, a downstream application level, and a task start time, a task end time, a task runtime, and a processing link level of the model task.

[0124] As an optional implementation, the preset rule in the important task acquisition module 22 comprises a first preset condition that a downstream application of a processing link where the model task is located is an important application.

[0125] The important task acquisition module 22 comprises a preset rule, and the important task acquisition module 22 is configured to acquire, according to the metadata, an important task from the model task that meets the preset rule.

[0126] The important task acquisition module 22 comprises a preset rule, and the important task acquisition module 22 is configured to acquire, according to the metadata, an important task from the model task that meets the preset rule.

[0127] If the judgment result is yes, the model task is taken as the important task.

[0128] As an optional implementation, the preset rule in the important task acquisition module 22 comprises a second preset condition that a number of downstream tasks depending on the model task is greater than a first number threshold.

[0129] The important task acquisition module 22 comprises a preset rule, and the important task acquisition module 22 is configured to acquire, according to the metadata, an important task from the model task that meets the preset rule.

[0130] The important task acquisition module 22 comprises a preset rule, and the important task acquisition module 22 is configured to acquire, according to the metadata, an important task from the model task that meets the preset rule.

[0131] If the judgment result is yes, the model task is taken as the important task.

[0132] As an optional implementation, the screening rule in the substandard task screening module 23 comprises a first screening condition that an achievement rate of the important task is less than an achievement rate threshold.

[0133] The substandard task screening module 23 comprises a screening rule, and the substandard task screening module 23 is configured to acquire, according to the metadata, a substandard task from the important task that meets the screening rule.

[0134] The substandard task screening module 23 comprises a screening rule, and the substandard task screening module 23 is configured to acquire, according to the metadata, a substandard task from the important task that meets the screening rule.

[0135] If the judgment result is yes, the important task is taken as the substandard task.

[0136] As an optional implementation, the determining, by the optimization strategy generation module 24, of the substandard reason of the substandard task according to the metadata specifically comprises:

[0137] obtaining, according to the metadata, a time difference between a start time of the substandard task and an end time of an upstream model task of the substandard task, and determining that the substandard reason of the substandard task is a scheduling time difference if the time difference is greater than a first time threshold;

[0138] obtaining, according to the metadata, a start time of an initial task on which the substandard task depends, and determining that the substandard reason of the substandard task is late initial task start if the start time is later than a preset time point;

[0139] obtaining, according to the metadata, a running time length of the substandard task, and determining that the substandard reason of the substandard task is long task running time if the running time length is greater than a second time threshold;

[0140] obtaining, according to the metadata, a processing link level of the substandard task, and determining that the substandard reason of the substandard task is high task processing link level if the processing link level is greater than a second quantity threshold.

[0141] As an optional implementation, the generating, by the optimization strategy generation module 24, of the optimization strategy of the substandard task according to the substandard reason specifically comprises:

[0142] if the substandard reason of the substandard task is determined to be the scheduling time difference, the optimization strategy is to advance the start time of the substandard task;

[0143] if the substandard reason of the substandard task is determined to be the late initial task start, the optimization strategy is to advance the start time of the initial task on which the substandard task depends;

[0144] if the substandard reason of the substandard task is determined to be the long task running time, the optimization strategy is to determine a reason for the long running time of the substandard task and generate a corresponding optimization strategy;

[0145] if the substandard reason of the substandard task is determined to be the high task processing link level, the optimization strategy is to perform task decoupling.

[0146] Figure 3 is a structural schematic diagram of an electronic device of a third embodiment of the present application. As shown in Figure 3 the electronic device 30 comprises a processor 31 and a memory 32 coupled to the processor 31.

[0147] The memory 32 stores program instructions for implementing the model task timeliness optimization method of any of the above embodiments.

[0148] The processor 31 is configured to execute the program instructions stored in the memory 32 to perform the model task timeliness optimization.

[0149] The processor 31 can also be referred to as a CPU (Central Processing Unit). The processor 31 can be an integrated circuit chip having a processing capability of signals. The processor 31 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application-Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0150] Referring to Figure 4 , Figure 4 FIG. 4 is a structural schematic diagram of a storage medium according to the fourth embodiment of the present application. The storage medium according to the embodiment of the present application stores program instructions 41 capable of implementing all the methods described above. The storage medium can be non-volatile or volatile. The program instructions 41 can be stored in the storage medium in the form of a software product, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The storage medium described above includes a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, and various media capable of storing program codes, or a terminal device such as a computer, a server, a mobile phone, a tablet, etc.

[0151] In the several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0152] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit. The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent flow transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

[0153] The above is only an embodiment of the present application, and it should be noted that, for those skilled in the art, improvements can be made without departing from the inventive concept, but these are within the protection scope of the present application.

Claims

1. A model task time optimization method, characterized by, The method comprises the following steps: acquiring metadata of a model task; acquiring an important task meeting preset rules from the model task according to the metadata, the preset rules comprising at least one preset condition, the important task being a model task meeting at least one preset condition; acquiring a substandard task meeting screening rules from the important task according to the metadata, the screening rules comprising at least one screening condition, the substandard task being the important task meeting at least one screening condition; determining a substandard reason of the substandard task according to the metadata, and generating an optimization strategy of the substandard task according to the substandard reason; wherein the preset rules comprise a first preset condition and / or a second preset condition; the first preset condition is that a downstream application of a processing link where the model task is located is an important application; and the step of acquiring an important task meeting preset rules from the model task according to the metadata comprises judging whether the downstream application of the processing link where the model task is located is an important application according to the metadata; if the result of the judgment is yes, the model task is regarded as the important task; the second preset condition is that the number of downstream tasks dependent on the model task is greater than a first quantity threshold; and the step of acquiring an important task meeting preset rules from the model task according to the metadata comprises judging whether the number of downstream tasks dependent on the model task is greater than a first quantity threshold according to the metadata; if the result of the judgment is yes, the model task is regarded as the important task; the screening rules comprise a first screening condition, the first screening condition being that the achievement rate of the important task is less than an achievement rate threshold; and the step of acquiring a substandard task meeting screening rules from the important task according to the metadata comprises judging whether the achievement rate of the important task is less than an achievement rate threshold according to the metadata; if the result of the judgment is yes, the important task is regarded as the substandard task.

2. The model task aging optimization method of claim 1, wherein, The metadata comprises the start time of an initial task on which the model task depends, the start time and the end time of an upstream model task on which the model task depends, the number of downstream tasks dependent on the model task, the level of a downstream application, and the start time, the end time, the runtime and the processing link level of the model task.

3. The model task aging optimization method of claim 1, wherein, The step of determining a substandard reason of the substandard task according to the metadata comprises the following steps: acquiring the time difference between the start time of the substandard task and the end time of the upstream model task on which the substandard task depends according to the metadata, and determining that the substandard reason of the substandard task is a scheduling time difference if the time difference is greater than a first time threshold; acquiring the start time of the initial task on which the substandard task depends according to the metadata, and determining that the substandard reason of the substandard task is that the initial task is started late if the start time is later than a preset time point; acquiring the runtime of the substandard task according to the metadata, and determining that the substandard reason of the substandard task is that the task is run for a long time if the runtime is greater than a second time threshold. According to the metadata, a processing link level of the substandard task is obtained, and if the processing link level is greater than a second quantity threshold, a substandard reason of the substandard task is determined as that the processing link level of the task is too many.

4. The model task aging optimization method of claim 1, wherein, The generating of the optimization strategy of the substandard task according to the substandard reason comprises: if the substandard reason of the substandard task is determined as that there is a scheduling time difference, the optimization strategy is to advance the starting time of the substandard task; if the substandard reason of the substandard task is determined as that the initial task is started late, the optimization strategy is to advance the starting time of the initial task on which the substandard task depends; if the substandard reason of the substandard task is determined as that the task running time is long, the optimization strategy is to determine the reason why the substandard task running time is long and generate a corresponding optimization strategy; if the substandard reason of the substandard task is determined as that the processing link level of the task is too many, the optimization strategy is to decouple the task.

5. The model task time expiration optimization apparatus, characterized by, Comprise: a metadata obtaining module, configured to obtain metadata of model tasks; an important task obtaining module, configured to obtain important tasks meeting preset rules from the model tasks according to the metadata, the preset rules comprising at least one preset condition, and the important tasks being model tasks at least meeting one preset condition; a substandard task screening module, configured to obtain substandard tasks meeting screening rules from the important tasks according to the metadata, the screening rules comprising at least one screening condition, and the substandard tasks being the important tasks at least meeting one screening condition; an optimization strategy generating module, configured to determine a substandard reason of the substandard tasks according to the metadata and generate an optimization strategy of the substandard tasks according to the substandard reason. The preset rules comprise a first preset condition and / or a second preset condition. The first preset condition is that a downstream application of a processing link where the model task is located is an important application; and the obtaining of the important tasks meeting the preset rules from the model tasks according to the metadata comprises determining whether the downstream application of the processing link where the model task is located is an important application according to the metadata; and if the determination result is yes, the model task is taken as the important task. The second preset condition is that a number of downstream tasks depending on the model task is greater than a first quantity threshold; and the obtaining of the important tasks meeting the preset rules from the model tasks according to the metadata comprises determining whether the number of downstream tasks depending on the model task is greater than the first quantity threshold according to the metadata; and if the determination result is yes, the model task is taken as the important task. The screening rules comprise a first screening condition, the first screening condition being that an achievement rate of the important task is less than an achievement rate threshold; and the obtaining of the substandard tasks meeting the screening rules from the important tasks according to the metadata comprises determining whether the achievement rate of the important task is less than the achievement rate threshold according to the metadata; and if the determination result is yes, the important task is taken as the substandard task.

6. An electronic device, comprising: The model task time efficiency optimization method comprises a processor and a memory coupled with the processor, wherein the memory stores program instructions executable by the processor; and the processor executes the program instructions stored in the memory to implement the model task time efficiency optimization method according to any one of claims 1-4.

7. A storage medium, characterized by The storage medium stores program instructions, and the program instructions are executed by a processor to implement the model task time efficiency optimization method according to any one of claims 1-4.

Images