Task processing method and device, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of computers and data processing, can be applied to the fields of medical treatment, finance and the like, and discloses a task processing method and device, computer equipment and a readable storage medium. The method comprises the steps of obtaining current attribute information of a task, and judging whether the task needs to be verified or not according to the current attribute information; if verification is needed, obtaining a current log of the task, and judging whether a query database of the task is an Oracle type or not according to the current log; if the task is the Oracle type, whether the task is in a stuck state or not is judged according to the current log; if the current log is in the stuck state, all historical logs of the task are obtained, and whether the ratio of the current stuck duration of the task to the historical average running duration of the task under the condition of the same query data size exceeds a preset ratio or not is judged according to the current log and all the historical logs; and if the preset ratio is exceeded, stopping the task and then restarting the task. According to the method, the task delay risk can be greatly reduced, so that the decision error risk is reduced.

Description

Technical Field

[0001] This invention relates to the field of computer and data processing technology, and in particular to a task processing method, apparatus, computer equipment and readable storage medium, which can be applied to fields such as medicine and finance. Background Technology

[0002] As business expands, applications remain online for longer periods, and the user base grows, the amount of data stored in the database also increases. To extract useful data, relevant big data components are needed to periodically query the database and analyze the content via scheduled tasks. For example, in the healthcare field, if a medical application wants to analyze the geographical distribution of a disease's incidence, a relevant big data component needs to query the database of patients with that disease and analyze their geographical distribution. Similarly, in the financial field, if a financial application wants to analyze the age distribution of users who purchase a financial product, a relevant big data component needs to query the database of users who purchase that financial product and analyze their age distribution.

[0003] However, various reasons (such as insufficient database performance, excessive database usage, and big data component crashes) can lead to task delays.

[0004] Delays occurring during the execution of business data extraction tasks can lead to extraction failures if left unaddressed, impacting business decision-making. These delays may be caused by various factors, such as increased business data volume or decreased database performance, and a single solution cannot address each cause. With tens of thousands of such tasks, manual delay analysis would incur excessive manpower costs. Summary of the Invention

[0005] This invention provides a task processing method, apparatus, computer device, and readable storage medium that systematically analyzes whether a task is functioning correctly and intervenes in a timely manner for stuck tasks, thereby reducing the risk of task delays.

[0006] Firstly, a task processing method is provided, which includes:

[0007] Get the current attribute information of the task, and determine whether the task needs to be verified based on the current attribute information;

[0008] If verification is required, obtain the current log of the task and determine whether the database queried by the task is of type Oracle based on the current log;

[0009] If it is an Oracle type, determine whether the task is stuck based on the current log;

[0010] If the task is stuck, retrieve all historical logs of the task and determine whether the ratio of the current stuck duration of the task to the historical average runtime of the task under the same query data volume exceeds the preset ratio based on the current log and all historical logs.

[0011] If the preset ratio is exceeded, the task will be terminated and then restarted.

[0012] Secondly, a task processing apparatus is provided, the task processing apparatus comprising:

[0013] The verification and judgment unit is used to obtain the current attribute information of the task and determine whether the task needs to be verified based on the current attribute information.

[0014] The type determination unit is used to obtain the current log of the task when verification is required, and to determine whether the database queried by the task is of type Oracle based on the current log.

[0015] The status judgment unit is used to determine whether the task is stuck based on the current log when it is of Oracle type;

[0016] The ratio judgment unit is used to obtain all historical logs of the task when it is stuck, and to judge whether the ratio of the current stuck time of the task to the historical average running time of the task under the same query data volume exceeds the preset ratio based on the current log and all historical logs.

[0017] The terminate / restart unit is used to terminate and restart a task when the preset ratio is exceeded.

[0018] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described task processing method.

[0019] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the recognition processing method described above.

[0020] In the above-described task processing method, apparatus, computer equipment, and readable storage medium, the current attribute information of the task is obtained, and it is determined whether the task needs to be verified based on the current attribute information. If verification is required, the current log of the task is obtained, and it is determined whether the database queried by the task is of Oracle type based on the current log. If it is of Oracle type, it is determined whether the task is stuck based on the current log. If it is stuck, it is determined whether the ratio of the current stuck duration of the task to the historical average runtime of the task under the same query data volume exceeds a preset ratio based on the current log and all historical logs of the task. If it exceeds the preset ratio, the task is terminated and then restarted.

[0021] The task processing method proposed in this invention performs real-time analysis of the task execution process and automatically processes tasks when relevant conditions are met, which can greatly reduce the risk of task delays and thus reduce the risk of decision-making errors. The task processing method of this invention compares historical task execution data to determine whether the task execution is normal, analyzes whether the task progress not updating is due to a freeze or data querying, and intervenes accurately according to the specific situation to avoid erroneous operations on the task. The task processing method proposed in this invention requires no manual intervention, reducing labor costs. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a task processing method according to an embodiment of the present invention;

[0024] Figure 2 This is a flowchart illustrating a task processing method in another embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the structure of a task processing device according to an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The task processing method proposed in this invention can be executed by a task processing device deployed on a server. When retrieving useful data from a database, relevant big data components are used to periodically query the database via scheduled tasks. Data retrieval is primarily scheduled for the early morning hours, therefore, it is generally not monitored in real-time. Task dependencies are complex and the dependency processes are lengthy; a stuck task can affect the retrieval of many subsequent tasks. The task processing device deployed on the server is used to analyze the execution of scheduled tasks, determine whether the tasks are functioning correctly, and intervene promptly in stuck tasks, thereby reducing the risk of task delays.

[0029] Please see Figure 1 As shown, Figure 1 This is a flowchart illustrating a task processing method according to an embodiment of the present invention. The method includes the following steps:

[0030] Step S110: Obtain the current attribute information of the task, and determine whether the task needs to be verified based on the current attribute information.

[0031] The device checks whether the task requires verification based on its current attribute information and verification conditions. If the task meets the verification conditions, it proceeds to the next step of processing; otherwise, no processing is required.

[0032] In some optional implementations, step S110, obtaining the current attribute information of the task and determining whether the task needs to be verified based on the current attribute information, includes: obtaining the current attribute information, which includes: the running status of the task, the module to which the task belongs, and the running time of the task; if the running status is running, the module to which the task belongs is an important module, and the running time exceeds a preset time threshold, it is determined that verification is required.

[0033] The device acquires the task's running status, the module to which the task belongs, and the task's running duration.

[0034] The running status of a task includes running and not running. Running indicates that the task has been started and has not yet been terminated; not running indicates that the task has not been started or has been terminated.

[0035] Tasks are categorized into important and unimportant modules. Important modules represent query results that are crucial to the analysis; unimportant modules represent query results that are less important to the analysis. This classification can be determined based on practical experience.

[0036] For running tasks, the runtime represents the duration from task startup to the current time. For terminated tasks, the runtime represents the duration from task startup to task termination. For tasks that have not started, the runtime attribute is empty.

[0037] The verification conditions are: running, critical modules, and runtime exceeding a preset time threshold. The preset time threshold can be determined based on actual conditions and experience. That is, the task only needs to proceed to the next step if all three conditions are met simultaneously; if the task does not meet any of the three conditions, no processing is required.

[0038] For example, if the device detects a task as running, belonging to an important module, and having exceeded a preset time threshold, then the task needs further processing. If the device detects a task as not running, belonging to an important module, and having not exceeded the preset time threshold, then the task does not need processing. If the device detects a task as running, belonging to a non-important module, and having not exceeded the preset time threshold, then the task does not need processing.

[0039] Step S120: If verification is required, obtain the current log of the task and determine whether the database queried by the task is of Oracle type based on the current log.

[0040] For tasks requiring verification, the device analyzes and extracts the current logs of the task to determine the type of database required for the query. If the database is Oracle, further processing is performed; otherwise, no processing is required.

[0041] In some optional implementations, after step S120, which determines whether the database being queried by the task is of Oracle type based on the current log, the method further includes: if it is Oracle type, determining the table and field information used by the task based on the current log; determining whether the table and field information has an index in the query database; if an index exists, determining whether the task is stuck based on the current log; if no index exists, providing a prompt to add an index.

[0042] If the queried database is Oracle, the device extracts the table and field information used by the task from the current log and determines whether indexes exist in the queried database. If an index exists, it proceeds to the next step; otherwise, it provides a prompt to add the index and contacts relevant personnel to evaluate its creation.

[0043] The presence of indexes on tables and fields has a significant impact on data query efficiency. The system should promptly determine whether indexes exist in the database for the tables and fields used in the analysis tasks. If not, it should contact relevant personnel to assess and add them, thereby improving query efficiency.

[0044] Step S130: If it is an Oracle type, determine whether the task is stuck based on the current log.

[0045] For tasks querying Oracle databases, the device continues to analyze the task's current logs to determine if the task is stuck. If it is stuck, further processing is performed; otherwise, no processing is required.

[0046] Specifically, the device determines whether the task is currently in a state where both map and reduce are at 0% based on the current logs. If both map and reduce are at 0%, the task is stuck; if neither map nor reduce is at 0%, or neither map nor reduce is at 0%, the task is not stuck.

[0047] Step S140: If the task is stuck, obtain all historical logs of the task, and determine whether the ratio of the current stuck duration of the task to the historical average runtime of the task under the same query data volume exceeds the preset ratio based on the current log and all historical logs.

[0048] For tasks that are stuck, the device continues to analyze the task's current log and all historical logs to determine whether the task needs to be terminated. The determination is based on whether the current stuck duration of the task exceeds a preset ratio of the task's historical average runtime under the same query data volume.

[0049] In some optional implementations, step S140, determining whether the ratio of the current stall duration of the task to the historical average runtime of the task under the same query data volume exceeds a preset ratio based on the current log and all historical logs, includes: determining the total stall duration of the task and the total number of data entries to be extracted based on the current log; determining the historical average runtime and the historical average amount of data read based on all historical logs; determining the current stall duration based on the total stall duration, the total number of data entries to be extracted, and the historical average amount of data read; and determining whether the ratio of the current stall duration to the historical average runtime exceeds a preset ratio.

[0050] Based on the current logs, the device can determine the total stall time of the task and the total number of data records to be extracted. Specifically, the total stall time is determined based on the current time and the write time when the map is at 0% and the reduce is at 0%; the total number of data records to be extracted is determined based on the task's query statement.

[0051] The device analyzes the current logs and determines the total stuck time by subtracting the current time from the write time when both map and reduce are at 0%.

[0052] For example, if the current time is 17:32:33, and the task's current map and reduce percentages are 0% and the write time is 17:28:21, then the total stuck time S is 4 minutes and 12 seconds.

[0053] The device analyzes and extracts SQL statements from the current log, and accesses the database to query the total number of data records required to be extracted based on the SQL statement's constraints.

[0054] For example: The device determines that the total number of data entries required to be extracted for the task is M.

[0055] Based on all historical logs, the device can determine the historical average runtime and historical average data read volume of a task. Specifically, for any historical log, the single historical runtime of the task is determined based on the write time when the historical map was 100% and the reduce was 100% and the write time when the historical map was 0% and the reduce was 0%; the historical average runtime is determined based on all single historical runtimes; for any historical log, the single historical data read volume of the task is determined based on historical query records; the historical average data read volume is determined based on all single historical data read volumes.

[0056] The device analyzes any historical log and determines the duration of a single historical run by subtracting the write time when the historical map is at 100% and the reduce is at 100% from the write time when the historical map is at 0% and the reduce is at 0%.

[0057] For example, for a historical log, the write time when the historical map is 100% and the reduce is 100% is 9 hours, 33 minutes, and 10 seconds, and the write time when the historical map is 0% and the reduce is 0% is 9 hours, 33 minutes, and 0 seconds. Then, the runtime of a single historical log is s. i It lasts for 10 seconds.

[0058] The average historical runtime is determined by calculating the runtime of each historical log entry and then averaging the results.

[0059] For example, the historical average runtime s is 30 seconds.

[0060] Because the amount of data may increase, further analysis of the data volume is needed, even after calculating the total task stall time and historical average runtime. The device queries relevant historical query information through any historical log of the task to determine the number of rows of data read in history as the single historical data read volume.

[0061] For example, for a historical log, the amount of data read in a single historical session is m. i .

[0062] After querying all historical logs to determine the amount of data read in a single historical period, the average value is taken to determine the average amount of historical data read.

[0063] For example, the average amount of historical data read is m.

[0064] The device can determine the current jam duration based on the total jam duration, the total number of data entries to be extracted, and the average amount of historical data read. Specifically, it determines the data growth factor based on the ratio of the total number of data entries to be extracted to the average amount of historical data read; and determines the current jam duration based on the ratio of the total jam duration to the data growth factor.

[0065] Compare the total number of data entries to be extracted with the average amount of historical data to determine if the data volume has increased and calculate the data growth rate.

[0066] For example, if the total number of data entries to be extracted is M, and the average amount of historical data read is m, then the data growth factor N = M / m.

[0067] Divide the total stuck time by the data growth factor to determine the current stuck time of the task.

[0068] For example, if the total stuck time is S and the data growth rate is N, the current stuck time t = S / N. For instance, if the total stuck time S is 4 minutes and 12 seconds and the data growth rate is 10 times, then the current stuck time t is 25.2 seconds.

[0069] The device then compares the current jam duration with the historical average runtime to determine whether the current jam duration exceeds a preset ratio of the historical average runtime. The preset ratio can be determined based on experience.

[0070] For example, the t / s result is compared with a preset ratio. If the current stall duration t is 25.2 seconds and the historical average run time s is 30 seconds, then the t / s result is 84%.

[0071] Step S150: If the preset ratio is exceeded, terminate the task and then restart the task.

[0072] For tasks exceeding the preset ratio, the device terminates the task and then restarts it. For tasks within the preset ratio, no processing is required.

[0073] For example, if the t / s result is 84% ​​and the preset ratio is 50%, then it is determined that the preset ratio has been exceeded. In this case, the device will automatically kill (terminate) the task and restart it.

[0074] The process of steps S110 to S150 proposed in this invention is executed periodically in a loop to analyze the operation of the task at any time and to promptly identify and intervene in any stuck task.

[0075] Depend on Figure 1 As shown in the method, the task processing method proposed in this invention obtains the current attribute information of the task and determines whether the task needs to be verified based on the current attribute information. If verification is required, the current log of the task is obtained, and it is determined whether the database queried by the task is of Oracle type based on the current log. If it is of Oracle type, it is determined whether the task is stuck based on the current log. If it is stuck, all historical logs of the task are obtained, and it is determined whether the ratio of the current stuck duration of the task to the historical average runtime of the task under the same query data volume exceeds a preset ratio based on the current log and all historical logs. If it exceeds the preset ratio, the task is terminated and then restarted.

[0076] The task processing method proposed in this invention performs real-time analysis of the task execution process and automatically processes tasks when relevant conditions are met, which can greatly reduce the risk of task delays and thus reduce the risk of decision-making errors. The task processing method of this invention compares historical task execution data to determine whether the task execution is normal, analyzes whether the task progress not updating is due to a freeze or data querying, and intervenes accurately according to the specific situation to avoid erroneous operations on the task. The task processing method proposed in this invention requires no manual intervention, reducing labor costs.

[0077] Please see Figure 2 As shown, Figure 2A flowchart illustrating a task processing method according to another embodiment of the present invention. The method includes the following steps:

[0078] Step S201: Obtain the task's running status, the module to which the task belongs, and the task's runtime. Proceed to step S202.

[0079] Step S202: Determine if the running status is "running". If yes, proceed to step S203; otherwise, proceed to step S217.

[0080] Step S203: Determine whether the runtime exceeds a preset time threshold. If yes, proceed to step S204; otherwise, proceed to step S217.

[0081] Step S204: Determine whether the module belongs to an important module. If yes, proceed to step S205; otherwise, proceed to step S217.

[0082] Step S205: Obtain the current log of the task. Proceed to step S206.

[0083] Step S206: Determine if the database being queried by the task is of type Oracle. If yes, proceed to step S207; otherwise, proceed to step S217.

[0084] Step S207: Determine the table and field information used by the task based on the current log. Proceed to step S208.

[0085] Step S208: Determine if the table and field information have an index in the query database. If yes, proceed to step S209; otherwise, proceed to step S210.

[0086] Step S209: Determine if the task is currently in a state where both map and reduce are at 0%. If yes, proceed to step S211; otherwise, proceed to step S217.

[0087] Step S210: Add a prompt to the feedback index.

[0088] Step S211: Obtain all historical logs for the task. Proceed to step S212.

[0089] Step S212: Determine the total stall duration based on the current time and the write time when both map and reduce are at 0%; determine the total number of data records to be extracted based on the task's query statement. Proceed to step S213.

[0090] Step S213: For any historical log, determine the single historical runtime of the task based on the write time when the historical map is 100% and the reduce is 100% and the write time when the historical map is 0% and the reduce is 0%; determine the historical average runtime based on all single historical runtimes; for any historical log, determine the single historical data read volume of the task based on historical query records; determine the historical average data read volume based on all single historical data read volumes. Proceed to step S214.

[0091] Step S214: Determine the data growth factor based on the ratio of the total number of data entries to be extracted to the average amount of historical data read; determine the current stall duration based on the ratio of the total stall duration to the data growth factor. Proceed to step S215.

[0092] Step S215: Determine whether the ratio of the current stuck duration to the historical average running duration exceeds a preset ratio. If yes, proceed to step S216; otherwise, proceed to step S217.

[0093] Step S216: Terminate the task and then restart the task.

[0094] Step S217: No processing performed.

[0095] The following are specific embodiments in a medical setting to illustrate the task processing method proposed in this invention.

[0096] In the medical field, if a medical application wants to analyze the geographical distribution of a particular disease, it requires relevant big data components to query the database of the medical application for basic information on patients suffering from that disease and analyze its geographical distribution. Therefore, the task is to query the database for basic information on patients suffering from this disease. This task will be analyzed.

[0097] If the task's running status is "running", the module is classified as an important module, or the running time has exceeded the preset time threshold, then proceed to the next step.

[0098] If the database being queried is determined to be Oracle, proceed to the next step.

[0099] If the tables and fields used in the task are found to have indexes in the query database, proceed to the next step. For example, the fields might include: disease name, place of residence, etc.

[0100] If the task is determined to be in a state where the map is 0% and the reduce is 0%, then proceed to the next step.

[0101] The total stall duration is determined based on the current time and the write time when the current map and reduce phases are both 0%. The total number of data records to be extracted is determined based on the task's query statement. The single historical runtime of the task is determined based on the historical write time when the map and reduce phases are both 100% and 100%. The historical average runtime is determined based on all single historical runtimes. The single historical data read volume of the task is determined based on historical query records. The historical average data read volume is determined based on all single historical data read volumes. The data growth factor is determined based on the ratio of the total number of data records to be extracted to the historical average data read volume. The current stall duration is determined based on the ratio of the total stall duration to the data growth factor.

[0102] If the ratio of the current stuck duration to the historical average running duration exceeds the preset ratio, proceed to the next step.

[0103] Restart the task after terminating it.

[0104] The following are specific examples of financial scenarios to illustrate the task processing method proposed in this invention.

[0105] In the financial sector, analyzing the age distribution of buyers of a specific financial product requires using big data components to retrieve basic information about these buyers from a database and then analyzing their age distribution. Therefore, the task is to retrieve basic information about buyers of this financial product from a database. This task will be analyzed.

[0106] If the task's running status is "running", the module is classified as an important module, or the running time has exceeded the preset time threshold, then proceed to the next step.

[0107] If the database being queried is determined to be Oracle, proceed to the next step.

[0108] If the tables and fields used in the task are found to have indexes in the query database, proceed to the next step. For example, the fields might include: financial product name, user's date of birth, etc.

[0109] If the task is determined to be in a state where the map is 0% and the reduce is 0%, then proceed to the next step.

[0110] The total stall duration is determined based on the current time and the write time when the current map and reduce phases are both 0%. The total number of data records to be extracted is determined based on the task's query statement. The single historical runtime of the task is determined based on the historical write time when the map and reduce phases are both 100% and 100%. The historical average runtime is determined based on all single historical runtimes. The single historical data read volume of the task is determined based on historical query records. The historical average data read volume is determined based on all single historical data read volumes. The data growth factor is determined based on the ratio of the total number of data records to be extracted to the historical average data read volume. The current stall duration is determined based on the ratio of the total stall duration to the data growth factor.

[0111] If the ratio of the current stuck duration to the historical average running duration is not preset, no action will be taken.

[0112] In one embodiment, a task processing apparatus is provided, which corresponds one-to-one with the task processing methods described in the above embodiments. For example... Figure 3 As shown, the task processing device 300 includes:

[0113] The verification and judgment unit 310 is used to obtain the current attribute information of the task and determine whether the task needs to be verified based on the current attribute information.

[0114] The type determination unit 320 is used to obtain the current log of the task when verification is required, and to determine whether the database queried by the task is of type Oracle based on the current log.

[0115] Status judgment unit 330 is used to determine whether the task is stuck based on the current log when it is of Oracle type;

[0116] The ratio judgment unit 340 is used to obtain all historical logs of the task when it is stuck, and judge whether the ratio of the current stuck time of the task to the historical average running time of the task under the same query data volume exceeds the preset ratio based on the current log and all historical logs.

[0117] The termination and rerun unit 350 is used to restart the task after terminating it when the preset ratio is exceeded.

[0118] In some optional implementations, in the above-described device 300, the verification and judgment unit 310 is specifically used to: obtain current attribute information, which includes: the running status of the task, the module affiliation of the task, and the running duration of the task; if the running status is running, the module affiliation is an important module, and the running duration exceeds a preset duration threshold, it is determined that verification is required.

[0119] In some optional implementations, the above-mentioned device 300 further includes an index determination unit, used to: if it is an Oracle type, determine the table and field information used by the task based on the current log; determine whether the table and field information has an index in the query database; if no index exists, provide a prompt to add an index.

[0120] In some optional embodiments, in the above-described apparatus 300, the ratio judgment unit 340 includes: a current task analysis module, used to determine the total stuck time of the task and the total number of data entries to be extracted based on the current log; a historical task analysis module, used to determine the historical average runtime and the historical average amount of data read based on all historical logs; a normalization module, used to determine the current stuck time based on the total stuck time, the total number of data entries to be extracted, and the historical average amount of data read; and a duration ratio judgment module, used to determine whether the ratio of the current stuck time to the historical average runtime exceeds a preset ratio.

[0121] In some optional implementations, in the above-described apparatus 300, the current task analysis module is specifically used to: determine the total stuck time based on the current time and the write time when the current map is 0% and the reduce is 0%; and determine the total number of data entries to be extracted based on the task's query statement.

[0122] In some optional implementations, in the above-described apparatus 300, the historical task analysis module is specifically used to: determine the single historical runtime of a task for any historical log based on the write time when the historical map is 100% and the reduce is 100% and the write time when the historical map is 0% and the reduce is 0%; determine the historical average runtime based on all single historical runtimes; determine the single historical read data volume of a task for any historical log based on historical query records; and determine the historical average read data volume based on all single historical read data volumes.

[0123] In some optional implementations, in the above-described device 300, the normalization module is specifically used to: determine the data growth factor based on the ratio of the total number of data entries to be extracted to the average amount of historical data read; and determine the current jamming time based on the ratio of the total jamming time to the data growth factor.

[0124] Specific limitations regarding the task processing device can be found in the limitations of the task processing method described above, and will not be repeated here. Each unit or module in the aforementioned task processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These units or modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each unit or module.

[0125] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external devices via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a task processing method.

[0126] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0127] Get the current attribute information of the task, and determine whether the task needs to be verified based on the current attribute information;

[0128] If verification is required, obtain the current log of the task and determine whether the database queried by the task is of type Oracle based on the current log;

[0129] If it is an Oracle type, determine whether the task is stuck based on the current log;

[0130] If the task is stuck, retrieve all historical logs of the task and determine whether the ratio of the current stuck duration of the task to the historical average runtime of the task under the same query data volume exceeds the preset ratio based on the current log and all historical logs.

[0131] If the preset ratio is exceeded, the task will be terminated and then restarted.

[0132] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0133] Get the current attribute information of the task, and determine whether the task needs to be verified based on the current attribute information;

[0134] If verification is required, obtain the current log of the task and determine whether the database queried by the task is of type Oracle based on the current log;

[0135] If it is an Oracle type, determine whether the task is stuck based on the current log;

[0136] If the task is stuck, retrieve all historical logs of the task and determine whether the ratio of the current stuck duration of the task to the historical average runtime of the task under the same query data volume exceeds the preset ratio based on the current log and all historical logs.

[0137] If the preset ratio is exceeded, the task will be terminated and then restarted.

[0138] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0140] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0141] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A task processing method, characterized in that, The task processing method includes: Obtain the current attribute information of the task, and determine whether the task needs to be verified based on the current attribute information; If verification is required, obtain the current log of the task and determine whether the database queried by the task is of Oracle type based on the current log; If it is an Oracle type, determine whether the task is stuck based on the current log; If the task is stuck, obtain all historical logs of the task, and determine whether the ratio of the current stuck duration of the task to the historical average runtime of the task under the same query data volume exceeds a preset ratio based on the current log and all the historical logs. If the preset ratio is exceeded, the task will be terminated and then restarted.

2. The task processing method according to claim 1, characterized in that, The step of obtaining the current attribute information of the task and determining whether the task needs verification based on the current attribute information includes: Obtain the current attribute information, which includes: the running status of the task, the module affiliation of the task, and the running duration of the task; If the running status is "running", the module is classified as an important module, and the running time exceeds a preset time threshold, it is determined that verification is required.

3. The task processing method according to claim 1, characterized in that, After the step of determining whether the database queried by the task is of type Oracle based on the current log, the method further includes: If it is an Oracle type, determine the table and field information used by the task based on the current log; Determine whether the table and field information have an index in the query database; If an index exists, determine whether the task is stuck based on the current log; If the index does not exist, provide a suggestion to add the index.

4. The task processing method according to claim 1, characterized in that, The step of determining whether the ratio of the current stall duration of the task to the historical average runtime of the task under the same query data volume exceeds a preset value based on the current log and all the historical logs includes: Based on the current log, determine the total stuck time of the task and the total number of data entries to be extracted; Based on all the historical logs, determine the historical average runtime and the historical average amount of data read; The current stuck duration is determined based on the total stuck duration, the total number of data entries to be extracted, and the average amount of historical data read. Determine whether the ratio of the current stuck duration to the historical average running duration exceeds a preset ratio.

5. The task processing method according to claim 4, characterized in that, The step of determining the total stuck time of the task and the total number of data entries to be extracted based on the current log includes: The total stuck time is determined based on the current time and the write time when the current map is 0% and the reduce is 0%. The total number of data entries to be extracted is determined based on the query statement of the task.

6. The task processing method according to claim 4, characterized in that, The step of determining the historical average runtime and the historical average amount of data read based on all the historical logs includes: For any of the aforementioned historical logs, the single historical runtime of the task is determined based on the write time when the historical map is 100% and the reduce is 100% and the write time when the historical map is 0% and the reduce is 0%. The historical average runtime is determined based on all the individual historical runtimes. For any of the aforementioned historical logs, determine the single historical data read volume for the task based on the historical query records; The average amount of historical read data is determined based on all the individual historical read data amounts.

7. The task processing method according to claim 4, characterized in that, The step of determining the current stuck duration based on the total stuck duration, the total number of data entries to be extracted, and the average amount of historical data read includes: The data growth factor is determined based on the ratio of the total number of data entries to be extracted to the average amount of historical data read. The current stuck duration is determined based on the ratio of the total stuck duration to the data growth factor.

8. A task processing device, characterized in that, The task processing device includes: The verification and judgment unit is used to obtain the current attribute information of the task and determine whether the task needs to be verified based on the current attribute information. The type determination unit is used to obtain the current log of the task when verification is required, and determine whether the database queried by the task is of Oracle type based on the current log; The status judgment unit is used to determine whether the task is stuck based on the current log when it is of Oracle type; The ratio judgment unit is used to obtain all historical logs of the task when it is stuck, and to determine whether the ratio of the current stuck time of the task to the historical average running time of the task under the same query data volume exceeds a preset ratio based on the current log and all the historical logs. The termination and rerun unit is used to terminate the task and then restart the task when the preset ratio is exceeded.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the task processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the task processing method as described in any one of claims 1 to 7.

Images