Task data management method, device and computer equipment based on Hadoop

Abstract

The present invention discloses a task data management and control method based on Hadoop, which includes: obtaining source information of a task process log of a first task management platform, identifying the target task name and target task necessary parameters therein; performing consistency comparison between the target task name and target task necessary parameters and the predecessor task name and predecessor task necessary parameters in each task record in a preset predecessor and successor task execution list; when the first task record existing in the predecessor and successor task execution list is consistent with the task name and target task necessary parameters of the target task, sending the execution instruction corresponding to the successor task in the first task record to a second task management platform. The present invention also provides a task data management and control device, a computer device and a computer-readable storage medium based on Hadoop. The present invention can realize real-time, efficient and accurate management and control of task data across task management platforms.

Description

Technical Field

[0001] The present invention relates to the field of process monitoring technology, and in particular to a task data management method, device, computer equipment and computer-readable storage medium based on Hadoop. Background Art

[0002] At present, many task management platforms that can be used to control and manage production tasks are applied to the daily production of enterprises. Of course, there are many production tasks, and different production tasks may be executed on different task management platforms. Therefore, multiple task management platforms may be required to manage and control different production task data separately to jointly complete the preset production tasks. For example, the existing OOZIE task management platform and LINKDO task management platform are both commonly used for the management of production task data, but there is no interface between the two task management platforms for the task data of the two platforms to interact. Therefore, when the preceding tasks and subsequent tasks decomposed from the production tasks are respectively assigned to the OOZIE task management platform and the LINKDO task management platform, it will be very troublesome to manage the task data between the two task management platforms. For example, it often happens that the preceding task executed by one of the task management platforms is abnormal, but the subsequent task that depends on the preceding task cannot be informed of the abnormal execution of the preceding task, then the subsequent task will run according to the normal time, which will cause the task data to be unable to be truly synchronized, and the task execution results are prone to deviations. Therefore, in the prior art, the control of task data across task management platforms cannot achieve accurate and synchronous effects. Summary of the invention

[0003] In view of this, the present invention proposes a method, device, computer equipment and computer-readable storage medium for user task data management based on Hadoop, which can obtain the source information of the task process log of the first task management platform, and then identify it to identify the target task name and the target task necessary parameters; then the target task name and the target task necessary parameters are compared for consistency with the predecessor task name and the predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list; when the predecessor task name and the predecessor task necessary parameters in the first task record in the predecessor and successor task execution list are consistent with the target task name and the target task necessary parameters, the execution instruction corresponding to the successor task in the first task record is sent to the second task management platform. In this way, real-time, efficient and accurate management and control of task data can be achieved across task management platforms.

[0004] First, to achieve the above purpose, the present invention provides a task data management method based on Hadoop, the method comprising:

[0005] Acquire source information of the task process log of the first task management platform; identify the source information of the task process log to identify the target task name and target task necessary parameters; compare the target task name and target task necessary parameters with the predecessor task name and predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list for consistency; when the predecessor task name and predecessor task necessary parameters in the first task record existing in the predecessor and successor task execution list are consistent with the target task name and target task necessary parameters, send the execution instruction corresponding to the successor task in the first task record to the second task management platform.

[0006] Optionally, the step of identifying the source information of the task process log and identifying the target task name and necessary parameters of the target task includes: converting the source information of the task process log into log text information; performing sentence segmentation on the log text information to obtain at least one task execution record statement; performing text recognition on each task execution record statement to identify the target task name and necessary parameters of the target task.

[0007] Optionally, the step of performing text recognition on each task execution record statement includes: identifying each task execution record statement according to a preset target task name keyword, thereby finding a first task execution record statement including the target task name; identifying the first task execution record statement according to a preset necessary parameter format, thereby identifying the target task necessary parameters corresponding to the target task name.

[0008] Optionally, the necessary parameters include time and operating status, and the necessary parameter formats include time format and operating status format.

[0009] In addition, to achieve the above purpose, the present invention also provides a task data management device based on Hadoop, the device comprising:

[0010] An acquisition module is used to acquire source information of a task process log of a first task management platform; an identification module is used to identify the source information of the task process log and identify a target task name and target task necessary parameters; a judgment module is used to compare the target task name and target task necessary parameters with the predecessor task name and predecessor task necessary parameters in each task record in a preset predecessor and successor task execution list for consistency; a sending module is used to send an execution instruction corresponding to a successor task in the first task record to a second task management platform when the predecessor task name and predecessor task necessary parameters in the first task record in the predecessor and successor task execution list are consistent with the target task name and target task necessary parameters.

[0011] Optionally, the identification module is also used to: convert the source information of the task process log into log text information; perform sentence segmentation on the log text information to obtain at least one task execution record statement; perform text recognition on each task execution record statement to identify the target task name and necessary parameters of the target task.

[0012] Optionally, the identification module is also used to: identify each task execution record statement according to a preset target task name keyword, thereby finding the first task execution record statement including the target task name; identify the first task execution record statement according to a preset necessary parameter format, thereby identifying the target task necessary parameters corresponding to the target task name.

[0013] Optionally, the necessary parameters include time and operating status, and the necessary parameter formats include time format and operating status format.

[0014] Furthermore, the present invention also proposes a computer device, which includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the steps of the Hadoop-based task data management method as described above are implemented.

[0015] Furthermore, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, which stores a computer program, and the computer program can be executed by at least one processor so that the at least one processor performs the steps of the Hadoop-based task data management method as described above.

[0016] Compared with the prior art, the Hadoop-based task data management method, device, computer equipment and computer-readable storage medium proposed in the present invention can obtain the source information of the task process log of the first task management platform, and then identify it to identify the target task name and the target task necessary parameters; then the target task name and the target task necessary parameters are compared with the predecessor task name and the predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list for consistency; when the predecessor task name and the predecessor task necessary parameters in the first task record in the predecessor and successor task execution list are consistent with the target task name and the target task necessary parameters, the execution instruction corresponding to the successor task in the first task record is sent to the second task management platform. In this way, real-time, efficient and accurate management and control of task data can be achieved across task management platforms. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic diagram of an optional hardware architecture of a computer device of the present invention;

[0018] Figure 2 It is a schematic diagram of program modules of an embodiment of a task data control device based on Hadoop of the present invention;

[0019] Figure 3 It is a flowchart of an embodiment of a task data control method based on Hadoop of the present invention.

[0020] Reference numerals:

[0021]

[0022]

[0023] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] It should be noted that the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0026] See also Figure 1 , which is a schematic diagram of an optional hardware architecture of the computer device 1 of the present invention.

[0027] In this embodiment, the computer device 1 may include, but is not limited to, a memory 11, a processor 12, and a network interface 13 that can be interconnected and communicated via a system bus.

[0028] The computer device 1 is connected to the network via the network interface 13 ( Figure 1The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), the Wideband Code Division Multiple Access (WCDMA), the 4G network, the 5G network, Bluetooth, Wi-Fi, the call network, etc.

[0029] It should be pointed out that Figure 1 The computer device 1 is shown only with components 11 - 13 , but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0030] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the computer device 1, such as a hard disk or memory of the computer device 1. In other embodiments, the memory 11 can also be an external storage device of the computer device 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the computer device 1. Of course, the memory 11 can also include both the internal storage unit of the computer device 1 and its external storage device. In this embodiment, the memory 11 is generally used to store the operating system and various application software installed on the computer device 1, such as the program code of the task data control device 200 based on Hadoop. In addition, the memory 11 can also be used to temporarily store various types of data that have been output or are to be output.

[0031] The processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 12 is generally used to control the overall operation of the computer device 1, such as performing control and processing related to data interaction or communication. In this embodiment, the processor 12 is used to run the program code stored in the memory 11 or process data, such as running the task data control device 200 based on Hadoop.

[0032] The network interface 13 may include a wireless network interface or a wired network interface, and the network interface 13 is generally used to establish a communication connection between the computer device 1 and other terminal devices such as a mobile terminal, a user device, a PC, and other task management platforms.

[0033] In this embodiment, when a task data control device 200 based on Hadoop is installed and running in the computer device 1, when the task data control device 200 based on Hadoop is running, it can directly obtain the task process log of the first task management platform, and then identify it to identify the target task name and the target task necessary parameters; then the target task name and the target task necessary parameters are compared with the predecessor task name and the predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list for consistency; when the predecessor task name and the predecessor task necessary parameters in the first task record in the predecessor and successor task execution list are consistent with the target task name and the target task necessary parameters, the execution instruction corresponding to the successor task in the first task record is sent to the second task management platform. In this way, real-time, efficient and accurate control of task data can be achieved across task management platforms.

[0034] So far, the application environment of each embodiment of the present invention and the hardware structure and function of the related equipment have been described in detail. Next, based on the above application environment and related equipment, various embodiments of the present invention are proposed.

[0035] First, the present invention proposes a task data management device 200 based on Hadoop.

[0036] See also Figure 2 , which is a program module diagram of an embodiment of a task data control device 200 based on Hadoop according to the present invention.

[0037] In this embodiment, the Hadoop-based task data control device 200 includes a series of computer program instructions stored in the memory 11. When the computer program instructions are executed by the processor 12, the Hadoop-based task data control functions of various embodiments of the present invention can be implemented. In some embodiments, based on the specific operations implemented by each part of the computer program instructions, the Hadoop-based task data control device 200 can be divided into one or more modules. For example, in Figure 2 In the example, the Hadoop-based task data management device 200 can be divided into an acquisition module 201, an identification module 202, a judgment module 203 and a sending module 204.

[0038] The acquisition module 201 is used to acquire source information of the task process log of the first task management platform.

[0039] Hadoop is a software framework that can perform distributed processing on a large amount of data. Therefore, a multi-task management platform based on Hadoop can be established on the computer device 1 based on Hadoop technology to perform joint management of task data. In this embodiment, the computer device 1 is connected to multiple task management platforms including at least a first task management platform and a second task management platform. The first task management platform and the second task management platform cannot directly interact with each other due to different programming languages ​​or different data formats.

[0040] Specifically, the computer device 1 is connected with the first task management platform, and periodically obtains the source information of the real-time task process log printed by the first task management platform. In the present embodiment, the operating system of the task management platform can monitor and record all the tasks running by itself, and then output the source information of the task process log in a standard format, such as the task execution record including the elements such as task name, time, and execution status, and save it in the subdirectory under the directory where the operating system is located, such as the task log folder. Of course, the task category and the elements to be monitored can also be defined according to the user's instructions. Therefore, the first task management platform can be pre-set to print out the task process log in real time during the execution of the task, and update and store it under the preset directory. Then, the acquisition module 201 obtains the latest source information of the task process log from the directory storing the task process log of the first task management platform according to the preset cycle, such as 30s / times. In this embodiment, the computer device 1 is configured to have log reading permissions for the first task management platform. Therefore, the acquisition module 201 can read the directory file storing the task process log of the first task management platform according to the preset period, and copy and receive the task execution log with a preset name, such as "XX month XX day - task process log".

[0041] The identification module 202 is used to identify the source information of the task process log, and identify the target task name and target task necessary parameters.

[0042] Specifically, the identification module 202 first converts the source information of the task process log into log text information, then performs sentence segmentation on the log text information to obtain at least one task execution record statement, and then performs text recognition on each task execution record statement to identify the target task name and target task necessary parameters. Among them, the identification module 202 identifies each task execution record statement according to the preset target task name keyword, thereby finding the first task execution record statement including the target task name, and then identifies the first task execution record statement according to the preset necessary parameter format, thereby identifying the target task necessary parameters corresponding to the target task name.

[0043] In the present embodiment, due to the different programming languages ​​or data formats used by different task management platforms. Therefore, the identification module 202 first converts the format of the task process log printed by the first task management platform into a text format. For example, if the task process log printed by the first task management platform is in JAVA format, then the identification module 202 converts the task process log in the JAVA format into a text format according to a preset JAVA-text format conversion tool, thereby converting the source information of the task process log to obtain the text information of the task process log. Then, the identification module 202 identifies the text information of the task process log according to the preset target task name. In the present embodiment, the task process log includes all the previous task process information recorded by the first task management platform, so the task process log can be identified, thereby identifying the process information of the target task. First, the identification module 202 performs sentence segmentation on the text information of the task process log, wherein the sentence segmentation is mainly to divide the text information of the task process log into short sentences, so the short sentence segmentation can be performed according to the punctuation of the text information of the task process log, thereby obtaining the sentence of each task process log. In the present embodiment, the sentence segmentation mainly identifies the ascii code value of the punctuation mark in the task process log, and then performs sentence segmentation, such as when the ascii code value "0X2E" is identified, it is judged as a period, and a line break is performed to segment the sentence. Then, the task name keyword of the preset target task is compared with each text sentence in the task process log, thereby obtaining the sentence including the target task name keyword. In the present embodiment, the task name keyword of the target task is directly the task name of the target task. Of course, in other embodiments, the task name keyword of the target task can also be used as the keyword of the task name according to the unique words, words or phrases included in the task name. That is to say, the statement corresponding to the target task name in the text information in the task process log can be identified according to the target task name keyword.

[0044] After identifying the statement corresponding to the target task name in the text information in the task process log, the identification module 202 then further identifies the necessary parameters included in each task execution record including the target task name. In the present embodiment, the necessary parameters of the task execution record include: time, and operating status. Therefore, the time and operating parameters in each statement of the identified task process log can be identified. Specifically, the identification process includes: identifying the time in the task execution statement according to the time format, such as "year-month-day-hour-minute-second"; identifying the execution status corresponding to the target task name in the task process log statement according to the operating status format, such as "operating status: ffff". In this way, the necessary parameters in each task execution record of the target task can be identified.

[0045] The judgment module 203 is used to compare the target task name and target task necessary parameters with the predecessor task name and predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list.

[0046] Specifically, the computer device 1 pre-sets a list of execution of preceding and following tasks, and the list of execution of preceding and following tasks includes the dependency relationship between any preceding task and the corresponding subsequent task in the production process, and the dependency relationship includes the first task management platform carrying the preceding task, the name of the preceding task, the execution status of the preceding task, and the second task management platform carrying the subsequent task, the name of the subsequent task, and the execution instruction corresponding to the subsequent task. Therefore, after the identification module 202 identifies the name of the preceding task and the corresponding necessary parameters of the task execution record, the judgment module 203 can further compare the target task name and the necessary parameters of the target task with the name of the preceding task and the necessary parameters of the preceding task in each task record in the preset list of execution of preceding and following tasks. In this embodiment, the judgment module 203 searches in the list of execution of preceding and following tasks according to the name of the target task in the task execution record, finds out the name of the target task according to the text comparison method, and when the target task is found as the name of the preceding task, the time and running status information of the target task in the task process log are further found, and the time and running status information are also identified by text comparison. In this embodiment, when the judgment module 203 finds out that the target task is stored in the execution list of the preceding and following tasks as a predecessor task, the running state corresponding to the target task in the task process log is compared with the corresponding predecessor task execution state of the target task as the predecessor task in the execution list of the preceding and following tasks. When they are consistent, it is judged that the target task is the predecessor task of the execution list of the preceding and following tasks, and has been executed, and the subsequent task corresponding to the predecessor task should be further executed. In other embodiments, each predecessor task in the execution list of the preceding and following tasks also includes an execution time. For example, after the execution time of the predecessor task reaches T hours, and then the running state reaches F, it is considered that the predecessor task is completed normally; if the execution time does not reach T hours, and the running state reaches F, it is considered that the predecessor task is abnormal. Then the subsequent task will only be executed after the predecessor task is completed normally. Therefore, the multi-task management system will also judge the execution time of the target task in the task process log.

[0047] The sending module 204 is used to send the execution instruction corresponding to the subsequent task in the first task record in the preceding and following task execution list to the second task management platform when the predecessor task name and the predecessor task necessary parameters in the first task record existing in the preceding and following task execution list are consistent with the target task name and the target task necessary parameters.

[0048] Specifically, when the judgment module 203 determines that the target task name and the running status information in the task execution record are the corresponding predecessor task name and predecessor task execution status in the predecessor and successor task execution list, the predecessor task is considered to have been completed. Therefore, the sending module 204 will send the execution instruction corresponding to the successor task in the first task record to the second task management platform according to the predecessor and successor task execution list. Therefore, the second task management platform can start executing the successor task.

[0049] As can be seen from the above, the computer device 1 is able to obtain the source information of the task process log of the first task management platform, and then identify it to identify the target task name and the target task necessary parameters; then the target task name and the target task necessary parameters are compared for consistency with the predecessor task name and the predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list; when the predecessor task name and the predecessor task necessary parameters in the first task record in the predecessor and successor task execution list are consistent with the target task name and the target task necessary parameters, the execution instruction corresponding to the successor task in the first task record is sent to the second task management platform. In this way, real-time, efficient and accurate management and control of task data can be achieved across task management platforms.

[0050] In addition, the present invention also proposes a task data management method based on Hadoop, which is applied to computer equipment.

[0051] See also Figure 3 FIG. 1 is a flow chart of an embodiment of a task data control method based on Hadoop of the present invention. In this embodiment, according to different requirements, Figure 3 The execution order of the steps in the flowchart shown may be changed, and some steps may be omitted.

[0052] Step S500: Acquire source information of the task process log of the first task management platform.

[0053] Hadoop is a software framework that can perform distributed processing on a large amount of data. Therefore, a multi-task management platform based on Hadoop can be established on the computer device based on Hadoop technology to perform joint management of task data. In this embodiment, the computer device is connected to multiple task management platforms including at least a first task management platform and a second task management platform. The first task management platform and the second task management platform cannot directly interact with each other due to different programming languages ​​or different data formats.

[0054] Specifically, the computer device is connected with the first task management platform, and periodically obtains the source information of the real-time task process log printed by the first task management platform. In the present embodiment, the operating system of the task management platform can monitor and record all the tasks running by itself, and then output the source information of the task process log in a standard format, such as the task execution record including the elements such as the task name, time, and execution status, and save it in the subdirectory under the directory where the operating system is located, such as the task log folder. Of course, the task category and the elements to be monitored can also be defined according to the user's instructions. Therefore, the first task management platform can be pre-set to print out the task process log in real time during the execution of the task, update and store it under the preset directory. Then, the computer device obtains the latest source information of the task process log from the directory storing the task process log of the first task management platform according to the preset cycle, such as 30s / times. In this embodiment, the computer device is configured to have log reading permissions for the first task management platform. Therefore, the computer device can read the directory file storing the task process log of the first task management platform according to the preset period, and copy and receive the task execution log with a preset name, such as "XX month XX day - task process log".

[0055] Step S502, identifying the source information of the task process log, and identifying the target task name and target task necessary parameters.

[0056] Specifically, the computer device first converts the source information of the task process log into log text information, then performs sentence segmentation on the log text information to obtain at least one task execution record statement, and then performs text recognition on each task execution record statement to identify the target task name and target task necessary parameters. Among them, the computer device recognizes each task execution record statement according to a preset target task name keyword, thereby finding the first task execution record statement including the target task name, and then recognizes the first task execution record statement according to a preset necessary parameter format, thereby identifying the target task necessary parameters corresponding to the target task name.

[0057] In the present embodiment, due to the different programming languages ​​or data formats used by different task management platforms. Therefore, the computer device first converts the format of the task process log printed by the first task management platform into a text format. For example, if the task process log printed by the first task management platform is in JAVA format, then the computer device converts the task process log in JAVA format into a text format according to a preset JAVA-text format conversion tool, thereby converting the source information of the task process log to obtain the text information of the task process log. Then, the computer device identifies the text information of the task process log according to the preset target task name. In the present embodiment, the task process log includes all the previous task process information recorded by the first task management platform, so the task process log can be identified, thereby identifying the process information of the target task. First, the computer device performs sentence segmentation on the text information of the task process log, wherein the sentence segmentation is mainly to divide the text information of the task process log into short sentences, so the short sentence segmentation can be performed according to the punctuation of the text information of the task process log, thereby obtaining the sentence of each task process log. In this embodiment, the sentence segmentation mainly identifies the ASCII code value of the punctuation mark in the task process log, and then performs sentence segmentation. For example, when the ASCII code value "0X2E" is identified, it is judged as a period, and a line break is performed to segment the sentence. Then, the task name keyword of the preset target task is compared with each text sentence in the task process log, thereby obtaining the sentence including the target task name keyword. In this embodiment, the task name keyword of the target task is directly the task name of the target task. Of course, in other embodiments, the task name keyword of the target task can also be used as the keyword of the task name according to the unique words, words or phrases included in the task name. That is to say, the statement corresponding to the target task name in the text information in the task process log can be identified according to the target task name keyword.

[0058] After identifying the statement corresponding to the target task name in the text information in the task process log, the computer device then further identifies the necessary parameters included in each task execution record including the target task name. In the present embodiment, the necessary parameters of the task execution record include: time, and operating status. Therefore, the time and operating parameters in each statement of the identified task process log can be identified. Specifically, the identification process includes: identifying the time in the task execution statement according to the time format, such as "year-month-day-hour-minute-second"; identifying the execution status corresponding to the target task name in the task process log statement according to the operating status format, such as "operating status: ffff". In this way, the necessary parameters in each task execution record of the target task can be identified.

[0059] Step S504 , performing consistency comparison between the target task name and target task necessary parameters and the predecessor task name and predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list.

[0060] Specifically, the computer device pre-sets a list of executions of preceding and following tasks, and the list of executions of preceding and following tasks includes the dependency relationship between any preceding task and the corresponding subsequent task in the production process, and the dependency relationship includes the first task management platform carrying the preceding task, the name of the preceding task, the execution status of the preceding task, and the second task management platform carrying the subsequent task, the name of the subsequent task, and the execution instructions corresponding to the subsequent task. Therefore, after the computer device identifies the name of the preceding task and the corresponding necessary parameters of the task execution record, the target task name and the necessary parameters of the target task can be further compared with the name of the preceding task and the necessary parameters of the preceding task in each task record in the preset list of executions of preceding and following tasks for consistency. In this embodiment, the computer device searches the list of executions of preceding and following tasks according to the name of the target task in the task execution record, and finds out the name of the target task according to the text comparison method. When the target task is found as the name of the preceding task, the time and running status information of the target task in the task process log are further found, and the time and running status information are also identified by text comparison. In this embodiment, when the computer device finds out that the target task is stored in the execution list of the preceding and following tasks as a predecessor task, the running state corresponding to the target task in the task process log is compared with the corresponding predecessor task execution state of the target task as the predecessor task in the execution list of the preceding and following tasks. When they are consistent, it is judged that the target task is the predecessor task of the execution list of the preceding and following tasks, and has been executed, and the subsequent task corresponding to the predecessor task should be further executed. In other embodiments, each predecessor task in the execution list of the preceding and following tasks also includes an execution time. For example, after the execution time of the predecessor task reaches T hours, and then the running state reaches F, it is considered that the predecessor task is completed normally; if the execution time does not reach T hours, and the running state reaches F, it is considered that the predecessor task is abnormal. Then the subsequent task will only be executed after the predecessor task is completed normally. Therefore, the multi-task management system will also judge the execution time of the target task in the task process log.

[0061] Step S504, when the predecessor task name and the predecessor task necessary parameters in the first task record in the predecessor and successor task execution list are consistent with the target task name and the target task necessary parameters, the execution instruction corresponding to the successor task in the first task record is sent to the second task management platform.

[0062] Specifically, when the computer device determines that the target task name and the running status information in the task execution record are the corresponding predecessor task name and predecessor task execution status in the predecessor and successor task execution list, the predecessor task is considered to have been completed. Therefore, the computer device will send the execution instruction corresponding to the successor task in the first task record to the second task management platform according to the predecessor and successor task execution list. Therefore, the second task management platform can start executing the successor task.

[0063] The Hadoop-based task data management and control method proposed in this embodiment can obtain the source information of the task process log of the first task management platform, and then identify it to identify the target task name and the target task necessary parameters; then the target task name and the target task necessary parameters are compared for consistency with the predecessor task name and the predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list; when the predecessor task name and the predecessor task necessary parameters in the first task record in the predecessor and successor task execution list are consistent with the target task name and the target task necessary parameters, the execution instruction corresponding to the successor task in the first task record is sent to the second task management platform. In this way, real-time, efficient and accurate management and control of task data can be achieved across task management platforms.

[0064] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0065] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0066] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A task data management method based on Hadoop, characterized in that: The method comprises the steps of: Obtaining source information of the task process log of the first task management platform; Identify the source information of the task process log, and identify the target task name and necessary parameters of the target task; Compare the target task name and target task necessary parameters with the predecessor task name and predecessor task necessary parameters in each task record in the preset predecessor and successor task execution list for consistency; When the name of the predecessor task and the necessary parameters of the predecessor task in the first task record in the execution list of the predecessor and successor tasks are consistent with the name of the target task and the necessary parameters of the target task, the execution instruction corresponding to the successor task in the first task record is sent to the second task management platform; The step of identifying the source information of the task process log and identifying the target task name and necessary parameters of the target task includes: Converting the source information of the task process log into log text information; Segment the log text information to obtain at least one task execution record statement; Perform text recognition on each task execution record statement to identify the target task name and necessary parameters of the target task; The step of performing text recognition on each task execution record statement includes: Identify each task execution record statement according to a preset target task name keyword, thereby finding a first task execution record statement including the target task name; The first task execution record statement is identified according to a preset necessary parameter format, thereby identifying the target task necessary parameters corresponding to the target task name.

2. The task data management method based on Hadoop according to claim 1, characterized in that: The necessary parameters include time and operating status, and the necessary parameter formats include time format and operating status format.

3. A task data control device based on Hadoop, characterized in that: The task data control device based on Hadoop includes: An acquisition module, used to acquire source information of a task process log of the first task management platform; An identification module, used to identify the source information of the task process log, and identify the target task name and the target task necessary parameters; A judgment module, used for comparing the target task name and target task necessary parameters with the predecessor task name and predecessor task necessary parameters in each task record in a preset predecessor and successor task execution list for consistency; A sending module, configured to send an execution instruction corresponding to a subsequent task in the first task record to a second task management platform when the name of the preceding task and the necessary parameters of the preceding task in the first task record in the execution list of the preceding and subsequent tasks are consistent with the name of the target task and the necessary parameters of the target task; The step of identifying the source information of the task process log and identifying the target task name and necessary parameters of the target task includes: Converting the source information of the task process log into log text information; Segment the log text information to obtain at least one task execution record statement; Perform text recognition on each task execution record statement to identify the target task name and necessary parameters of the target task; The step of performing text recognition on each task execution record statement includes: Identify each task execution record statement according to a preset target task name keyword, thereby finding a first task execution record statement including the target task name; The first task execution record statement is identified according to a preset necessary parameter format, thereby identifying the target task necessary parameters corresponding to the target task name.

4. The task data control device based on Hadoop according to claim 3, characterized in that: The necessary parameters include time and operating status, and the necessary parameter formats include time format and operating status format.

5. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program that can be run on the processor, and when the computer program is executed by the processor, the steps of the Hadoop-based task data management method as described in any one of claims 1-2 are implemented.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program can be executed by at least one processor to enable the at least one processor to perform the steps of the Hadoop-based task data management method according to any one of claims 1 to 2.

Images