Data processing job monitoring method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing job monitoring method and device, electronic equipment and a storage medium, and relates to the technical field of computers, the method comprises the steps that monitoring configuration information and push information corresponding to a target monitoring group are acquired, the target monitoring group comprises a job flow, the job flow comprises a data processing job, and the data processing job is pushed to the target monitoring group according to the monitoring configuration information; and generating a target monitoring task corresponding to each job flow, allocating corresponding task execution information to the target monitoring task, executing the target monitoring task according to the task execution information to monitor the job flow and / or the data processing job to obtain a monitoring result, and pushing the monitoring result according to the pushing information. The task execution states of multiple data processing jobs can be detected in batches, so that the monitoring efficiency of the data processing jobs can be improved, and the timeliness requirement of data processing job monitoring is effectively met.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a monitoring method, device, electronic device and storage medium for a data processing operation. Background Art

[0002] Banks or other financial institutions need to check the running status of data batch processing jobs on a daily basis to ensure that data can be processed accurately and in a timely manner, thereby preventing abnormal operation from affecting the normal operation of the system.

[0003] In the related art, the running status of data batch processing jobs mainly relies on manual work, which requires operation and maintenance personnel to spend a lot of time and energy to check the running status of the jobs one by one, increasing the workload of operation and maintenance personnel and making it difficult to meet the timeliness requirements of the inspection work. Summary of the invention

[0004] The present disclosure proposes a monitoring method, device, electronic device, storage medium and computer program product for data processing operations, aiming to solve technical problems in related technologies at least to a certain extent.

[0005] The first aspect of the present disclosure proposes a method for monitoring data processing jobs, including: obtaining monitoring configuration information and push information corresponding to a target monitoring group, wherein the target monitoring group includes: a job flow, and the job flow includes: a data processing job; generating a target monitoring task corresponding to each job flow according to the monitoring configuration information, and assigning corresponding task execution information to the target monitoring task; executing the target monitoring task according to the task execution information to monitor the job flow and / or the data processing job to obtain monitoring results; and pushing the monitoring results according to the push information.

[0006] The second aspect of the present disclosure proposes a monitoring device for data processing jobs, including: an acquisition module, used to acquire monitoring configuration information and push information corresponding to a target monitoring group, wherein the target monitoring group includes: a job flow, and the job flow includes: a data processing job; a generation module, used to generate a target monitoring task corresponding to each job flow according to the monitoring configuration information, and assign corresponding task execution information to the target monitoring task; a monitoring module, used to execute the target monitoring task according to the task execution information to monitor the job flow and / or the data processing job to obtain monitoring results; and a push module, used to push the monitoring results according to the push information.

[0007] The third aspect of the present disclosure provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a method for monitoring a data processing job.

[0008] A fourth aspect of the present disclosure provides a computer-readable storage medium, which, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform a monitoring method for a data processing job.

[0009] The fifth aspect of the present disclosure provides a computer program product, including a computer program, characterized in that the computer program is used to monitor data processing operations executed by a processor.

[0010] The data processing job monitoring method, device, electronic device, storage medium and computer program product proposed in this embodiment have at least the following beneficial effects: by obtaining monitoring configuration information and push information corresponding to the target monitoring group, wherein the target monitoring group includes: job flow, the job flow includes: data processing job, according to the monitoring configuration information, a target monitoring task corresponding to each job flow is generated, and corresponding task execution information is assigned to the target monitoring task, the target monitoring task is executed according to the task execution information to monitor the job flow and / or data processing job to obtain monitoring results, and the monitoring results are pushed according to the push information, thereby enabling batch detection of the task execution status of multiple data processing jobs, thereby improving the monitoring efficiency of the data processing job and effectively meeting the timeliness requirements of the data processing job monitoring.

[0011] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0013] Figure 1 is a flowchart of a method for monitoring a data processing job according to the first embodiment of the present disclosure;

[0014] Figure 2 is a flow chart of a method for monitoring a data processing job according to a second embodiment of the present disclosure;

[0015] Figure 3 is a flowchart of a method for monitoring a data processing job according to a third embodiment of the present disclosure;

[0016] Figure 4 is a block diagram of a monitoring device for a data processing operation according to the present disclosure;

[0017] Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present disclosure, and are not to be construed as limitations of the present disclosure. On the contrary, the embodiments of the present disclosure include all changes, modifications, and equivalents that fall within the spirit and connotation of the appended claims.

[0019] It should be noted that the executor of the data processing job monitoring method of this embodiment can be a data processing job monitoring device, which can be implemented by software and / or hardware, and can be configured in an electronic device, which can include but is not limited to a terminal, a server, etc.

[0020] It should be noted that the process of obtaining, storing, using, and processing information in the disclosed technical solution complies with the relevant provisions of national laws and regulations and does not violate public order and good morals.

[0021] It should be noted that in the embodiments of the present application, there may be existing solutions in the industry such as certain software, components, models, etc., which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the relevant content of the solution.

[0022] Figure 1 is a flow chart of a method for monitoring a data processing operation according to the first embodiment of the present disclosure, such as Figure 1 As shown, the method includes:

[0023] S101: Acquire monitoring configuration information and push information corresponding to a target monitoring group, wherein the target monitoring group includes: an operation flow, and the operation flow includes: a data processing operation.

[0024] Among them, the target monitoring group refers to the job group whose job execution status is currently to be monitored. The target monitoring group includes: at least one job flow, each job flow includes: multiple data processing jobs, and the data processing jobs can be, for example: data analysis jobs, data storage jobs, etc., without any restriction.

[0025] The monitoring configuration information corresponding to the target monitoring group refers to information configured for the target monitoring group and used for batch monitoring of the execution status of data processing jobs in the target monitoring group.

[0026] In the disclosed embodiment, the monitoring configuration information includes: monitoring frequency and detection time. The monitoring time may be, for example, the monitoring start time, the monitoring end time, etc. The monitoring frequency may be, for example, the monitoring frequency, etc., and there is no limitation on this.

[0027] Among them, the push information corresponding to the target monitoring group means that after batch monitoring of the execution status of the data processing jobs in the target monitoring group is carried out to obtain the monitoring results, the monitoring results need to be pushed to the relevant users according to the push information, and the push information refers to the information referenced in the process of pushing the monitoring results. The push information can be, for example, the user identification information of the push user, the user email address, the telephone number, etc., and can also be, for example, the push time, the push frequency, etc., and there is no restriction on this.

[0028] In some embodiments, obtaining monitoring configuration information and push information corresponding to the target monitoring group can be to determine at least one data processing job whose execution status is currently to be monitored from multiple data processing jobs, and grouping the at least one data processing job to be monitored based on the job description information of each data processing job to obtain at least one job flow, and use the one or more job flows determined above as the target monitoring group.

[0029] In the disclosed embodiment, after obtaining the target monitoring group, it is possible to determine the job start execution time and / or job execution time corresponding to the target monitoring group, and use the aforementioned determined job start execution time and / or job execution time together as monitoring configuration information. It is also possible to determine user description information of users related to the target monitoring group, and use the user description information as push information corresponding to the target monitoring group, and there is no limitation on this.

[0030] S102: Generate a target monitoring task corresponding to each workflow according to the monitoring configuration information, and assign corresponding task execution information to the target monitoring task.

[0031] The target monitoring task refers to a task for monitoring the job execution status of each job flow in the target monitoring group.

[0032] The task execution information refers to the information that needs to be referenced during the execution of the target monitoring task.

[0033] In the embodiment of the present disclosure, the task execution information includes: task execution time and task execution status, and the task execution time can be, for example, the time when the task starts executing, the time required for task execution, etc. The task execution status can include: task execution completed, task execution not completed, etc., without limitation.

[0034] In the disclosed embodiment, after obtaining the monitoring configuration information and push information corresponding to the target monitoring group, a target monitoring task corresponding to each workflow may be generated according to the monitoring configuration information, and corresponding task execution information may be allocated to the target monitoring task.

[0035] In some embodiments, a target monitoring task corresponding to each job flow is generated based on the monitoring configuration information. The monitoring configuration information can be parsed, and the multiple job flows are grouped and classified based on the parsing results. The multiple job flows in each group of job flows are sorted based on the job dependencies between the multiple job flows in each group of job flows. The target monitoring task is then generated based on the job flow identifier of each group of job flows after sorting, the job flow execution order and the monitoring configuration information.

[0036] In the disclosed embodiment, after the target monitoring task corresponding to each workflow is generated according to the monitoring configuration information, the monitoring frequency, monitoring time and other information in the monitoring configuration information may be used as the task execution information.

[0037] In other embodiments, the large model may be combined to generate target monitoring tasks corresponding to each job flow based on the monitoring configuration information, and assign corresponding task execution information to the target monitoring tasks. That is, the monitoring configuration information may be input into the large model, and the large model may process the monitoring configuration information to generate target monitoring tasks corresponding to each job flow based on the monitoring configuration information, and task execution information corresponding to each target monitoring task.

[0038] S103: Execute the target monitoring task according to the task execution information to monitor the job flow and / or data processing job to obtain a monitoring result.

[0039] Among them, the monitoring result is obtained after monitoring the execution status of the job flow and / or the data processing job, which is used to describe the execution status of the job flow and / or the data processing job. The monitoring result can be, for example, that multiple data processing jobs in the job flow are all executed, the data processing job is not executed, etc., and there is no limitation on this.

[0040] In some embodiments, a target monitoring task is executed according to task execution information to monitor the job flow and / or data processing job to obtain a monitoring result. The job flow and / or data processing job can be monitored based on the monitoring start time described in the task execution information, and the monitoring result of the job flow and / or data processing job can be determined based on the monitoring period described in the task execution information. There is no limitation on this.

[0041] In other embodiments, a target monitoring task is executed according to the task execution information to monitor the job flow and / or data processing job and obtain monitoring results. It is also possible to select a monitoring task execution tool and configure the task execution information in the task execution tool so that the monitoring task execution tool can execute the target monitoring task according to the task execution tool, thereby realizing monitoring of the job flow and / or data processing job and obtaining monitoring results. There is no limitation to this.

[0042] S104: Push the monitoring result according to the push information.

[0043] In the disclosed embodiment, after executing the target monitoring task according to the task execution information to monitor the job flow and / or the data processing job and obtaining the monitoring result, the detection result can be pushed according to the push information.

[0044] For example, in some embodiments, the push information includes a user mailbox corresponding to the target monitoring group. The monitoring results may be sent to the user mailbox corresponding to the target monitoring group after the monitoring results are obtained. There is no limitation to this.

[0045] For example, in some other embodiments, the push information includes push frequency and push method (broadcast push). The monitoring results are pushed according to the push information. The monitoring results can be pushed regularly in the form of broadcast push at the push frequency described in the push information. In this way, it can be ensured that users related to the target monitoring group can promptly know the monitoring results of the target monitoring group.

[0046] In the disclosed embodiment, the push information includes: content template, push method and push object.

[0047] Among them, the content template can be, for example: "Target monitoring group identification; number of executed jobs-job identification information; number of jobs being executed-job identification information; number of unexecuted jobs-job identification information". The content template can be adaptively set according to business needs in actual business scenarios, and there is no restriction on this.

[0048] The push method may be, for example, email push, broadcast push, SMS push, etc., and there is no limitation on this.

[0049] The push object may be, for example, a user related to the target monitoring group, or a business platform related to the target monitoring group, etc., and there is no limitation on this.

[0050] Optionally, in some embodiments, the monitoring result is pushed according to the push information, and the target result may be generated according to the content template and the monitoring result, and the target result may be pushed to the push object using a target push method.

[0051] That is to say, in the embodiments of the present disclosure, the content format of the monitoring results can be adjusted based on the content template, and the adjusted monitoring results can be pushed using the target push method, thereby ensuring that the target push results with standardized format can be pushed to the push object in a target push method that the push object can accept.

[0052] In the disclosed embodiment, by obtaining monitoring configuration information and push information corresponding to the target monitoring group, wherein the target monitoring group includes: an operation flow, and the operation flow includes: a data processing operation, a target monitoring task corresponding to each operation flow is generated according to the monitoring configuration information, and corresponding task execution information is assigned to the target monitoring task, and the target monitoring task is executed according to the task execution information to monitor the operation flow and / or the data processing operation to obtain the monitoring result, and the monitoring result is pushed according to the push information, thereby being able to batch detect the task execution status of multiple data processing operations, thereby being able to improve the monitoring efficiency of the data processing operations and effectively meet the timeliness requirements of the data processing operation monitoring.

[0053] Figure 2 is a flow chart of a method for monitoring a data processing operation according to a second embodiment of the present disclosure, such as Figure 2 As shown, the method includes:

[0054] S201: Acquire monitoring configuration information and push information corresponding to a target monitoring group, wherein the target monitoring group includes: an operation flow, and the operation flow includes: a data processing operation.

[0055] S202: Generate a target monitoring task corresponding to each workflow according to the monitoring configuration information, and allocate corresponding task execution information to the target monitoring task.

[0056] The specific description of S201 - S202 can be found in the above embodiment, which will not be repeated here.

[0057] S203: Determine that the current time reaches the task execution time and the task execution status is not executed, and obtain the target monitoring content corresponding to the target monitoring group.

[0058] The target monitoring content includes: information of the operation flow and / or information of the data processing operation, and the monitoring type.

[0059] Among them, the information of the job flow can be, for example, the identification information of the job flow, such as the job flow name, the job flow number, etc., and the information of the data processing job can be, for example, the information of the data processing job, such as the data processing job name, the data processing job number, etc., without limitation.

[0060] The monitoring type is used to indicate whether the content to be monitored is an entire job flow or a certain data processing job.

[0061] That is to say, in the embodiments of the present disclosure, after generating a target monitoring task corresponding to each job flow according to the monitoring configuration information and assigning corresponding task execution information to the target monitoring task, it is determined whether the current time has reached the task execution time, and when the current time has reached the task execution time and it is determined that the task execution status is not executed, the job flow information and / or data processing job information corresponding to the target monitoring group, and the monitoring type are obtained, and the aforementioned job flow information and / or data processing job information corresponding to the target monitoring group, and the monitoring type are used as the target monitoring content, without any restriction on this.

[0062] Optionally, in some embodiments, obtaining the target monitoring content corresponding to the target monitoring group may be obtaining a monitoring group content configuration table, wherein the monitoring group content configuration table includes: information of multiple candidate monitoring groups, and candidate monitoring content corresponding to the information of the candidate monitoring groups, determining the candidate monitoring group that is the same as the target monitoring group based on the information of the candidate monitoring group, and determining the candidate monitoring content corresponding to the information of the candidate monitoring group that is the same as the target monitoring group as the target monitoring content.

[0063] The information of the candidate monitoring group is used to describe the candidate monitoring group. The information of the candidate monitoring group may be, for example, identification information of the candidate monitoring group, such as the name of the candidate monitoring group, the number of the candidate monitoring group, etc., and there is no limitation on this.

[0064] The candidate monitoring content refers to the operation flow and / or data processing operation in the candidate monitoring group.

[0065] The monitoring group content configuration table may be as shown in Table 1 below:

[0066] Table 1

[0067]

[0068] That is to say, in the embodiment of the present disclosure, a monitoring group content configuration table may be obtained, and a candidate monitoring group that is identical to a target monitoring group may be determined based on the information of the candidate monitoring group in the monitoring group content configuration table, and the candidate monitoring content corresponding to the information of the candidate monitoring group that is identical to the target monitoring group may be determined as the target monitoring content.

[0069] For example, after obtaining the monitoring group content configuration table shown in Table 1 above, a candidate monitoring group with the same monitoring group ID and monitoring group name as the target monitoring group is selected, and the job flow and / or data processing job in the candidate monitoring group is used as the target monitoring content.

[0070] S204: Monitor the job flow and / or data processing job according to the monitoring type to obtain a monitoring result.

[0071] In the disclosed embodiment, after determining that the current time has reached the task execution time and the task execution status is not executed, and obtaining the target monitoring content corresponding to the target monitoring group, the job flow and / or data processing job can be monitored according to the monitoring type to obtain the monitoring results.

[0072] In the disclosed embodiment, the execution status of the job flow and / or the data processing job may be monitored according to the job flow that needs to be monitored or a data processing job indicated by the monitoring type to obtain a monitoring result.

[0073] That is to say, in the embodiments of the present disclosure, when the monitoring type indicates that what needs to be monitored is the job flow in the target monitoring group, the job flow in the target monitoring group is monitored; when the monitoring type indicates that what needs to be monitored is the data processing job in the target monitoring group, the data processing job in the target monitoring group is monitored to obtain the monitoring results.

[0074] S205: Push the monitoring results according to the push information.

[0075] The specific description of S205 can be found in the above embodiment, which will not be repeated here.

[0076] In the disclosed embodiment, by obtaining monitoring configuration information and push information corresponding to the target monitoring group, wherein the target monitoring group includes: job flow, and the job flow includes: data processing job, then according to the monitoring configuration information, a target monitoring task corresponding to each job flow is generated, and the corresponding task execution information is assigned to the target monitoring task, then it is determined that the current time reaches the task execution time, and the task execution status is not executed, the target monitoring content corresponding to the target monitoring group is obtained, and then the job flow and / or data processing job is monitored according to the monitoring type to obtain the monitoring results, and then the monitoring results are pushed according to the push information, thereby, the task execution status of multiple data processing jobs can be batch detected, thereby improving the monitoring efficiency of the data processing job, effectively meeting the timeliness requirements of the data processing job monitoring, and the monitoring results can be pushed to the push object in a timely manner.

[0077] Figure 3 is a flow chart of a method for monitoring a data processing operation according to a third embodiment of the present disclosure, such as Figure 3 As shown, the method includes:

[0078] S301: Acquire a monitoring group task configuration table, wherein the monitoring group task configuration table includes: information of multiple candidate monitoring groups, and candidate configuration information corresponding to the information of the candidate monitoring groups.

[0079] The monitoring group task configuration table includes: information of multiple candidate monitoring groups, and candidate configuration information corresponding to the information of the candidate monitoring groups.

[0080] The information of the candidate monitoring group refers to information used to describe the candidate monitoring group, and the information may be, for example, identification information of the candidate monitoring group, such as: candidate monitoring group name, candidate monitoring group number, etc., without limitation.

[0081] The candidate configuration information corresponding to the information of the candidate monitoring group may be, for example, the monitoring time, monitoring frequency, etc. related to the candidate monitoring group pre-configured for the candidate detection group, and there is no limitation on this.

[0082] An example of the task configuration table is shown in Table 2 below:

[0083] Table 2

[0084]

[0085] That is to say, in the embodiments of the present disclosure, the monitoring group task configuration table can be configured according to the business needs in the actual scenario, and then, the monitoring method for executing subsequent data processing operations can be triggered based on the detection group task configuration table. For details, please refer to the subsequent embodiments and will not be repeated here.

[0086] S302: Determine a candidate monitoring group that is the same as the target monitoring group based on the information of the candidate monitoring groups.

[0087] In the disclosed embodiment, after obtaining the monitoring group task configuration table, the candidate monitoring group that is the same as the target monitoring group can be determined based on the information of the candidate monitoring group.

[0088] That is to say, in the embodiment of the present disclosure, after determining the monitoring group task configuration table, the information of the candidate monitoring group that is the same as the target monitoring group information is determined, thereby determining the candidate monitoring group that is the same as the target monitoring group.

[0089] S303: Determine the candidate configuration information corresponding to the information of the candidate monitoring group that is the same as the target monitoring group as the monitoring configuration information.

[0090] In the embodiment of the present disclosure, after determining the candidate monitoring group identical to the target monitoring group according to the information of the candidate monitoring group, the candidate configuration information corresponding to the information of the candidate monitoring group identical to the target monitoring group may be determined as the monitoring configuration information.

[0091] S304: Acquire a monitoring result notification configuration table, wherein the monitoring result notification configuration table includes: information of multiple candidate monitoring groups, and candidate push information corresponding to the information of the candidate monitoring groups.

[0092] The monitoring result notification configuration table includes: information of multiple candidate monitoring groups, and candidate push information corresponding to the information of the candidate monitoring groups. An example of the monitoring result notification configuration table is shown in Table 3 below:

[0093] Table 3

[0094]

[0095] As shown in Table 3 above, the information of the candidate monitoring group in the monitoring result notification configuration table may refer to identification information of the candidate monitoring group, such as monitoring group ID, monitoring group name, etc., and there is no limitation on this.

[0096] Among them, as shown in the above Table 3, the candidate push information in the monitoring result notification configuration table can be notification content, notification target mobile phone number, notification target name, etc., and there is no restriction on this.

[0097] S306: Determine the candidate monitoring group that is the same as the target monitoring group based on the information of the candidate monitoring groups.

[0098] In the embodiment of the present disclosure, after obtaining the monitoring result notification configuration table, the candidate monitoring group that is the same as the target monitoring group may be determined based on the information of the candidate monitoring group.

[0099] That is to say, in the embodiment of the present disclosure, after determining the monitoring group task configuration table, the information of the candidate monitoring group that is the same as the target monitoring group information is determined, thereby determining the candidate monitoring group that is the same as the target monitoring group.

[0100] S307: Determine the candidate push information corresponding to the information of the candidate monitoring group that is the same as the target monitoring group as the push information.

[0101] In the disclosed embodiment, after determining the candidate monitoring group identical to the target monitoring group according to the information of the candidate monitoring group, the candidate push information corresponding to the information of the candidate monitoring group identical to the target monitoring group may be determined as the push information.

[0102] S308: Generate a target monitoring task corresponding to each workflow according to the monitoring configuration information, and assign corresponding task execution information to the target monitoring task.

[0103] S309: Execute the target monitoring task according to the task execution information to monitor the job flow and / or data processing job to obtain a monitoring result.

[0104] S310: Push monitoring results according to the push information.

[0105] The specific description of S308 - S310 can be found in the above embodiment, which will not be repeated here.

[0106] In the disclosed embodiment, a monitoring group task configuration table is obtained, wherein the monitoring group task configuration table includes: information of multiple candidate monitoring groups, and candidate configuration information corresponding to the information of the candidate monitoring groups; a candidate monitoring group identical to the target monitoring group is determined according to the information of the candidate monitoring groups; and the candidate configuration information corresponding to the information of the candidate monitoring group identical to the target monitoring group is determined as the monitoring configuration information; a monitoring result notification configuration table is obtained, wherein the monitoring result notification configuration table includes: information of multiple candidate monitoring groups, and candidate push information corresponding to the information of the candidate monitoring groups; a candidate monitoring group identical to the target monitoring group is determined according to the information of the candidate monitoring groups; and the candidate push information corresponding to the information of the candidate monitoring group identical to the target monitoring group is determined as the push information; a target monitoring task corresponding to each job flow is generated according to the monitoring configuration information, and corresponding task execution information is assigned to the target monitoring task; the target monitoring task is executed according to the task execution information to monitor the job flow and / or the data processing job to obtain the monitoring result; and the monitoring result is pushed according to the push information. Thus, the task execution status of multiple data processing jobs can be detected in batches, thereby improving the monitoring efficiency of the data processing jobs, effectively meeting the timeliness requirements of the data processing job monitoring, and timely pushing the monitoring result to the push object.

[0107] Figure 4 is a block diagram of a monitoring device for data processing operations according to the present disclosure, such as Figure 4 As shown, the monitoring device 40 for the data processing operation includes:

[0108] The acquisition module 401 is used to acquire monitoring configuration information and push information corresponding to a target monitoring group, wherein the target monitoring group includes: an operation flow, and the operation flow includes: a data processing operation;

[0109] A generating module 402, for generating a target monitoring task corresponding to each job flow according to the monitoring configuration information, and assigning corresponding task execution information to the target monitoring task;

[0110] A monitoring module 403 is used to execute a target monitoring task according to the task execution information to monitor the job flow and / or the data processing job to obtain a monitoring result; and

[0111] The push module 404 is used to push the monitoring results according to the push information.

[0112] In some embodiments of the present disclosure, the acquisition module 401 is further used to:

[0113] Acquire a monitoring group task configuration table, wherein the monitoring group task configuration table includes: information of multiple candidate monitoring groups, and candidate configuration information corresponding to the information of the candidate monitoring groups;

[0114] Determine, based on the information of the candidate monitoring groups, a candidate monitoring group that is identical to the target monitoring group;

[0115] The candidate configuration information corresponding to the information of the candidate monitoring group that is the same as the target monitoring group is determined as the monitoring configuration information.

[0116] In some embodiments of the present disclosure,

[0117] The monitoring configuration information includes at least one of the following: monitoring frequency, monitoring time;

[0118] The task execution information includes at least one of the following: task execution time and task execution status.

[0119] In some embodiments of the present disclosure, the task execution information includes: task execution time and task execution status; wherein the monitoring module 403 is further used to:

[0120] Determine that the current time reaches the task execution time, and the task execution status is not executed, and obtain the target monitoring content corresponding to the target monitoring group, wherein the target monitoring content includes: information of the job flow and / or information of the data processing job, and the monitoring type;

[0121] The job flow and / or data processing job is monitored according to the monitoring type to obtain monitoring results.

[0122] In some embodiments of the present disclosure, the monitoring module 403 is further configured to:

[0123] Obtaining a monitoring group content configuration table, wherein the monitoring group content configuration table includes: information of a plurality of candidate monitoring groups, and candidate monitoring contents corresponding to the information of the candidate monitoring groups;

[0124] Determine, based on the information of the candidate monitoring groups, a candidate monitoring group that is identical to the target monitoring group;

[0125] The candidate monitoring content corresponding to the information of the candidate monitoring group that is the same as the target monitoring group is determined as the target monitoring content.

[0126] In some embodiments of the present disclosure, the acquisition module 401 is further used to:

[0127] Obtaining a monitoring result notification configuration table, wherein the monitoring result notification configuration table includes: information of multiple candidate monitoring groups, and candidate push information corresponding to the information of the candidate monitoring groups;

[0128] Determine, based on the information of the candidate monitoring groups, a candidate monitoring group that is identical to the target monitoring group;

[0129] The candidate push information corresponding to the information of the candidate monitoring group that is the same as the target monitoring group is determined as the push information.

[0130] In some embodiments of the present disclosure, the push information includes: content template, push method and push object; wherein the push module 404 is further used to:

[0131] Generate target results based on content templates and monitoring results;

[0132] Use the target push method to push the target result to the push object.

[0133] In the disclosed embodiment, by obtaining monitoring configuration information and push information corresponding to the target monitoring group, wherein the target monitoring group includes: an operation flow, and the operation flow includes: a data processing operation, a target monitoring task corresponding to each operation flow is generated according to the monitoring configuration information, and corresponding task execution information is assigned to the target monitoring task, and the target monitoring task is executed according to the task execution information to monitor the operation flow and / or the data processing operation to obtain the monitoring result, and the monitoring result is pushed according to the push information, thereby being able to batch detect the task execution status of multiple data processing operations, thereby being able to improve the monitoring efficiency of the data processing operations and effectively meet the timeliness requirements of the data processing operation monitoring.

[0134] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a computer-readable storage medium, and a computer program product.

[0135] Figure 5 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present disclosure is shown.

[0136] Figure 5 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0137] like Figure 5 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a memory 28, and a bus 18 that connects various system components (including the memory 28 and the processing unit 16).

[0138] The bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnection (PCI) bus.

[0139] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0140] The memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 Not shown, often called a "hard drive").

[0141] although Figure 5 Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a compact disc read only memory (hereinafter referred to as: CD-ROM), a digital versatile disc read only memory (hereinafter referred to as: DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0142] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in the memory 28, such program modules 42 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0143] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), may also communicate with one or more devices that enable the human body to interact with the electronic device 12, and / or communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 22. In addition, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 through a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0144] The processing unit 16 executes various functional applications and parameter information determination by running the program stored in the memory 28, such as implementing the business data storage method mentioned in the above embodiment, or implementing the business data acquisition method mentioned in the above embodiment.

[0145] It should be noted that, in the description of the present disclosure, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "plurality" is two or more.

[0146] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0147] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0148] A person of ordinary skill in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0149] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0150] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0151] Although the embodiments of the present disclosure have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A method for monitoring a data processing operation, characterized in that: include: Acquire monitoring configuration information and push information corresponding to a target monitoring group, wherein the target monitoring group includes: an operation flow, and the operation flow includes: a data processing operation; Generate a target monitoring task corresponding to each of the job flows according to the monitoring configuration information, and assign corresponding task execution information to the target monitoring task; executing the target monitoring task according to the task execution information to monitor the job flow and / or the data processing job to obtain a monitoring result; and The monitoring result is pushed according to the push information.

2. The method according to claim 1, characterized in that The acquiring of monitoring configuration information corresponding to the target monitoring group includes: Acquire a monitoring group task configuration table, wherein the monitoring group task configuration table includes: information of multiple candidate monitoring groups, and candidate configuration information corresponding to the information of the candidate monitoring groups; Determine, based on the information of the candidate monitoring group, a candidate monitoring group that is the same as the target monitoring group; The candidate configuration information corresponding to the information of the candidate monitoring group that is the same as the target monitoring group is determined as the monitoring configuration information.

3. The method according to claim 1, characterized in that: in, The monitoring configuration information includes at least one of the following: monitoring frequency and monitoring time; The task execution information includes at least one of the following: task execution time and task execution status.

4. The method according to claim 3, characterized in that The task execution information includes: task execution time and task execution status; wherein, executing the target monitoring task according to the task execution information to monitor the job flow and / or the data processing job to obtain a monitoring result includes: Determine that the current time reaches the task execution time, and the task execution status is not executed, and obtain the target monitoring content corresponding to the target monitoring group, wherein the target monitoring content includes: information of the job flow and / or information of the data processing job, and monitoring type; The job flow and / or the data processing job is monitored according to the monitoring type to obtain a monitoring result.

5. The method according to claim 4, characterized in that The obtaining of target monitoring content corresponding to the target monitoring group includes: Acquire a monitoring group content configuration table, wherein the monitoring group content configuration table includes: information of a plurality of candidate monitoring groups, and candidate monitoring contents corresponding to the information of the candidate monitoring groups; Determine, based on the information of the candidate monitoring group, a candidate monitoring group that is the same as the target monitoring group; The candidate monitoring content corresponding to the information of the candidate monitoring group that is the same as the target monitoring group is determined as the target monitoring content.

6. The method according to claim 1, characterized in that Get the push information corresponding to the target monitoring group, including: Acquire a monitoring result notification configuration table, wherein the monitoring result notification configuration table includes: information of multiple candidate monitoring groups, and candidate push information corresponding to the information of the candidate monitoring groups; Determine, based on the information of the candidate monitoring group, a candidate monitoring group that is the same as the target monitoring group; The candidate push information corresponding to the information of the candidate monitoring group that is the same as the target monitoring group is determined as the push information.

7. The method according to claim 1, characterized in that The push information includes: content template, push method and push object; wherein, pushing the monitoring result according to the push information includes: Generate a target result according to the content template and the monitoring result; The target result is pushed to the push object using the target push method.

8. A monitoring device for data processing operations, characterized in that: The device comprises: An acquisition module, used to acquire monitoring configuration information and push information corresponding to a target monitoring group, wherein the target monitoring group includes: an operation flow, and the operation flow includes: a data processing operation; A generating module, configured to generate a target monitoring task corresponding to each of the job flows according to the monitoring configuration information, and to allocate corresponding task execution information to the target monitoring task; a monitoring module, configured to execute the target monitoring task according to the task execution information, so as to monitor the job flow and / or the data processing job and obtain a monitoring result; and A push module is used to push the monitoring result according to the push information.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images