Monitoring operation and maintenance method, system and equipment for tasks on resource scheduler cluster and medium

Abstract

The invention discloses a method, a system, equipment and a medium for monitoring operation and maintenance of tasks on a resource scheduler cluster, and relates to the technical field of computers, and the method comprises the following steps: when a task client submits the tasks to the resource scheduler cluster, reading configuration information in the tasks, and writing the configuration information into a task submission table, the configuration information comprises environment variables configured with priorities; after a heartbeat request of a task center hop operator is received, heartbeat information of the task submission table is updated, and the heartbeat operator is a heartbeat operator added by the task after the data source operator; and according to the heartbeat information in the task submission table, determining a target task meeting a restarting condition and a target priority, and restarting the target task according to the target priority. According to the method and the device, the problem that the operator false death in the Flink task cannot be monitored in the prior art can be solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a monitoring and operation method, system, device and medium for tasks on a resource scheduler cluster. Background Art

[0002] The existing monitoring and operation solutions for Flink tasks deployed on the Yarn cluster are based on detecting whether the Task within the Flink task is in a failed state or whether the Flink task is in a running state on the Yarn cluster to restart the task, but both strategies have defects.

[0003] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of this application is to provide a method, system, device and medium for monitoring and operating tasks on a resource scheduler cluster, aiming to solve the technical problem that the existing monitoring and operating solutions for Flink tasks deployed on a Yarn cluster are defective.

[0005] To achieve the above objectives, the present application proposes a method for monitoring and operating tasks on a resource scheduler cluster, the method comprising:

[0006] When the task client submits the task to the resource scheduler cluster, the configuration information in the task is read and the configuration information is written into the task submission table, wherein the configuration information includes environment variables configured with priorities;

[0007] After receiving the heartbeat request of the heartbeat operator in the task, updating the heartbeat information of the task submission table, wherein the heartbeat operator is the heartbeat operator added after the data source operator of the task;

[0008] Determine a target task and a target priority that meet the restart condition according to the heartbeat information in the task submission table, and restart the target task according to the target priority.

[0009] In one embodiment, after receiving a heartbeat request from a heartbeat operator in a task, the step of updating the heartbeat information of the task submission table, wherein the heartbeat operator is a heartbeat operator added after the data source operator of the task, comprises:

[0010] After obtaining the heartbeat request of the heartbeat operator in the task on the resource scheduler cluster through regular scheduling, the heartbeat information of the task submission table is updated, wherein the task includes multiple data source operators and multiple heartbeat operators, and the scheduling thread pool started in the open function of the heartbeat operator is used to perform heartbeat requests regularly, and the heartbeat request includes the task name and the operator name of the data source operator in the task.

[0011] In one embodiment, when the task client submits the task to the resource scheduler cluster, after the step of reading the configuration information in the task and writing the configuration information into the task submission table, the step further includes:

[0012] The task status on the resource scheduler cluster is obtained regularly, and the task status of each task in the task submission table is updated.

[0013] In one embodiment, the step of determining the target task and the target priority that meet the restart condition according to the heartbeat information in the task submission table includes:

[0014] Traverse the task submission table according to the number of restarts, and determine whether the task submission time of the task in the task submission table is greater than the preset heartbeat period;

[0015] When it is determined that the task submission time is greater than the preset heartbeat period, the target task in the task submission table that meets the restart condition is determined according to the heartbeat information and task status in the task submission table, and the target priority of the target task is determined according to the task submission table.

[0016] In one embodiment, the step of determining the target task in the task submission table that meets the restart condition according to the heartbeat information and the task status in the task submission table includes:

[0017] Determine the normal operation task according to the task status in the task submission table;

[0018] Determine whether the heartbeat information of each data flow of the normal running task is updated in the task submission table;

[0019] If the heartbeat information of each data stream of the normal operating task in the task submission table is not updated, the normal operating task is determined as a target task that meets the restart condition.

[0020] In one embodiment, the step of restarting the target task according to the target priority includes:

[0021] Determine a target restart queue for the target task according to the target priority, and add the target task to the target restart queue;

[0022] After selecting the target task in the target restart queue through the task submission thread pool, the target task is restarted.

[0023] In one embodiment, after selecting the target task in the target restart queue through the task submission thread pool, the step of restarting the target task includes:

[0024] After selecting the target task in the target restart queue through the task submission thread pool, the corresponding record of the target task in the task submission table is locked, and the corresponding record of the target task in the task submission table is unlocked after the target task is submitted to the resource scheduler cluster.

[0025] In addition, to achieve the above purpose, the present application also proposes a monitoring and operation system for tasks on a resource scheduler cluster, wherein the monitoring and operation system includes a task publishing module, a task monitoring module, and a task restart scheduling module; wherein,

[0026] The task publishing module is used to read the configuration information in the task when the task client submits the task to the resource scheduler cluster, and write the configuration information into the task submission table, wherein the configuration information includes environment variables configured with priorities;

[0027] The task monitoring module is used to update the heartbeat information of the task submission table after receiving the heartbeat request of the heartbeat operator in the task, wherein the heartbeat operator is the heartbeat operator added after the data source operator of the task;

[0028] The task restart scheduling module is used to determine the target task and the target priority that meet the restart condition according to the heartbeat information in the task submission table, and restart the target task according to the target priority.

[0029] In addition, to achieve the above-mentioned purpose, the present application also proposes a monitoring and operation and maintenance device for tasks on a resource scheduler cluster, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the monitoring and operation and maintenance method for tasks on a resource scheduler cluster as described above.

[0030] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the monitoring and operation method of tasks on the resource scheduler cluster as described above are implemented.

[0031] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the monitoring and operation method of tasks on the resource scheduler cluster as described above.

[0032] One or more technical solutions proposed in this application have at least the following technical effects:

[0033] The monitoring and operation method, system, device and medium of tasks on a resource scheduler cluster proposed in the present application, when a task client submits a task to a resource scheduler cluster, reads configuration information in the task and writes the configuration information into a task submission table, wherein the configuration information includes environment variables configured with priorities; after receiving a heartbeat request of a heartbeat operator in the task, updates the heartbeat information of the task submission table, wherein the heartbeat operator is a heartbeat operator added to the task after the data source operator; determines a target task and a target priority that meet the restart condition according to the heartbeat information in the task submission table, and restarts the target task according to the target priority, solving the technical problem that the existing monitoring and operation schemes for Flink tasks deployed on a resource scheduler cluster are defective. Compared with the prior art, the present application realizes an automatic monitoring and operation scheme for abnormal Flink tasks on a resource scheduler cluster by configuring priorities in environment variables of Flink tasks and adding heartbeat operators in Flink tasks, which can not only solve the problem that the operator pseudo-death in Flink tasks cannot be monitored under the prior art, but also can first restore Flink tasks with high priorities when resources are insufficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0036] Figure 1 A flowchart of the first embodiment of the method for monitoring and operating tasks on a resource scheduler cluster of the present application is provided;

[0037] Figure 2 A flow chart of adding a heartbeat operator to a Flink task provided in Example 1 of the monitoring and operation method for tasks on a resource scheduler cluster of this application;

[0038] Figure 3 A specific flow chart of the first embodiment of the method for monitoring and operating tasks on a resource scheduler cluster of the present application;

[0039] Figure 4 A flowchart diagram of the second embodiment of the method for monitoring and operating tasks on a resource scheduler cluster of the present application;

[0040] Figure 5 A schematic diagram of a priority restart process provided for Embodiment 2 of the method for monitoring and operating tasks on a resource scheduler cluster of the present application;

[0041] Figure 6 A schematic diagram of the module structure of a task monitoring and operation system on a resource scheduler cluster according to an embodiment of the present application;

[0042] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the method for monitoring and operating tasks on a resource scheduler cluster in an embodiment of the present application.

[0043] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0045] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0046] The main solution of the embodiment of the present application is: when the task client submits the task to the resource scheduler cluster, read the configuration information in the task, and write the configuration information into the task submission table, wherein the configuration information includes environment variables configured with priorities; after receiving the heartbeat request of the heartbeat operator in the task, update the heartbeat information of the task submission table, wherein the heartbeat operator is the heartbeat operator added to the task after the data source operator; determine the target task and the target priority that meet the restart condition according to the heartbeat information in the task submission table, and restart the target task according to the target priority.

[0047] It can be seen from the above embodiments that the present application reads the configuration information in the task and writes the configuration information into the task submission table when the task client submits the task to the resource scheduler cluster, wherein the configuration information includes environment variables configured with priorities; after receiving the heartbeat request of the heartbeat operator in the task, the heartbeat information of the task submission table is updated, wherein the heartbeat operator is the heartbeat operator added to the task after the data source operator; the target task and the target priority that meet the restart condition are determined according to the heartbeat information in the task submission table, and the target task is restarted according to the target priority, thereby solving the technical problem that the existing monitoring and operation and maintenance solutions for Flink tasks deployed on the resource scheduler cluster are defective. Compared with the prior art, the present application configures the priority in the environment variables of the Flink task and adds a heartbeat operator in the Flink task, thereby realizing an automated monitoring and operation and maintenance solution for abnormal Flink tasks on the resource scheduler cluster, which can not only solve the problem that the operator in the Flink task cannot be monitored under the prior art, but also can restore the Flink task with high priority first when resources are insufficient.

[0048] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a monitoring and operation and maintenance device for tasks on a resource scheduler cluster, etc. The following takes the monitoring and operation and maintenance device for tasks on a resource scheduler cluster as an example to illustrate this embodiment and the following embodiments.

[0049] Based on this, the embodiment of the present application provides a monitoring and operation method for tasks on a resource scheduler cluster, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for monitoring and operating tasks on a resource scheduler cluster of the present application.

[0050] In this embodiment, the monitoring and operation method of tasks on the resource scheduler cluster includes steps S10 to S30:

[0051] Step S10, when the task client submits the task to the resource scheduler cluster, read the configuration information in the task and write the configuration information into the task submission table, wherein the configuration information includes environment variables configured with priorities;

[0052] It should be noted that the task refers to the Flink task on the Yarn cluster (that is, the resource scheduler cluster). The configuration information also includes the task name, parallelism, and memory size of the Flink task.

[0053] It is understandable that when a Flink task is submitted to the cluster, the Flink task publishing module in the monitoring and operation system can read the configuration information such as environment variables, parallelism, and memory size in the Flink task, and write it into the Flink task submission table.

[0054] It should be noted that the Flink task submission table contains the task name of the Flink task, the running status of the Flink task on the Yarn cluster, the heartbeat information of the Flink task, and the configuration information of the Flink task.

[0055] In a feasible implementation, when the task client submits the task to the resource scheduler cluster, the step of reading the configuration information within the task and writing the configuration information into the task submission table also includes: periodically obtaining the task status on the resource scheduler cluster and updating the task status of the task in the task submission table.

[0056] It should be noted that the task status of the Flink task needs to be pulled from the Yarn cluster regularly and written into the Flink task submission table.

[0057] Step S20, after receiving the heartbeat request of the heartbeat operator in the task, updating the heartbeat information of the task submission table, wherein the heartbeat operator is the heartbeat operator added by the Flink task after the data source operator;

[0058] It should be noted that after receiving the heartbeat request of the heartbeat operator in the Flink task, the corresponding heartbeat information in the task submission table can be updated according to the task name and the operator name of the data source contained in the heartbeat request (updating the heartbeat information means updating the heartbeat to the latest timestamp).

[0059] In a feasible implementation, after receiving the heartbeat request of the heartbeat operator in the task, the heartbeat information of the task submission table is updated, wherein the step of the heartbeat operator being a heartbeat operator added after the data source operator of the task includes: after obtaining the heartbeat request of the heartbeat operator in the task on the resource scheduler cluster through regular scheduling, the heartbeat information of the task submission table is updated, wherein the task includes multiple data source operators and multiple heartbeat operators, and the scheduling thread pool started in the open function of the heartbeat operator is used to perform heartbeat requests regularly, and the heartbeat request includes the task name and the operator name of the data source operator in the task.

[0060] It should be noted that if Figure 2As shown in the figure, each Flink task includes multiple data source operators and multiple heartbeat operators. A heartbeat operator is added after each data source operator. A scheduling thread pool is started in the open function of the heartbeat operator. Requests can be made to the Flink task monitoring application (that is, the task monitoring module in the monitoring operation and maintenance system) regularly. The request body includes the Flink task name and the data source operator name.

[0061] Step S30, determining a target task and a target priority that meet the restart condition according to the heartbeat information in the task submission table, and restarting the target task according to the target priority.

[0062] It should be noted that the target task is the Flink task that needs to be restarted. After the target task is obtained, the target priority of the target task can be determined according to the environment variables with priority configured in the task submission table.

[0063] In a feasible implementation, the step of determining the target task and target priority that meet the restart condition based on the heartbeat information in the task submission table includes: traversing the task submission table according to the number of restarts, and determining whether the task submission time of the task in the task submission table is greater than a preset heartbeat cycle; when it is determined that the task submission time is greater than the preset heartbeat cycle, determining the target task in the task submission table that meets the restart condition based on the heartbeat information and task status in the task submission table, and determining the target priority of the target task based on the task submission table.

[0064] It should be noted that when traversing the task submission table, tasks with fewer restart times are traversed first.

[0065] It should be noted that the Flink task needs to be restarted depending on the following restart conditions:

[0066] 1. Ignore the first three heartbeat cycles of Flink task submission. Since there is a time difference between Flink task submission and Flink task running on the Yarn cluster, skip the restart judgment in the first three heartbeat cycles of Flink task submission (that is, the Flink task does not need to be restarted) to avoid false judgment and infinite loop.

[0067] 2. If the corresponding timestamp of any data source in the Flink task is not updated within n heartbeat cycles, or the Flink task is not running on the Yarn cluster, it is determined that the Flink task needs to be restarted.

[0068] It can be understood that the preset heartbeat cycle can be set to 3 heartbeat cycles. The task submission time is the time difference between the submission of the Flink task and the running of the Flink task on the Yarn cluster. When the task submission time of the Flink task is less than or equal to the preset heartbeat cycle, the restart judgment of the Flink task is skipped, which can avoid misjudgment and infinite loop. When the task submission time of the Flink task is greater than the preset heartbeat cycle, it is necessary to first determine whether the Flink task is in a suspended state according to the task status and heartbeat information in the task submission table (the suspended state means that the task status of the Flink task in the task submission table is still in the running state, but the logic of the operator inside the Flink task is not executed). When the Flink task is in a suspended state, the Flink task needs to be restarted. When the Flink task is not in a suspended state, there is no need to restart the Flink task. That is, the target task that meets the restart condition is the Flink task in a suspended state and needs to be restarted.

[0069] In a feasible implementation, the step of determining the target task in the task submission table that meets the restart condition based on the heartbeat information and task status in the task submission table includes: determining the normal operating task based on the task status in the task submission table; judging whether the heartbeat information of each data stream of the normal operating task in the task submission table has been updated; if the heartbeat information of each data stream of the normal operating task in the task submission table has not been updated, determining the normal operating task as the target task that meets the restart condition.

[0070] It should be noted that the heartbeat of a Flink task can only be judged to be normal when the heartbeat information of each data stream in the Flink task has been updated.

[0071] In the specific implementation, we can first determine which Flink tasks are in normal operation (that is, normal operation tasks) based on the task status in the task submission table, and then determine whether the heartbeat information of the strips of data streams in the normal operation task in the task submission table have all been updated. When the heartbeat information of each data stream in the normal operation task in the task submission table has all been updated, it can be determined that the Flink task is not the target task (that is, the heartbeat of the Flink task is normal). When the heartbeat information of the strips of data streams in the normal operation task in the task submission table has not all been updated, it can be determined that the Flink task is the target task.

[0072] In the specific implementation, Figure 3As shown in the figure, when a Flink task is submitted to the cluster, the Flink task publishing application reads the environment variables in the Flink task, as well as configuration information such as parallelism and memory size, and writes them into the Flink task submission table. The Flink task monitoring application obtains heartbeat information, task submission time, and task status in the Yarn cluster through regular scheduling, determines whether the task is in a suspended state, and finally restarts the Flink task according to priority.

[0073] This embodiment reads the configuration information in the task and writes the configuration information into the task submission table when the task client submits the task to the resource scheduler cluster, wherein the configuration information includes environment variables configured with priorities; after receiving the heartbeat request of the heartbeat operator in the task, the heartbeat information of the task submission table is updated, wherein the heartbeat operator is the heartbeat operator added after the data source operator of the task; the target task and the target priority that meet the restart condition are determined according to the heartbeat information in the task submission table, and the target task is restarted according to the target priority, thereby solving the technical problem that the existing monitoring and operation and maintenance solutions for Flink tasks deployed on the Yarn cluster are defective. Compared with the prior art, the present application implements an automatic monitoring and operation and maintenance solution for abnormal Flink tasks on the Yarn cluster by configuring the priority in the environment variables of the Flink task and adding a heartbeat operator in the Flink task. It can not only solve the problem that the operator pseudo-death in the Flink task cannot be monitored under the prior art, but also can restore the Flink task with high priority first when there are insufficient resources.

[0074] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 4 , step S30 also includes steps S301-S302:

[0075] Step S301, determining a target restart queue of the target task according to the target priority, and adding the target task to the target restart queue;

[0076] It should be noted that the target task can be determined by traversing the Flink tasks in the task submission table according to the number of Flink task restarts, and then the target task can be placed in the corresponding restart queue according to the target priority (the restart queue includes a high priority queue, a medium priority queue, and a low priority queue).

[0077] It is understandable that after the target task is added to the priority queue, it needs to wait for selection by the task submission thread pool. The target task selected by the task submission thread pool will be submitted to the Yarn cluster.

[0078] Step S302: After selecting a target task in the target restart queue through a task submission thread pool, restart the target task.

[0079] It should be noted that the task submission thread pool preferably selects the target task in the target restart queue according to the selection strategy, and the priority selection strategy is to first select the Flink task in the highest priority queue, and first select the Flink task with the least restart times in the same priority queue.

[0080] In a feasible implementation, after the target task in the target restart queue is selected by the task submission thread pool, the step of restarting the target task includes: after the target task in the target restart queue is selected by the task submission thread pool, the corresponding record of the target task in the task submission table is locked, and after the target task is submitted to the resource scheduler cluster, the corresponding record of the target task in the task submission table is unlocked.

[0081] It should be noted that when restarting a task, a lock is added to the corresponding record in the Flink task submission table, and then the Flink task is submitted to the Yarn cluster. After the submission is completed, regardless of success or failure, the client releases the lock on the corresponding record in the Flink task submission table and increases the number of restarts by one.

[0082] In the specific implementation, Figure 5 As shown, the Flink task submission table can be traversed in ascending order of the number of Flink task restarts. After determining that the task meets the above restart conditions, it is placed in the corresponding restart queue according to the priority corresponding to the Flink task. After traversing the Flink task submission table, the task submission thread pool gives priority to tasks in the highest priority queue, and first takes tasks with fewer restarts in the same priority task queue. When restarting a task, lock the corresponding record in the Flink task submission table, and then submit the Flink task to the Yarn cluster. After the submission is completed, regardless of success or failure, the client releases the lock on the corresponding record in the Flink task submission table and increases the number of restarts by one.

[0083] This embodiment determines the target restart queue of the target task according to the target priority, and adds the target task to the target restart queue; after selecting the target task in the target restart queue through the task submission thread pool, the target task is restarted. When the Flink task fails through scheduling monitoring, this embodiment will try to restart the task, and assign it to different task restart queues according to the different priorities of the Flink task configuration, and restart the Flink task with a higher priority first to avoid the problem that important tasks cannot be restored due to physical machine failures.

[0084] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the monitoring and operation method of tasks on the resource scheduler cluster of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0085] This application also provides a monitoring and operation system for tasks on a resource scheduler cluster. Please refer to Figure 6 The monitoring and operation system includes a task publishing module 10, a task monitoring module 20 and a task restart scheduling module 30; wherein,

[0086] The task publishing module 10 is used to read the configuration information in the task when the task client submits the task to the resource scheduler cluster, and write the configuration information into the task submission table, wherein the configuration information includes environment variables configured with priorities;

[0087] The task monitoring module 20 is used to update the heartbeat information of the task submission table after receiving the heartbeat request of the heartbeat operator in the task, wherein the heartbeat operator is the heartbeat operator added after the data source operator of the task;

[0088] The task restart scheduling module 30 is used to determine the target task and the target priority that meet the restart condition according to the heartbeat information in the task submission table, and restart the target task according to the target priority.

[0089] The monitoring and operation system for tasks on the resource scheduler cluster provided by the present application adopts the monitoring and operation method for tasks on the resource scheduler cluster in the above-mentioned embodiments, which can solve the technical problem that the monitoring and operation schemes for Flink tasks deployed on the Yarn cluster are defective. Compared with the prior art, the beneficial effects of the monitoring and operation system for tasks on the resource scheduler cluster provided by the present application are the same as the beneficial effects of the monitoring and operation method for tasks on the resource scheduler cluster provided by the above-mentioned embodiments, and other technical features of the monitoring and operation system for tasks on the resource scheduler cluster are the same as the features disclosed in the above-mentioned embodiment methods, which will not be repeated here.

[0090] The present application provides a monitoring and operation and maintenance device for tasks on a resource scheduler cluster, and the monitoring and operation and maintenance device for tasks on a resource scheduler cluster includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the monitoring and operation and maintenance method for tasks on the resource scheduler cluster in the above-mentioned embodiment one.

[0091] Reference below Figure 7, which shows a schematic diagram of the structure of a monitoring and operation equipment for tasks on a resource scheduler cluster suitable for implementing an embodiment of the present application. The monitoring and operation equipment for tasks on a resource scheduler cluster in an embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The monitoring and operation equipment for tasks on the resource scheduler cluster shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0092] like Figure 7 As shown, the monitoring and operation equipment of the tasks on the resource scheduler cluster may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 to the random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the monitoring and operation equipment of the tasks on the resource scheduler cluster are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 1009. The communication device 1009 can allow the monitoring and operation and maintenance equipment of the tasks on the resource scheduler cluster to communicate wirelessly or wired with other devices to exchange data. Although the figure shows the monitoring and operation and maintenance equipment of the tasks on the resource scheduler cluster with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

[0093] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0094] The monitoring and operation and maintenance device for tasks on the resource scheduler cluster provided by the present application adopts the monitoring and operation and maintenance method for tasks on the resource scheduler cluster in the above-mentioned embodiment, which can solve the technical problem that the monitoring and operation and maintenance schemes for Flink tasks deployed on the Yarn cluster are all defective. Compared with the prior art, the beneficial effects of the monitoring and operation and maintenance device for tasks on the resource scheduler cluster provided by the present application are the same as the beneficial effects of the monitoring and operation and maintenance method for tasks on the resource scheduler cluster provided by the above-mentioned embodiment, and the other technical features of the monitoring and operation and maintenance device for tasks on the resource scheduler cluster are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0095] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0096] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0097] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, the computer-readable program instructions being used to execute the monitoring and operation method for tasks on a resource scheduler cluster in the above-mentioned embodiment.

[0098] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0099] The computer-readable storage medium may be included in the monitoring and operation and maintenance device of the task on the resource scheduler cluster; or may exist independently without being assembled into the monitoring and operation and maintenance device of the task on the resource scheduler cluster.

[0100] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the monitoring and operation and maintenance equipment of the task on the resource scheduler cluster, the monitoring and operation and maintenance equipment of the task on the resource scheduler cluster: when the task client submits the task to the resource scheduler cluster, reads the configuration information in the task, and writes the configuration information into the task submission table, wherein the configuration information includes environment variables configured with priorities; after receiving the heartbeat request of the heartbeat operator in the task, updates the heartbeat information of the task submission table, wherein the heartbeat operator is the heartbeat operator added to the task after the data source operator; determines the target task and the target priority that meet the restart condition according to the heartbeat information in the task submission table, and restarts the target task according to the target priority.

[0101] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0103] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0104] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the monitoring and operation method of the above-mentioned resource scheduler cluster task, and can solve the technical problem that the monitoring and operation schemes for Flink tasks deployed on the Yarn cluster are defective. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the monitoring and operation method of the resource scheduler cluster task provided in the above-mentioned embodiment, and will not be repeated here.

[0105] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the monitoring and operation method for tasks on the resource scheduler cluster as described above.

[0106] The computer program product provided by this application can solve the technical problem that the monitoring and operation and maintenance solutions for Flink tasks deployed on the Yarn cluster are defective. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the monitoring and operation and maintenance method for tasks on the resource scheduler cluster provided by the above embodiment, which will not be repeated here.

[0107] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for monitoring and operating tasks on a resource scheduler cluster, characterized in that: The monitoring operation and maintenance method comprises: When the task client submits the task to the resource scheduler cluster, the configuration information in the task is read and the configuration information is written into the task submission table, wherein the configuration information includes environment variables configured with priorities; After receiving the heartbeat request of the heartbeat operator in the task, updating the heartbeat information of the task submission table, wherein the heartbeat operator is the heartbeat operator added after the data source operator of the task; Determine a target task and a target priority that meet the restart condition according to the heartbeat information in the task submission table, and restart the target task according to the target priority.

2. The method according to claim 1, characterized in that After receiving the heartbeat request of the heartbeat operator in the task, the step of updating the heartbeat information of the task submission table, wherein the heartbeat operator is a heartbeat operator added after the data source operator of the task, comprises: After obtaining the heartbeat request of the heartbeat operator in the task on the resource scheduler cluster through regular scheduling, the heartbeat information of the task submission table is updated, wherein the task includes multiple data source operators and multiple heartbeat operators, and the scheduling thread pool started in the open function of the heartbeat operator is used to perform heartbeat requests regularly, and the heartbeat request includes the task name and the operator name of the data source operator in the task.

3. The method according to claim 1, characterized in that When the task client submits the task to the resource scheduler cluster, the step of reading the configuration information in the task and writing the configuration information into the task submission table further includes: The task status on the resource scheduler cluster is obtained regularly, and the task status of each task in the task submission table is updated.

4. The method according to claim 3, characterized in that The step of determining the target task and the target priority that meet the restart condition according to the heartbeat information in the task submission table comprises: Traverse the task submission table according to the number of restarts, and determine whether the task submission time of the task in the task submission table is greater than the preset heartbeat period; When it is determined that the task submission time is greater than the preset heartbeat period, the target task in the task submission table that meets the restart condition is determined according to the heartbeat information and task status in the task submission table, and the target priority of the target task is determined according to the task submission table.

5. The method according to claim 4, characterized in that The step of determining the target task in the task submission table that meets the restart condition according to the heartbeat information and the task status in the task submission table comprises: Determine the normal operation task according to the task status in the task submission table; Determine whether the heartbeat information of each data flow of the normal running task is updated in the task submission table; If the heartbeat information of each data stream of the normal operating task in the task submission table is not updated, the normal operating task is determined as a target task that meets the restart condition.

6. The method according to claim 1, characterized in that The step of restarting the target task according to the target priority comprises: Determine a target restart queue for the target task according to the target priority, and add the target task to the target restart queue; After selecting the target task in the target restart queue through the task submission thread pool, the target task is restarted.

7. The method according to claim 6, characterized in that After selecting the target task in the target restart queue through the task submission thread pool, the step of restarting the target task includes: After selecting the target task in the target restart queue through the task submission thread pool, the corresponding record of the target task in the task submission table is locked, and the corresponding record of the target task in the task submission table is unlocked after the target task is submitted to the resource scheduler cluster.

8. A monitoring and operation system for tasks on a resource scheduler cluster, characterized in that: The monitoring and operation system includes a task publishing module, a task monitoring module and a task restart scheduling module; wherein, The task publishing module is used to read the configuration information in the task when the task client submits the task to the resource scheduler cluster, and write the configuration information into the task submission table, wherein the configuration information includes environment variables configured with priorities; The task monitoring module is used to update the heartbeat information of the task submission table after receiving the heartbeat request of the heartbeat operator in the task, wherein the heartbeat operator is the heartbeat operator added after the data source operator of the task; The task restart scheduling module is used to determine the target task and the target priority that meet the restart condition according to the heartbeat information in the task submission table, and restart the target task according to the target priority.

9. A monitoring and operation device for tasks on a resource scheduler cluster, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the method for monitoring and operating tasks on a resource scheduler cluster as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the method for monitoring and operating tasks on a resource scheduler cluster as described in any one of claims 1 to 7 are implemented.

Images