Resource monitoring method and device based on YARN, and terminal equipment
A resource monitoring and resource technology, applied in the field of big data, can solve problems such as busy system, inability to execute tasks, insufficient operating resources, etc., and achieve the effect of avoiding long-term occupation, improving resource utilization and task operating efficiency
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Method used
Image
Examples
Embodiment 1
[0041] figure 1 It shows a schematic flowchart of the first YARN-based resource monitoring method provided by the embodiment of the present application. The execution subject of the embodiment of the present application is a terminal device, and the terminal device can access the YARN interface of the YARN-based distributed system. Details as follows:
[0042] In S101 , within the target time period, the running state information of each target task being executed is obtained through the YARN interface every first preset time period, and the running state information includes the running time and the resource size occupied by the running.
[0043] The YARN interface in the embodiment of this application specifically refers to the representational state transfer application interface (Representational State Transfer API, REST API) provided by the Hadoop YARN Web service, which is used to obtain the cluster, node and application program information of the YARN-based distributed ...
Embodiment 2
[0064] figure 2 It shows a schematic flowchart of the second YARN-based resource monitoring method provided by the embodiment of the present application. The execution subject of the embodiment of the present application is a terminal device, and the terminal device can access the YARN interface of the YARN-based distributed system. Details as follows:
[0065] The YARN-based resource monitoring method in the embodiment of the present application includes both the steps from step S101 to step S102 described in the first embodiment, and the step of storing operation log information. The detailed content of step S101 to step S102 in the embodiment of the present application is completely the same as that in the first embodiment, and will not be repeated here. The operation log information storage steps of the embodiment of the present application are described in detail as follows:
[0066] S201: If a task whose execution is completed is detected, store the running log inform...
Embodiment 3
[0079] image 3 It shows a schematic structural diagram of a YARN-based resource monitoring device provided by the embodiment of the present application. For the convenience of description, only the parts related to the embodiment of the present application are shown:
[0080] The YARN-based resource monitoring device includes: a running state information acquisition unit 31 and an abnormal task termination unit 32 . in:
[0081] The running state information acquisition unit 31 is used to obtain the running state information of each target task in execution through the YARN interface every first preset time period within the target time period, and the running state information includes the running time length and the running time. resource size;
[0082] The abnormal task terminating unit 32 is configured to terminate the abnormal task if it is detected that there is an abnormal task in the target task, wherein the abnormal task is that the running time exceeds the running...
PUM
Login to View More Abstract
Description
Claims
Application Information
Login to View More