Early warning system of data development platform realized based on flink
A technology for developing platforms and early warning systems. It is applied in the fields of electrical digital data processing, generation of response errors, and instruments. It can solve problems such as hidden dangers of the platform, lack of algorithms, and low accuracy of fault diagnosis results, so as to achieve rapid and accurate positioning and improve accuracy. The effect of reducing the false alarm rate of faults
Inactive Publication Date: 2022-01-11
北京科杰科技有限公司
View PDF2 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
The current early warning system cannot issue an early warning before a fault, but can only perform fault diagnosis after the fault occurs, which brings great hidden dangers to the platform, and there is no algorithm support, and the conclusion is drawn based on incomplete information. very low accuracy
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0014] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution, and provides detailed implementation and specific operation process, but the protection scope of the present invention is not limited to the present invention. Example.
[0015] This embodiment provides a data development platform early warning system based on flink, such as figure 1 shown, including:
[0016] Message queue module: used to receive pushed indicator data, and store them in different message partitions in the system database according to the processing task type of the data development platform; the indicator data includes related indicators such as CPU and memory used by the program, and network io;
[0017] Data real-time prediction module: used to use the flink real-time computing engine to use the accumulated data to predict the next data when the data is received normally and ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses an early warning system of data development platform realized based on flink, which is characterized in that time sequence data during business processing is received and forwarded by a message queue, the received data is processed and analyzed in real time by using an flink real-time calculation engine, and meanwhile, different data sensitivity levels are divided for different businesses; processing different data on the data with different sensitivity levels, so that the accuracy of fault detection is improved, and meanwhile, the fault false alarm rate is reduced. The invention also predicts the data of the next time point, compares the data with the collected real-time data and the historical data stored in the database, judges the fault, and achieves the quick and accurate positioning of the fault occurrence point of the platform and the fault generation reason. At present, early warning can be carried out on the fault 72 hours before the platform fault occurs, the accuracy rate is 90% or above, and the accuracy rate of the conclusion of the generation reason of the fault reaches 95% or above.
Description
technical field [0001] The invention relates to the field of data center fault diagnosis, in particular to a data development platform early warning system based on flink. Background technique [0002] When the data platform processes a large number of data tasks, there will be excessive storage capacity, high memory usage or other problems, which will cause platform failures, ranging from reducing work efficiency to causing platform congestion and stalling, affecting business processing and data. analyze. In response to this situation, the industry attaches great importance to it, and the requirements for platform security, stable long-term and full-load operation are becoming more and more urgent. It is hoped that we can understand the status of the platform system in a timely manner, and achieve early detection and early prevention of faults through detection and prediction of status trends. The purpose of solving the problem is to avoid platform blocking and improve pro...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F11/07
CPCG06F11/079
Inventor 于洋高经郡道玉明
Owner 北京科杰科技有限公司