Streaming type statistical definition and operation method based on configuration file

Abstract

The invention discloses a streaming type statistical definition and operation method based on a configuration file. The method comprises the following steps: creating a configuration file, wherein a content of the configuration file includes task information and a computing model; editing the task information, wherein the task information includes an attribute, a task name, a task type and a task description; editing the computing model, wherein constituent elements required by a computing model descriptive statistical process includes a data source definition, a mode definition, a statistical operation definition and an action definition; verifying the validity of the configuration file; carrying out validity management on the configuration file, wherein all streaming type statistical configuration files are uniformly managed by separate task configuration files and can take effect only being increased to the valid streaming type statistical configuration files in the task configuration file; and operating streaming type statistical tasks, wherein start parameters of a statistical service start time bandwidth include a task number, and in a statistical task execution process, all data are of a JSON format and are accessed via an attribute name. The method disclosed by the invention avoids the code maintenance, can be defined flexibly and is applied to scheduling an automation system.

Description

technical field [0001] The invention relates to a definition and operation method of flow statistics based on a configuration file, and belongs to the technical field of electric power system automation. Background technique [0002] With the development of cloud computing and big data technology, the use of big data technology to obtain potentially useful information from massive data has attracted widespread attention. Log data, as raw big data, is getting more and more attention for its value. In the scheduling automation system, the current log data only stays for the single purpose of post-event fault diagnosis, and its main analysis method still stays at the stage of manual log filtering and retrieval with simple scripts. In order to fully tap the value of the logs in the dispatch automation system, conduct data statistics on the key indicators of the dispatch automation system, and provide a data basis for assessing the health of the system in a quantitative manner, ...

AI Technical Summary

Problems solved by technology

This design method has high flexibility and wide adaptability, but the price is that adding new statistical tasks requires calling the corresponding interface to complete code editing and compiling, which is more complicated to use and high in learning costs.

For the scheduling automation system, the realization of the streaming statistics function based on the open source streaming computing framework is very intrusive to the original system in terms of deployment, and on the other hand, it is not conducive to the use of users

Images