Electric power data association query method and device based on intelligent filtering

Abstract

The invention relates to the technical field of big data processing, in particular to an electric power data association query method and device based on intelligent filtering. An electric power data association query method based on intelligent filtering comprises the following steps: inputting sample data to form a sample training set to be subjected to association query; cleaning sample data; extracting characteristic parameters; constructing a filtering strategy combination library; implementing a filtering strategy; accurate matching is carried out; the method has the advantages that firstly, the problems of large calculation amount and poor timeliness of correlation matching of more than a hundred million level of power data are solved, and the quality and efficiency of unstructured data matching are greatly improved; secondly, optimized filtering combinations can be generated for different types of data, the problem that non-matched data cannot be effectively eliminated by a single filtering strategy is avoided, and the adaptability of the filtering strategy to heterogeneous power data is improved; and thirdly, an operable correlation matching evaluation algorithm is constructed, the filtering method is evaluated from two aspects of matching quality and calculation efficiency, and the engineering practical value is achieved.

Description

Technical field: [0001] The present invention relates to the technical field of big data processing, in particular to a method and device for associated query of power data based on intelligent filtering. Background technique: [0002] Electric power big data mainly comes from all aspects of power generation, transmission, transformation (distribution), and use, covering business fields such as power production, power grid operation, enterprise management, and marketing services. For a long time, the power system has been divided into majors to carry out informatization construction, resulting in most of the electric power majors forming their own systems. The data are independent of each other and are stored in different platform systems, forming an information island. It mainly embodies three types of characteristics: First, the rapid development of "cloudification" of server equipment has made mass storage possible, the data buried points of various electric power majors ...

AI Technical Summary

Problems solved by technology

Third, there are differences in the data models and storage specifications of different professions, and the description of the same entity is inconsistent. In addition, some data is manually entered, and the traditional precise matching method has defects such as low coverage, and it is difficult to effectively realize the data association of the same entity.

The existing data association technology is mainly based on the filtering-verification framework, but the filtering method lacks the targeted development and adjustment of electric power data, and there are generally problems of low accuracy and low integrity of the associated data in the precise matching process

In 2019, the Electric Power Technology Journal published the article "Address Fuzzy Matching Technology Based on Edit Distance", which solved the problem of homophonic and precise matching in electric power customer service data, but for massive data, the data model has low operating efficiency and is difficult to meet the needs of engineering practice

[0004] When power data is correlated and queried, firstly, precise matching methods are usually used in data correlated queries, resulting in a large amount of approximate data not being effectively matched, and the overall matching hit rate is low; secondly, for highly dissimilar data, the traditional precise matching method is still used. A lot of computing resources are wasted, there is a lack of efficient filtering methods for highly dissimilar data, the overall operation process is inefficient, takes a long time, and is highly dependent on the expansion of computing and storage resources

[0005] Although filtering methods are used to improve efficiency in relational queries in other industries, the effect of filtering methods on different texts varies greatly, and it is difficult to provide an optimized filtering combination strategy for multi-source heterogeneous power data

Simply apply the filtering method, and the calculation efficiency is difficult to be effectively guaranteed

Images