Data processing method and system based on data resource directory construction

A data resource and data processing technology, which is applied in the direction of electronic digital data processing, database management system, special data processing applications, etc., can solve the problem that database storage technology cannot meet the performance requirements of data storage, query, etc., and the ability of parallel processing of large data is low. Problems such as high construction costs have achieved the effects of reducing operational complexity, improving management and use levels, and speeding up implementation

Inactive Publication Date: 2020-05-15
国家电网有限公司大数据中心
View PDF4 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Because different businesses have different data access performance requirements, a single database storage technology cannot meet the performance requirements of all businesses for data storage and query, and the ability to process large data in parallel is low and the construction cost is high. Encountered technical bottlenecks when dealing with massive multi-source heterogeneous data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and system based on data resource directory construction
  • Data processing method and system based on data resource directory construction
  • Data processing method and system based on data resource directory construction

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0067] Existing types of databases include relational databases, columnar databases, real-time databases, and MPP databases. Each type of database has its own advantages and disadvantages in the storage of intelligent data resource catalog construction. Among them, relational databases are stored in rows. Relational databases are good at random read operations and are not suitable for big data. They are mainly used in occasions where the real-time performance of data access is not high. Columnar storage is suitable for low-latency read-write access and high-concurrency access requests. The data storage and management based on column storage is fast in loading speed, easy to compress, and easy to do aggregation analysis, which is suitable for application functions such as statistics and analysis under the scale of large data. Columnar databases are organized, indexed, and stored according to key-value pairs (Key-Value Pair), which is suitable for semi-structured data storage wi...

Embodiment 2

[0109] Based on the same inventive concept, the embodiment of the present invention also provides a data processing system based on the data resource directory, including:

[0110] The classification storage module is used to classify and store the big data of electric power informatization collected from the construction of each data resource catalog and according to the data type;

[0111] The processing module is used to select the corresponding data from the data stored according to the data type according to the business requirements and the construction of each data resource directory, centrally store and process them, and form business data;

[0112] An application service module, configured to provide the business data to users;

[0113] Wherein, the data resource directory is constructed and stored based on database types; the data types include model data, operating data and statistical analysis data.

[0114] In this embodiment, the processing module includes:

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data processing method and system based on data resource directory construction. The method comprises the following steps: classifying electric power informatization big datacollected in each data resource directory construction and storing the data according to data types; selecting corresponding data from the data stored according to the data types according to the service requirements and the data resource directories, and storing and processing the data in a centralized manner to form service data; providing the service data for a user, wherein the data resource directory construction is based on database types for storage, and the data types comprise model data, operation data and statistical analysis data. According to the invention, the data performance ofregulating and controlling different services of big data in multiple aspects is improved.

Description

technical field [0001] The invention relates to the field of data storage, in particular to a data processing method and system constructed based on a data resource directory. Background technique [0002] With the large-scale construction of the UHV AC-DC hybrid power grid and the comprehensive advancement of the "big operation" system, the service scope of the power grid dispatching control system is rapidly expanding, the data sources are increasing, the data types are diverse, and the data scale and storage pressure are increasing sharply. The requirements for the efficient storage and fast processing of massive multi-source data are constantly increasing in regulatory business. At present, the database used in the power dispatching control system is mainly a relational database. A relational database is a database based on a relational model. It uses mathematical concepts and methods such as set algebra to process the data in the database. It is stable, safety features...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/25G06F16/215G06F16/27G06F16/28G06F16/2457
CPCG06F16/22G06F16/254G06F16/215G06F16/27G06F16/284G06F16/2457
Inventor 梁云丹高灵超周春雷王宏刚聂玲姚黎明何笛宋文婷
Owner 国家电网有限公司大数据中心
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap