Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data cleaning and repairing method

A technology of data cleaning and repairing method, which is applied in the direction of electronic digital data processing, structured data retrieval, database management system, etc., and can solve problems such as inaccurate results of auxiliary decision-making advanced applications, different time scales, measurement residual pollution, etc. To achieve the effect of ensuring high concurrency, preventing data leakage, and ensuring rapidity

Pending Publication Date: 2019-05-21
CHINA ELECTRIC POWER RES INST +2
View PDF0 Cites 14 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In view of the poor quality of current distribution network data, different time scales, serious measurement residual pollution, and closed data between systems, it has seriously affected the normal development of current distribution network planning, operation and maintenance, and maintenance, directly leading to auxiliary decision-making and advanced For the problem of inaccurate application results, a new data cleaning and repair method is urgently needed. The present invention can efficiently distinguish data errors and bad data, prevent data omissions and misjudgments, ensure repair efficiency, and meet the requirements of upper-level business application data; Different data error formation mechanisms quickly evaluate quality problems and call corresponding repair modules to realize data quality improvement and repair functions. Differentiated and flexible loading, flexible and adjustable granularity, while ensuring the rapidity and high concurrency of the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data cleaning and repairing method
  • Data cleaning and repairing method
  • Data cleaning and repairing method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments, wherein the schematic embodiments and descriptions are only used to explain the present invention, but are not intended to limit the present invention.

[0023] Such as figure 1 As shown, the data cleaning and repairing system applied to a data cleaning and repairing method of the present invention is described in detail, and the system includes: a physical system, a repairing system, a data quality evaluation system, a data service system, a business application system, and a data visualization system , management and control layer system;

[0024] The repair system includes: an access layer module, a conversion layer module, a cleaning layer module and an offline layer module; the repair system is used for data monitoring extraction and data repair; wherein: data monitoring extraction includes real-time online data and static historical data ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a data cleaning and repairing method, which comprises the following steps of: for data errors generated in acquisition and data transmission generated by power distribution network terminal equipment, setting an access module to carry out data cleaning and repairing; For data errors generated by information equipment in the power distribution network information system, setting a conversion module for data cleaning and repairing; For data interaction among a plurality of business information systems of the power distribution network, data badness generated in data processing, setting a rule base cleaning module; For cleaning of historical data stored in a warehouse, setting an offline module for data cleaning and repairing, and carrying out logic checking on the whole historical data. According to the method, data errors and bad data conditions can be efficiently judged, data missed judgment and wrong judgment are prevented, the repair efficiency is ensured, and the upper-layer business application data requirement is met.

Description

【Technical field】 [0001] The invention belongs to the technical field of distributed data processing, and in particular relates to a data cleaning and repairing method. 【Background technique】 [0002] The quality of data is related to the stable operation of the distribution network. If there is no certain data quality improvement method for massive low-value density data, even the most accurate processing logic and the most efficient parallel strategy will not obtain correct results. At present, many data preprocessing and cleaning systems in the field of information science have been introduced into the field of distribution network big data research, but these systems can only be applied to large-scale data with simple Internet logic or regular application scenarios such as image audio and video. It cannot handle the high-speed real-time data of the whole process generated by the continuous and highly automated production process of the power system and the high-dimension...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/25G06F16/215
Inventor 盛万兴刁赢龙胡丽娟刘科研孟晓丽何开元贾东梨叶学顺董伟杰白牧可
Owner CHINA ELECTRIC POWER RES INST
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com