Electricity consumption data batch high speed processing method and system based on distributed offline technology

A power data, high-speed processing technology, applied in the direction of electrical digital data processing, special data processing applications, structured data retrieval, etc., can solve the problems that cannot meet the requirements of massive data storage, computing, and lack of high-speed massive data storage computing models, etc. Achieve the effect of improving system stability, increasing processing capacity, and improving execution efficiency

Inactive Publication Date: 2017-03-15
STATE GRID ZHEJIANG ELECTRIC POWER +2
View PDF3 Cites 38 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] However, at present, there is a lack of high-speed and massive data storage and computing models. The current electricity consumption information collection business data model, with the needs of high-fre

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Electricity consumption data batch high speed processing method and system based on distributed offline technology
  • Electricity consumption data batch high speed processing method and system based on distributed offline technology
  • Electricity consumption data batch high speed processing method and system based on distributed offline technology

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0038]At present, the acquisition system can be widely used in company marketing and many other professions. Through the application of remote automatic meter reading, the transformation of electricity bill reading and collection mode is realized, supporting the full implementation of tiered electricity prices and peak-valley electricity prices, and the automatic meter reading and accounting ratio reaches over 92%; through the application of fee control functions, the efficiency of electricity fee recovery is effectively improved, reducing Electricity cost risk; through the online monitoring of the metering device, the evaluation of the operation status of the on-site metering device can be realized, which plays an important role in ensuring the safe and stable operation of the power grid, anti-stealing electricity, and reducing publi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides an electricity consumption data batch high speed processing method and system based on distributed offline technology, and relates to a data batch processing method system. At present, there is a lacking of the high speed mass data storage and calculation model, and not all the types and forms of mass data storage and calculation can be fulfilled. The method comprises the steps of (1) an electricity consumption front sampler sending the sampled electricity consumption data to a Kafka data queue for buffering in a real-time mode, (2) Storm cluster reading the electricity consumption information in the Kafka queue, and storing the information in a real-time mode to a Hbase, (3) Spark extracting the electricity consumption information to be processed from the Hbase, and uploading the information to a Hive data sheet, (4) through Spark conducting the offline operation of related Hive data sheet to acquire the electricity consumption of the current period, and processing related abnormalities. The method combines the advantages of both the Storm and the Spark, enhances the overall processing capacity. From the dynamic migration technology of transaction security node, a transaction security protocol of complete task node migration is proposed. On the foundation of the guarantee of no packet loss, no repetition in the migration process, at the same time the execution efficiency of the migration itself is enhanced, and the system stability is elevated.

Description

technical field [0001] The present invention relates to a data batch processing method and system, in particular to a batch high-speed processing method and system for power data based on distributed offline technology. Background technique [0002] The electricity consumption information collection system consists of a system master station, a transmission channel, a collection terminal and a smart meter. [0003] The system master station is responsible for the collection, storage, analysis, processing and application of electricity consumption information of the entire system, and is composed of communication subsystems, databases, business applications, interface applications, etc. Most of the network provincial companies adopt the provincial-level centralized deployment method for construction. [0004] The transmission channel is divided into the remote communication channel between the system master station and the terminal, and the local communication channel betwee...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F9/48G06F17/30
CPCG06F9/4881G06F16/27Y02D10/00
Inventor 蒋锦霞朱青山裴旭斌戴波王红凯何东闫娇娇
Owner STATE GRID ZHEJIANG ELECTRIC POWER
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap