Migration balancing policy based electricity-consuming information distributed file storage method and apparatus

A technology of distributed files and electricity consumption information, applied in the fields of electricity digital data processing, special data processing applications, climate sustainability, etc., can solve the problems of data processing, storage and management needs differences, etc. The effect of high utilization and meeting storage requirements

Inactive Publication Date: 2015-12-23
SHANDONG UNIV +1
View PDF4 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

(2) There are various types of data. The data involved in the electricity consumption information collection system include structured, semi-structured and unstructured data such as basic file data, original data frames, basic business data, complex business data, and statistical analysis data. There are also large differences in data processing, storage and management requirements

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Migration balancing policy based electricity-consuming information distributed file storage method and apparatus
  • Migration balancing policy based electricity-consuming information distributed file storage method and apparatus
  • Migration balancing policy based electricity-consuming information distributed file storage method and apparatus

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0038] Such as figure 1 As shown, it is a distributed file storage method of power consumption information based on the migration balance strategy of the present invention, which includes the following steps:

[0039] The A data acquisition module parses the original data frame of the electricity consumption data sent by the acquisition terminal of the electricity consumption information acquisition system according to its channel type and reduction type to obtain the original data. The original data is stored in the local disk module in the form of a file for caching, and The original data frame is uploaded to the front-end communication platform relational database and MongoDB database;

[0040] B detects the number, size and last write time of local disk files. If the file size reaches the set threshold or the file is not updated within the set time...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention discloses a migration balancing policy based electricity-consuming information distributed file storage method and apparatus. The method comprises: a data acquisition module analyzing original data frames to obtain original data, storing the original data into a local disk module in the form of a file, and uploading the original data frames to a relational database and a MongoDB database; detecting information of the file in the local disk module, and selecting whether to upload the file to a data storage module; the MongoDB database acquiring a node load where a slice is located in mongos by means of routing and by using a migration balancing policy to implement balancing from two aspects: a data volume and a load; a big data management engine regularly monitoring file information that is not subjected to data saving calculation, extracting the file from the MongoDB database, and using time as a tag; converting file data extracted from the MongoDB database into data character string streams, and carrying out calculation to obtain service data; and finally, performing data persistence. Therefore, efficient storage of electricity-consuming information big data is implemented, and electricity-consuming big data storage capacity and storage rate demands of an electricity-consuming acquisition system are met.

Description

technical field [0001] The invention relates to the field of intelligent electricity utilization, in particular to a distributed file storage method and device for electricity utilization information based on a migration balance strategy. Background technique [0002] With the continuous deepening of smart grid construction and the advancement of the "large concentration" construction model, the number of power users and terminals has increased, and user electricity consumption data has become a typical industry big data. The power consumption data of power users presents big data characteristics such as large scale, diversity and heterogeneity, which are specifically manifested in: (1) The data scale is huge. Taking Shandong Province as an example, since the operation of the power consumption information collection system in 2007, the cumulative data volume is about 1.4PB; after the realization of "full coverage" in 2014, the daily growth of system power consumption data is...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F17/30
CPCY02D10/00
Inventor 史玉良李庆忠张晖管永明吕梁
Owner SHANDONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap