Method and device for collecting decentralized big data

A technology of decentralization and collection method, applied in transmission systems, electrical components, etc., can solve problems such as the inability to fully utilize the processing performance of distributed computing clusters, and the low efficiency of sensor data processing, so as to improve transmission and storage efficiency, ensure The effect of authenticity

Active Publication Date: 2021-08-13
STATE GRID CORP OF CHINA +1
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, when there are many application scenarios and the sensing devices involved in each application scenario are complex, the processing efficiency of the central processing server for a large amount of sensing data is low, and the processing performance of the distributed computing cluster cannot be fully utilized.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for collecting decentralized big data
  • Method and device for collecting decentralized big data
  • Method and device for collecting decentralized big data

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] see figure 1 A flowchart of a method for collecting decentralized big data is shown, the method is applied to a gateway, and specifically includes the following steps:

[0048] S101. Collect data detected by the sensor;

[0049] S102. Process the data to form processed data;

[0050] S103. Using a symmetric encryption key to encrypt the processed data to form an encrypted file;

[0051] S104. Select a preset asymmetric encryption public key to encrypt the symmetric encryption key, and produce a header file according to the number of the application scene of the processed data, and the asymmetric encryption public key and the header file form a combined file ;

[0052] S105. Transmit the encrypted file and combined file to the cloud storage server.

[0053] Further, the data is processed to form processed data, including:

[0054] The data is constructed into a data matrix, and the dimensionality reduction of the data matrix is ​​a dimensionality reduction data set,...

Embodiment 2

[0057] see figure 2 A flow chart of a method for collecting decentralized big data is shown, the method is applied to a distributed server, and specifically includes the following steps:

[0058] S201. Download the encrypted file and combined file on the cloud server;

[0059] S202. Use the asymmetric encryption public key in the combination file to decrypt the symmetric encryption key;

[0060]S203. Find a server matching the processed data according to the number in the header file in the combined file, and decrypt the encrypted file by using the symmetric encryption key.

[0061] Further, according to the number in the header file in the combined file, find the server matching the processed data, and use the symmetric encryption key to decrypt the encrypted file, including:

[0062] The encrypted file is decrypted into a reduced-dimensional data set by using the symmetric encryption key.

Embodiment 3

[0064] Regarding the method for collecting decentralized big data provided in the foregoing embodiments, the embodiment of the present invention provides a device for collecting decentralized big data, which is applied to a gateway, see image 3 Shown is a structural block diagram of a device for collecting decentralized big data, which includes the following parts:

[0065] Acquisition module 31, is used for collecting the data that sensor detects;

[0066] A processing module 32, configured to process the data to form processed data;

[0067] An encryption module 33, configured to encrypt processed data using a symmetric encryption key to form an encrypted file;

[0068] The combination module 34 is used to select a preset asymmetric encryption public key to encrypt the symmetric encryption key, and produce a header file according to the number of the application scene of the processed data, the asymmetric encryption public key and the header The file forms a combined file...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention provides a method and device for collecting decentralized big data, without the central server distributing the sensing data collected by the gateway to different distributed servers according to tasks; the integration and analysis of the sensing data is free from The performance of the central server is limited; at the same time, during the process of data interaction between the gateway and the distributed server through the cloud storage server, the sensing data is encrypted through a combination of symmetric encryption algorithm and asymmetric encryption algorithm to ensure the authenticity of the sensing data; At the same time, the gateway adopts the design idea of ​​edge computing to process the sensing data, compress the sensing data under the premise of ensuring the analysis value of the data, and improve the efficiency of transmission and storage.

Description

technical field [0001] The invention relates to the technical field of big data collection, in particular to a method and device for collecting decentralized big data. Background technique [0002] Big data refers to the collection of data that cannot be captured, managed and processed by conventional software tools within an acceptable time frame. [0003] Big data analysis is a technology that conducts statistics and analysis of all data, and is used to obtain or predict answers through existing or unknown correlations between multiple data. [0004] Big data analysis technology has high application value in scenarios such as smart cities and industrial systems, but scenarios such as smart cities are subdivided into several specific application scenarios, such as smart transportation, smart agriculture, and smart water affairs. The application scenarios are all equipped with sensing devices suitable for or dedicated application scenarios. Sensing devices in multiple appl...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/06
CPCH04L63/045
Inventor 罗弦胡率廖荣涛焦尧毅曾玉荣徐焕查志勇詹伟龙霏刘芬王逸兮冯浩代荡荡
Owner STATE GRID CORP OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap