Unlock instant, AI-driven research and patent intelligence for your innovation.

A data bleaching method and device

A data bleaching and data technology, applied in the field of data processing, can solve the problems of complex data statistics process, low efficiency, no data processing process, etc., to achieve the effect of solving complexity and improving efficiency

Active Publication Date: 2021-11-23
BANK OF CHINA
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the existing data bleaching scripts mainly bleach the data one by one, which will reduce the bleaching efficiency of big data, and directly import the bleached data into another system or environment for analysis, without a general data processing process. Make the data statistics process complicated and inefficient

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data bleaching method and device
  • A data bleaching method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0042] see figure 1 A data bleaching method provided in Embodiment 1 of the present invention includes:

[0043] S11. Acquire source data, and store the source data in a distributed storage manner;

[0044] Since there are multiple types of source data, and each type of data is based on a different transmission protocol, when the source data is acquired in the embodiment of the present invention, it is necessary to advance the data unified interface that only supports multiple data transmission modes, so that Various types of source data are obtained, and different data types can be processed. At the same time, encrypted source data transmission is supported, that is, a corresponding decryption method can be set in the present invention, and if the source data is in encrypted mode, the source data can be decrypted.

[0045] Since the source data is directly stored on a single storage node in the prior art, the source data can only be accessed through the storage node. If the...

Embodiment 2

[0068] Referring to embodiment one of the present invention and figure 1 For the specific process of steps S11 to S14 described in the above, Embodiment 2 of the present invention also provides a data bleaching device, including:

[0069] An acquisition module 1, configured to acquire source data, and store the source data in a distributed storage manner;

[0070] The bleaching module 2 is used to bleach the stored source data in blocks to obtain bleached data;

[0071] A storage module 3, configured to generate an index of the bleached data, and store the bleached data in a database according to the index;

[0072] The extraction module 4 is configured to search for an index corresponding to the category according to the category of the data to be counted, extract the bleached data in the database according to the index, and perform statistics on the extracted bleached data.

[0073] Specifically, the bleaching module includes:

[0074] a block unit, configured to block th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data bleaching method and device. The method includes: acquiring source data, storing the source data in a distributed storage manner; performing block bleaching on the stored source data to obtain bleached data ; generate the bleached data index, store the bleached data in the database according to the index; search for the index corresponding to the category according to the category of the data to be counted, and extract the data in the database according to the index Data after bleaching, and perform statistics on the extracted data after bleaching. The invention realizes the purpose of improving the data bleaching efficiency and conveniently performing statistical analysis on the bleached data.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a data bleaching method and device. Background technique [0002] With the development of information technology, the amount of information data is increasing, and people can access massive data information through various channels. Usually data contains various information, some of which are not what the user wants, or some information contains sensitive information, so the data needs to be bleached, and then the data information can be better used. [0003] The existing data bleaching system usually stores the data to be bleached in the intermediate processing platform environment, then writes the bleaching script according to the bleaching requirements, runs the bleaching script on the intermediate platform environment, and bleaches the data to be bleached to generate bleached data. Finally, import the bleached data into another environment for statistical analysis, and obtain ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/2458G06F16/215G06F16/25G06F16/27
CPCG06F16/215G06F16/258G06F16/27
Inventor 许凌超陈志李汉涛
Owner BANK OF CHINA