Unlock instant, AI-driven research and patent intelligence for your innovation.

Data processing device and method

A data processing device and data technology, applied in the field of communication, can solve the problems of consuming computer software and hardware resources, and achieve the effects of reducing consumption, increasing processing speed, and simplifying the processing process

Active Publication Date: 2015-07-22
HUAWEI TECH CO LTD
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] At present, the use of existing technologies to simplify the processing of large amounts of data still consumes a large amount of hardware and software resources in the computer, such as computer memory and central processing unit (CPU, Central Processing Unit) resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing device and method
  • Data processing device and method
  • Data processing device and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0049] The present invention provides a data processing device and method, which are used to simplify the process of processing a large number of data files by threads through a hash algorithm, improve the processing speed of threads on files, reduce the consumption of computer resources, and increase the calculation speed of hash algorithms .

[0050] Such as figure 1 As shown, the embodiment of the present invention provides a data processing device, the device includes:

[0051] The determination unit 11 is used to determine any line of data in the file to be processed, determine the feature value of the line of data according to the preset key information of the line of data, and locate the row data in the preset array structure according to the feature value. elements of the identity;

[0052] The processing unit 12 is configured to determine whether the element located by the determination unit 11 is occupied, and if so, determine the row data corresponding to the row ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a data processing device and method. The data processing device and method are used for simplifying the process that a thread processes a large amount of data files, increasing the file processing speed for the thread, reducing the consumption of computer resources and increasing the computation speed of a Hash algorithm. The data processing method comprises the steps that for any one row of data of a file to be processed, the feature value of the row of data is determined according to preset key information of the row of data, and according to the feature value, an element used for storing the identification of the row of data is located in a preset array structure; whether the located element is occupied or not is judged, if yes, the row of data and the row of data corresponding to the identification occupying the element are determined as rows of data meeting preset conditions, and if not, the identification of the row of data is stored in the element.

Description

technical field [0001] The present invention relates to the communication field, in particular to a data processing device and method. Background technique [0002] When processing a large amount of data, it is usually necessary to deduplicate the large amount of data, that is, in a large amount of data, find two or more associated data segments according to the key information of each data segment, and The data segment is processed accordingly. [0003] Taking the business process of data auditing as an example, the process mainly includes Customer Relationship Management (CRM, Customer Relationship Management), Convergent Billing System (CBS, Convergent Billing System) and auditing system. CRM and CBS are responsible for providing a large number of data files, The audit system is responsible for auditing these data files, that is, to find two or more associated data segments in all files according to the key information of each data segment, and analyze the same and diffe...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50G06F9/38G06F17/30
Inventor 吴万里
Owner HUAWEI TECH CO LTD