Data processing method and device
A data processing and data technology, applied in electrical digital data processing, special data processing applications, instruments, etc., can solve the problems of high cost of reading data by computer, inability to obtain calculation results, long data processing time, etc., to improve data processing. performance, overcoming the high overhead of reading data, and reducing the effect of computing overhead
Inactive Publication Date: 2015-05-27
ADVANCED NEW TECH CO LTD
View PDF9 Cites 3 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0006] In order to overcome the defects of high computer IO and low computing efficiency when processing a large amount of data, the main purpose of this application is to provide a data processing method and device to solve the problem of improving the computing performance and computing efficiency in the process of computer data processing The problem
[0007] The solution provided by this application can further overcome the defects of low computing efficiency, long data processing time, or inability to obtain calculation results caused by excessive computer computing and too concentrated values, and solve the problem of improving computing efficiency and shortening data processing time. ; Further, it can also overcome the large defect of computer reading data overhead due to the large amount of data, and solve the problem of saving computer reading data overhead
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0025] The main idea of this application is that in the process of data processing by the computer system for massive data, first count the number of repetitions of the same data in the database table to obtain the count value table, and then obtain the data distribution analysis based on the count value table ( probing) related results. Further, the total count value, average value, maximum value, minimum value, mode, and cumulative value of each data in the database table can be calculated according to the count value of each data in the count value table, and According to the total count value and the count value and cumulative value of each data, the quantile interval of each data is obtained. Furthermore, the data distribution of massive data can also be analyzed for subsequent data mining. For details, please refer to the following description of the data processing method of this application.
[0026]In the operation of this application, first count and obtain the c...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The application relates to a data processing method and a data processing device. The data processing method includes: scanning one piece or multiple pieces of data stored in a database table; counting the number of pieces of repeated data in the one piece or the multiple pieces of the data based on scanning, and confirming a count value of each piece of the repeated data; calculating a result related to analysis data distribution according to each piece of the repeated data and the count value of each piece of the repeated data. The data processing method and the data processing device which are achieved based on the repeated data avoid scanning on all the data, effectively reduce computer IO, significantly reduce the computation data size, improve computation efficiency, shorten data processing time, and reduce a computer running burden, and furthermore rapidly obtain a quantile and reduce computation cost by using a quantile range. Accordingly, the data processing method and the data processing device achieve the purposes of improving computation performance during the computer data processing process and improving the computation efficiency.
Description
technical field [0001] The present application relates to the field of computer processing, and more specifically relates to a data processing method and device. Background technique [0002] In today's big data era, data collection and processing technologies are developing rapidly, and the amount of data collected by computers is huge. The number of records in database tables used to store data exceeds 100 million, or even tens of billions. Wherein, each record item can be used to store one or more data. The data can be numeric. To analyze and mine the value of these data (values), usually the first step is to use computers to operate or process these data (values) to explore the distribution of data. [0003] In the traditional process of probing data distribution, it is necessary to scan the database table multiple times during data processing to calculate the calculation results related to the data contained in the database table, such as: mean, maximum, minimum, vari...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/24535G06F16/278
Inventor 吕春建
Owner ADVANCED NEW TECH CO LTD