Data processing method, device and system

A data processing and source data technology, applied in the field of computer communication, can solve problems such as high labor costs and lack of efficiency, and achieve the effect of improving efficiency

Active Publication Date: 2015-03-25
TENCENT TECH (SHENZHEN) CO LTD +1
View PDF3 Cites 25 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the existing data processing technology only abstracts the statistical framework, which means that the statistical logic it provides is only suitable for the processing of homogeneous data, but not for heterogeneous data from different data sources. To enter a type of data, or to change the access data, it is necessary to change the code of the access and processing analysis module, and provide the corresponding data access and data processing logic through manual coding, which has high labor costs and lacks efficiency.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device and system
  • Data processing method, device and system
  • Data processing method, device and system

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0024] see figure 1 , shows a flow chart of the data processing method provided by the first embodiment of the present invention. Such as figure 1 As shown, the data processing method provided in this embodiment includes the following steps:

[0025] Step 11, the server obtains the source data to be counted, and pre-allocates a processing process for the source data to be counted according to the preset processing process allocation rules and all the processes started in the server.

[0026] Specifically, the server obtains the source data to be counted from the local storage or other network storage servers, and calculates all the statistical analysis processes started in the server according to the resource identification (ID) of each process, and then allocates them according to the preset processing processes All the processes that have been started in the server are calculated by the rules and pre-allocated processing processes for the source data to be counted. Wherei...

no. 2 example

[0036] see figure 2 , shows a flow chart of the data processing method provided by the second embodiment of the present invention. Such as figure 2 As shown, the data processing method provided in this embodiment includes the following steps:

[0037] In step 21, the server queries and acquires global configuration rule information for statistical analysis of source data.

[0038] Specifically, the server queries and acquires global configuration rule information used for statistical analysis of the source data from the configuration server. The global configuration rule information may specifically include preset processing process assignment rules, preset source data processing rules, preset data stream processing rules, and preset output control rules.

[0039] Step 22, the server obtains the source data to be counted from the source data directory in the preset source data processing rules.

[0040] The preset source data processing rules include the source data dire...

no. 3 example

[0069] see Image 6 , shows a flow chart of the data processing method provided by the third embodiment of the present invention. Such as Image 6 As shown, the data processing method provided in this embodiment includes the following steps:

[0070] Step 61, the configuration server acquires global configuration rule information, and sends the global configuration rule information to the statistics server.

[0071] The global configuration rule information includes preset processing process allocation rules, preset source data processing rules, and preset data flow processing rules.

[0072] Specifically, the configuration server obtains the global configuration rule information input by the user for statistical analysis of the source data, and provides the statistics server with a query server for the global configuration rule information at any time. When the configuration server receives a query request from the statistics server, according to the query Request to send ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data processing method. The method includes the steps that a server acquires source data to be subjected to statistics, and a processing course is pre-allocated for the source data to be subjected to statistics according to a preset processing course allocation rule and all courses having been started in the server; the obtained source data are converted into data elements in a uniform format according to a preset source data processing rule; the converted data elements are distributed to the corresponding processing courses, and the data elements are combined and subjected to statistics by the processing courses according to a preset dataflow processing rule; the combined data elements having been subjected to statistics are output. The invention further provides a data processing device and system. Through the data processing method, device and system, the data processing efficiency can be greatly improved.

Description

technical field [0001] The present invention relates to the technical field of computer communication, in particular to a data processing method, device and system. Background technique [0002] The usual method for statistical analysis of data is to receive the data of the external module, process it in the storage medium (for example, memory, disk, database, network storage, etc.), and then output the result data. However, the existing data processing technology only abstracts the statistical framework, which means that the statistical logic it provides is only suitable for the processing of homogeneous data, but not for heterogeneous data from different data sources. Entering a class of data, or changing the access data, requires code changes to the access and processing analysis modules, and provides the corresponding data access and data processing logic through manual coding, which has high labor costs and lacks efficiency. Contents of the invention [0003] In view...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30G06F9/46
CPCG06F16/907
Inventor 何涛杨帅罗伟
Owner TENCENT TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap