Data processing method, device and system

A data processing and data technology, applied in the transmission system, electrical components, etc., can solve the problems of being unable to undertake massive data processing, data warehouses undertake heavy tasks, and scalability bottlenecks, so as to maintain relative independence and increase data processing capacity , low cost effect

Active Publication Date: 2016-08-03
CHINA MOBILE COMM GRP CO LTD
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this way, the data warehouse will be burdened with heavy tasks and high performance pressure; the architecture determines that its scalability has a bottleneck, and may not be able to handle massive data processing in the future; the cost of expansion is high, and the rate of obsolescence is low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device and system
  • Data processing method, device and system
  • Data processing method, device and system

Examples

Experimental program
Comparison scheme
Effect test

no. 1 example

[0036] Such as figure 1 As shown, the first embodiment of the data processing method of the present invention includes the following steps:

[0037] Step 102: The interface machine receives data from multiple data sources;

[0038] Step 104, the cloud computing platform extracts data from the interface machine, or the interface machine sends the data to the cloud computing platform;

[0039] Step 106: The cloud computing platform processes the acquired data, and sends the processed data to the data processing server;

[0040] Step 108: The data processing server transmits the processed data to the data warehouse.

[0041] In this embodiment, the cloud computing platform can perform all data processing functions of the data processing server, including: data inspection, cleaning, conversion, aggregation, association, and so on.

[0042] In the first method embodiment, by acquiring data from the cloud computing platform and processing the data, the data processing server transmits the proc...

Embodiment 1

[0072] Such as Figure 4 As shown, the 34 embodiment of the cloud computing platform of the present invention includes: a receiving module 341 for data obtained from an interface machine; a processing module 342 for processing the extracted data and sending the processed data to the data processing server .

[0073] The cloud computing platform of this embodiment acquires and processes data, and the data processing server transmits the processed data, increases the amount of data processing, realizes massive parallel data processing, and does not make too many changes to the existing architecture to maintain the cloud The relative independence of the platform makes it easy to transform. The cloud computing platform has low cost, low investment, and high rate of interest, which can ease the pressure of system expansion and reduce system costs.

[0074] Among them, the receiving module 341 is used to obtain data with a data amount greater than a preset threshold from the interface m...

Embodiment 2

[0079] Such as Figure 5 As shown, an embodiment of the data processing server 36 of the present invention includes:

[0080] The transmission module 363 is used to receive the data processed by the cloud computing platform 34 and transmit it to the data warehouse.

[0081] The data processing server of this embodiment is only used to transmit processed data. In this way, the amount of data processing is increased, massive parallel data processing is realized, and the existing architecture will not be changed too much to maintain the relative balance of the cloud platform. Independence, easy to transform. The cloud computing platform has low cost, low investment, and high rate of interest, which can ease the pressure of system expansion and reduce system costs.

[0082] Preferably, the data processing server 36 in this embodiment further includes:

[0083] The receiving module 361 is configured to obtain data whose data amount is less than a preset threshold value from the interface ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method, device and system. The data processing method includes: the cloud computing platform processes the acquired data, and sends the processed data to a data processing server; the data processing server transmits the data processed by the cloud computing platform to a data warehouse. The data processing method, device and system of the present invention extract and process data through the cloud computing platform, and the data processing server transmits the processed data to the data warehouse, thereby increasing the amount of data processing and realizing massive data processing, and will not Make too many changes to the existing architecture, maintain the relative independence of the cloud platform, and be easy to transform. The cloud computing platform has low cost, small investment, and high interest rate, which can relieve the pressure of system expansion and reduce system cost.

Description

Technical field [0001] The invention relates to a business support technology, in particular to a data processing method, device and system. Background technique [0002] Data processing system architecture: including interface machine, ETL server and data warehouse. The interface machine collects data from multiple data sources, the ETL server extracts the data from the interface machine and performs conversion processing, and then loads the data into the data warehouse. [0003] ETL is the abbreviation of Extraction-Transformation-Loading, and the Chinese name is data extraction, transformation and loading. The ETL server is responsible for extracting data from distributed and heterogeneous data sources, such as relational data, flat data files, etc., to the temporary intermediate layer for cleaning, conversion, and integration, and finally loading into the data warehouse or data mart to become online analysis processing , The basis of data mining. [0004] Conversion (also call...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08
Inventor 高丹徐萌邓超江志雄罗治国孙少陵陶涛段云峰何鸿凌谢帆赵静张韬吴明励崖卉王之恒邓青周福伟
Owner CHINA MOBILE COMM GRP CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap