Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data processing method and system thereof

A data processing and data technology, applied in the field of data processing methods and systems, can solve problems such as questioning data integrity and increasing workload, and achieve the effects of flexible transfer, reducing data volume, and saving cluster resources

Active Publication Date: 2019-03-01
BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Those who do not understand the cold data pulling rules will question the integrity of the data, causing users to find the reason for the abnormal data. After the reason is found, they need to submit an application to pull back the cold data before it can be used normally, which greatly increases workload

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and system thereof
  • Data processing method and system thereof
  • Data processing method and system thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and fully convey the concept of example embodiments to those skilled in the art. The drawings are merely schematic illustrations of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted.

[0032] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to give a thorough understanding of embodiments of the invention. However, those skilled in the art will appreciate...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a data processing method and a system of the same and belongs to the technical field of data processing. The method comprises the steps that cold data in a data warehouse is transferred to and stored in a cold data cluster; a query statement received by the data warehouse is monitored, a table name in the query statement and a data partitioning scope of a table corresponding to the table name are compared with the same data partitioning scope of a corresponding table in the data warehouse; according to a comparison result, whether all the data inquired by the query statement is stored in the data warehouse is judged; and when the data inquired by the query statement is not completely located in the data warehouse, the cold data which is stored in the corresponding table of the cold data cluster and is not located in the data partitioning scope of the data warehouse is automatically pulled back to the data warehouse. According to the invention, flexibility of cold data storage can be enhanced; and data transfer between the cold data cluster and the data warehouse can be controlled automatically.

Description

technical field [0001] The present invention relates to data processing technology, in particular to a data processing method and system thereof. Background technique [0002] With the rapid development of the Internet and e-commerce, the diversification of business forms, and the increase in people's dependence on the Internet, the amount of data carried by enterprise data warehouses has shown explosive growth. Here comes a very big challenge. [0003] Some long-term data are not frequently called, but may be used occasionally, so these data cannot be deleted. As time goes by, such data will increase. Not only is it a waste of space to store, but if it is not used properly , will greatly reduce the query efficiency. How to effectively manage massive data has become a topic that cluster managers pay more and more attention to. [0004] Most of the fact tables in the data warehouse are partitioned and stored according to certain business hours. The same business time is s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22G06F16/245
CPCG06F16/22G06F16/245
Inventor 侍建超
Owner BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com