Data processing method and device

A data processing and data technology, applied in the field of data processing, can solve the problems of large delay in the data mart, nowhere to query the data situation, and difficult maintenance, etc., to achieve the effect of ensuring real-time performance

Active Publication Date: 2018-01-05
BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD +1
View PDF5 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, some requirements with high real-time requirements cannot be met, such as: order risk control requirements (if the behavior of swiping orders is quickly identified, logistics costs can be reduced to a certain extent);
[0009] 2. At present, the delay of the data mart is relatively large. The

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device
  • Data processing method and device
  • Data processing method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The following describes exemplary embodiments of the present invention with reference to the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and should be regarded as merely exemplary. Therefore, those of ordinary skill in the art should realize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Likewise, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] figure 1 It is a schematic diagram of the main steps of a data processing method according to an embodiment of the present invention. Such as figure 1 As shown, the data processing method of the present invention mainly includes the following steps S11 to S13.

[0041] Step S11: access processing, according to the set access frequency, periodically add online dat...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a data processing method and device. The data processing method comprises the following steps: access processing, synchronously adding an online data record to a corresponding data table under a first catalogue of a distributed file system of a data market periodically according to set access frequency; merge processing which is independent from the access processing, periodically reading the newly added data record in the data table saved under the first catalogue, and merging with the previous data saved in the corresponding data table under a second catalogue of the distributed file system, and recording the current time as the latest update time whenever the data table under the second catalogue is updated; query processing, merging the data table under the second catalogue with the data record newly added after the latest update time when receiving a data query request, and executing the processing of the data query request on the merged data.

Description

Technical field [0001] The present invention relates to the technical field of computers and computer software, in particular to a method and device for data processing. Background technique [0002] Data analysis refers to the process of analyzing a large amount of collected data with appropriate analysis methods, extracting useful information and forming conclusions, and then conducting detailed research and generalization of the data. In practical applications, data analysis can help people make judgments in order to take appropriate actions. [0003] Since the collected data is generally stored in a data warehouse, the data warehouse is the basis for business analysis and decision-making. A data warehouse is a collection of data used by data developers. After the collected data is stored in the data warehouse, data developers then use Hive for data association, analysis and mining of the data stored in the data warehouse. [0004] The data mart is a special data warehouse for ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/30
Inventor 袁建军王彦明刘业辉雷强
Owner BEIJING JINGDONG SHANGKE INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap