Unlock instant, AI-driven research and patent intelligence for your innovation.

Data processing method and device and storage medium

A data processing device and data processing technology, applied in the field of data processing, can solve problems such as inability to directly analyze data, query, and reduce data query efficiency, and achieve the effects of saving operating procedures, comprehensive query and analysis, and good adaptability

Active Publication Date: 2021-04-30
CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD +1
View PDF2 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Since it is impossible to directly query various data sources, nor can it directly analyze the data, the data processing cycle is too long and the efficiency of data query is reduced.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method and device and storage medium
  • Data processing method and device and storage medium
  • Data processing method and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0076] The present application will be described in further detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, not to limit the present application.

[0077] In order to better understand the embodiments of the present application, the nouns and terms involved in the embodiments of the present application and the prior art related to the present application are briefly described below.

[0078] 1) Data Cube

[0079] Data Cube is a technical framework for data analysis and indexing. It is a powerful tool for processing big data, and can perform real-time indexing of metadata with any number of keywords. After the metadata is analyzed through the data cube, the efficiency of data query and retrieval can be greatly accelerated.

[0080] 2) Isomorphic data source

[0081] Data sources that have the same database type, support ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The embodiment of the invention discloses a data processing method, and the method comprises the steps: obtaining the data of at least one data source based on Spark, and forming an elastic distributed data set (RDD) corresponding to the data source; generating a first registration temporary table based on the RDD corresponding to the data source; constructing an association relationship between the at least two first registration temporary tables, and generating a first data model; wherein the first data model is used for data query. The invention further discloses a data processing device and a storage medium. Through the embodiment of the invention, the data processing period can be shortened, and the data query efficiency is improved.

Description

technical field [0001] The present application relates to data processing technology, and in particular to a data processing method, device and storage medium. Background technique [0002] In the prior art, when analyzing big data, the technicians first need to use professional tools to extract the required data from the data source, after data conversion and cleaning, etc., load the data into a data warehouse, and then establish an analysis The data model allows users to perform data analysis based on the model. Since it is impossible to directly query various data sources, nor can it directly analyze the data, the data processing cycle is too long and the efficiency of data query is reduced. Contents of the invention [0003] In order to solve existing technical problems, embodiments of the present application provide a data processing method, device, and storage medium, which can at least solve the above-mentioned problems existing in the prior art. [0004] An embod...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/22G06F16/2453G06F16/2455
CPCG06F16/2282G06F16/2453G06F16/24553
Inventor 杨文峰
Owner CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD