Unlock instant, AI-driven research and patent intelligence for your innovation.

Big data processing method, device, equipment and medium

A big data processing and data technology, applied in the field of big data, can solve the problems of scattered technology stack, high cost and complexity, and achieve the effect of improving efficiency and performance

Active Publication Date: 2021-04-02
PING AN TECH (SHENZHEN) CO LTD
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] Big data computing usually involves a large amount of data, and the scene is complex
Therefore, the data processing system of the entire big data platform is a complex system that needs to be combined by different products and technologies to cover various scenarios, which will bring considerable challenges to data engineers or architects
[0003] Data engineers need to master the programming syntax and debugging skills of corresponding tools. Data architects need to have an in-depth understanding of various technical principles involved, and summarize the best solutions for use. Various technologies and products also need corresponding familiarity. Personnel, not only the cost is high, but the technology stack is too scattered

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data processing method, device, equipment and medium
  • Big data processing method, device, equipment and medium
  • Big data processing method, device, equipment and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059]In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0060] Such as figure 1 Shown is a flowchart of a preferred embodiment of the big data processing method of the present invention. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0061] The big data processing method is applied to one or more electronic devices, and the electronic device is a device that can automatically perform numerical calculation and / or information processing according to preset or stored instructions, and its hardware includes but not Limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention relates to the field of big data, and provides a big data processing method, device, equipment and medium, which can define the SQL grammatical structure, analyze the data to be processed according to the SQL grammatical structure, generate an abstract syntax tree, realize the unification of SQL grammatical, and shield Various grammatical differences, transform the abstract syntax tree to obtain the first execution plan, optimize the abstract syntax tree according to the execution environment, and obtain the second execution plan, not only further optimize the statement, but also optimize again according to the execution environment , start the execution engine corresponding to the target interface to execute the first execution plan and the second execution plan to automatically identify the target interface, and then call the corresponding big data engine to process data through the target interface, so as to realize the unified automatic processing of big data, effectively The efficiency of big data processing is improved, and the performance of big data systems is improved. The present invention also relates to block chain technology, and the abstract syntax tree can be stored in the block chain.

Description

technical field [0001] The present invention relates to the field of big data technology, in particular to a big data processing method, device, equipment and medium. Background technique [0002] Big data computing usually involves a large amount of data and the scenarios are complex. Therefore, the data processing system of the entire big data platform is a complex system that needs to be combined with different products and technologies to cover various scenarios, which will bring considerable challenges to data engineers or architects. [0003] Data engineers need to master the programming syntax and debugging skills of corresponding tools. Data architects need to have an in-depth understanding of various technical principles involved, and summarize the best solutions for use. Various technologies and products also need corresponding familiarity. Personnel, not only the cost is high, but also the technology stack is too scattered. Contents of the invention [0004] I...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/242G06F40/205G06F40/253G06F40/284G06F21/64
CPCG06F21/64G06F16/2433G06F40/205G06F40/253G06F40/284
Inventor 蔡满天张茜凌海挺杜均
Owner PING AN TECH (SHENZHEN) CO LTD