SQL data real-time processing method and device based on Flink, computer equipment and medium

A technology of real-time data processing and computer programs, applied in the field of data processing, can solve problems such as real-time processing of complex SQL data, and achieve complex results

Pending Publication Date: 2020-09-15
CHINA PING AN PROPERTY INSURANCE CO LTD
View PDF0 Cites 7 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] Based on this, it is necessary to address the above technical problems. This application provides a Flink-based real-time SQL data processing method, device, computer equipment and storage media to solve the complex technical problems of real-time processing of SQL data in the prior art.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • SQL data real-time processing method and device based on Flink, computer equipment and medium
  • SQL data real-time processing method and device based on Flink, computer equipment and medium
  • SQL data real-time processing method and device based on Flink, computer equipment and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of the application; the terms used herein in the description of the application are only to describe specific embodiments The purpose is not to limit the present application; the terms "comprising" and "having" and any variations thereof in the specification and claims of the present application and the description of the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

[0027] Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrenc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the field of data processing, and discloses an SQL data real-time processing method and device based on Flink, computer equipment and a readable storage medium. The method comprises the steps of storing data input by a user in tidb through a client, then classifying SQL statements in an sql data table in an flink cluster to obtain classified SQL statements, and correspondingly packaging the classified SQL statements to generate a job operator; and the job operators are connected in series according to the logic of the SQL statement to generate an operator flow, and theoperator flow is submitted to a taskmanger for execution, so that real-time processing of SQL data is realized. By adopting the method to encapsulate the SQL statement, a user can replace writing ofa large number of Java codes with a small number of SQL codes, and the technical problem that in the prior art, implementation of SQL data processing through flink is complex is solved. The inventionfurther relates to a blockchain technology, and the data input by the user can be stored in the blockchain node.

Description

technical field [0001] The present application relates to the field of data processing, in particular to a method, device, computer equipment and storage medium for real-time processing of SQL data based on Flink. Background technique [0002] Flink SQL is a development language that conforms to standard SQL semantics and is designed for Flink real-time computing to simplify the computing model and lower the threshold for users to use real-time computing. The cost of open source Flink programming development is high, the learning cost is high, the debugging is complicated, and the online cycle is long. Different development methods are used for each data source connection, which consumes a lot of learning costs and development time. When Flink is applied to the real-time processing of SQL data, in order to realize functions such as source, sink, and dimension tables, users need to inherit various functions (functions) of Flink for development, and dimension tables also need...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/242G06F16/22G06F16/28
CPCG06F16/2282G06F16/2433G06F16/285
Inventor 邓煜
Owner CHINA PING AN PROPERTY INSURANCE CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap