Method for realizing Update function in Trino Iceberg connection

A technology of function and function logic, applied in the field of non-temporary computer-readable storage media, can solve the problems of not supporting UPDATE operation, unable to realize batch update of data, etc., to achieve the effect of eliminating inconsistent caliber and reducing development cost

Pending Publication Date: 2022-06-03
杭州玳数科技有限公司
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

As one of the data lake solutions, Iceberg supports the Update syntax of the Iceberg Connector, which is an important part of Trino's connection to the data lake solution. However, in related technologies, Trino does not support the UPDATE operation on the Iceberg table, which makes it impossible for Trino to connect to the Iceberg data. Realize the batch update function of data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for realizing Update function in Trino Iceberg connection

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0020] figure 1 is a flowchart of a method for implementing the Update function in a Trino Iceberg connection according to an embodiment of the present invention, such as figure 1 As shown, the method includes the following steps:

[0021] S1, receive and parse the Update Sql task submitted by the user through the Trino Engine Sql execution engine to obtain an executable plan.

[0022] Further, receive and parse the Update Sql task ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method for realizing an Update function in Trino Iceberg connection, which comprises the following steps of: receiving and analyzing an Update Sql task submitted by a user through a Trino Engine Sql execution engine so as to obtain an executable plan; the method comprises the following steps of: supporting Update function logic of Iceberg metadata and bottom layer data through an Update PageSource software programming interface provided by Trino; and according to the Update function logic, carrying out data UPDATE operation on the bottom layer data of the Iceberg through a RowDelta application programming interface provided by the Iceberg. According to the method, the Update function can be supported when the Iceberg data lake is docked through Trino, so that a user can conveniently and efficiently complete more batch processing scenes.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a method for implementing the Update function in a Trino Iceberg connection, a computer device and a non-transitory computer-readable storage medium. Background technique [0002] Iceberg is an open tabular format for large analytical datasets that uses a high-performance table format similar to SQL (Structured Query Language) tables that can be added to Spark, a A fast and general computing engine designed for large-scale data processing), Trino (a fast distributed SQL query engine for big data analysis), Flink (an open source stream processing framework) and Hive (a data warehouse tool) and other computing engines, and supports operations such as adding, deleting, checking, and modifying the table format. [0003] At present, the only computing engines that can support UPDATE operations on Iceberg tables are Spark and Flink, which provide more computing scenari...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/242
CPCG06F16/2365G06F16/2433Y02D10/00
Inventor 兰洋
Owner 杭州玳数科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap