Data platform metadata automatic generation method based on large model

By standardizing and statically parsing SQL code, and combining it with deep semantic understanding of large models, the problem of low metadata generation efficiency in complex data platforms by traditional methods is solved, and efficient and accurate data lineage generation is achieved.

CN120873206AInactive Publication Date: 2025-10-31ZHEJIANG NON-LINEAR DIGITAL TECH CO LTD
0 Cites 9 Cited by

Patent Information

Application Number
CN202511018594.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-23
Publication Date
2025-10-31
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional methods of metadata management and data lineage generation are inefficient, time-consuming, and difficult to guarantee the real-time performance and accuracy of data when dealing with large-scale and complex enterprise-level data platforms, especially when dealing with complex SQL code, where it is difficult to accurately parse the data lineage.

Method used

We adopt an automatic metadata generation method for data platforms based on large models. By standardizing and statically parsing the original SQL code, we identify the code type, perform preliminary lineage analysis, use a large language model for deep semantic understanding, integrate simple and complex lineage relationships, and generate complete data platform metadata.

Benefits of technology

It significantly improves the accuracy and completeness of metadata generation, makes up for the shortcomings of traditional parsing tools in understanding complex semantics, and improves the efficiency and accuracy of metadata generation.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of metadata generation, and discloses a data platform metadata automatic generation method based on a large model, which comprises the following steps of: firstly, performing unified standardization and preliminary grammatical analysis on an original SQL (Structured Query Language) code to obtain a structured intermediate representation; on the basis, preliminary blood relationship analysis based on rules is carried out, and simple column references are quickly identified and processed. For complex expressions which are difficult to accurately analyze by a traditional method, code snippets and context information of the complex expressions are accurately extracted and submitted to a large language model for deep semantic understanding and complex blood relationship analysis. And finally, integrating the complex consanguinity analyzed by the large model with the initial consanguinity list to form a comprehensive and accurate field-level consanguinity, and further generating complete data platform metadata. In this way, the defect that a traditional analysis tool understands complex semantics is effectively overcome, and the accuracy and integrity of metadata generation are remarkably improved.
Need to check novelty before this filing date? Find Prior Art