Metadata management method and system based on large model

Abstract

The invention belongs to the technical field of metadata management. According to the metadata management method and system based on the large model, by integrating the large model technology into the whole metadata management process, the functions of intelligent supplementation of metadata information, intelligent blood relationship analysis, intelligent query of metadata, intelligent number finding and the like are achieved, and the efficiency and accuracy of metadata management are greatly improved; according to the method, more convenient and intelligent data consumption experience is provided for the user in modes of visually displaying the blood relationship of the data, intelligently recommending related data objects and the like, manual intervention and the error rate are reduced, and the data management cost is reduced; and through intelligent metadata management, the data is easier to understand and utilize, the potential value of the data can be mined, and powerful support is provided for enterprise decision making.

Description

Technical Field

[0001] The present invention relates to the technical field of metadata management, and in particular to a metadata management method and system based on a large model. Background Art

[0002] The statements in this section merely provide background art related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of big data technology, the scale of enterprise data has exploded, and the data types have become increasingly diversified. Metadata, as the "data" of data, plays a vital role in data governance, data management, and data utilization. However, traditional metadata management methods have many shortcomings.

[0004] On the one hand, due to the irregularities in source system design and management, enterprises can only obtain the most basic technical metadata information when collecting metadata, such as database name, schema name, table name, and field name. Although this information is important, it is far from meeting the needs of enterprises for in-depth understanding and utilization of data. In order to improve other core metadata information, such as the Chinese name, business scope, description, and label of the table, and the Chinese name, description, value description, and sensitivity level of the field, enterprises need to invest a lot of manpower and time.

[0005] On the other hand, with the development of business and technology, data processing logic is becoming increasingly complex, database types are diverse, and processing scripts are numerous, which brings great challenges to data lineage analysis. Traditional lineage analysis methods are often unable to cope with these complex scenarios, resulting in low success rate and accuracy of lineage analysis.

[0006] At the same time, traditional metadata management and retrieval rely on structured organizational forms and keyword retrieval methods, which are more effective in scenarios with less data volume and lower business complexity, and can help users quickly locate the data resources they need. For example, in the early construction of data maps, traditional keyword retrieval played an important role. However, with the continuous development of big data technology and the increasing complexity of business needs, the scale of data has exploded and the data structure has become increasingly diversified. In this case, traditional keyword retrieval methods have begun to expose obvious limitations, causing users to be submerged in a large number of irrelevant or redundant documents and schemas, which require a lot of time and energy to screen and read.

[0007] In addition, when users have questions about the specific meaning, purpose or structure of data, they usually need to consult complex documents or rely on manual consultation with data producers or developers, which is not only inefficient but may also lead to misunderstandings due to poor communication. Summary of the invention

[0008] In order to address the deficiencies of the prior art, the present invention provides a metadata management method and system based on a large model, which automatically fills in other core metadata information based on the most basic technical metadata information and business sample data, thereby reducing manual input and time costs and improving the success rate and accuracy of data lineage analysis.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides a metadata management method based on a big model.

[0011] A metadata management method based on a large model includes the following processes:

[0012] Design meta-model and metadata knowledge graph schema, and generate metadata knowledge graph database;

[0013] A base large model is selected, and fine-tuning training and verification of the base large model is performed according to the instruction data set and the prompt word template to obtain a metadata intelligent filling large model, wherein the metadata intelligent filling large model is used to complete the metadata when the metadata changes;

[0014] Use the data lineage intelligent analysis model to analyze the data processing scripts in data development, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs;

[0015] Write the blood relationship obtained by the data blood relationship intelligent analysis model into the metadata knowledge graph database, update the metadata completed by the metadata intelligent filling model into the metadata knowledge graph database, write the metadata information of data standards, data indicators, data labels in the data governance platform and their association relationship data with tables and fields into the metadata knowledge graph database to obtain the metadata knowledge graph;

[0016] The metadata knowledge graph is vectorized through a semantic vector model to obtain a metadata vector library, and intelligent metadata retrieval and intelligent data search are performed based on the base model, the metadata knowledge graph and the metadata vector library.

[0017] In a second aspect, the present invention provides a metadata management system based on a large model.

[0018] A metadata management system based on a big model, comprising: a base big model, a metadata intelligent filling big model and a data lineage intelligent parsing big model;

[0019] The metadata intelligent filling model obtains metadata from multi-source heterogeneous data through the metadata collection engine, and the data lineage intelligent analysis model obtains data processing scripts from the data development platform; among them, the metadata intelligent filling model is generated after fine-tuning training and verification of the base model based on the instruction data set and prompt word template;

[0020] The base model, metadata intelligent filling model and data lineage intelligent analysis model are integrated into the existing metadata management process. When metadata changes, the metadata intelligent filling model is called to complete the metadata.

[0021] Use the data lineage intelligent analysis model to analyze the data processing scripts in data development, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs;

[0022] Write the blood relationship obtained by the data blood relationship intelligent analysis model into the metadata knowledge graph database, update the metadata completed by the metadata intelligent filling model into the metadata knowledge graph database, write the metadata information of data standards, data indicators, data labels in the data governance platform and their association relationship data with tables and fields into the metadata knowledge graph database to obtain the metadata knowledge graph;

[0023] The metadata knowledge graph is vectorized through a semantic vector model to obtain a metadata vector library, and intelligent metadata retrieval and intelligent data search are performed based on the base model, the metadata knowledge graph and the metadata vector library.

[0024] In a third aspect, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium;

[0025] a processor adapted to execute a computer program;

[0026] A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the metadata management method based on the large model as described in the first aspect of the present invention is implemented.

[0027] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by a processor and executing the large model-based metadata management method as described in the first aspect of the present invention.

[0028] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, and when the computer program is executed by a processor, it implements the metadata management method based on the big model as described in the first aspect of the present invention.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention integrates big model technology into the entire metadata management process to realize functions such as intelligent completion of metadata information, intelligent analysis of blood relationships, intelligent query of metadata, and intelligent number finding, which greatly improves the efficiency and accuracy of metadata management.

[0031] 2. The present invention provides users with a more convenient and intelligent data consumption experience by visually displaying data blood relationships and intelligently recommending related data objects, thereby reducing manual intervention and error rates and lowering data management costs.

[0032] 3. The present invention makes data easier to understand and use through intelligent metadata management, helps to explore the potential value of data, and provides strong support for corporate decision-making.

[0033] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0035] Figure 1 A schematic diagram of the principle of the metadata management method based on a large model provided in Example 1 of the present invention;

[0036] Figure 2 A schematic diagram of fine-tuning, training and verification of the metadata intelligent filling large model provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0038] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0039] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0040] Embodiment 1:

[0041] This implementation proposes a metadata management method based on a large model to achieve intelligent management and efficient use of metadata. The specific technical problems to be solved include: (1) How to automatically fill in other core metadata information based on the most basic technical metadata information and business sample data to reduce manual input and time costs; (2) How to improve the success rate and accuracy of data lineage analysis, especially in scenarios with complex codes, heterogeneous databases, and multi-type processing scripts; (3) How to use a large language model to build and optimize an intelligent data analysis capability to completely revolutionize the existing data query and understanding model.

[0042] The metadata management method of this implementation is as follows: Figure 1 As shown, it mainly includes: metamodel management, metadata collection, metadata management, metadata intelligent retrieval and intelligent data search. Its core lies in the intelligent completion of metadata business information based on the big model, intelligent analysis of blood relationship, metadata knowledge graph construction and intelligent semantic retrieval of metadata. More specifically, it includes the following contents:

[0043] Step 1: Metamodel design.

[0044] According to the needs of metadata management, the metamodel design and metadata knowledge graph schema design are carried out to generate the graph database of metadata knowledge graph. Among them, nodes include: tables, fields, tasks, APIs, data elements, code sets, data indicators, data labels, etc.; edges include: upstream and downstream relationships, positive and negative correlation relationships, association relationships, etc.; each node and edge contains corresponding attribute information, such as the node's unique identifier, name, description, etc., and the edge's unique identifier, relationship description, correlation coefficient, etc.

[0045] Step 2: Select the base model.

[0046] Based on the five dimensions of Chinese comprehension ability, domain knowledge comprehension ability, logical reasoning ability, information generation ability, and instruction attachment ability, experts score the output results of the big model and select the best big model as the base big model for use in subsequent processes. The base big model is then locally deployed for easy subsequent use.

[0047] Step 3: Fine-tune, train, and validate the large model with metadata intelligent filling.

[0048] Since enterprise data involves a large amount of domain knowledge of the company and has specific requirements for output formats, it is necessary to make some fine-tuning based on the selected base model to obtain the metadata intelligent filling model, and then deploy the metadata intelligent filling model and integrate it into the metadata management process, such as Figure 2 shown.

[0049] Step 3.1: Create a training instruction set and design a prompt word template.

[0050] Sort out the metadata information of the existing standard data sets, including data sets in national standards, industry standards, landmark standards, and data sets with metadata specifications in the data warehouse, and then design a prompt word template. The prompt word template needs to clarify the role. The example is as follows:

[0051] You are a data management expert and you understand xx business very well. Please answer based on the known information and questions. When answering, please follow the JSON format specification to return data and do not explain anything other than the question.

[0052] The format is:

[0053]

[0054]

[0055] Finally, we sorted out the instruction data set with standardized question-answer structure based on the prompt word template and the standard data set, including questions and answers. Examples of questions in the instruction data set are:

[0056] The table "PERFTRACE_INFO" is known. Its English fields include: TASKID, PROCESSID, SQL, INITVALUE, FINALVALUE, COLUMNTYPE, EXECTORSTATUS, DELNUM, TOTALCOSTS, STATETIMESTAMP, ENDTIMESTAMP, BYTESPEEDPERSEBYTESPEEDPERSECOND, RECORDSPEEDPERSECOND , TOTALREADRECORDTOTALREADRECORDS ,TOTALERRORRECORDTOTALERRORRECORDS;

[0057] Please explain the Chinese name and business meaning of the table "PERFTRACE_INFO", and give the Chinese name and business meaning of each field. The Chinese name of the table, the business meaning of the table, the Chinese name of the field and the business meaning of the field should be within 200 characters. Please answer according to the specification and return all the parsing in json format. No fields can be missing.

[0058] The answers in the instruction data set are organized according to the required JSON format specifications.

[0059] Step 3.2: Model fine-tuning, training and validation.

[0060] Based on the base big model, the LORA algorithm is used to fine-tune the big model training of the instruction data set, and a domain big model that automatically generates metadata information is constructed. Then some data sets are randomly selected as verification data sets to test the constructed domain big model, and business experts are invited to manually review the test results to verify whether the content quality generated by the domain big model meets the standards. If so, the successfully verified domain big model will be set as the metadata intelligent filling big model.

[0061] Step 3.3: Deployment and application of metadata intelligent filling large model.

[0062] The metadata intelligent filling model is integrated into the existing metadata management process. The metadata collection engine periodically collects business system metadata (that is, collects metadata from multi-source heterogeneous data). If the metadata changes, the metadata intelligent filling model is called to complete the metadata.

[0063] Step 4: Intelligent analysis of data lineage.

[0064] Use the data lineage intelligent analysis model to analyze the complex data processing scripts in the data development platform, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs. Then, integrate the data lineage intelligent analysis model into the existing metadata management process. It can be understood that the data development platform in this implementation method can preferably adopt the data development platform in CN116910078A, a data management platform with metadata as the core and an implementation method.

[0065] Step 5: Obtain global metadata information and build a metadata knowledge graph.

[0066] Write the blood relationship between tables and tables, tables and tasks, and tables and APIs parsed in step 4 into the metadata knowledge graph database created in step 1; update the metadata completed in step 3 into the metadata knowledge graph database created in step 1; write the metadata information of data standards, data indicators, and data labels in the data governance platform and the relationship data between them and tables and fields into the metadata knowledge graph database created in step 1. It can be understood that the data governance platform in CN116910078A, a data management platform with metadata as the core and its implementation method, can be preferably adopted.

[0067] Step 6: Build metadata vector library.

[0068] Through the semantic vector model (for example, Zhiyuan BGE Chinese and English semantic vector model), the metadata knowledge graph in step 5 is vectorized to facilitate the use in subsequent steps.

[0069] Step 7: Intelligent retrieval of metadata.

[0070] Combined with the data map function, intelligent retrieval of metadata helps users find the required data based on their needs. Specifically, it includes:

[0071] Step 7.1: Problem understanding.

[0072] The system uses a large base model to perform semantic analysis on user queries and understand the real intention behind them. Even if users express the same intention in different ways, the system can accurately capture it.

[0073] Step 7.2: Generate semantic vectors.

[0074] Vectorize the intent understood by the base large model in step 7.1 to obtain a semantic vector, perform semantic search based on the semantic vector, and then judge the user's intent based on the semantic search results, so as to provide search results that better meet the needs.

[0075] Step 7.3: Ranking of recommended answers.

[0076] Combine the keyword search with the results of the semantic search in step 7.2, sort the search results according to relevance, and recommend the answer that best meets the user's needs to the top.

[0077] Step 7.4: Model deployment and application.

[0078] Integrate the metadata intelligent retrieval model into the existing metadata management process. When retrieving metadata in the data map function, you can accurately locate and quickly find data in specific scenarios.

[0079] Step 8: Intelligent number finding.

[0080] The smart data search function helps users better apply data, especially how to conduct detailed business analysis and definition interpretation of selected data. The scenario of smart data search is for queries that go beyond the scope of simple keyword search. The system uses the ability of intelligent data analysis to identify and interpret complex data structures and business logic, and supports in-depth analysis and interpretation of tables, fields, data standards, data indicators, and data tags to meet users' needs for data understanding. Specifically, it includes the following processes:

[0081] Step 8.1: Problem understanding.

[0082] The system uses a large base model to perform semantic analysis on users' questions and understand the real needs behind them. Even if users express the same intention in different ways, the system can accurately capture it.

[0083] Step 8.2: Generate semantic vectors.

[0084] Vectorize the intent understood by the base large model in step 8.1 to generate a semantic vector, perform semantic search based on the semantic vector, and then judge the user's intent based on the semantic search results, so as to provide search results that better meet the needs.

[0085] Step 8.3: Ranking of recommended answers.

[0086] The system reorders the results of the semantic search in step 8.2 and recommends the answer that best meets the user's needs to the top.

[0087] Step 8.4: Summarize and refine.

[0088] Use the base big model to summarize the answers to the query, extract key information concisely and clearly, and help users find what they need faster.

[0089] Step 8.5: Model deployment and application.

[0090] Integrate the base big model (with intelligent number-finding function) into the existing metadata management process to achieve understanding of complex data structures and business logic. Through the intelligent number-finding function, users' needs for data understanding can be met through conversational answers.

[0091] Step 9: Continuous optimization and iterative upgrade of the model.

[0092] Track the latest research trends, try new technologies, and regularly retrain or fine-tune the base big model, metadata intelligent filling big model, and data lineage intelligent parsing big model.

[0093] Embodiment 2:

[0094] This implementation method provides a metadata management system based on a big model, including: a base big model, a metadata intelligent filling big model and a data lineage intelligent analysis big model. Each big model can be deployed in a terminal device or in a distributed manner. Specifically, the above-mentioned big models can be embedded in the metadata management process, and the specific method in Example 1 is used to implement metadata management, which will not be repeated here.

[0095] Embodiment 3:

[0096] This implementation provides an electronic device, which includes a processor, a communication interface, and a computer-readable storage medium, wherein the processor, the communication interface, and the computer-readable storage medium may be connected via a bus or other means.

[0097] Among them, the communication interface is used to receive and send data, the computer-readable storage medium can be stored in the memory of the electronic device, the computer-readable storage medium is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer-readable storage medium.

[0098] A processor (or CPU (Central Processing Unit)) is the computing core and control core of an electronic device, which is suitable for implementing one or more instructions, and specifically suitable for loading and executing one or more instructions to implement corresponding method flows or corresponding functions.

[0099] The processor is configured to perform the following process:

[0100] Design meta-model and metadata knowledge graph schema, and generate metadata knowledge graph database;

[0101] A base large model is selected, and fine-tuning training and verification of the base large model is performed according to the instruction data set and the prompt word template to obtain a metadata intelligent filling large model, wherein the metadata intelligent filling large model is used to complete the metadata when the metadata changes;

[0102] Use the data lineage intelligent analysis model to analyze the data processing scripts in data development, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs;

[0103] Write the blood relationship obtained by the data blood relationship intelligent analysis model into the metadata knowledge graph database, update the metadata completed by the metadata intelligent filling model into the metadata knowledge graph database, write the metadata information of data standards, data indicators, data labels in the data governance platform and their association relationship data with tables and fields into the metadata knowledge graph database to obtain the metadata knowledge graph;

[0104] The metadata knowledge graph is vectorized through a semantic vector model to obtain a metadata vector library, and intelligent metadata retrieval and intelligent data search are performed based on the base model, the metadata knowledge graph and the metadata vector library.

[0105] The detailed process is shown in the specific method in Example 1 and will not be repeated here.

[0106] Embodiment 4:

[0107] This implementation provides a computer-readable storage medium (Memory), which is a memory device in an electronic device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the electronic device and, of course, extended storage media supported by the electronic device. The computer-readable storage medium provides a storage space that stores the processing system of the electronic device.

[0108] In addition, the storage space also stores one or more instructions suitable for being loaded and executed by the processor, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here may be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory; optionally, it may also be at least one computer-readable storage medium located away from the aforementioned processor.

[0109] In one embodiment, the computer-readable storage medium stores one or more instructions; the processor loads and executes the one or more instructions stored in the computer-readable storage medium to implement the following process:

[0110] Design meta-model and metadata knowledge graph schema, and generate metadata knowledge graph database;

[0111] A base large model is selected, and fine-tuning training and verification of the base large model is performed according to the instruction data set and the prompt word template to obtain a metadata intelligent filling large model, wherein the metadata intelligent filling large model is used to complete the metadata when the metadata changes;

[0112] Use the data lineage intelligent analysis model to analyze the data processing scripts in data development, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs;

[0113] Write the blood relationship obtained by the data blood relationship intelligent analysis model into the metadata knowledge graph database, update the metadata completed by the metadata intelligent filling model into the metadata knowledge graph database, write the metadata information of data standards, data indicators, data labels in the data governance platform and their association relationship data with tables and fields into the metadata knowledge graph database to obtain the metadata knowledge graph;

[0114] The metadata knowledge graph is vectorized through a semantic vector model to obtain a metadata vector library, and intelligent metadata retrieval and intelligent data search are performed based on the base model, the metadata knowledge graph and the metadata vector library.

[0115] The detailed process is shown in the specific method in Example 1 and will not be repeated here.

[0116] Embodiment 5:

[0117] The present implementation provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the following process:

[0118] Design meta-model and metadata knowledge graph schema, and generate metadata knowledge graph database;

[0119] A base large model is selected, and fine-tuning training and verification of the base large model is performed according to the instruction data set and the prompt word template to obtain a metadata intelligent filling large model, wherein the metadata intelligent filling large model is used to complete the metadata when the metadata changes;

[0120] Use the data lineage intelligent analysis model to analyze the data processing scripts in data development, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs;

[0121] Write the blood relationship obtained by the data blood relationship intelligent analysis model into the metadata knowledge graph database, update the metadata completed by the metadata intelligent filling model into the metadata knowledge graph database, write the metadata information of data standards, data indicators, data labels in the data governance platform and their association relationship data with tables and fields into the metadata knowledge graph database to obtain the metadata knowledge graph;

[0122] The metadata knowledge graph is vectorized through a semantic vector model to obtain a metadata vector library, and intelligent metadata retrieval and intelligent data search are performed based on the base model, the metadata knowledge graph and the metadata vector library.

[0123] The detailed process is shown in the specific method in Example 1 and will not be repeated here.

[0124] A person skilled in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0125] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instruction can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instruction can be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (for example, coaxial cable, optical fiber, digital line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server, a data center, etc. that contains one or more available media integration. Available media can be magnetic media (for example, floppy disk, hard disk, tape), optical media (for example, DVD), or semiconductor media (for example, solid-state drive (Solid State Disk, SSD)) and the like.

[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A metadata management method based on a large model, characterized in that: The process includes: Design meta-model and metadata knowledge graph schema, and generate metadata knowledge graph database; Select a base large model, perform fine-tuning training and verification on the base large model according to the instruction data set and the prompt word template, and obtain a metadata intelligent filling large model, wherein the metadata intelligent filling large model is used to complete the metadata when the metadata changes; Use the data lineage intelligent analysis model to analyze the data processing scripts in data development, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs; Write the blood relationship obtained by the data blood relationship intelligent analysis model into the metadata knowledge graph database, update the metadata completed by the metadata intelligent filling model into the metadata knowledge graph database, write the metadata information of data standards, data indicators, data labels in the data governance platform and their association relationship data with tables and fields into the metadata knowledge graph database to obtain the metadata knowledge graph; The metadata knowledge graph is vectorized through a semantic vector model to obtain a metadata vector library, and intelligent metadata retrieval and intelligent data search are performed based on the base model, the metadata knowledge graph and the metadata vector library.

2. The metadata management method based on a large model as claimed in claim 1, characterized in that: Design the metamodel and metadata knowledge graph schema, where nodes include: tables, fields, tasks, APIs, data elements, code sets, data indicators and data labels; edges include: upstream and downstream relationships, positive and negative correlation relationships, and association relationships; each node and edge contains corresponding attribute information.

3. The metadata management method based on a large model as claimed in claim 1, characterized in that: Based on the base big model, the LORA algorithm is used to fine-tune the base big model training on the instruction data set, and a domain big model that automatically generates metadata information is constructed. A data set is randomly selected as a verification data set to test the constructed domain big model. Combined with the manual verification results, a metadata intelligent filling big model with content quality that meets the standards is obtained. The metadata intelligent filling model is integrated into the metadata management process. The metadata collection engine periodically collects business system metadata. If the metadata changes, the metadata intelligent filling model is called to complete the metadata.

4. The metadata management method based on a large model as claimed in claim 1, characterized in that: Perform intelligent retrieval of metadata, including: Use the base big model to perform semantic analysis on user queries to understand the real intention behind them; Vectorizing the true intent to generate a semantic vector, and performing semantic retrieval based on the semantic vector; Combining the results of keyword search and semantic search, the search results are sorted according to relevance, and the answers that best match the real intention are recommended to the top.

5. The metadata management method based on a large model as claimed in claim 1, characterized in that: Perform intelligent number search, including: Use the base big model to perform semantic analysis on users' questions and understand the real needs behind them; Vectorize the real needs understood through the base big model to generate semantic vectors, and perform semantic retrieval based on the semantic vectors; Re-rank the results of semantic retrieval and recommend the answer that best meets the user's needs to the top; Use the base model to summarize the answers that best meet user needs and extract key information to help users find what they need.

6. The metadata management method based on a large model as claimed in claim 1, characterized in that: Based on the five dimensions of Chinese comprehension ability, domain knowledge comprehension ability, logical reasoning ability, information generation ability and instruction attachment ability, experts score the output results of different large models and select the best large model as the base large model.

7. A metadata management system based on a large model, characterized in that: include: The base model, metadata intelligent filling model and data lineage intelligent analysis model; The metadata intelligent filling model obtains metadata from multi-source heterogeneous data through the metadata collection engine, and the data lineage intelligent analysis model obtains data processing scripts from the data development platform; among them, the metadata intelligent filling model is generated after fine-tuning training and verification of the base model based on the instruction data set and prompt word template; The base model, metadata intelligent filling model and data lineage intelligent analysis model are integrated into the existing metadata management process. When metadata changes, the metadata intelligent filling model is called to complete the metadata. Use the data lineage intelligent analysis model to analyze the data processing scripts in data development, and analyze the lineage relationships between tables, tables and tasks, and tables and APIs; Write the blood relationship obtained by the data blood relationship intelligent analysis model into the metadata knowledge graph database, update the metadata completed by the metadata intelligent filling model into the metadata knowledge graph database, write the metadata information of data standards, data indicators, data labels in the data governance platform and their association relationship data with tables and fields into the metadata knowledge graph database to obtain the metadata knowledge graph; The metadata knowledge graph is vectorized through a semantic vector model to obtain a metadata vector library, and intelligent metadata retrieval and intelligent data search are performed based on the base model, the metadata knowledge graph and the metadata vector library.

8. A computer device, characterized in that: include: A processor and a computer readable storage medium; a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the large model-based metadata management method as described in any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the large model-based metadata management method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the big model-based metadata management method is implemented.

Images