Large language model driven ODS layer full-process data resource evaluation modeling system, method and device, storage medium and program product

Abstract

The invention relates to the technical field of database information management, discloses a big language model driven ODS layer full-process data resource evaluation modeling system, method and device, a storage medium and a program product, and is used for solving the technical problems of low modeling efficiency and poor effect in the prior art. Comprising a data input module used for capturing a table structure and table metadata of a newly-added business table corresponding to a target iteration version in real time; the modeling evaluation module is used for calling a large language model to fuse historical evaluation records and resource classification rules, evaluating data resources contained in the newly-added service table and outputting a resource classification result and a modeling evaluation conclusion; the data modeling module is used for calling a large language model to generate field description for each field contained in the newly added business table when modeling needs to be executed, and analyzing business source codes related to enumeration fields to generate field description of the enumeration fields when the fields are enumeration fields; and based on the table structure, the table metadata and the field description, generating a data resource model on the ODS layer.

Description

Technical Field

[0001] This invention relates to the field of database information management technology, and in particular to a large language model-driven ODS layer full-process data resource evaluation and modeling system, method, electronic device, computer storage medium, and computer program product. Background Technology

[0002] With the acceleration of enterprise digital transformation in recent years, the amount of data has grown exponentially, and this data may contain a wealth of useful information. Therefore, to organize and analyze this massive amount of data, enterprises typically rely on basic scripts and manual operations for data management. Current technologies for digital resource inventory and modeling primarily involve manually or using scripts with fixed rules to inventory and classify new business tables, completing the preliminary step of data resource assessment and modeling. The assessment results then determine whether modeling is necessary. This approach is not only time-consuming but also susceptible to human factors, lacking intelligent assessment (e.g., ignoring the relationships between tables), leading to a high rate of misjudgment in modeling decisions (e.g., misjudging core transaction tables as "not requiring modeling"). It also has poor scalability, as the rule-based scripts cannot adapt to business growth, requiring continuous and significant manpower for maintenance, resulting in a low degree of automation.

[0003] Furthermore, similar challenges arise when maintaining table and field descriptions in data warehouses, such as the ODS (Operational Data Store) area, during modeling. Due to reliance on manual maintenance, the descriptions of the same field in different tables may be inconsistent (e.g., a user id might be described as "user number" or "customer unique identifier"), leading to misunderstandings. For instance, looking up enumerated fields is particularly time-consuming, often requiring contacting the development manager to locate the corresponding enumerated value through the project code, increasing communication costs and time consumption. All these factors contribute to the waste of useful information hidden within massive amounts of data, resulting in a lack of intelligent support for data resource assessment and modeling, leading to inefficiency and an inability to meet requirements.

[0004] Therefore, there is an urgent need for a data resource assessment system and method at the ODS layer that can achieve full-process, automated data resource assessment through artificial intelligence algorithms, so as to meet enterprises' quality and efficiency requirements for data resource management. Summary of the Invention

[0005] The main objective of this invention is to solve the technical problem that the data resource assessment and modeling in the prior art relies heavily on manual labor, making it impossible to automate and intelligently inventory massive amounts of data resources, resulting in poor inventory efficiency and unsatisfactory inventory results.

[0006] The first aspect of this invention provides a large language model-driven end-to-end data resource evaluation and modeling system for the ODS layer, comprising: The data input module is used to capture the new business tables corresponding to the target iteration version in real time, and obtain the table structure and table metadata of the new business tables; The modeling and evaluation module is used to call the large language model to integrate historical evaluation records and the resource classification rules contained in the preset resource classification knowledge base, evaluate the data resources contained in the newly added business table, and output the resource classification results and modeling and evaluation conclusions. The data modeling module is used to generate field descriptions for each field in the newly added business table when the modeling evaluation conclusion output by the modeling evaluation module indicates that the data resources of the newly added business table need to be modeled. Specifically, when the field is an enumerated field, the module calls the large language model to analyze the business source code related to the enumerated field and generate the field description of the enumerated field. Based on the table structure, table metadata, and field descriptions of the newly added business table, a data resource model is generated at the ODS layer.

[0007] Optionally, in a first implementation of the first aspect of the present invention, the data modeling module is specifically used for: Find the version control information corresponding to the enumerated fields; Based on the version control information, obtain the code information, entity class object information, and table creation metadata information of the operation class logic layer; The large language model is invoked to generate annotation information based on the code information, the entity class object information, and the table creation metadata information, and the annotation information is merged to generate the field description of the enumeration field.

[0008] Alternatively, in a second implementation of the first aspect of the invention, The fields also include ordinary fields; the data modeling module is further specifically used for: When the field is a regular field, the field name is obtained from the table metadata, and the field name is split and divided to obtain multiple tokens; By combining the resource classification knowledge base, the large language model is called to determine whether each word element belongs to the standard word root. If it does not belong to the standard word root, the standard word root is used to replace it to obtain the standard word element sequence. After translating the standard lexical sequence, the large language model is invoked to output ordinary field descriptions in natural language based on Chinese language rules.

[0009] Optionally, in a third implementation of the first aspect of the present invention, the data modeling module is further configured to: The generated ordinary field description is compared and evaluated with the original field descriptions pre-stored in the data catalog system to determine whether the similarity between the ordinary field description and the original field description is lower than a preset similarity threshold. If the similarity is lower than the preset similarity threshold, the description of the ordinary field is retained; If the similarity is higher than the preset similarity threshold, the original field description will be used to replace the ordinary field description.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, the data modeling module is further configured to: Call the large language model to determine whether each field in the newly added business table belongs to the standard fields; If it is not a standard field, then determine whether it is an enumerated field; If it belongs to an enumerated field, then the enumerated field will be described as a standardized field. If it is neither a standard field nor an enumerated field, then it is described as a standardized field based on the ordinary field description.

[0011] Optionally, in a fifth implementation of the first aspect of the present invention, the data input module is specifically used for: The operation and maintenance management system monitors data definition language changes in real time, captures multiple newly added business tables, and obtains the table structure of the newly added business tables. Based on iteration version information, retrieve a candidate list of new business tables corresponding to the target iteration version; Based on the candidate list, the table metadata of the new business table corresponding to the target iteration version is found through the data catalog system.

[0012] Optionally, in a sixth implementation of the first aspect of the present invention, the historical evaluation record includes historical evaluation records of historical business tables that are similar to the database, table name, or table description in the table metadata of the newly added business table. The modeling and evaluation module is specifically used for: Extract the source database name, table name, table description, and field details of the table metadata of the newly added business table; The system architecture and business domain information of the source database for obtaining the table metadata of the newly added business table; Retrieve resource classification rules from the preset resource classification knowledge base; The large language model is invoked to search for historical evaluation records based on the table name, table description, field details, system architecture, and business domain information of the newly added business table; Based on the historical assessment records and resource classification rules, a comprehensive assessment is conducted to generate resource classification results and modeling assessment conclusions.

[0013] A second aspect of this invention provides a large language model-driven end-to-end data resource evaluation and modeling method for the ODS layer, comprising: Real-time capture of newly added business tables corresponding to the target iteration version, and acquisition of table structure and table metadata of the newly added business tables; The system calls upon a large language model to integrate historical evaluation records with the resource classification rules contained in a preset resource classification knowledge base, evaluates the data resources contained in the newly added business table, and outputs resource classification results and modeling evaluation conclusions. When the modeling evaluation conclusion indicates that the data resources of the newly added business table need to be modeled, the large language model is invoked to generate field descriptions for each field contained in the newly added business table; wherein, when the field is an enumerated field, the large language model is invoked to analyze the business source code related to the enumerated field and generate the field description of the enumerated field. Based on the table structure, table metadata, and field descriptions of the newly added business table, a data resource model is generated at the ODS layer.

[0014] A third aspect of the present invention provides a large language model-driven ODS layer full-process data resource evaluation and modeling device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the large language model-driven ODS layer full-process data resource evaluation and modeling device to execute the steps of the above-described large language model-driven ODS layer full-process data resource evaluation and modeling method.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described large language model-driven ODS layer full-process data resource evaluation and modeling method.

[0016] The fifth aspect of the present invention provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the large language model-driven ODS layer full-process data resource evaluation and modeling method described above.

[0017] The technical solution provided by this invention includes a large language model-driven ODS layer full-process data resource evaluation and modeling system comprising: a data input module, used to capture the new business table corresponding to the target iteration version in real time and obtain the table structure and table metadata of the new business table; a modeling and evaluation module, used to call the large language model to integrate historical evaluation records with the resource classification rules contained in the preset resource classification knowledge base, evaluate the data resources contained in the new business table, and output the resource classification results and modeling and evaluation conclusions; and a data modeling module, used to call the large language model to generate field descriptions for each field contained in the new business table when the modeling and evaluation conclusion output by the modeling and evaluation module indicates that the data resources of the new business table need to be modeled; wherein, when the field is an enumerated field, the large language model is called to analyze the business source code related to the enumerated field and generate the field description of the enumerated field; and a data resource model is generated at the ODS layer based on the table structure, table metadata, and field descriptions of the new business table.

[0018] The beneficial effects of this system are: it enables automatic evaluation and modeling of data resources based on a large language model, improving the automation and efficiency of data resource evaluation and modeling, while also enhancing the accuracy and consistency of the modeling results. Furthermore, the method, electronic device, computer-readable storage medium, and computer program product provided by this invention also solve corresponding technical problems. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating the first embodiment of the ODS layer full-process data resource evaluation and modeling method driven by a large language model in this invention. Figure 2 This is a flowchart illustrating the second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by a large language model in this invention. Figure 3 This is a schematic diagram of the process for obtaining table metadata of newly added business tables in the second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by a large language model in this invention. Figure 4 This is a flowchart illustrating the reasoning and classification conclusions and the necessity of modeling in the second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by the large language model in this invention. Figure 5 This is a schematic diagram of the first process for generating field descriptions in the second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by a large language model in this invention. Figure 6 This is a schematic diagram of the second process for generating field descriptions in the second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by a large language model in this invention. Figure 7 This is a schematic diagram of the judgment process for enumerated fields in the second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by the large language model in this invention. Figure 8 This is a schematic diagram of the generation process of the field description of the enumerated field in the second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by the large language model in this invention. Figure 9 This is a schematic diagram of an embodiment of the ODS layer full-process data resource evaluation and modeling system driven by a large language model in this invention. Detailed Implementation

[0020] Exemplary embodiments of the invention will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limiting the invention to the embodiments set forth herein. Rather, these exemplary embodiments are provided to make the invention more comprehensive and complete, and to facilitate a full communication of the inventive concept to those skilled in the art. The same reference numerals in the drawings denote the same or similar elements, components, or parts, and therefore repeated descriptions of them will be omitted.

[0021] Subject to the technical concept of this invention, the features, structures, characteristics or other details described in a particular embodiment may be combined in one or more other embodiments in a suitable manner.

[0022] In the description of specific embodiments, the features, structures, characteristics, or other details described in this invention are intended to enable those skilled in the art to fully understand the embodiments. However, it is not excluded that those skilled in the art can practice the technical solutions of this invention without one or more of the specific features, structures, characteristics, or other details.

[0023] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The terms “and / or” or “and / or” include all combinations of any one or more of the listed items.

[0026] See appendix Figure 1 The first embodiment of the large language model-driven ODS layer full-process data resource evaluation and modeling method in this invention includes: S101. Capture the newly added business tables corresponding to the target iteration version in real time, and obtain the table structure and table metadata of the newly added business tables; It is understood that the executing entity of this invention can be a full-process data resource evaluation and modeling device or system driven by a large language model at the ODS (Operational Data Store) layer, or it can be a terminal or a server; the specifics are not limited here. This embodiment of the invention uses a server as an example for illustration.

[0027] After receiving a data resource assessment and modeling request, the server first obtains the data objects that need to be assessed and modeled. In this embodiment, data resources can be batched according to each information iteration version, and the newly added business tables corresponding to the target iteration version can be captured in real time through a pre-built data fetching pipeline, and the table structure and metadata of the newly added business tables can be obtained. Furthermore, the data captured in this embodiment is all data that has been authorized or anonymized.

[0028] In this embodiment, when capturing newly added business tables, information from the operation and maintenance management system needs to be integrated. The operation and maintenance management system monitors data definition language changes in real time, capturing multiple newly added business tables to obtain their table structure and information. The table structure describes the table's constituent units, such as partition information. Furthermore, this embodiment also captures a candidate list of newly added business tables corresponding to the target iteration version based on iteration version information. Based on this candidate list, the table metadata of the newly added business tables corresponding to the target iteration version is searched in the data catalog system.

[0029] S102. Call the large language model to integrate historical evaluation records with the resource classification rules contained in the preset resource classification knowledge base, evaluate the data resources contained in the newly added business table, and output the resource classification results and modeling evaluation conclusions. In this step, the historical evaluation records include historical evaluation modeling information related to historical business tables that are similar to those in the database, table name, or table description of the table metadata of the newly added business table. The historical evaluation records can be constructed and stored based on databases and knowledge graphs. When obtaining historical evaluation modeling records of historical business tables that are similar to those in the database, table name, or table description of the table metadata of the newly added business table, they can be retrieved using cosine similarity algorithms or other similarity algorithms.

[0030] When performing reasoning and judgment, it is first necessary to obtain historical evaluation records and resource classification rules contained in a preset resource classification knowledge base. Specifically, this includes: extracting the table name, table description, and field details from the table metadata of the newly added business table; obtaining the system architecture and business domain information of the database to which the table metadata of the newly added business table belongs; calling the large language model to search for historical evaluation modeling records based on the table name, table description, field details, system architecture, and business domain information; and calling resource classification rules from the preset resource classification knowledge base. The resource classification knowledge base can be a predefined set of rules pre-built based on business domain information and the importance level information of each data point in the business scenario.

[0031] After obtaining this information, the large language model is called again to conduct a comprehensive evaluation based on the historical evaluation modeling records and resource classification rules, generating resource classification results and modeling evaluation conclusions.

[0032] The Large Language Model (LLM) described in this invention is a deep learning model trained on a large amount of text data, which enables the model to generate natural language text or understand the meaning of language text. In this embodiment, it can output reasoning conclusions based on multiple data and preset rules.

[0033] S103. When the modeling evaluation conclusion is that the data resources of the newly added business table need to be modeled, the large language model is called to generate field descriptions for each field contained in the newly added business table. When the modeling evaluation concludes that modeling is required, the large language model is invoked to generate standardized field descriptions for the fields of the newly added business table. In this embodiment, fields will be processed in three categories: standard fields, enumerated fields, and ordinary fields. Standard fields are fields that conform to the predefined names, types, and semantics in the data warehouse architecture standard; enumerated fields are fields whose values ​​are limited to a predefined finite set, such as status codes and type identifiers; ordinary fields are fields that neither match standard fields nor belong to enumerated fields. In this embodiment, when a field belongs to an enumerated field, the large language model is invoked to analyze the relevant business source code of the enumerated field and generate a field description for the enumerated field.

[0034] In one specific implementation, it is necessary to first determine whether each field of the newly added business table has a matching standard field in the preset data warehouse architecture standard. This preset data warehouse architecture standard can be built based on MCP (Model Context Protocol). If no matching standard field is found, it is determined whether the field is an enumeration field. If it is an enumeration field, a large language model is invoked to analyze the relevant business source code and generate a field description for the enumeration field. If it is neither a standard field nor an enumeration field, a field description is generated using the description method for ordinary fields. When generating ordinary field descriptions, the semantic understanding capabilities of the large language model can be utilized to perform word segmentation and semantic parsing on the original field names, generating standardized field descriptions corresponding to the ordinary fields, thus generating accurate and clear data annotations for the fields of the newly added business table.

[0035] S104. Based on the table structure, table metadata, and field descriptions of the newly added business tables, generate a data resource model at the ODS layer.

[0036] After generating field descriptions based on the large language model, the table metadata can be updated based on the field descriptions, and the updated table metadata can be injected into the previously obtained table structure, ultimately forming a standardized and highly readable data dictionary document at the ODS layer.

[0037] The ODS layer full-process data resource assessment and modeling system driven by a large language model provided in this invention can automatically perform data resource assessment and modeling through a large language model-driven approach. This achieves the technical effects of improving the efficiency, accuracy, and consistency of data resource assessment and modeling results.

[0038] See appendix Figure 2 -Appendix Figure 7 The second embodiment of the ODS layer full-process data resource evaluation and modeling method driven by a large language model in this invention includes: S201. Monitor data definition language changes in real time through the operation and maintenance management system, capture multiple newly added business tables, and obtain the table structure of the newly added business tables. It is understood that the executing entity of this invention can be a large language model-driven ODS layer full-process data resource evaluation and modeling device or system, or it can be a terminal or a server; the specifics are not limited here. This embodiment of the invention uses a server as the executing entity, and takes the implementation of an automatic data resource evaluation and modeling process in the ODS (Operational Data Store) area as an example for illustration. Here, ODS is a type of data warehouse layer, typically used to support daily business operations and decision-making processes. It integrates data from different sources, storing a copy of the data tables from the source systems intact, and serves as the source of data for subsequent data warehouse processing. In a preferred embodiment, the data resources mentioned in this embodiment can refer to data resources legally owned or controlled by the enterprise that are expected to bring economic benefits, also known as data assets.

[0039] This embodiment requires the pre-construction of a real-time data pipeline through the collaboration of multiple systems. After receiving a data resource assessment and modeling request, the server captures new data in real time through the constructed real-time data pipeline.

[0040] In one specific implementation, see Appendix Figure 3 This embodiment provides a solution for building a real-time data pipeline through an operation and maintenance management system and a data catalog system. In this step, the operation and maintenance management system is first connected to monitor DDL (Data Definition Language) statement change events submitted by various departments in real time, automatically capturing newly added business tables to obtain their table structure and information. The table structure describes the constituent units of the table, such as partition information. Specifically, the DDL statement change event can be an operation such as creating a new table (CREATETABLE), capturing the newly added business table.

[0041] S202. Retrieve a candidate list of new business tables corresponding to the target iteration version based on iteration version information; In this embodiment, when monitoring newly added business tables in the operation and maintenance management system, a candidate list of incremental tables is also generated according to the iteration version; for example, a list of incremental business tables is generated according to the weekly release window of the version. By annotating the newly added business tables of the target iteration version in this way, the business table information captured in this instance can be used, and further, deduplication and version backtracking can be supported.

[0042] S203. Based on the candidate list, locate the table metadata of the new business table corresponding to the target iteration version through the data catalog system; Please refer to the appendix for further details. Figure 3The real-time data pipeline constructed in this embodiment also includes an integrated data catalog system. This data catalog system associates source table descriptions and field description information pools, fusing data from both sources to provide metadata information related to the newly added business tables. The source table descriptions can be obtained from a MySQL information schema database, and the field description information pool can be a pre-defined business dictionary description database.

[0043] Based on the candidate list from the previous step, the corresponding metadata information is searched in the data catalog system to enhance the metadata of the newly added business table obtained in step S201, thus obtaining more metadata information. Combined with the target iteration version from step S201, the table metadata of the newly added business table corresponding to the target iteration version can be obtained; at this point, the specific data to be processed is obtained.

[0044] S204. Call the large language model to integrate historical evaluation records with the resource classification rules contained in the preset resource classification knowledge base, and evaluate the data resources contained in the newly added business table. S205. Output the resource classification results and determine whether modeling is necessary; In this embodiment, after obtaining the table metadata of the newly added business table, the solution can call the large language model to infer the approximate data resource type based on the relevant information content of the business table metadata, and determine whether it is information that needs to be modeled.

[0045] Specifically, the intelligent reasoning model constructed using the Large Language Model (LLM) technology can integrate information from multiple knowledge bases to output conclusions and suggestions.

[0046] In one specific implementation, the large language model described in this embodiment can integrate historical evaluation records with information contained in a preset resource classification knowledge base. Based on the content of the table metadata of a newly added business table, it infers the resource classification result and determines whether modeling is necessary. The historical evaluation records may include the database name or business domain information of the database from which the table metadata of the newly added business table originates, as well as historical evaluation modeling information of historical business tables with similar table names or descriptions. Specific steps include: extracting the table name, table description, and field details from the table metadata of the newly added business table; obtaining the system architecture and business domain information of the database from which the table metadata of the newly added business table originates; calling resource classification rules from the preset resource classification knowledge base; calling the large language model to search for historical evaluation modeling records based on the table name, table description, field details, system architecture, and business domain information; and performing a comprehensive evaluation based on the historical evaluation modeling records and resource classification rules to generate resource classification results and modeling evaluation conclusions.

[0047] See appendix Figure 4 In one specific implementation, the process of steps S204 and S205 is divided into three levels: the first level is the data input layer, the second level is the large language model processing layer, and the third level is the output layer.

[0048] The first-level data input layer needs to acquire data information from multiple sources, including: (1) Query source table information and field information: The source table information and field information are extracted by obtaining the table metadata of the new business table in step S203, specifically including: obtaining the basic information (such as table name, table description) and field details (field name, type, description) of the new business table to be evaluated from the data catalog system based on the table metadata of the new business table. (2) Query data preview: Obtain sample data of the table from the data catalog system to assist the large language model in judging data characteristics.

[0049] (3) Query database information: Obtain the source database metadata of the table metadata from the data catalog (such as system architecture, database type, business domain) to assist the large language model in judging data characteristics; (4) Query historical evaluation records: Retrieve historical evaluation and modeling records under the same system / same library, focusing on: Historical cases with similar table names, such as “user_info” and “user_detail”, which have a high degree of semantic similarity, need to be focused on and extracted. Tables describing similar historical cases, such as historical cases that all contain the description of "user transaction records"; Evaluation and modeling experience of business relationship tables, such as the processing rules for upstream and downstream dependency tables; (5) Query classification rules: Standardized table classification rules, such as business domain division and sensitive table definition, can be obtained from the built-in resource classification knowledge base.

[0050] The second-level large language model processing layer then calls the large language model and performs inference by integrating various information obtained from the data input layer, including: (1) Library information / system architecture: Understand the technical environment and business context of the table. Library information and system architecture serve as the basis for reasoning in the large language model, ensuring that the reuse of historical experience conforms to the technical environment of the current table; (2) Table-related information: metadata and data characteristics of the current table; (3) Historical evaluation records: Prioritize the reuse of evaluation modeling logic from similar tables under the same system / library; (4) Resource classification rules (obtained from the resource classification knowledge base): Output classification suggestions based on the company's standardized classification rules; The third layer is the output layer, which generates data table resource classification conclusions. For example, it outputs classification levels (such as primary classification and secondary classification) based on importance or preset classification rules, outputs a conclusion on whether evaluation and modeling are necessary, and outputs modeling instructions related to the modeling conclusions.

[0051] In this embodiment, the resource classification knowledge base mainly contains rules and bases related to the modeling conclusions derived from the data resource classification system and custom-defined rules. For example, the core business data is defined as: key business data items that directly affect the completion of business operations, and data that is directly related to the main business content in the business process.

[0052] S206. When the modeling evaluation conclusion is that the data resources of the newly added business table need to be modeled, the large language model is called to generate standardized field descriptions for each field contained in the newly added business table. Taking the method of evaluating and modeling resource data in the ODS layer in this embodiment as an example, the ODS field description to be generated in this step is an important part of ODS modeling. In this embodiment, the description can be intelligently generated in stages by leveraging the capabilities of the large language model, thereby accurately filling in the field description.

[0053] In one implementation, the field description includes standard field description, enumerated field description, and ordinary field description. The standard field can seamlessly connect to the enterprise data standard knowledge base, enabling intelligent matching and precise mapping between the field description and a unified semantic standard, ensuring consistency of core terminology. The enumerated field can be obtained through deep analysis of code semantics, using natural language understanding technology to intelligently analyze the inherent logic and business meaning of enumerated code values, automatically generating accurate and readable corresponding descriptions, eliminating "code black holes." The ordinary field is generated by integrating table-level descriptions, intelligent matching of standard word roots, and advanced semantic word segmentation technology to obtain highly relevant field descriptions, which are then rigorously verified through similarity comparison to ensure the accuracy and business relevance of the descriptions.

[0054] See appendix Figure 5 and appendix Figure 6 The specific process of calling the large language model to generate standardized field descriptions for the fields of the newly added business table includes: S2061. Call the large language model to determine whether each field of the newly added business table matches the standard fields in the preset data warehouse architecture standard. S2062. If it matches a standard field, then the standard field description shall be used; S2063. If it does not match the standard field, then determine whether it is an enumerated field; First, key metadata is obtained through the data catalog system to provide basic data support for large language model inference, including: (1) Table structure and field information: Obtain the table name, description and field details (field name, type and comment) of the new business table to be evaluated and modeled. (2) Data Sample Preview: Extract sample data from the table to assist the large language model in recognizing data distribution and feature patterns; (3) Standard field information: obtained from the company's data warehouse specification field standards; (4) Standard root word information: The root word standard is standardized through the company's data warehouse information. Specifically, it can be constructed based on industry terminology and the company's internal dictionary, including a standardized mapping table of nouns and verbs; (5) Field description context information: Obtain metadata such as the system architecture, library type and business domain of the database to which the table belongs, and enhance the understanding of the data background by the large language model.

[0055] Subsequently, the table field information is extracted and matched with standard fields to determine whether the table contains standard fields. If a match is found, the standard field description is used completely; otherwise, the process proceeds to step S2064 to determine whether it belongs to an enumerated field.

[0056] The standard fields mentioned above are built upon the MCP (Model Context Protocol). Specifically, MCP is an open protocol designed to achieve seamless integration between large language models and external data sources, tools, and services. Its goal is to create a universal standard that simplifies and unifies the development and integration of AI applications; MCP standardizes the way applications provide context, helping large language models generate higher-quality responses.

[0057] In one specific implementation, determining whether a value is an enumeration involves three aspects of analysis: naming characteristic checking, structural analysis, and data analysis. If both the naming characteristic checking and structural analysis confirm the enumeration characteristics, then the immediate query interface is invoked to query the data grouping (e.g., using a GROUP BY statement). See Appendix. Figure 7 The specific judgment process described above achieves automatic identification and classification of field types by intelligently integrating table structure metadata and field enumeration value statistics. The process covers calling MCP to query table information, initial screening of LLM field types, generating field enumeration value query SQL (Structured Query Language), calling MCP to execute SQL data query, LLM enumeration type confirmation and standardized format output, forming a closed-loop automated classification path.

[0058] S2064. If it belongs to an enumeration field, then generate an enumeration field description by analyzing the business source code; When generating enumeration field descriptions in S2064, the version control information corresponding to the enumeration field is first retrieved. This version control information can be obtained through the Git version control system. For example, the Git API can be used to map table names to corresponding code branches, extracting operation classes, entity classes, and DDL statements. Then, based on the version control information, the code information of the operation class logic layer, entity class object information, and table creation metadata information are obtained. The large language model is then called to integrate the code information, entity class object information, and table creation metadata information to generate the enumeration field description. Specifically, the understanding capabilities of the large language model can be leveraged to generate accurate and clear data annotations for enumeration fields.

[0059] See appendix Figure 8 The process for describing a specific enumeration field is as follows: Upon initiation of the process, the system first queries the project information in the version control system corresponding to the target database table through the MCP service to determine whether the corresponding project code exists. The version control system can be a Git version control system. In this embodiment, a standardized interface with Git and database metadata is established through the MCP protocol to enable dynamic access to external data sources.

[0060] Enumerated field description process branch A: (A-1) When relevant Git project information exists, structured information is obtained. This requires obtaining three key metadata types simultaneously: Retrieve code information for operation classes, such as Service, Controller, and other business logic layers; Retrieve entity class information, such as POJO and Bean, and the entity object information mapped to the database table structure; Retrieve table creation metadata information: such as column names, types, comments, etc. in DDL statements; (A-2) Next, LLM processing and annotation generation are performed, specifically including: Enumeration class traversal search: Provide all code snippets containing operation classes to the LLM for judgment, and find the enumeration class used by the field in the code snippet; Enumeration class annotation information generation: LLM comprehensive analysis of operation class code snippets combined with enumeration class code content helps to understand the specific meaning and optional value range of the enumeration field in the business code, and generates accurate annotations; Entity class annotation information generation: Find the variable annotations corresponding to the enumeration fields from the entity class definition code snippet, use LLM to understand the meaning of the code annotations to generate standardized enumeration annotations, and supplement them when no field enumeration class is found; Metadata information is used to generate annotations: Standardized enumeration annotations are generated from table metadata information, such as field names and field descriptions, through LLM understanding. Combined with entity class annotation information, the more detailed the description, the better. When no field enumeration class is found, it is added.

[0061] (A-3) Finally, aggregate and output the results: integrate all field descriptions generated by LLM to form a complete and uniformly formatted field description.

[0062] Enumerated field description process branch B: When no Git project information is found or a query fails, the system directly outputs a "No project information or error" message in a standardized format, and the process terminates. This ensures the robustness of the process and provides clear feedback on the execution status.

[0063] S2065. If it is not an enumerated field, then perform word segmentation and semantic parsing by field name to generate a normal field description.

[0064] In S2065, if the judgment result belongs to neither a standard field nor an enumerated field, this field is treated as a regular field and a regular field description is generated. Specifically, the field name is split into multiple tokens; a large language model combined with a resource classification knowledge base is called to determine whether each token belongs to a standard word root. If it does not belong to a standard word root, a standard word root is used to replace it, resulting in a standard token sequence; after translating the standard token sequence, the large language model is called to output a regular field description in natural language form based on Chinese language rules.

[0065] To illustrate with a concrete example, the field name is first split into tokens according to word segmentation rules. For instance, when the input field name is "user_age_int_not_null", the most basic string splitting function in programming languages ​​is used to perform the splitting. In Python, a specific example would be: "user_age_int_not_null".split('_'); the output tokens could be: ["user", "age", "int", "not", "null"]. The split tokens then undergo preliminary processing to prepare for subsequent classification. First, they are all converted to lowercase to ensure that subsequent matching is not affected by case, such as converting "User" to "user". Next, empty strings are filtered to prevent empty tokens caused by consecutive underscores (such as a__b). Then, numbers and special characters are processed, identified and preserved; for example, "255" in "varchar_255" needs to be retained. Finally, a cleaned list of tokens is output.

[0066] After obtaining the list of tokens, it is necessary to obtain the mapping of the technical dictionary. For example, in the context of a database, the word "user" is not translated but retained as part of the field name; in special cases, it can be translated as "用户". Here, translation is chosen to keep the sentence smooth; "int" is mapped to the Chinese technical term "整数类型"; "not null" is mapped to the Chinese technical terms "非空" or "不为空". Next, perform translation according to the technical dictionary, and output a unified sequence to get ["用户", "年龄", "整数类型", "非", "空"].

[0067] Subsequently, apply Chinese grammar rules and combine the knowledge base data warehouse field description standards to assemble the structured semantic representation into a natural language description, including: (1) Preliminary sentence construction: The user age field is of integer type and is constrained to be non-empty; (2) Fluency optimization: Term optimization, conjunction addition, logic simplification; (3) Final result generation: "The user age is of integer type and non-empty".

[0068] In a preferred implementation manner, it further includes: comparing and evaluating the generated general field description with the original field description pre-stored in the data catalog system to determine whether the similarity between the general field description and the original field description is lower than a preset similarity threshold; if the similarity is lower than the preset similarity threshold, retain the general field description; if the similarity is higher than the preset similarity threshold, use the original field description to replace the general field description. For example, based on the deep understanding of language semantics by the large language model, directly analyze and compare the intentions, themes, and key information of two paragraphs, and output an integer similarity score from 1 to 10; among them, 1 - 3 points indicate a great semantic difference, 4 - 6 points indicate a certain similarity but obvious differences, and 7 - 10 points indicate a high semantic similarity or identity; then if the similarity is higher than 7 points, directly use the original description, otherwise return the description generated by the large language model in the previous step. When calculating semantic similarity, it includes but is not limited to using the cosine similarity algorithm or calculating through the BERT model.

[0069] Take a specific example to illustrate. For example, the field name is: "user_no", and the original field description in its DDL statement is: "用户号", and the recommended description obtained from the output of the large language model in the previous step is: "用户编号". However, semantically, the content of "用户号" and "用户编号" is highly similar, so reuse the original field description and use "用户号" instead of "用户编号".

[0070] S207. Generate a data resource model in the ODS layer based on the table structure, table metadata, and standardized field descriptions of the newly added business table.

[0071] In this step, the metadata of the newly added business table is combined with the data produced throughout the entire process, including field descriptions of standard fields, enumerated fields, and ordinary fields. A data resource model is constructed, accurate and clear data annotations are generated, and finally a standardized and highly readable ODS layer data dictionary document is formed.

[0072] The ODS layer full-process data resource assessment and modeling system driven by a large language model provided in this invention can automatically perform data resource assessment and modeling through a large language model-driven approach. This improves the efficiency of data resource assessment and modeling. In actual testing, when dealing with large datasets, the system can reduce the modeling time per table from 20 minutes to 30 seconds, improving the efficiency of enumeration value mapping and increasing modeling efficiency by up to 15 times. Furthermore, it optimizes the accuracy of data resource assessment and modeling, ensuring consistency in model construction and avoiding misunderstandings caused by manual assessment and modeling. It also enhances self-service and flexibility, allowing even non-technical users to complete over 90% of the modeling tasks independently, simplifying operations. This invention effectively promotes the transformation of enterprise data governance from "manually driven" to "artificial intelligence self-service," significantly reducing operational costs.

[0073] The above describes the method for full-process data resource evaluation and modeling of the ODS layer driven by a large language model in the embodiments of the present invention. The following describes the system for full-process data resource evaluation and modeling of the ODS layer driven by a large language model in the embodiments of the present invention. (See appendix.) Figure 9 An embodiment of the large language model-driven ODS layer full-process data resource evaluation and modeling system of the present invention includes: The data input module 901 is used to capture the newly added business table corresponding to the target iteration version in real time, and obtain the table structure and table metadata of the newly added business table; The modeling evaluation module 902 is used to call the large language model to integrate historical evaluation records and the resource classification rules contained in the preset resource classification knowledge base, evaluate the data resources contained in the newly added business table, and output the resource classification results and modeling evaluation conclusions. The data modeling module 903 is used to generate field descriptions for each field in the newly added business table when the modeling evaluation conclusion output by the modeling evaluation module 902 indicates that the data resources of the newly added business table need to be modeled. Specifically, when the field is an enumerated field, the module calls the large language model to analyze the business source code related to the enumerated field and generate the field description of the enumerated field. Based on the table structure, table metadata, and field descriptions of the newly added business table, a data resource model is generated at the ODS layer.

[0074] The ODS layer full-process data resource evaluation and modeling system driven by a large language model provided in this invention can automatically evaluate and model data resources based on the large language model. This achieves the technical effects of improving the automation level and efficiency of data resource evaluation and modeling, as well as improving the accuracy and consistency of modeling results.

[0075] In another embodiment of this application, the data modeling module 903 is specifically used for: Find the version control information corresponding to the enumerated fields; Based on the version control information, obtain the code information, entity class object information, and table creation metadata information of the operation class logic layer; The large language model is invoked to generate annotation information based on the code information, the entity class object information, and the table creation metadata information, and the annotation information is merged to generate the field description of the enumeration field.

[0076] In another embodiment of this application, the field further includes a general field; the data modeling module 903 is specifically used for: When the field is a regular field, the field name is obtained from the table metadata, and the field name is split and divided to obtain multiple tokens; By combining the resource classification knowledge base, the large language model is called to determine whether each word element belongs to the standard word root. If it does not belong to the standard word root, the standard word root is used to replace it to obtain the standard word element sequence. After translating the standard lexical sequence, the large language model is invoked to output ordinary field descriptions in natural language based on Chinese language rules.

[0077] In another embodiment of this application, the data modeling module 903 is further configured to: The generated ordinary field description is compared and evaluated with the original field descriptions pre-stored in the data catalog system to determine whether the similarity between the ordinary field description and the original field description is lower than a preset similarity threshold. If the similarity is lower than the preset similarity threshold, the description of the ordinary field is retained; If the similarity is higher than the preset similarity threshold, the original field description will be used to replace the ordinary field description.

[0078] In another embodiment of this application, the data modeling module 903 is further configured to: Call the large language model to determine whether each field in the newly added business table belongs to the standard fields; If it is not a standard field, then determine whether it is an enumerated field; If it belongs to an enumerated field, then the enumerated field will be described as a standardized field. If it is neither a standard field nor an enumerated field, then it is described as a standardized field based on the ordinary field description.

[0079] In another embodiment of this application, the data input module 901 is specifically used for: The operation and maintenance management system monitors data definition language changes in real time, captures multiple newly added business tables, and obtains the table structure of the newly added business tables. Based on iteration version information, retrieve a candidate list of new business tables corresponding to the target iteration version; Based on the candidate list, the table metadata of the new business table corresponding to the target iteration version is found through the data catalog system.

[0080] In another embodiment of this application, the historical evaluation record includes historical evaluation records of historical business tables that are similar to the database, table name, or table description in the table metadata of the newly added business table; The modeling and evaluation module 902 is specifically used for: Extract the source database name, table name, table description, and field details of the table metadata of the newly added business table; The system architecture and business domain information of the source database for obtaining the table metadata of the newly added business table; Retrieve resource classification rules from the preset resource classification knowledge base; The large language model is invoked to search for historical evaluation records based on the table name, table description, field details, system architecture, and business domain information of the newly added business table; Based on the historical assessment records and resource classification rules, a comprehensive assessment is conducted to generate resource classification results and modeling assessment conclusions.

[0081] In another embodiment of this application, the specific implementation details of the large language model-driven ODS layer full-process data resource evaluation and modeling system during execution can be found in the specific descriptions in the foregoing method embodiments, and will not be repeated here.

[0082] The ODS layer full-process data resource evaluation and modeling system driven by a large language model provided in this invention can automatically evaluate and model data resources based on the large language model. This achieves the technical effects of improving the automation level and efficiency of data resource evaluation and modeling, as well as improving the accuracy and consistency of modeling results.

[0083] Based on the same inventive concept, embodiments of this specification also provide an electronic device for ODS layer full-process data resource evaluation and modeling driven by a large language model. The electronic device for ODS layer full-process data resource evaluation and modeling driven by a large language model includes: a memory and at least one processor. The memory stores instructions. The at least one processor calls the instructions in the memory to cause the electronic device for ODS layer full-process data resource evaluation and modeling driven by a large language model to perform the steps of the ODS layer full-process data resource evaluation and modeling method as described in the above embodiments.

[0084] Based on the same inventive concept, the present invention also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the large language model-driven ODS layer full-process data resource evaluation and modeling method as described in any of the above embodiments.

[0085] A computer program implementing the steps of a large language model-driven ODS layer full-process data resource evaluation and modeling method as described in the above embodiments can be stored on one or more computer-readable media. The computer-readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0086] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0087] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0088] In summary, the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that in practice, general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0089] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0090] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0091] If the technical solution of this application involves user personal information, the product applying the technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product applying the technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, the personal information processing rules are clearly informed through signs / information, and authorization is obtained through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0092] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A large language model-driven end-to-end data resource evaluation and modeling system for the ODS layer, characterized in that, include: The data input module is used to capture the new business tables corresponding to the target iteration version in real time, and obtain the table structure and table metadata of the new business tables; The modeling and evaluation module is used to call the large language model to integrate historical evaluation records and the resource classification rules contained in the preset resource classification knowledge base, evaluate the data resources contained in the newly added business table, and output the resource classification results and modeling and evaluation conclusions. The data modeling module is used to generate field descriptions for each field in the newly added business table when the modeling evaluation conclusion output by the modeling evaluation module indicates that the data resources of the newly added business table need to be modeled. Specifically, when the field is an enumerated field, the module calls the large language model to analyze the business source code related to the enumerated field and generate the field description of the enumerated field. Based on the table structure, table metadata, and field descriptions of the newly added business table, a data resource model is generated at the ODS layer.

2. The large language model-driven ODS layer full-process data resource evaluation and modeling system according to claim 1, characterized in that, The data modeling module is specifically used for: Find the version control information corresponding to the enumerated fields; Based on the version control information, obtain the code information, entity class object information, and table creation metadata information of the operation class logic layer; The large language model is invoked to generate annotation information based on the code information, the entity class object information, and the table creation metadata information, and the annotation information is merged to generate the field description of the enumeration field.

3. The large language model-driven ODS layer full-process data resource evaluation and modeling system according to claim 1, characterized in that, The fields also include ordinary fields; The data modeling module is also specifically used for: When the field is a regular field, the field name is obtained from the table metadata, and the field name is split and divided to obtain multiple tokens; By combining the resource classification knowledge base, the large language model is called to determine whether each word element belongs to the standard word root. If it does not belong to the standard word root, the standard word root is used to replace it to obtain the standard word element sequence. After translating the standard lexical sequence, the large language model is invoked to output ordinary field descriptions in natural language based on Chinese language rules.

4. The large language model-driven ODS layer full-process data resource evaluation and modeling system according to claim 3, characterized in that, The data modeling module is also specifically used for: The generated ordinary field description is compared and evaluated with the original field descriptions pre-stored in the data catalog system to determine whether the similarity between the ordinary field description and the original field description is lower than a preset similarity threshold. If the similarity is lower than the preset similarity threshold, the description of the ordinary field is retained; If the similarity is higher than the preset similarity threshold, the original field description will be used to replace the ordinary field description.

5. The large language model-driven ODS layer full-process data resource evaluation and modeling system according to claim 3, characterized in that, The data modeling module is also specifically used for: Call the large language model to determine whether each field in the newly added business table belongs to the standard fields; If it is not a standard field, then determine whether it is an enumerated field; If it belongs to an enumerated field, then the enumerated field will be described as a standardized field. If it is neither a standard field nor an enumerated field, then it is described as a standardized field based on the ordinary field description.

6. The large language model-driven ODS layer full-process data resource evaluation and modeling system according to any one of claims 1-5, characterized in that, The data input module is specifically used for: The operation and maintenance management system monitors data definition language changes in real time, captures multiple newly added business tables, and obtains the table structure of the newly added business tables. Based on iteration version information, retrieve a candidate list of new business tables corresponding to the target iteration version; Based on the candidate list, the table metadata of the new business table corresponding to the target iteration version is found through the data catalog system.

7. The large language model-driven ODS layer full-process data resource evaluation and modeling system according to claim 6, characterized in that, The historical evaluation records include historical evaluation records of historical business tables that are similar to the database, table name, or table description in the table metadata of the newly added business table. The modeling and evaluation module is specifically used for: Extract the source database name, table name, table description, and field details of the table metadata of the newly added business table; The system architecture and business domain information of the source database for obtaining the table metadata of the newly added business table; Retrieve resource classification rules from the preset resource classification knowledge base; The large language model is invoked to search for historical evaluation records based on the table name, table description, field details, system architecture, and business domain information of the newly added business table; Based on the historical assessment records and resource classification rules, a comprehensive assessment is conducted to generate resource classification results and modeling assessment conclusions.

8. A large language model-driven method for full-process data resource evaluation and modeling at the ODS layer, characterized in that, include: Real-time capture of newly added business tables corresponding to the target iteration version, and acquisition of table structure and table metadata of the newly added business tables; The system calls upon a large language model to integrate historical evaluation records with the resource classification rules contained in a preset resource classification knowledge base, evaluates the data resources contained in the newly added business table, and outputs resource classification results and modeling evaluation conclusions. When the modeling evaluation conclusion indicates that the data resources of the newly added business table need to be modeled, the large language model is invoked to generate field descriptions for each field contained in the newly added business table; wherein, when the field is an enumerated field, the large language model is invoked to analyze the business source code related to the enumerated field and generate the field description of the enumerated field. Based on the table structure, table metadata, and field descriptions of the newly added business table, a data resource model is generated at the ODS layer.

9. The method for full-process data resource evaluation and modeling of the ODS layer driven by a large language model according to claim 8, characterized in that, The process of calling the large language model to analyze the business source code related to the enumeration fields and generating field descriptions for the enumeration fields includes: Find the version control information corresponding to the enumerated fields; Based on the version control information, obtain the code information, entity class object information, and table creation metadata information of the operation class logic layer; The large language model is invoked to generate annotation information based on the code information, the entity class object information, and the table creation metadata information, and the annotation information is merged to obtain the field description of the enumerated field.

10. The method for full-process data resource evaluation and modeling of the ODS layer driven by a large language model according to claim 8, characterized in that, The fields also include ordinary fields; The large language model-driven ODS layer full-process data resource evaluation and modeling method also includes: When the field is a regular field, the field name is obtained from the table metadata, and the field name is split and divided to obtain multiple tokens; By combining the resource classification knowledge base, the large language model is called to determine whether each word element belongs to the standard word root. If it does not belong to the standard word root, the standard word root is used to replace it to obtain the standard word element sequence. After translating the standard lexical sequence, the large language model is invoked to output ordinary field descriptions in natural language based on Chinese language rules.

11. The method for full-process data resource evaluation and modeling of the ODS layer driven by a large language model according to claim 10, characterized in that, After the large language model is invoked to output ordinary field descriptions in natural language form based on Chinese language rules, the following is also included: The generated ordinary field description is compared and evaluated with the original field descriptions pre-stored in the data catalog system to determine whether the similarity between the ordinary field description and the original field description is lower than a preset similarity threshold. If the similarity is lower than the preset similarity threshold, the description of the ordinary field is retained; If the similarity is higher than the preset similarity threshold, the original field description will be used to replace the ordinary field description.

12. The method for full-process data resource evaluation and modeling of the ODS layer driven by a large language model according to claim 10, characterized in that, Also includes: Call the large language model to determine whether each field in the newly added business table belongs to the standard fields; If it is not a standard field, then determine whether it is an enumerated field; If it belongs to an enumerated field, then the enumerated field will be described as a standardized field. If it is neither a standard field nor an enumerated field, then it is described as a standardized field based on the ordinary field description.

13. The method for full-process data resource evaluation and modeling of the ODS layer driven by a large language model according to any one of claims 8-12, characterized in that, The real-time capture of the newly added business tables corresponding to the target iteration version, and the acquisition of the table structure and table metadata of the newly added business tables, includes: The operation and maintenance management system monitors data definition language changes in real time, captures multiple newly added business tables, and obtains the table structure of the newly added business tables. Based on iteration version information, retrieve a candidate list of new business tables corresponding to the target iteration version; Based on the candidate list, the table metadata of the new business table corresponding to the target iteration version is found through the data catalog system.

14. The method for full-process data resource evaluation and modeling of the ODS layer driven by a large language model according to claim 13, characterized in that, The historical evaluation records include historical evaluation records of historical business tables that are similar to the database, table name, or table description in the table metadata of the newly added business table. The process of invoking a large language model to integrate historical evaluation records with the resource classification rules contained in a preset resource classification knowledge base, evaluating the data resources contained in the newly added business table, and outputting resource classification results and modeling evaluation conclusions includes: Extract the source database name, table name, table description, and field details of the table metadata of the newly added business table; The system architecture and business domain information of the source database for obtaining the table metadata of the newly added business table; Retrieve resource classification rules from the preset resource classification knowledge base; The large language model is invoked to search for historical evaluation records based on the table name, table description, field details, system architecture, and business domain information of the newly added business table; Based on the historical assessment records and resource classification rules, a comprehensive assessment is conducted to generate resource classification results and modeling assessment conclusions.

15. A large language model-driven end-to-end data resource evaluation and modeling device for the ODS layer, characterized in that, The large language model-driven ODS layer full-process data resource evaluation and modeling device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the large language model-driven ODS layer full-process data resource evaluation and modeling device to perform the steps of the large language model-driven ODS layer full-process data resource evaluation and modeling method as described in any one of claims 8-14.

16. A computer-readable storage medium storing a computer program / instructions thereon, characterized in that, When the program / instruction is executed by the processor, it implements the steps of the large language model-driven ODS layer full-process data resource evaluation and modeling method as described in any one of claims 8-14.

17. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the ODS layer full-process data resource evaluation and modeling method driven by the large language model as described in any one of claims 8-14 are implemented.

Images