A Method and System for Automatic Modeling of Digital Main Lines Based on Large Language Model
By using a digital mainline automatic modeling method based on a large language model, and leveraging NLP embedding models and large language models for data vectorization and SQL generation, the inefficiency of traditional data integration and modeling methods is solved, achieving efficient and accurate data integration and modeling.
Patent Information
- Application Number
- CN202511032779.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-25
- Publication Date
- 2025-10-31
- Estimated Expiration
- 2045-07-25
AI Technical Summary
Traditional data integration and modeling methods face problems such as large differences in data formats and encoding, time-consuming and labor-intensive manual matching, and low modeling efficiency, which are particularly difficult to integrate and utilize efficiently in the digital transformation of enterprises with massive amounts of data.
We adopt an automatic modeling method for digital main lines based on a large language model. We use NLP embedding models to vectorize database metadata, intelligently match similar fields in the vector database through cosine similarity calculation, and automatically generate SQL statements for data integration and modeling using a large language model.
It achieves efficient and accurate data field matching, reduces manual intervention, improves data integration efficiency and accuracy, simplifies data migration steps, and lowers the technical threshold.
Smart Images

Figure CN120523828B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer science and technology, and in particular to an automatic modeling method and system for digital main lines based on large language models. Background Technology
[0002] A digital thread is an information framework that connects data flows, enabling the efficient integration of various data throughout the product lifecycle to form a unified digital model spanning the entire lifecycle. For enterprises to use these digital models, they need to extract data from their existing business databases and populate the models—a process called modeling. In today's wave of digital transformation, enterprises and organizations have accumulated massive amounts of data, scattered across different databases. Effective integration of this data is crucial to building a digital thread, thereby achieving efficient data utilization and value extraction.
[0003] However, traditional data integration and modeling methods face many challenges. Databases from different sources differ in data format, encoding, and data type, making direct data interaction and integration very difficult. During data integration, determining the correspondence between data fields in different databases usually relies on manual work, which is both time-consuming and labor-intensive. The matching relationship between data tables and data fields needs to be searched, identified, and verified one by one. Traditional data modeling methods often require professional data engineers and analysts to spend a lot of time on data cleaning, schema design, relationship definition, and other tasks, resulting in low modeling efficiency. Summary of the Invention
[0004] To address the aforementioned issues, this invention proposes an automatic modeling method and system for digital main lines based on a large language model.
[0005] To achieve the above objectives, the following technical solution was adopted:
[0006] A method and system for automatic modeling of digital main lines based on a large language model, wherein the automatic modeling system for digital main lines based on a large language model includes a database system, an embedding model, a vector database, a matching processing unit, a front-end interface, a prompt word generation module, and a large language model interface.
[0007] The database system described herein is used to store data from standard databases and business databases, and to perform data extraction, standardization processing, and execution of SQL statements.
[0008] The embedding model is an NLP embedding model, which is an artificial intelligence model trained on massive amounts of text data and capable of vectorizing text information.
[0009] The vector database is used to store vectorized and mapped standard database metadata, which is then used for searching the vectorized fields to be matched during the business metadata matching process.
[0010] The matching processing unit is used to search the vectorized fields to be matched in the vector database, sort the search results, select candidate tables and candidate fields, and filter the candidate fields.
[0011] The front-end interface is used to present the candidate matching relationships obtained by the matching processing unit and to receive confirmation information input by the user.
[0012] The prompt word generation module is used to convert data matching relationships into prompt word text.
[0013] The large language model interface is used to call the large language model to generate SQL statements for data transfer.
[0014] The automatic modeling method for digital main lines based on large language models includes the following steps:
[0015] S1: Extract data from the standard database and the database to be matched, and convert the format;
[0016] The database system extracts structured metadata information from the standard database and the database to be matched.
[0017] The database system performs standardized parsing on the extracted metadata information and then returns a metadata result set. This system receives and parses the metadata result set returned by the database system, and organizes the parsed metadata information into a standardized format according to predefined rules and structures.
[0018] The standardized format is a standardized, machine-readable data exchange format.
[0019] The core metadata information extracted from the database system includes the database name, database description, table name, table description, and data field names. The extraction method includes the following steps:
[0020] S11: Standard interface for connecting to the database. This system connects and interacts through the standard access interface provided by the database system.
[0021] S12 queries the system directory. This system sends predefined metadata query statements that conform to the SQL standard database language to the database.
[0022] S13: Retrieve metadata result set. After the database system executes the query, it will return a result set containing metadata information.
[0023] S2: Use an embedding model to vectorize and store data information from a standard database in a vector database;
[0024] An embedding model is introduced into the system. The embedding model vectorizes the text fields of the standardized database and maps the vectorized fields to the vector database.
[0025] The embedding model, through training, enables computers to quantify the semantic similarity between texts using mathematical calculations. Embedding models typically determine vector similarity by calculating cosine similarity, using the following formula: in, A vectorized representation of standard database fields. A is a vectorized representation of the database fields to be matched. i B i is the component of the vector in the i-th dimension, n is the vector dimension of the embedding model, and i is the index of the vector dimension, which is used to ensure that the formula can align the two vectors by dimension. The calculated result is in the interval [-1, 1]. The larger the value, the higher the similarity.
[0026] In this system, the embedding model uses a two-stage embedding method to vectorize the fields in the standardized database. The two embeddings produce two vector representations, each stored separately in two vector databases: a standard table vector database and a standard field vector database. The steps of the two-stage embedding are as follows:
[0027] S21: First embedding: In order to vectorize the context information at the table level in the standard database, the name of the data table to which the target field belongs and the description information of the data table are concatenated and combined, and then the whole is input into the embedding model.
[0028] S22: The first embedding output result. The embedding model generates a table-level semantic vector, which contains the overall business background and context information of the data table to which the field belongs;
[0029] S23: The second embedding involves vectorizing the field information in the standard database by concatenating and combining the field name and its description information, and then inputting the whole into the embedding model.
[0030] S24: The second embedding output result. The embedding model generates a field semantic vector, which directly represents all the information of the field.
[0031] S3: Vectorize the business database data information to be matched, search for similar fields in the vector database, and select candidate fields;
[0032] The embedding model vectorizes the fields in the business database to be matched, and then the matching processing unit performs a second matching on the vectorized fields. The vectorized fields are then used to search the vector database to obtain candidate data tables and candidate fields. The candidate fields are then further filtered to obtain the final matching result of the fields in the business database to be matched.
[0033] In the secondary matching process, the user pre-sets the number of candidate tables 'a' and the number of candidate fields 'b'. The vectorized fields to be matched obtained from the secondary embedding in step S3 are compared with the vectorized fields mapped in the vector database in step S2. Cosine similarity is calculated, and the results are sorted according to the similarity scores. The results are then returned. The steps of the secondary matching are as follows:
[0034] S31: Table vector search, which uses vectors generated from table information to search in a standard table vector database, i.e., to perform matching between tables;
[0035] S32: Obtain the candidate table, calculate the cosine similarity between the candidate table and the data table to which the field to be matched belongs, sort them according to the similarity score, and select the a database tables with the highest similarity scores as candidate data tables.
[0036] S33: Field vector search, using vectors generated from field information to search in a standard field vector database;
[0037] S34: Obtain candidate fields, calculate the cosine similarity between candidate fields and fields to be matched, sort them according to similarity scores, and select the b data fields with the highest similarity scores as candidate fields.
[0038] After obtaining the candidate table and candidate fields, further filtering is performed. Each candidate field is traversed. If the data table to which the candidate field belongs is a candidate table, the next candidate field is traversed. Otherwise, the candidate field is removed from the results. Finally, the final matching result of the field to be matched is obtained.
[0039] S4: The user checks the similarity fields of the candidates to confirm the mapping relationship;
[0040] The program loops through each field in the database to be matched, executing S3 to obtain a set of candidate matching relationships. Each field to be matched corresponds to 0 to b candidate fields. The matching processing unit of this system saves the matching relationships in the database system and displays them to the user through the front-end interface. The user performs a final check based on their own knowledge and finally determines the mapping matching relationship.
[0041] Furthermore, the matching relationships are saved in a standardized format, and all the matching relationships together form a list.
[0042] If there are no candidate fields or the candidate fields are all incorrect, it means that there are fields in the business database and the standard database that cannot be matched. Leave the result blank or enter the correct comments or field names according to specific business requirements.
[0043] S5: Input the data matching relationship into the large language model to generate data transfer SQL;
[0044] Once the final data mapping relationship is determined, the user directly converts the determined data matching relationship into text through the prompt word generation module, inserts prompt words, and then directly inputs the prompt words into the large language model through the large language model interface. The large language model uses its own programming tools to write SQL statements based on the text information and returns them to the system.
[0045] The prompts include task descriptions, biographical examples, and text emphasizing key details.
[0046] S6: Execute SQL to achieve data integration and automatic modeling;
[0047] This system directly inputs the SQL statements returned by the large language model in S5 into the database system. The database system then performs data operations on the SQL statements to achieve data integration and model creation.
[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0049] The database system of this system can automatically standardize the data extracted from the standard database and the business database according to preset rules, unify the data format, encoding and type, lay a solid foundation for subsequent data integration and modeling, and effectively eliminate data heterogeneity barriers.
[0050] This system incorporates an NLP embedding model to vectorize database metadata. By calculating vector similarity scores, it can intelligently and efficiently search and match similar data fields in the vector database. The matching processing unit sorts the results according to the similarity scores, selects the most similar vector tables and fields as candidates, and filters them to ensure accurate matching. This automated matching significantly reduces manual intervention, both reducing the workload of manual matching and avoiding errors caused by human negligence.
[0051] This invention significantly reduces the workload of manual field matching and improves data integration efficiency by utilizing an automated semantic matching method. By embedding fields semantically using an embedding model and combining this with vector database lookup, synonymous fields can be quickly identified, accelerating the data integration process.
[0052] Compared to traditional simple string matching, this invention's field semantic-based matching method achieves a breakthrough from form to connotation, significantly improving matching accuracy. This method transforms fields into vector form and measures similarity by calculating the distance between vectors. This quantitative evaluation method provides a precise and intuitive basis for judging field similarity.
[0053] This invention utilizes a large-scale language model to automatically generate the SQL statements required for data migration, greatly simplifying the data migration process and lowering the technical threshold. Automated SQL statement generation reduces the probability of human error and lightens the burden on developers. Attached Figure Description
[0054] Figure 1 This is a flowchart illustrating the steps of the automatic digital mainline modeling method and system based on a large language model according to the present invention.
[0055] Figure 2 This is a flowchart of the data extraction process in step S1 of the present invention, which is based on a large language model and is an automatic digital mainline modeling method and system.
[0056] Figure 3 This is a flowchart of the secondary embedding process in step S2 of the present invention, which is based on a large language model and is an automatic modeling method and system for digital main lines.
[0057] Figure 4 This is a flowchart of the secondary matching process in step S3 of the present invention, which is based on a large language model and is an automatic modeling method and system for digital main lines. Detailed Implementation
[0058] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0059] An automatic modeling system for digital main lines based on a large language model includes a database system, an embedding model, a vector database, a matching processing unit, a front-end interface, a prompt word generation module, and a large language model interface;
[0060] The database system described herein is used to store data from standard databases and business databases, and to perform data extraction, standardization processing, and execution of SQL statements.
[0061] The embedding model is an NLP embedding model, which is an artificial intelligence model trained on massive amounts of text data and capable of vectorizing text information.
[0062] The vector database is used to store vectorized and mapped standard database metadata, which is then used for searching the vectorized fields to be matched during the business metadata matching process.
[0063] The matching processing unit is used to search the vectorized fields to be matched in the vector database, sort the search results, select candidate tables and candidate fields, and filter the candidate fields.
[0064] The front-end interface is used to present the candidate matching relationships obtained by the matching processing unit and to receive confirmation information input by the user.
[0065] The prompt word generation module is used to convert data matching relationships into prompt word text.
[0066] The large language model interface is used to call the large language model to generate SQL statements for data transfer.
[0067] like Figure 1 As shown, an automatic modeling method for digital main lines based on large language models includes the following steps:
[0068] S1: Extract data from the standard database and the database to be matched, and convert the format;
[0069] The database system extracts structured metadata information from the standard database and the database to be matched.
[0070] The database system performs standardized parsing on the extracted metadata information and then returns a metadata result set. This system receives and parses the metadata result set returned by the database system, and organizes the parsed metadata information into a standardized format according to predefined rules and structures.
[0071] A standardized format is a standardized, machine-readable data exchange format.
[0072] The database system will extract the following five core metadata information:
[0073] Database name: A unique name that identifies a specific database instance.
[0074] Database description: A textual description of the overall functionality, stored content, or business domain of the database.
[0075] Table name: The unique identifier for each specific table within the database.
[0076] Table description: A textual description of the data content, purpose, or business meaning contained in a single table.
[0077] Data field name: The unique identifier name of each specific field (column) in the data table.
[0078] like Figure 2 As shown, the metadata extraction method includes the following steps:
[0079] S11: Standard interface for connecting to the database. This system connects and interacts through the standard access interface provided by the database system.
[0080] S12 queries the system directory. This system sends predefined metadata query statements that conform to the SQL standard or a specific database language to the database.
[0081] S13: Retrieve metadata result set. After the database system executes the query, it will return a result set containing metadata information.
[0082] Furthermore, the standardized output format used in this extraction method is JSON.
[0083] S2: Use an embedding model to vectorize and store data information from a standard database in a vector database;
[0084] An embedding model is introduced into the system. This model vectorizes the text fields in the standardized database and maps the vectorized fields to a vector database. The core objective of introducing the embedding model is to transform textual information (table name, table description, field name, field description) into numerical vectors with semantic representation capabilities. This process is called "vectorization" or "embedding".
[0085] The fundamental purpose of vectorization is to enable computers to more accurately identify data fields from different data sources that have the same actual meaning but different expressions, based on semantics rather than just literal names, thus laying a solid foundation for subsequent automated field matching.
[0086] Embedding models possess a key characteristic: semantically similar text segments, after transformation by the model, exhibit high similarity in their corresponding vectors within a high-dimensional space; conversely, texts with significant semantic differences show low vector similarity. This consistency between semantic and vector similarity is acquired through training, enabling computers to quantify the semantic similarity between texts mathematically. Embedding models typically determine vector similarity by calculating cosine similarity, using the following formula: in, A vectorized representation of standard database fields. A is a vectorized representation of the database fields to be matched. i B i is the component of the vector in the i-th dimension, n is the vector dimension of the embedding model, and i is the index of the vector dimension, which is used to ensure that the formula can align the two vectors by dimension. The calculated result is in the interval [-1, 1]. The larger the value, the higher the similarity.
[0087] To capture the semantic information of fields more comprehensively and accurately, especially to address the common issues of ambiguous field names and diverse expressions in database environments, the embedding model in this system uses secondary embedding to vectorize fields in the standardized database, vectorizing information at different levels.
[0088] like Figure 3 As shown, the steps of secondary embedding are as follows:
[0089] S21: First embedding: In order to vectorize the context information at the table level in the standard database, the name of the data table to which the target field belongs and the description information of the data table are concatenated and combined, and then the whole is input into the embedding model.
[0090] S22: The first embedding output result. The embedding model generates a table-level semantic vector, which contains the overall business background and context information of the data table in which the field is located.
[0091] S23: The second embedding involves vectorizing the field information in the standard database by concatenating and combining the field name and its description information, and then inputting the whole into the embedding model.
[0092] S24: The second embedding output result. The embedding model generates a field semantic vector, which directly represents all the information of the field.
[0093] The purpose and function of the first embedding is to capture the "environmental semantics" of a field. The meaning of the same field in different tables will vary depending on the description of the table in which it is located. This helps to distinguish fields with the same name but different actual meanings in different tables. The purpose and function of the second embedding is to capture the semantic information of the field itself. Different fields may have the same or similar semantic information. This helps to identify which fields have different names but highly consistent core semantics.
[0094] The two embeddings generate two vector representations, each stored separately in two vector databases: a standard table vector database and a standard field vector database. This method of storing data in two separate vector databases not only improves the accuracy of field matching but also enhances the robustness of the system, especially when dealing with large-scale databases with multiple similar fields.
[0095] S3: Vectorize the business database data information to be matched, search for similar fields in the vector database, and select candidate fields;
[0096] The embedded model vectorizes the fields in the business database to be matched, and then the matching processing unit performs a second matching on the vectorized fields. The vectorized fields are then used to search the vector database to obtain candidate data tables and candidate fields. The candidate fields are then further filtered to obtain the final matching result of the fields in the business database to be matched.
[0097] During the secondary matching process, the user pre-sets the number of candidate tables 'a' and the number of candidate fields 'b'. The vectorized fields obtained from the secondary embedding in step S3 are compared with the vectorized fields mapped in the vector database in step S2. The results are then sorted according to the similarity scores and returned.
[0098] like Figure 4 As shown, the steps of quadratic matching are:
[0099] S31: Table vector search, which uses vectors generated from table information to search in a standard table vector database, i.e., to perform matching between tables;
[0100] S32: Obtain the candidate table, calculate the cosine similarity between the candidate table and the data table to which the field to be matched belongs, sort them according to the similarity score, and select the a database tables with the highest similarity scores as candidate data tables.
[0101] S33: Field vector search, using vectors generated from field information to search in a standard field vector database;
[0102] S34: Obtain candidate fields, calculate the cosine similarity between candidate fields and fields to be matched, sort them according to similarity scores, and select the b data fields with the highest similarity scores as candidate fields.
[0103] After obtaining the candidate tables and candidate fields, further filtering is performed. Each candidate field is iterated over; if the data table to which the candidate field belongs is also a candidate table, the process continues to the next candidate field; otherwise, the candidate field is removed from the results. Following this removal rule, b fields are filtered out, ultimately resulting in 0 to b candidate fields, which serve as the final matching results for the fields to be matched. This step ensures that the field information and table information are semantically matched, preventing situations where the same field exists in unrelated tables and can still be matched.
[0104] S4: The user checks the similarity fields of the candidates to confirm the mapping relationship;
[0105] The program loops through each field in the database to be matched, executing S3 to obtain a set of candidate matching relationships. Each field to be matched corresponds to 0 to b candidate fields. The matching processing unit of this system saves the matching relationships in the database system and displays them to the user through the front-end interface. The user performs a final check based on their own knowledge and finally determines the mapping matching relationship.
[0106] Specifically, the front-end interface displays the fields to be matched and the filtered candidate fields. Then, the user selects the fields as if taking a multiple-choice test to determine the final matching relationship, and then the final matching relationship is saved in the database.
[0107] Furthermore, the matching relationships are saved in the standard JSON format. Each JSON object stores a set of matching relationship data (including standard library data tables, data fields, matching business data tables, data fields, and similarity scores). All the matching relationship JSONs together form a list.
[0108] If the final mapping match has no candidate fields or the candidate fields are all incorrect, it means that there are fields in the business database and the standard database that cannot be matched. The user should leave the result blank and enter the correct comments or field names according to the specific business requirements.
[0109] S5: Input the data matching relationship into the large language model to generate data transfer SQL;
[0110] Once the final data mapping relationship is determined, the user directly converts the determined data matching relationship into text through the prompt word generation module, inserts prompt words, and then directly inputs the prompt words into the large language model through the large language model interface. The large language model uses its own programming tools to write SQL statements based on the text information and returns them to the system.
[0111] The prompts include task descriptions, examples of people involved, and text emphasizing key details.
[0112] S6: Execute SQL to achieve data integration and automatic modeling;
[0113] This system directly inputs the SQL statements returned by the large language model in S5 into the database system. The database system then performs data operations on the SQL statements to achieve data integration and model creation.
[0114] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Those skilled in the art may find other optimizations and additional functions in this application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. An automatic modeling method for digital main lines based on large language models, characterized in that: Includes the following steps: S1: Extract data from the standard database and the database to be matched, and convert the format; The database system extracts structured metadata information from the standard database and the database to be matched; The database system performs standardized parsing on the extracted metadata information, converting the metadata information into a standardized format; The standardized format is a standardized, machine-readable data exchange format; S2: Use an embedding model to vectorize and store data information from a standard database in a vector database; The vectorization of fields in the standardized database through the embedding model adopts a two-stage embedding. The two embeddings produce two vector representations, and each vector representation is stored in two dedicated vector databases: the standard table vector database and the standard field vector database. The steps of the secondary embedding are as follows: S21: First embedding: In order to vectorize the context information at the table level in the standard database, the name of the data table to which the target field belongs and the description information of the data table are concatenated and combined, and then the whole is input into the embedding model. S22: The first embedding output result. The embedding model generates a table-level semantic vector, which contains the overall business background and context information of the data table in which the field is located. S23: The second embedding involves vectorizing the field information in the standard database by concatenating and combining the field name and its description information, and then inputting the whole into the embedding model. S24: The second embedding output result. The embedding model generates a field semantic vector, which directly represents all the information of the field. S3: Vectorize the business database data information to be matched, search for similar fields in the vector database, and select candidate fields; The user pre-sets the number of candidate tables 'a' and the number of candidate fields 'b'. The embedded model vectorizes the fields in the business database to be matched. Then, the matching processing unit performs a second matching on the vectorized fields. The vectorized fields are used to search the vector database to obtain candidate data tables and candidate fields. The candidate fields are then further filtered to obtain the final matching result of the field to be matched. The steps of the secondary matching are as follows: S31: Table vector search, which uses vectors generated from table information to search in a standard table vector database, i.e., to perform matching between tables; S32: Obtain the candidate table, calculate the cosine similarity between the candidate table and the data table to which the field to be matched belongs, sort them according to the similarity score, and select the a database tables with the highest similarity scores as candidate data tables. S33: Field vector search, using vectors generated from field information to search in a standard field vector database; S34: Obtain candidate fields, calculate the cosine similarity between candidate fields and fields to be matched, sort them according to similarity scores, and select the b data fields with the highest similarity scores as candidate fields; S4: The user checks the similarity fields of the candidates to confirm the mapping relationship; The program loops and executes S3 for each field to be matched in the database to obtain a set of candidate matching relationships. That is, each field to be matched corresponds to 0 to b candidate fields. The matching processing unit saves the matching relationship in the database system and displays it to the user through the front-end interface. The user makes a final check based on their own knowledge and finally determines the mapping matching relationship. If there are no candidate fields or the candidate fields are all incorrect in the front-end interface, it means that there are fields that cannot be matched between the business database and the standard database. Leave the result blank or enter the correct comments or field names according to the specific business requirements. S5: Input the data matching relationship into the large language model to generate data transfer SQL; Once the final data mapping relationship is determined, the user directly converts the determined data matching relationship into text through the prompt word generation module, inserts prompt words, and then directly inputs the prompt words into the large language model through the large language model interface. The large language model uses its own programming tools to write SQL statements based on the text information and returns them. S6: Execute SQL to achieve data integration and automatic modeling; The SQL statements returned by the large language model in S5 can be directly input into the database system, and the database system can perform data operations on the SQL statements to achieve data integration and model creation.
2. The automatic modeling method for digital main lines based on a large language model as described in claim 1, characterized in that: In step S1, the core metadata information extracted by the database system includes the database name, database description, table name, table description, and data field name. The extraction method includes the following steps: S11: Connect to the standard database interface, and connect and interact through the standard access interface provided by the database system; S12: Query the database system catalog and send predefined metadata query statements that conform to the SQL standard or a specific database language to the database; S13: Retrieve metadata result set. After the database system executes the query, it will return a result set containing metadata information.
3. The automatic modeling method for digital main lines based on large language models as described in claim 2, characterized in that: It receives and parses the metadata result set returned by the database system, and organizes the parsed metadata information into a standardized format according to predefined rules and structures.
4. The automatic modeling method for digital main lines based on large language models as described in claim 1, characterized in that: Embedded models can determine vector similarity by calculating cosine similarity. The formula is as follows: in, A vectorized representation of standard database fields. A is a vectorized representation of the database fields to be matched. i B i is the component of the vector in the i-th dimension, n is the vector dimension of the embedding model, and i is the index of the vector dimension, which is used to ensure that the formula can align the two vectors by dimension. The calculated result is in the interval [-1, 1]. The larger the value, the higher the similarity.
5. The automatic modeling method for digital main lines based on a large language model as described in claim 1, characterized in that: In step S3, during the secondary matching process, the vectorized field obtained from the secondary embedding in step S3 is compared with the vectorized field mapped in the vector database in step S2. The results are then sorted according to the similarity scores and returned.
6. The automatic modeling method for digital main lines based on large language models as described in claim 1, characterized in that: In step S3, after obtaining the candidate table and candidate fields, further filtering is performed. Each candidate field is traversed. If the data table to which the candidate field belongs is a candidate table, the next candidate field is traversed. Otherwise, the candidate field is removed from the results, and the final matching result of the matching field is finally obtained.
7. The automatic modeling method for digital main lines based on large language models as described in claim 1, characterized in that: In step S4, the matching relationships are saved in a standardized format, and all the matching relationships together form a list.
8. The automatic modeling method for digital main lines based on large language models as described in claim 1, characterized in that: In step S5, the prompts include task descriptions, examples of people, and text emphasizing key details.
9. An automatic digital mainline modeling system based on a large language model that implements the method of any one of claims 1-8, characterized in that: The automatic modeling system for digital main lines based on a large language model includes a database system, an embedded model, a vector database, a matching processing unit, a front-end interface, a prompt word generation module, and a large language model interface. The database system described herein is used to store data from standard databases and business databases, and to perform data extraction, standardization processing, and execution of SQL statements. The embedding model is an NLP embedding model, which is an artificial intelligence model trained on massive amounts of text data and is capable of vectorizing text information. The vector database is used to store vectorized and mapped standard database metadata, which is used for searching the vectorized fields to be matched during the business metadata matching process. The matching processing unit is used to search the vectorized fields to be matched in the vector database, sort the search results, select candidate tables and candidate fields, and filter the candidate fields. The front-end interface is used to present the candidate matching relationships obtained by the matching processing unit and to receive confirmation information input by the user. The prompt word generation module is used to convert data matching relationships into prompt word text; The large language model interface is used to call the large language model to generate SQL statements for data transfer.
Citation Information
Patent Citations
Method and computer program product for automated modeling of structure of scheme phase
CN118504298A
Database operation statement generation method and device, electronic equipment and medium
CN119884145A