A method and device for using a large model to match a database in the data governance industry

Abstract

The application provides a method and device for matching a database using a large model in the data governance industry, which can improve database query efficiency. The method for matching a database using a large model in the data governance industry comprises the following steps: extracting features of a target database and table information under the target database; performing vectorization processing on the features of the target database and the table information under the target database; based on the features of the target database and the table information under the target database after the vectorization processing, establishing a library vector library and a table vector library; performing vectorization processing on query information to obtain a vectorized query vector result; based on the similarity between the query vector result and the library vector library, determining a target database corresponding to a library vector library to be queried; calculating the similarity between the query vector result and a table vector library under the target database corresponding to the library vector library to be queried, so as to determine a table vector library to be queried; and based on the table information corresponding to the table vector library to be queried, outputting a query result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database query, in particular to a method and device for matching a database using a large model in the data governance industry. BACKGROUND

[0002] With the rapid development of big data and artificial intelligence technology, data has become the core resource of enterprise operation and decision-making. As a key link to ensure data quality, security and compliance, data governance is becoming increasingly important. As the foundation of the development of the big data industry, databases have evolved from simple data storage to complex data management systems. In the big data environment, a database may contain multiple databases and tables, and the amount and complexity of data in the database are large. Traditional matching methods often require a lot of time for traversal and comparison, making it difficult to quickly and accurately match the correct database and table from multiple databases and tables. SUMMARY

[0003] To solve the above technical problems, the present application is proposed. The embodiments of the present application provide a method and device for matching a database using a large model in the data governance industry, which can improve the database query efficiency.

[0004] According to one aspect of the present application, a method for matching a database using a large model in the data governance industry is provided, comprising: extracting features of a target database and table information under the target database; performing vectorization processing on the features of the target database and the table information under the target database; based on the vectorization-processed features of the target database and the table information under the target database, establishing a database vector library and a table vector library; performing vectorization processing on query information to obtain a vectorized query vector result; based on the similarity between the query vector result and the database vector library, determining the target database corresponding to the database vector library to be queried; calculating the similarity between the query vector result and the table vector library under the target database corresponding to the database vector library to be queried, to determine the table vector library to be queried; and based on the table information corresponding to the table vector library to be queried, outputting a query result.

[0005] In an embodiment, based on the similarity between the query vector result and the database vector library, the target database corresponding to the database vector library to be queried is determined, comprising: for each target database, calculating the cosine similarity between the query vector result and the database vector library; wherein the closer the value of the cosine similarity is to 1, the more similar the query vector result and the database vector library are, and the closer the value of the cosine similarity is to -1, the less similar the query vector result and the database vector library are; selecting the record with the smallest absolute value of cosine distance to obtain a selected database result; and determining the corresponding target database based on the selected database result.

[0006] In an embodiment, after obtaining the selected result by selecting the record with the minimum absolute value of cosine distance, the method for matching a database using a large model in the data governance industry comprises: based on the selected result, obtaining a table vector library to be queried under the corresponding target database; wherein the similarity between the query vector result and the table vector library to be queried under the corresponding target database is calculated to determine the table vector library to be queried, comprising: using the query vector result as a condition to retrieve the table vector library to be queried; based on the cosine similarity algorithm, a similarity list of the query vector result and the table vector library is obtained; wherein the elements in the similarity list include the similarity distance between the table information in the table vector library and the query vector result; all similarity distances in the similarity list are sorted; when there is a similarity distance greater than or equal to a preset threshold in the table vector library to be queried, and the number of similarity distances greater than or equal to the preset threshold is greater than a preset number, the table vector library to be queried is recorded in the first selected table result.

[0007] In an embodiment, the method for matching a database using a large model in the data governance industry further comprises: based on the foreign key relationship, establishing an association relationship between tables and tables; the primary table and the corresponding foreign key associated table are sorted as a key-value pair for storage.

[0008] In an embodiment, the method for matching a database using a large model in the data governance industry further comprises: when the query information exists in multiple table joint queries, obtaining the foreign key associated table associated with the table vector library to be queried; recording the foreign key associated table in the first selected table result to form a second selected table result; wherein based on the table information corresponding to the table vector library to be queried, the query result is output, comprising: based on the first selected table result and the second selected table result, the query result is output.

[0009] In an embodiment, the features of the target database include name and remarks; the vectorization processing of the features of the target database and the table information under the target database comprises: reading the name of the target database and its remarks to form a first description text; reading the table information under the target database; extracting the content features of the table information and forming a second description text; splicing the first description text and the second description text to form a spliced text content, and performing vectorization processing on the spliced text content; wherein based on the vectorization-processed features of the target database and the table information under the target database, the library vector library and the table vector library are established, comprising: using the vectorization-processed spliced text content as an index to construct the library vector library.

[0010] In an embodiment, the features of the target database and the table information under the target database are subjected to vectorization processing, including: extracting detailed content of the table information under the target database; splicing and correcting the detailed content to obtain a third corrected description text; and subjecting the third description text to vectorization processing to generate a vector result and establish an index; wherein, based on the features of the target database and the table information under the target database subjected to vectorization processing, a library vector library and a table vector library are established, including: based on the vector result, a table vector library is established; wherein each feature of the target database corresponds to a table vector library, and each table vector library corresponds to at least one data table.

[0011] In an embodiment, the detailed content includes: table name, table note, column name and column note; wherein, the detailed content is spliced and corrected to obtain a third corrected description text, including: the table name, the table note, the column name and the column note are spliced into a preliminary text; the preliminary text is cleaned and the data format is cropped to obtain the third corrected description text.

[0012] In an embodiment, the query information is subjected to vectorization processing to obtain a vectorized query vector result, including: based on a large language model, extracting key elements in the natural language of the query information; wherein, the key elements include time, place and query object; the extracted key elements are spliced into a first query text; the punctuation in the first query text is removed and the case is unified to obtain a second query text; the second query text is subjected to vectorization processing to obtain a vectorized query vector result.

[0013] According to another aspect of the present application, there is provided an apparatus for matching databases using large models in the data governance industry, including: an extraction module for extracting features of a target database and table information under the target database; a processing module for subjecting the features of the target database and the table information under the target database to vectorization processing; an establishment module for establishing a library vector library and a table vector library based on the features of the target database and the table information under the target database subjected to vectorization processing; a vectorization module for subjecting query information to vectorization processing to obtain a vectorized query vector result; a determination module for determining a target database corresponding to a library vector library to be queried based on the similarity of the query vector result and the library vector library; a calculation module for calculating the similarity of the query vector result and the table vector library under the target database corresponding to the library vector library to be queried to determine a table vector library to be queried; and an output module for outputting a query result based on the table information corresponding to the table vector library to be queried.

[0014] The application provides a method and device for matching a database using a large model in the data governance industry. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0016] Figure 1 is a flowchart of a method for matching a database using a large model in the data governance industry according to an example embodiment of the present application.

[0017] Figure 2 is a structural diagram of a data structure according to an example embodiment of the present application.

[0018] Figure 3 is a structural diagram of a device for matching a database using a large model in the data governance industry according to an example embodiment of the present application. DETAILED DESCRIPTION

[0019] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent to those skilled in the art that the described embodiments are merely a portion of the embodiments of the present application and that the present application is not limited to the described embodiments.

[0020] Figure 1 is a flowchart of a method for matching a database using a large model in the data governance industry according to an example embodiment of the present application. Figure 1 As shown in

[0021] Step 100: Extracting features of a target database and table information under the target database.

[0022] The multiple databases in the environment (e.g., databases to be queried or databases to be used for querying) are set as target databases, and the features of each target database are extracted for recalling the corresponding databases in the context of multiple database retrieval.

[0023] In some embodiments, the features of the target database can include the name and its remarks, and the table information under the target database can include the table name, table remarks, column name and column remarks. By vectorizing the features, the difficulty of subsequent calculation and retrieval can be reduced, and the efficiency can be improved.

[0024] Step 200: Vectorizing the features of the target database and the table information under the target database.

[0025] The features of the target database are vectorized respectively, and all the table information under each database is vectorized, so that the database can be matched first, and then the corresponding data table can be matched. The efficiency and accuracy of matching the database are improved.

[0026] Step 300: Based on the vectorized features of the target database and the table information under the target database, a library vector library and a table vector library are established.

[0027] For example, the features of each database are extracted and processed by a large language model to form a description text, and then vectorized to construct a library vector library for database recall. The detailed information of each table in each database is extracted, preprocessed, vectorized, and indexed to establish a table vector library.

[0028] Step 400: Vectorizing the query information to obtain a vectorized query vector result.

[0029] After obtaining the query information input by the user, the key elements in the natural language of the user query can be extracted using a large language model, such as time, entity, etc., to understand the user's query intention and reduce the impact of the difference in expression or tone on the semantics of the query statement. The processed natural language is reassembled into a new query text.

[0030] Step 500: Based on the similarity between the query vector result and the library vector library, determine the target database corresponding to the library vector library to be queried.

[0031] The similarity between the query vector result and the library vector library is evaluated, and the library vector library with the highest similarity is selected as the library vector library to be queried, and the corresponding target database is recalled from the library vector library as an index.

[0032] Step 600: Calculate the similarity between the query vector result and the table vector library under the target database corresponding to the library vector library to be queried to determine the table vector library to be queried.

[0033] According to the selected library vector library, the table vector library with the highest similarity is retrieved to further determine the table result of the query.

[0034] Step 700: output the query result based on the table information corresponding to the table vector library to be queried.

[0035] According to the table information corresponding to the table vector library, the relevant table or data is found and returned, that is, the query result. By converting the database table information into a vector form and storing it in the table vector library, the system can efficiently process complex query requests and quickly return the results related to the user's needs.

[0036] In an embodiment, step 500 can include: for each target database, calculating the cosine similarity between the query vector result and the library vector library; wherein the closer the value of the cosine similarity is to 1, the more similar the query vector result is to the library vector library, and the closer the value of the cosine similarity is to -1, the less similar the query vector result is to the library vector library; selecting the record with the smallest absolute value of cosine distance to obtain the selected library result; and determining the corresponding target database based on the selected library result.

[0037] The query vector result obtained in step 400 is compared with the library vector library established in step 300 for similarity. For each database vector, the cosine similarity with the query vector result is calculated. Cosine similarity is a commonly used measurement method that evaluates the similarity of two vectors by calculating the cosine value of the included angle between them. The cosine value of a 0-degree angle is 1, while the cosine value of any other angle is not greater than 1; and its minimum value is -1. Thus, the cosine value of the angle between two vectors determines whether the two vectors point in the same direction. When two vectors have the same direction, the value of the cosine similarity is 1; when the included angle between two vectors is 90°, the value of the cosine similarity is 0; and when two vectors point in completely opposite directions, the value of the cosine similarity is -1. This result is independent of the length of the vector and only related to the pointing direction of the vector. Cosine similarity is usually used in positive space, so the value given is between -1 and 1. The closer the value is to 1, the more similar the two vectors are; the closer the value is to -1, the less similar the two vectors are. The record with the smallest absolute value of cosine distance is selected to obtain the selected library result. That is, by calculating the cosine similarity with the query vector result, the most similar target database is selected, and after the target database is determined, the data table can be selected.

[0038] In an embodiment, after obtaining the selected library result by selecting the record with the minimum absolute value of cosine distance, the method for matching the database using the large model in the data governance industry can comprise: based on the selected library result, obtaining a table vector library to be queried under the corresponding target database; wherein the step 600 can be adjusted to: using the query vector result as a condition to search the table vector library to be queried; based on the cosine similarity algorithm, obtaining a similarity list of the query vector result and the table vector library; wherein the elements in the similarity list include the similarity distance between the table information in the table vector library and the query vector result; sorting all similarity distances in the similarity list; when there is a similarity distance greater than or equal to a preset threshold in the table vector library to be queried, and the number of similarity distances greater than or equal to the preset threshold is greater than a preset number, recording the table vector library to be queried in the first selected table result.

[0039] For example, Figure 2 is a structural diagram of a data structure provided by an exemplary embodiment of the present application, referring to Figure 2 , KEY is the database name, VAL is the data table vector library, the database name includes DB1, DB2, DB3, DB4 and DB5, and the data table vector library includes Vecdb1, Vecdb2, Vecdb3, Vecdb4 and Vecdb5. If the selected library in step 500 is DB2, the table vector library Vecdb2 is obtained at this time, and the query vector result obtained in step 400 is used as a condition to search the table vector library Vecdb2. The same cosine algorithm as calculating the similarity of the library vector library can be used to calculate the similarity of the query vector result and the table vector library, and obtain a similarity list sim, which contains the similarity distance between all table information and the table ebd. All elements in the similarity list sim are sorted and a reasonable threshold is set for screening. Only the tables with a similarity greater than the threshold will be recorded in the final selected table result TBs (the first selected table result).

[0040] In an embodiment, the method for matching the database using the large model in the data governance industry can further comprise: based on the foreign key relationship, establishing an association relationship between tables; and storing the primary table and the corresponding foreign key association table as a key-value pair.

[0041] When performing a database query, especially a query involving multiple table associations, in order to ensure the completeness and accuracy of the query results, the information of other tables associated with the main query table can be supplemented by using the foreign key (FK) relationship. The foreign key information can be obtained through metadata tables (such as INFORMATION_SCHEMA) or by intercepting the Create table statement, etc. The recalled table TBs and his foreign key associated table are sorted into key-value pairs. For example, the foreign keys of table t1 are t2 and t3, and the record can be as follows: t1: [t2, t3].

[0042] In an embodiment, the method of matching a database using a large model in the data governance industry can further include: when the query information involves multiple table associations, obtaining the foreign key associated table associated with the table vector library to be queried; recording the foreign key associated table in the first selected table result to form a second selected table result; wherein the step 700 can be adjusted as follows: based on the first selected table result and the second selected table result, output the query result.

[0043] For example, by constructing a prompt question big model, the query involves multiple table associations, such as selecting the corresponding table FKs, if the table already exists in TBs, there is no need to add the table to FKs, in the selected table result TBs and the selected table result FKs, according to the actual situation or actual demand, finally output the query result. That is, in addition to single table query, multiple table association can also be established, the range of query is improved, and the query accuracy is improved.

[0044] In some embodiments, the features of the target database include name and note; the above step 200 can include: reading the name of the target database and its note to form a first description text; reading the table information under the target database; extracting the content features of the table information and forming a second description text; splicing the first description text and the second description text to form a spliced text content, and performing vectorization processing on the spliced text content; wherein the above step 300 can be adjusted as follows: using the vectorization processed spliced text content as an index to construct a library vector library.

[0045] For example, first connect to the target database through the DBMS (database management system) component, read the name of the target database and its note to form a preliminary description text d1 (first description text), then read the table information under the database, and let the big model summarize all the table information through the prompt technology to obtain the content involved in the database in the table information, form a text d2 (second description text). Splice the contents of d1 and d2 to form a text d, and perform vectorization processing on the content of text d. Use the vectorized vector as an index to construct a vector database vec_db.

[0046] As an example, embedding technology can be used to convert key data into vector form and build a vector database. The vector database can store and index a large amount of vectorized data to facilitate fast retrieval. The management and operation of the vector library can use a variety of mature solutions such as vector database systems such as Faiss and Chromadb, which are specifically designed to handle large-scale vector storage, indexing, and similarity search, and can significantly improve processing speed and accuracy to meet the complex similarity retrieval needs of modern applications. Embedding technology can use large models for vectorization processing, such as common embedding models such as text2vec, bge, bert, etc. The vector database should at least include: stored vector results as indexes, actual stored data (table structure) and traceability information of the table structure.

[0047] In an embodiment, the above step 200 can include extracting detailed content of table information under the target database; splicing and correcting the detailed content to obtain a corrected third description text; and performing vectorization processing on the third description text to generate a vector result and establish an index. The above step 300 can be adjusted to: based on the vector result, establishing a table vector library; wherein the characteristics of each target database correspond to a table vector library, and each table vector library corresponds to at least one data table.

[0048] Detailed information of each table in each database is extracted, including table name, table remark, column name and column remark, and these information is spliced into a text t. The spliced text t is preprocessed and corrected, and then the corrected text is vectorized to generate a vector result and establish an index. The saved content is the full information of the data table, such as the create table statement.

[0049] After each database is separately established to save the vector database, a hashtable data structure as shown in Figure 2 is finally constructed to store the created vector library.

[0050] In an embodiment, the detailed content includes: table name, table remark, column name and column remark; wherein the splicing and correction of the detailed content to obtain the corrected third description text includes: splicing the table name, table remark, column name and column remark into a preliminary text; and cleaning the preliminary text and cutting the data format to obtain the corrected third description text.

[0051] The detailed content is spliced and corrected, and the correction includes cleaning irrelevant content, punctuation and other information in the remarks and cutting the data format, removing all punctuation, and unifying the case, such as cutting varchar (255) into varchar, etc. After the correction is completed, the result is spliced into a new query text, i.e. the third description text.

[0052] In an embodiment, the step 400 can include: extracting key elements in the natural language of the query information based on a large language model; wherein the key elements include time, place and query object; concatenating the extracted key elements into a first query text; removing punctuation in the first query text and unifying case to obtain a second query text; and performing vectorization processing on the second query text to obtain a vectorized query vector result.

[0053] Using a large language model (LLM) to extract key elements in the natural language of the user query, such as time, entity, etc., to understand the user's query intent and reduce the impact of query sentence semantics due to differences in expression or tone. Large language models are complex neural networks trained on large amounts of data, capable of capturing and modeling the complexity and diversity of language. These models typically use a transformer architecture, which is a model based on self-attention mechanisms that can handle sequential data such as text. The self-attention mechanism allows the model to consider other words in the entire text sequence when processing a single word, better understanding the context. After concatenating the extracted key elements into the first query text, all punctuation is removed and the case is unified. Then use a large model (such as Bert, GPT, etc.) to extract and identify key information and elements, and continue to use the large model to identify entities such as time, place, query object, etc. in the first query text. Then the result is re-spliced into the second query text. The second query text is vectorized using the same embedding method / model as before to obtain the query vector result.

[0054] Figure 3 is a structural schematic diagram of an apparatus for matching databases using a large model in the data governance industry provided by an exemplary embodiment of the present application, as Figure 3 shown, the apparatus 3 for matching databases using a large model in the data governance industry includes: an extraction module 31 for extracting features of a target database and table information under the target database; a processing module 32 for vectorizing the features of the target database and the table information under the target database; an establishment module 33 for establishing a library vector library and a table vector library based on the vectorized features of the target database and the table information under the target database; a vectorization module 34 for vectorizing query information to obtain a vectorized query vector result; a determination module 35 for determining the target database corresponding to the library vector library to be queried based on the similarity between the query vector result and the library vector library; a calculation module 36 for calculating the similarity between the query vector result and the table vector library under the target database corresponding to the library vector library to be queried to determine the table vector library to be queried; and an output module 37 for outputting a query result based on the table information corresponding to the table vector library to be queried.

[0055] The application provides a device for matching databases using large models in the data governance industry. By vectorizing databases and table information, the device can quickly identify the relevance and similarity between data, significantly reducing matching time, improving overall data processing speed, and improving the accuracy and timeliness of database matching.

[0056] In an embodiment, the determination module 35 can be configured to: for each target database, calculate the cosine similarity between the query vector result and the library vector library; wherein the closer the value of the cosine similarity is to 1, the more similar the query vector result is to the library vector library, and the closer the value of the cosine similarity is to -1, the less similar the query vector result is to the library vector library; select the record with the smallest absolute value of the cosine distance to obtain the selected library result; and determine the corresponding target database based on the selected library result.

[0057] In an embodiment, the device 3 for matching databases using large models in the data governance industry can be configured to: based on the selected library result, obtain the table vector library to be queried under the corresponding target database; wherein the calculation module 36 can be correspondingly configured to: use the query vector result as a condition to search the table vector library to be queried; based on the cosine similarity algorithm, obtain a similarity list of the query vector result and the table vector library; wherein the elements in the similarity list include the similarity distance between the table information in the table vector library and the query vector result; sort all similarity distances in the similarity list; when there is a similarity distance greater than or equal to a preset threshold in the table vector library to be queried, and the number of similarity distances greater than or equal to the preset threshold is greater than a preset number, record the table vector library to be queried in the first selected table result.

[0058] In an embodiment, the device 3 for matching databases using large models in the data governance industry can be configured to: based on the foreign key relationship, establish an association relationship between tables; and store the primary table and the corresponding foreign key association table as a key-value pair.

[0059] In an embodiment, the device 3 for matching databases using large models in the data governance industry can be configured to: when the query information involves multi-table association, obtain the foreign key association table associated with the table vector library to be queried; record the foreign key association table in the first selected table result to form a second selected table result; wherein the output module 37 can be correspondingly configured to: based on the first selected table result and the second selected table result, output the query result.

[0060] In an embodiment, the features of the target database include a name and a note; the processing module 32 can be configured to: read the name of the target database and its note to form a first description text; read the table information under the target database; extract the content features of the table information and form a second description text; splice the first description text and the second description text to form spliced text content, and perform vectorization processing on the spliced text content; wherein the establishing module 33 can be configured to: take the vectorization-processed spliced text content as an index to construct a library vector library.

[0061] In an embodiment, the processing module 32 described above can be configured to: extract detailed content of the table information under the target database; splice and correct the detailed content to obtain a third description text after correction; perform vectorization processing on the third description text to generate a vector result and establish an index; wherein the establishing module 33 described above can be correspondingly configured to: establish a table vector library based on the vector result; wherein the features of each target database correspond to a table vector library, and each table vector library corresponds to at least one data table.

[0062] In an embodiment, the detailed content includes: a table name, a table note, a column name, and a column note; the processing module 32 described above can be further configured to: splice the table name, the table note, the column name, and the column note into a preliminary text; clean up the preliminary text and crop the data format to obtain the third description text after correction.

[0063] In an embodiment, the vectorization module 34 described above can be configured to: extract key elements in the natural language of the query information based on a large language model; wherein the key elements include time, place, and query object; splice the extracted key elements into a first query text; remove punctuation in the first query text and unify the case to obtain a second query text; perform vectorization processing on the second query text to obtain a vectorized query vector result.

[0064] The embodiment of the application provides a device for matching a database using a large model in a data governance industry. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware layer, in addition to CPU, memory, network interface, and non-volatile memory, the device in the embodiment can also include other hardware, such as a forwarding chip responsible for processing packets, etc. Taking software implementation as an example, as a device in a logical sense, it is formed by the CPU of the device reading the corresponding computer program instructions in the non-volatile memory into the memory and running.

[0065] According to another aspect of the application, a computer-readable storage medium is provided, which stores a computer program for executing the method of matching a database using a large model in a data governance industry according to any of the above embodiments.

[0066] In addition to the methods and apparatus described above, embodiments of the present application can also be a computer program product that includes computer program instructions operable to cause a processor to perform the steps described in the "Example Method" section above of the present specification for the methods of using large model matching databases in the data governance industry according to various embodiments of the present application.

[0067] According to another aspect of the present application, there is provided an electronic device comprising: a processor; a memory for storing processor-executable instructions; the processor for performing the methods of using large model matching databases in the data governance industry of any of the embodiments described above.

[0068] In addition, embodiments of the present application can also be a computer readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform the steps described in the "Example Method" section above of the present specification for the methods of using large model matching databases in the data governance industry according to various embodiments of the present application.

[0069] The above only shows the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the scope of protection of the present application.

Claims

1. A method of using a large model matching database in the data governance industry, characterized by, The method comprises the following steps: extracting the features of the target database and the table information under the target database; vectorizing the features of the target database and the table information under the target database; based on the vectorized features of the target database and the table information under the target database, establishing a library vector library and a table vector library; vectorizing the query information to obtain a vectorized query vector result; based on the similarity between the query vector result and the library vector library, determining the target database corresponding to the library vector library to be queried; calculating the similarity between the query vector result and the table vector library under the target database corresponding to the library vector library to be queried, to determine the table vector library to be queried; based on the table information corresponding to the table vector library to be queried, outputting a query result; based on the similarity between the query vector result and the library vector library, determining the target database corresponding to the library vector library to be queried, comprising: for each target database, calculating the cosine similarity between the query vector result and the library vector library; wherein the closer the value of the cosine similarity is to 1, the more similar the query vector result and the library vector library are, and the closer the value of the cosine similarity is to -1, the less similar the query vector result and the library vector library are; selecting the record with the smallest absolute value of cosine distance to obtain a selected library result; based on the selected library result, determining the corresponding target database; after selecting the record with the smallest absolute value of cosine distance to obtain the selected library result, the method for matching a database using a large model in the data management industry comprises: based on the selected library result, obtaining the table vector library to be queried under the corresponding target database; wherein, calculating the similarity between the query vector result and the table vector library under the target database corresponding to the library vector library to be queried, to determine the table vector library to be queried, comprises: using the query vector result as a condition to retrieve the table vector library to be queried; based on the cosine similarity algorithm, obtaining a similarity list of the query vector result and the table vector library; wherein the elements in the similarity list include the similarity distance between the table information in the table vector library and the query vector result; sorting all similarity distances in the similarity list; when there is a similarity distance greater than or equal to a preset threshold in the table vector library to be queried, and the number of similarity distances greater than or equal to the preset threshold is greater than a preset number, recording the table vector library to be queried in a first selected table result.

2. The method of matching a database using a large model in the data governance industry according to claim 1, wherein, The method for matching a database using a large model in the data management industry further comprises: based on the foreign key relationship, establishing an association relationship between tables and tables; storing the primary table and the corresponding foreign key association table as a key-value pair.

3. The method of matching a database using a large model in the data governance industry according to claim 2, wherein, The method for matching a database using a large model in the data management industry further comprises: when the query information involves multi-table joint query, obtaining the foreign key association table associated with the table vector library to be queried; recording the foreign key association table in the first selected table result to form a second selected table result; wherein, based on the table information corresponding to the table vector library to be queried, outputting a query result, comprises: based on the first selected table result and the second selected table result, outputting a query result.

4. The method of matching a database using a large model in the data governance industry according to claim 1, wherein, The characteristics of the target database include a name and a note; the characteristics of the target database and table information under the target database are vectorized, including: Reading the name of the target database and its note to form a first description text; Reading the table information under the target database; Extracting the content features of the table information and forming a second description text; Splicing the first description text and the second description text to form a spliced text content, and vectorizing the spliced text content; Wherein, based on the vectorized characteristics of the target database and the table information under the target database, a library vector library and a table vector library are established, including: Taking the vectorized spliced text content as an index, a library vector library is constructed.

5. The method of matching a database using a large model in the data governance industry according to claim 1, wherein, The characteristics of the target database and the table information under the target database are vectorized, including: Extracting the detailed content of the table information under the target database; Splicing and correcting the detailed content to obtain a corrected third description text; Vectorizing the third description text to generate a vector result and establish an index; Wherein, based on the vectorized characteristics of the target database and the table information under the target database, a library vector library and a table vector library are established, including: Based on the vector result, a table vector library is established; wherein each characteristic of the target database corresponds to a table vector library, and each table vector library corresponds to at least one data table.

6. The method of matching a database using a large model in the data governance industry according to claim 5, wherein, The detailed content includes table name, table note, column name and column note; wherein, the detailed content is spliced and corrected to obtain a corrected third description text, including: Splicing the table name, the table note, the column name and the column note into a preliminary text; Cleaning and data format cropping the preliminary text to obtain a corrected third description text.

7. The method of matching a database using a large model in the data governance industry according to claim 1, wherein, The query information is vectorized to obtain a vectorized query vector result, including: Based on a large language model, key elements in the natural language of the query information are extracted; wherein, the key elements include time, place and query object; Splicing the extracted key elements into a first query text; Removing punctuation in the first query text and unifying case to obtain a second query text; Vectorizing the second query text to obtain a vectorized query vector result.

Images