Library table parallel retrieval method, system and medium for large language model SQL generation

Abstract

The application provides a library table parallel retrieval method, system and medium for large language model SQL generation, the method comprising: obtaining SQL generation requirement information proposed by a user, and extracting core keywords in the SQL generation requirement information; inputting the core keywords into a text embedding model to generate a query vector; performing similarity calculation in a library knowledge vector table and a table knowledge vector table according to the query vector to obtain an effective library set and an effective table set, wherein the dimensions of the query vector, the vectors in the library knowledge vector table and the vectors in the table knowledge vector table are all the same; constructing candidate pairs according to the effective library set and the effective table set, calculating the weight score of each candidate pair, and then outputting a retrieval result after sorting the candidate pairs according to the weight score; and the application effectively improves the accuracy of large language model retrieval and SQL statement generation.

Description

Technical Field

[0001] This application belongs to the field of large language model technology, and relates to a retrieval method for large language models, particularly to a parallel retrieval method, system and medium for database tables generated by large language model SQL. Background Technology

[0002] With the increasing application of large language models in the field of Structured Query Language (SQL) generation, large language models (LLMs) can rely on database and table knowledge bases to obtain the relationship information between "database-table-field" and generate SQL statements that conform to business logic. However, existing database and table knowledge base construction and retrieval schemes have the following shortcomings:

[0003] 1. Single dimension of knowledge base creation: Traditional solutions mostly only embedding and storing "table-level information" (such as table structure and field comments), without associating and retrieving "library-level information" (such as library business domain and core purpose) with the table, which makes it impossible for LLM to perceive the business attributes of the library to which the table belongs;

[0004] 2. Poor Search Fault Tolerance: Mainstream search engines employ a "single search table" logic—only filtering tables matching the query requirements within the table-level embedding, without any search or validation of the database to which the table belongs. In this approach, if multiple databases contain tables with similar names and fields (e.g., "user_info" and "order_info" both contain the "user_id" field, but their business semantics are completely different), table retrieval alone cannot distinguish the business affiliation of the tables, easily leading to the misselection of tables from non-target databases as matching results. Furthermore, if the table's embedding features are misjudged due to vague field descriptions (e.g., only labeling "user_info" without specifying the business scenario), there is no database information to assist in validation, ultimately causing the LLM to generate SQL that does not conform to business logic based on incorrect table information.

[0005] 3. Insufficient Result Accuracy: Table retrieval relies solely on the table's structural information (such as field names and types) to generate matching results. It fails to consider the business relationships between tables (e.g., the "order_id" association between the "order table" and the "payment table"), nor does it incorporate sample SQL reference information (e.g., standard SQL statements for querying "order payment status" from historical business records). In this situation, LLM can only generate SQL based on isolated table structure information, easily leading to "table association logic errors" (e.g., misusing non-related fields to associate tables) or "query semantic deviations" (e.g., needing to count "paid orders" but failing to add a payment status filter). This results in generated SQL that cannot meet actual business needs, significantly reducing accuracy. Summary of the Invention

[0006] The purpose of this application is to provide a parallel retrieval method, system, and medium for generating SQL from large language models, in order to solve the problem of low accuracy in SQL generation from large language models in the prior art.

[0007] Firstly, this application provides a parallel retrieval method for database tables generated from large language model SQL, the method comprising:

[0008] Obtain the SQL generation requirements submitted by the user, and extract the core keywords from the SQL generation requirements;

[0009] The core keywords are input into the text embedding model to generate query vectors;

[0010] Based on the query vector, similarity calculations are performed in the library knowledge vector table and the table knowledge vector table to obtain a valid library set and a valid table set, wherein the query vector has the same dimension as the vectors in the library knowledge vector table and the vectors in the table knowledge vector table;

[0011] Candidate pairs are constructed based on the set of valid libraries and the set of valid tables. After calculating the weight score of each candidate pair, the candidate pairs are sorted according to the weight scores and the search results are output.

[0012] In one implementation of the first aspect, the method for creating the library knowledge vector table and the table knowledge vector table includes the following process:

[0013] Collect database-level and table-level information from the target database;

[0014] After embedding feature encoding of the library-level information and the table-level information using a text embedding model, fixed-dimensional library knowledge vectors and table knowledge vectors are generated.

[0015] Create a library knowledge vector table and a table knowledge vector table in the vector database, store the library knowledge vectors in the library knowledge vector table, and store the table knowledge vectors in the table knowledge vector table.

[0016] In one implementation of the first aspect, the library-level information includes library ID, library business domain, library core purpose, and library associated system, and the table-level information includes table ID, library ID, table business semantics, core field set, table business scenario, inter-table relationships, and table sample SQL set.

[0017] In one implementation of the first aspect, the dimensions of both the library knowledge vector and the table knowledge vector are the same as the output dimension of the text embedding model.

[0018] In one implementation of the first aspect, the step of calculating similarity in the library knowledge vector table and the table knowledge vector table based on the query vector to obtain a valid library set and a valid table set includes:

[0019] Calculate the first similarity between the library knowledge vector in the library knowledge vector table and the query vector;

[0020] Select the library knowledge vectors whose first similarity is not lower than the preset library threshold to form an effective library set;

[0021] Calculate the second similarity between the table knowledge vector in the table knowledge vector table and the query vector;

[0022] Select the table knowledge vectors whose second similarity is not lower than the preset table threshold to form an effective table set;

[0023] The valid library set includes library ID and library structure information. The library structure information includes library business domain, core library purpose, and library associated systems. The valid table set includes table ID, table structure information, and its parent library ID. The table structure information includes table business semantics, core field set, table business scenario, inter-table relationships, and table sample SQL set.

[0024] In one implementation of the first aspect, constructing candidate pairs based on the set of valid libraries and the set of valid tables includes:

[0025] Obtain the library ID corresponding to each table knowledge vector in the valid table set, and the library ID of each library knowledge vector in the valid library set;

[0026] Associating the belonging library ID with the library ID, when the belonging library ID in the valid table set exists in the library ID of the library knowledge vector in the valid library set, the library ID, the table ID, the library structure information, and the table structure information are formed into a candidate pair;

[0027] When the library ID in the set of valid tables does not exist in the library ID of the library knowledge vector in the set of valid tables, the table ID, the table structure information, and the library ID in the set of valid tables are used to form the candidate pair;

[0028] When the table knowledge vector corresponding to the library ID in the set of valid libraries is not in the set of valid tables, the library ID and the library structure information are used to form the candidate pair.

[0029] In one implementation of the first aspect, the calculation process of the weight score of the candidate pair satisfies the following formula:

[0030] When the library ID belonging to the valid table set exists in the library ID of the library knowledge vector in the valid library set, the calculation process of the weight score of the candidate pair satisfies the following formula:

[0031] ;

[0032] When the library ID belonging to the valid table set does not exist in the library ID of the library knowledge vector in the valid library set, the calculation process of the weight score of the candidate pair satisfies the following formula:

[0033] ;

[0034] When the table knowledge vector corresponding to the library ID in the valid library set is not in the valid table set, the calculation process of the weight score of the candidate pair satisfies the following formula:

[0035] ;

[0036] in, The weight score represents the candidate pair. This represents the first similarity of the library knowledge vectors in the effective library set. This represents the second similarity of the knowledge vectors of the tables in the set of valid tables. Indicates the first parameter. Indicates the second parameter. This indicates the third parameter.

[0037] In one implementation of the first aspect, when the library ID in the valid table set exists in the library ID of the library knowledge vector in the valid library set, if there are at least two table knowledge vectors in the valid table set whose similarity difference is less than the difference threshold and whose libraries are different, then the first parameter is reduced.

[0038] This invention also discloses a parallel retrieval system for database tables used in large language model SQL generation, comprising:

[0039] The information extraction module is used to obtain the SQL generation requirement information submitted by the user and extract the core keywords from the SQL generation requirement information;

[0040] The generation module is used to generate query vectors after inputting the core keywords into the text embedding model;

[0041] The calculation module is used to perform similarity calculations in the library knowledge vector table and the table knowledge vector table according to the query vector to obtain a valid library set and a valid table set, wherein the query vector has the same dimension as the vectors in the library knowledge vector table and the vectors in the table knowledge vector table;

[0042] The sorting output module is used to construct candidate pairs based on the set of valid libraries and the set of valid tables, calculate the weight score of each candidate pair, sort the candidate pairs according to the weight scores, and then output the retrieval results.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described parallel retrieval method for generating database tables for large language model SQL.

[0044] As described above, the parallel retrieval method, system, and medium for generating large language model SQL described in this application have the following beneficial effects:

[0045] By employing multi-dimensional database knowledge and parallel database retrieval to generate SQL statements, and by improving the accuracy of retrieval results through database-level association, this technology solves the problem of poor fault tolerance and low accuracy of retrieval results caused by single-dimensional retrieval in existing technologies. It improves the accuracy of SQL statement generation, enhances the user experience, and can further improve the accuracy of retrieval results by dynamically adjusting weights during the retrieval process, thus meeting the needs of different usage scenarios. Attached Figure Description

[0046] Figure 1 The flowchart shown is a process for parallel retrieval of database tables for generating large language model SQL, as described in an embodiment of this application.

[0047] Figure 2 This diagram illustrates the process of establishing the library knowledge vector table and the table knowledge vector table in the parallel retrieval method for generating SQL from a large language model, as described in this embodiment of the application.

[0048] Figure 3 The flowchart shown is a process flow of the parallel retrieval system for database tables used in generating large language model SQL, as described in an embodiment of this application. Detailed Implementation

[0049] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0050] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0051] See Figures 1 to 3 The following embodiments of this application provide a method, system, and medium for parallel retrieval of database tables for generating SQL from large language models. By employing multi-dimensional database table knowledge and parallel database table retrieval, the method generates SQL statements and improves the accuracy of retrieval results through database table dimension association. This solves the problem of poor fault tolerance and low accuracy of retrieval results caused by single-dimensional retrieval in the prior art, improves the accuracy of SQL statement generation, enhances the user experience, and can further improve the accuracy of retrieval results by dynamically adjusting weights during the retrieval process, thus meeting the needs of different usage scenarios.

[0052] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0053] like Figure 1 As shown in the figure, this embodiment provides a parallel retrieval method for database tables generated by a large language model SQL. The method includes the following steps:

[0054] S101. Obtain the SQL generation requirement information proposed by the user, and extract the core keywords from the SQL generation requirement information.

[0055] When a user submits a request to generate an SQL query, the core keywords are extracted so that corresponding query vectors can be generated based on these keywords later.

[0056] Specifically, when generating SQL to query a user's real-name authentication status, extract the core keyword "user real-name authentication status".

[0057] S102. The core keywords are input into the text embedding model to generate a query vector.

[0058] By inputting the extracted core keywords into the text embedding model, query vectors are generated, which facilitates the subsequent generation of corresponding results based on the generated query vectors.

[0059] S103. Based on the query vector, perform similarity calculations in the library knowledge vector table and the table knowledge vector table respectively to obtain a valid library set and a valid table set, wherein the query vector has the same dimension as the vectors in the library knowledge vector table and the vectors in the table knowledge vector table.

[0060] In this embodiment, after generating the corresponding query vector according to the user's query requirements, similarity calculation is performed on the established library knowledge vector table and table knowledge vector table to obtain the effective library set and effective table set respectively. This enables comprehensive retrieval and result generation from the library and table dimensions, improving the problem of inaccurate retrieval results caused by single-dimensional retrieval.

[0061] In some embodiments, reference Figure 2 The method for creating the library knowledge vector table and the table knowledge vector table includes the following process:

[0062] S201. Collect database-level and table-level information of the target database;

[0063] S202. After using a text embedding model to encode the library-level information and the table-level information using embedding features, a fixed-dimensional library knowledge vector and table knowledge vector are generated.

[0064] S203. Create a library knowledge vector table and a table knowledge vector table in the vector database, store the library knowledge vectors in the library knowledge vector table, and store the table knowledge vectors in the table knowledge vector table.

[0065] In this embodiment, before performing parallel retrieval of the database and tables, a database knowledge vector table and a table knowledge vector table are first established to facilitate subsequent parallel retrieval. Specifically, during the establishment process, database-level information and table-level information of the target database are first collected. The database-level information includes the database ID, database business domain, database core purpose, and database association system. The table-level information includes the table ID, the ID of the database to which it belongs, the table business semantics, the set of core fields, the table business scenario, the inter-table relationships, and the set of sample SQL statements for the table.

[0066] For example, the database business domain includes a user center domain and an order transaction domain. The core uses of the database include storing C-segment user registration data and supporting login authentication. The database associated systems include the core e-commerce system and the payment system. The database ID is used to associate database-level information. The table business semantics include storing user real-name authentication information and ID card verification results. The core field set includes field semantics, such as user unique identifier and authentication status. The table business scenarios include user withdrawals and membership upgrades. The table relationships include table association fields. The table sample SQL set includes SQL statements frequently used in actual business operations. Since the above database-level and table-level information are directly collected from existing technologies, they will not be elaborated further here.

[0067] After collecting library-level and table-level information, a text embedding model is used to encode the embedding features of the library-level and table-level information respectively to generate fixed-dimensional library knowledge vectors and table knowledge vectors. Then, a library knowledge vector table and a table knowledge vector table are created in the vector database, and the library knowledge vectors are stored in the library knowledge vector table and the table knowledge vectors are stored in the table knowledge vector table, thereby completing the process of establishing the library knowledge vector table and the table knowledge vector table.

[0068] It should be noted that the dimensions of the library knowledge vector and the table knowledge vector are the same as the output dimension of the text embedding model, which makes it more convenient to calculate similarity later.

[0069] The text embedding model used when creating the library knowledge vector table and the table knowledge vector table, as well as the text embedding model used to generate query vectors based on core keywords, are the same model to ensure the accuracy of subsequent results.

[0070] In further embodiments, the step of calculating similarity in the library knowledge vector table and the table knowledge vector table based on the query vector to obtain a valid library set and a valid table set includes:

[0071] Calculate the first similarity between the library knowledge vector in the library knowledge vector table and the query vector;

[0072] Select the library knowledge vectors whose first similarity is not lower than the preset library threshold to form an effective library set;

[0073] Calculate the second similarity between the table knowledge vector in the table knowledge vector table and the query vector;

[0074] Select the table knowledge vectors whose second similarity is not lower than the preset table threshold to form an effective table set;

[0075] The valid library set includes library ID and library structure information. The library structure information includes library business domain, core library purpose, and library associated systems. The valid table set includes table ID, table structure information, and its parent library ID. The table structure information includes table business semantics, core field set, table business scenario, inter-table relationships, and table sample SQL set.

[0076] After obtaining the query vector, the similarity between the query vector and the knowledge vector in the library and table is calculated respectively, so as to select the library knowledge vector and table knowledge vector that meet the similarity requirements for subsequent retrieval results output.

[0077] Specifically, the first similarity between the library knowledge vector and the query vector in the library knowledge vector table and the second similarity between the table knowledge vector and the query vector in the table knowledge vector table are calculated using a similarity calculation method. Library knowledge vectors with a first similarity not lower than a preset library threshold are selected to form a valid library set, and table knowledge vectors with a second similarity not lower than a preset table threshold are selected to form a valid table set.

[0078] For example, the preset library threshold is 0.5, and the preset table threshold is 0.55. The first similarity and the second similarity are obtained by calculating the similarity between the query vector and the vectors in the library knowledge vector table and the table knowledge vector table, respectively. Library knowledge vectors with a first similarity greater than or equal to 0.5 are selected and a valid library set is established. Table knowledge vectors with a second similarity greater than or equal to 0.55 are selected and a valid table set is established.

[0079] Furthermore, the valid library set includes library ID and library structure information, and the valid table set includes table ID, table structure information, and the library ID to which it belongs, which facilitates the subsequent establishment of candidate pairs based on library-table associations.

[0080] S104. Construct candidate pairs based on the set of valid libraries and the set of valid tables, calculate the weight score of each candidate pair, sort the candidate pairs according to the weight score, and then output the retrieval results.

[0081] In some other embodiments, constructing candidate pairs based on the set of valid libraries and the set of valid tables includes:

[0082] Obtain the library ID corresponding to each table knowledge vector in the valid table set, and the library ID of each library knowledge vector in the valid library set;

[0083] Associating the belonging library ID with the library ID, when the belonging library ID in the valid table set exists in the library ID of the library knowledge vector in the valid library set, the library ID, the table ID, the library structure information, and the table structure information are formed into a candidate pair;

[0084] When the library ID in the set of valid tables does not exist in the library ID of the library knowledge vector in the set of valid tables, the table ID, the table structure information, and the library ID in the set of valid tables are used to form the candidate pair;

[0085] When the table knowledge vector corresponding to the library ID in the set of valid libraries is not in the set of valid tables, the library ID and the library structure information are used to form the candidate pair.

[0086] In this embodiment, during the process of constructing candidate pairs based on the valid database set and the valid table set, the database ID corresponding to the table knowledge vector in the valid table set and the database ID of the database knowledge vector in the valid database set are first obtained respectively. Then, the selected database ID and database ID are associated. When the database ID in the valid table set exists in the database ID of the database knowledge vector in the valid database set, a database valid-table valid candidate pair is constructed. That is, the database ID, the table ID, the database structure information, and the table structure information are combined to form a candidate pair, providing the LLM with the core information of "correct business domain + table feature matching", which is the main basis for SQL generation.

[0087] When the library ID in the set of valid tables does not exist in the library knowledge vector of the set of valid libraries, the component library invalid - table valid candidate pair is formed. That is, the table ID, the table structure information and the library ID in the set of valid tables are used to form the candidate pair. By retaining the results of table feature matching but library feature not yet associated, the output result avoids the loss of useful table knowledge vector due to insufficient features of library information vector.

[0088] When the table knowledge vector corresponding to the library ID in the set of valid libraries is not in the set of valid tables, a library valid-table invalid candidate pair is constructed. That is, the library ID and the library structured information are used to form the candidate pair. By constructing the library valid-table invalid candidate pair, the library information vector is used as the business scope anchor point to guide the LLM to supplement the query keywords and avoid no output results.

[0089] In some other embodiments, the calculation process of the weight score of the candidate pair satisfies the following formula:

[0090] When the library ID belonging to a candidate pair exists in the library knowledge vector of the valid library set, the calculation process of the weight score of the candidate pair satisfies the following formula.

[0091] ;

[0092] When the library ID belonging to the valid table set does not exist in the library ID of the library knowledge vector in the valid library set, the calculation process of the weight score of the candidate pair satisfies the following formula:

[0093] ;

[0094] When the table knowledge vector corresponding to the library ID in the valid library set is not in the valid table set, the calculation process of the weight score of the candidate pair satisfies the following formula:

[0095] ;

[0096] in, The weight score represents the candidate pair. This represents the first similarity of the library knowledge vectors in the effective library set. This represents the second similarity of the knowledge vectors of the tables in the set of valid tables. Indicates the first parameter. Indicates the second parameter. This indicates the third parameter.

[0097] The weight score of each candidate pair can be calculated using the above formula, so that the candidate pairs can be sorted according to the weight score in the subsequent process.

[0098] It should be noted that in the above formula, when the library ID belonging to the valid table set exists in the library ID of the library knowledge vector in the valid library set to construct a library valid-table valid candidate pair, the first parameter... The value of is between 0.5 and 1, thus making the weight of the vector... Weights greater than the library vector 1- Prioritize ensuring that table features match SQL generation requirements, and use database weights to assist in verifying the correctness of the business domain. When the database ID belonging to a table in the valid table set does not exist in the database ID of the database knowledge vector in the valid database set, the table score is reduced to reflect the risk of "questionable database ownership" while retaining the core value of the table, scoring the type of valid table-valid database candidate pair. When the table knowledge vector corresponding to the database ID in the valid database set is not in the valid table set, the score is further weakened by multiplying by 0.5 to avoid "valid database" masking the defect of "invalid table," and the score is significantly lower than the previous two candidate pair types.

[0099] For example, the first parameter The value range is from 0.55 to 0.65, the second parameter The value range of the first parameter is 0.8 to 0.9, and the value range of the third parameter is 0.7 to 0.8.

[0100] In some further embodiments, when the library ID in the valid table set exists in the library ID of the library knowledge vector in the valid library set, if there are at least two table knowledge vectors in the valid table set whose second similarity difference is less than the difference threshold and whose libraries are different, then the first parameter is reduced.

[0101] For example, consider the weight score S of knowledge vector A. A The value is 0.82, representing the S-value of knowledge vector a. a =0.80, then ΔS_table = 0.02), and when the two tables belong to different databases, table features cannot effectively distinguish priorities, and the distinguishability needs to be strengthened by increasing the database weight. Therefore, the formula for calculating the weight score should be adjusted, and the first parameter should be changed... The weight of the library vector is reduced, thereby increasing its weight in the calculation of the weight score. This ensures that when the table similarity is close, the "business domain matching degree" of the library becomes the key distinguishing criterion, and its business affiliation is more in line with the query requirements, so it should be given higher priority.

[0102] This invention also provides a parallel retrieval system for database tables generated from large language model SQL, see reference. Figure 3 ,include:

[0103] The information extraction module 301 is used to obtain the SQL generation requirement information proposed by the user and extract the core keywords from the SQL generation requirement information;

[0104] The generation module 302 is used to generate a query vector after inputting the core keywords into the text embedding model, wherein the query vector has the same dimension as the library knowledge vector table and the table knowledge vector table;

[0105] The calculation module 303 is used to perform similarity calculations in the library knowledge vector table and the table knowledge vector table respectively based on the query vector to obtain the effective library set and the effective table set;

[0106] The sorting output module 304 is used to construct candidate pairs based on the effective library set and the effective table set, calculate the weight score of each candidate pair, sort the candidate pairs based on the weight scores, and then output the retrieval results.

[0107] The parallel retrieval system for database tables used for generating large language model SQL can implement the parallel retrieval method for generating large language model SQL described in this application. However, the implementation device for the parallel retrieval method for generating large language model SQL described in this application includes, but is not limited to, the structure of the parallel retrieval system for generating large language model SQL listed in this embodiment. All structural modifications and substitutions of the prior art made based on the principles of this application are included within the protection scope of this application.

[0108] It should be noted that the structure and principle of each of the above modules correspond one-to-one with the steps in the parallel retrieval method of database tables for generating SQL for large language models. The specific working principle can also be found in the description of the parallel retrieval method of database tables for generating SQL for large language models in the previous embodiments, so it will not be repeated here.

[0109] The present invention further discloses a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described parallel retrieval method for generating database tables for large language model SQL.

[0110] The scope of protection of the parallel retrieval method for database tables for generating large language model SQL described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0111] This invention also provides an electronic device, which includes a processor and a memory; the memory is used to store a computer program; the processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-described parallel retrieval method for generating database tables for large language model SQL.

[0112] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. This available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0113] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0114] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0115] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0117] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A library table parallel retrieval method for large language model SQL generation, characterized in that, The method includes: Obtain the SQL generation requirements submitted by the user, and extract the core keywords from the SQL generation requirements; The core keywords are input into the text embedding model to generate query vectors; Based on the query vector, similarity calculations are performed in the library knowledge vector table and the table knowledge vector table to obtain a valid library set and a valid table set, wherein the query vector has the same dimension as the vectors in the library knowledge vector table and the vectors in the table knowledge vector table; Candidate pairs are constructed based on the set of valid libraries and the set of valid tables. After calculating the weight score of each candidate pair, the candidate pairs are sorted according to the weight score and the search results are output. The step of calculating similarity in the library knowledge vector table and the table knowledge vector table based on the query vector to obtain the effective library set and the effective table set includes: Calculate the first similarity between the library knowledge vector in the library knowledge vector table and the query vector; Select the library knowledge vectors whose first similarity is not lower than the preset library threshold to form an effective library set; Calculate the second similarity between the table knowledge vector in the table knowledge vector table and the query vector; Select the table knowledge vectors whose second similarity is not lower than the preset table threshold to form an effective table set; The effective library set includes library ID and library structure information. The library structure information includes library business domain, library core purpose and library associated system. The effective table set includes table ID, table structure information and its library ID. The table structure information includes table business semantics, core field set, table business scenario, inter-table relationship and table sample SQL set. The step of constructing candidate pairs based on the set of valid libraries and the set of valid tables includes: Obtain the library ID corresponding to each table knowledge vector in the valid table set, and the library ID of each library knowledge vector in the valid library set; Associating the belonging library ID with the library ID, when the belonging library ID in the valid table set exists in the library ID of the library knowledge vector in the valid library set, the library ID, the table ID, the library structure information, and the table structure information are formed into a candidate pair; When the library ID in the set of valid tables does not exist in the library ID of the library knowledge vector in the set of valid tables, the table ID, the table structure information, and the library ID in the set of valid tables are used to form the candidate pair; When the table knowledge vector corresponding to the library ID in the set of valid libraries is not in the set of valid tables, the library ID and the library structure information are used to form the candidate pair.

2. The library table parallel retrieval method for large language model SQL generation according to claim 1, characterized in that, The method for creating the library knowledge vector table and the table knowledge vector table includes the following process: Collect database-level and table-level information from the target database; After embedding feature encoding of the library-level information and the table-level information using a text embedding model, fixed-dimensional library knowledge vectors and table knowledge vectors are generated. Create a library knowledge vector table and a table knowledge vector table in the vector database, store the library knowledge vectors in the library knowledge vector table, and store the table knowledge vectors in the table knowledge vector table.

3. The parallel retrieval method for database tables used in generating large language model SQL according to claim 2, characterized in that, The database-level information includes database ID, database business domain, database core purpose, and database associated systems. The table-level information includes table ID, database ID, table business semantics, core field set, table business scenario, inter-table relationships, and table sample SQL set.

4. The parallel retrieval method for database tables used in generating large language model SQL according to claim 2, characterized in that, The dimensions of both the library knowledge vector and the table knowledge vector are the same as the output dimension of the text embedding model.

5. The parallel retrieval method for database tables used in generating large language model SQL according to claim 1, characterized in that, The calculation process of the weight score of the candidate pair satisfies the following formula: When the library ID belonging to the valid table set exists in the library ID of the library knowledge vector in the valid library set, the calculation process of the weight score of the candidate pair satisfies the following formula: ； When the library ID belonging to the valid table set does not exist in the library ID of the library knowledge vector in the valid library set, the calculation process of the weight score of the candidate pair satisfies the following formula: ； When the table knowledge vector corresponding to the library ID in the valid library set is not in the valid table set, the calculation process of the weight score of the candidate pair satisfies the following formula: ； in, The weight score represents the candidate pair. This represents the first similarity of the library knowledge vectors in the effective library set. This represents the second similarity of the knowledge vectors of the tables in the set of valid tables. Indicates the first parameter. Indicates the second parameter. This indicates the third parameter.

6. The parallel retrieval method for database tables used in generating large language model SQL according to claim 5, characterized in that, When the library ID in the valid table set exists in the library ID of the library knowledge vector in the valid library set, if there are at least two table knowledge vectors in the valid table set whose second similarity difference is less than the difference threshold and whose libraries are different, then the first parameter is reduced.

7. A parallel retrieval system for database tables used in generating large language model SQL, characterized in that, include: The information extraction module is used to obtain the SQL generation requirement information submitted by the user and extract the core keywords from the SQL generation requirement information; The generation module is used to generate query vectors after inputting the core keywords into the text embedding model; The calculation module is used to perform similarity calculations in the library knowledge vector table and the table knowledge vector table according to the query vector to obtain a valid library set and a valid table set, wherein the query vector has the same dimension as the vectors in the library knowledge vector table and the vectors in the table knowledge vector table; The sorting output module is used to construct candidate pairs based on the set of valid libraries and the set of valid tables, calculate the weight score of each candidate pair, sort the candidate pairs according to the weight score, and then output the retrieval results. The step of calculating similarity in the library knowledge vector table and the table knowledge vector table based on the query vector to obtain the effective library set and the effective table set includes: Calculate the first similarity between the library knowledge vector in the library knowledge vector table and the query vector; Select the library knowledge vectors whose first similarity is not lower than the preset library threshold to form an effective library set; Calculate the second similarity between the table knowledge vector in the table knowledge vector table and the query vector; Select the table knowledge vectors whose second similarity is not lower than the preset table threshold to form an effective table set; The effective library set includes library ID and library structure information. The library structure information includes library business domain, library core purpose and library associated system. The effective table set includes table ID, table structure information and its library ID. The table structure information includes table business semantics, core field set, table business scenario, inter-table relationship and table sample SQL set. The step of constructing candidate pairs based on the set of valid libraries and the set of valid tables includes: Obtain the library ID corresponding to each table knowledge vector in the valid table set, and the library ID of each library knowledge vector in the valid library set; Associating the belonging library ID with the library ID, when the belonging library ID in the valid table set exists in the library ID of the library knowledge vector in the valid library set, the library ID, the table ID, the library structure information, and the table structure information are formed into a candidate pair; When the library ID in the set of valid tables does not exist in the library ID of the library knowledge vector in the set of valid tables, the table ID, the table structure information, and the library ID in the set of valid tables are used to form the candidate pair; When the table knowledge vector corresponding to the library ID in the set of valid libraries is not in the set of valid tables, the library ID and the library structure information are used to form the candidate pair.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the parallel retrieval method for database tables for generating large language model SQL as described in any one of claims 1 to 6.

Images