Database table recall method, device and system and storage medium

Abstract

The embodiment of the invention relates to the technical field of machine learning, and provides a database table recall method, device and system and a storage medium. The method is applied to an intelligent analysis platform and comprises the steps of obtaining a query text submitted by a user; performing word segmentation processing on the query text to obtain a plurality of initial word segments; performing synonym replacement on the plurality of initial word segments to obtain a first standard word segment conforming to the standard expression; filtering the first standard word segment into a second standard word segment conforming to a preset feature vector dimension; performing table field recall on the second standard word segments to determine a field corresponding to each second standard word segment in a database, and determining a database table where the field is located; determining a database table of which the corresponding frequency of the second standard word segment and the field in the database meets a preset frequency threshold value as a to-be-recalled target database table for the query text; generating a structured query statement for recall based on the target database table; and recalling the target database table from the database based on the structured query statement.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a database table recall method, device, system and storage medium. Background Art

[0002] ChatBi is a tool based on natural language processing and machine learning technology that enables computers to understand human language and generate SQL statements based on user input, thereby realizing intelligent dialogue between users and database systems.

[0003] Most database systems today are oriented toward a specific business field, and the existing ChatBi system lacks a set of mature rules for converting user queries into professional terminology. As a result, ChatBi is still unable to extract and understand professional vocabulary that appears in user queries. At the same time, ChatBi does not make sufficient use of emerging large model technologies. Currently, the end-to-end Text-2-sql model is still used for natural language to generate SQL statements. This type of model lacks a deep understanding of human natural language, resulting in a low accuracy rate for SQL statements generated by ChatBi, which leads to insufficient completeness and accuracy in data recalled based on SQL statements. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a database table recall method, device, system and storage medium to solve the technical defects that the accuracy of the SQL statements generated by ChatBi in the prior art is low, resulting in insufficient completeness and accuracy of the data recalled based on the SQL statements.

[0005] In order to achieve the above-mentioned purpose, the first aspect of the present application provides a database table recall method, which is applied to an intelligent analysis platform, and the recall method includes: obtaining a query text submitted by a user; performing word segmentation processing on the query text to obtain multiple initial word segments; performing synonym replacement on the multiple initial word segments to obtain a first standard word segment that meets the standard expression; filtering the first standard word segment into a second standard word segment that meets the preset feature vector dimension; performing table field recall on the second standard word segment to determine the field corresponding to each second standard word segment in the database, and determine the database table where the field is located; determining the database table in the database whose number of correspondences between the second standard word segment and the field meets the preset number threshold as the target database table to be recalled for the query text; generating a structured query statement for recall based on the target database table; and recalling the target database table from the database based on the structured query statement.

[0006] In an embodiment of the present application, performing synonym replacement processing on multiple initial word segments to obtain a first standard word segment that meets the standard expression includes: for each initial word segment, dividing the multiple initial word segments into a first initial word segment with a matching synonym pair and a second initial word segment without a matching synonym pair based on a preset synonym library, wherein the preset synonym library includes multiple synonym pairs, each synonym pair includes a synonym set consisting of multiple mutually synonymous word segments and a standard word segment corresponding to the synonym set; for each first initial word segment, replacing each first initial word segment with a synonym set; The word segment is replaced with a standard word segment included in a synonym pair matching each first initial word segment; it is determined whether each second initial word segment has an implicit synonym relationship with any synonym pair; for any second initial word segment, when it is determined that the second initial word segment has an implicit synonym relationship with any synonym pair, the any synonym pair is determined as a synonym pair matching the second initial word segment, and the second initial word segment is replaced with a standard word segment included in the any synonym pair; the replaced first initial word segment and the replaced second initial word segment are determined as first standard word segments that meet the standard expression.

[0007] In an embodiment of the present application, determining whether each second initial word segment has an implicit synonymy relationship with any synonym pair includes: performing text vector embedding processing on each second initial word segment and the standard word segment included in each synonym pair to obtain a first embedding vector corresponding to each standard word segment and a second embedding vector corresponding to each second initial word segment; for any second initial word segment, when a similarity value between the second embedding vector of the second initial word segment and the first embedding vector of any synonym pair is greater than a preset threshold, determining that an implicit synonymy relationship exists between the second initial word segment and the any synonym pair; for any second initial word segment, when a similarity value between the second embedding vector of the second initial word segment and the first embedding vector of any synonym pair is less than or equal to a preset threshold, determining that no implicit synonymy relationship exists between the second initial word segment and the any synonym pair.

[0008] In an embodiment of the present application, the recall method further includes: for any second initial word segment, if there is no implicit synonymy relationship between the second initial word segment and all synonym pairs, determining the second initial word segment as a first standard word segment that meets the standard expression.

[0009] In an embodiment of the present application, the recall method also includes: determining a first field in the database for which a dimension value recall requirement exists, and obtaining the dimension value corresponding to each first field; constructing a dimension value query library for all first fields based on a dictionary tree, the dimension value query library including the correspondence between each first field and the corresponding dimension value; for each second standard word segment, inputting each second standard word segment into the dictionary tree to determine whether there is a dimension value corresponding to each second standard word segment in the dimension value query library based on the dictionary tree; for any second standard word segment, when it is determined that there is a dimension value corresponding to the second standard word segment in the dimension value query library, the first field corresponding to the corresponding dimension value is determined as the field corresponding to the second standard word segment in the database.

[0010] In an embodiment of the present application, table field recall is performed on the second standard word segment to determine the field corresponding to each second standard word segment in the database, and determining the database table where the field is located includes: for all database tables in the database, obtaining the second field included in each database table and the synonyms corresponding to each second field; vector embedding each second field and the synonyms corresponding to each second field, and storing the generated vector data in the vector database; determining the similarity score between each second standard word segment and each vector data in the vector database; for any second standard word segment, sorting each vector data in descending order based on the obtained similarity score, and determining the second field corresponding to the vector data whose similarity score is arranged within a preset numerical range as the field corresponding to the second standard word segment in the database, and determining the database table where the field is located.

[0011] In an embodiment of the present application, filtering the first standard word segment into a second standard word segment that meets the preset feature vector dimension includes: filtering the first standard word segment into a second standard word segment that meets the preset feature vector dimension based on an n-gram algorithm.

[0012] In an embodiment of the present application, the recall method also includes: after determining the target database table to be recalled for the query text, displaying the target database table to be recalled; after receiving a selection instruction triggered by the user, generating a structured query statement for recall based on the selection instruction, wherein the selection instruction is generated based on the database table selected by the user from the target database table to be recalled.

[0013] In an embodiment of the present application, the recall method also includes: after determining the target database table to be recalled for the query text, inputting the query text and the target database table into a decision model to filter a database table from the target database table based on the decision model; and generating a structured query statement for recall based on the filtered database table.

[0014] In an embodiment of the present application, the recall method also includes: defining a weight value for each field in each database table; after determining the target database table to be recalled for the query text, determining the total weight value of each database table based on the number of correspondences between the second standard word segment and the field in the database; and generating a structured query statement for recall based on the database table with the largest total weight value.

[0015] A second aspect of the present application provides a database table recall device, comprising: a memory configured to store instructions; The processor is configured to call the instructions from the memory and implement the above-mentioned database table recall method when executing the instructions.

[0016] A third aspect of the present application provides a database table recall system, including the above-mentioned database table recall device.

[0017] A fourth aspect of the present application provides a machine-readable storage medium having instructions stored thereon, which, when executed by a processor, configure the processor to execute the above-mentioned database table recall method.

[0018] The above technical solution obtains the query text submitted by the user, performs word segmentation processing on the query text to obtain multiple initial word segments, performs synonym replacement on the multiple initial word segments to obtain a first standard word segment that meets the standard expression, filters the first standard word segment into a second standard word segment that meets the preset feature vector dimension, recalls the second standard word segment to determine the field corresponding to each second standard word segment in the database, and determines the database table where the field is located, determines the database table in the database where the number of correspondences between the second standard word segment and the field meets the preset number threshold as the target database table to be recalled for the query text, generates a structured query statement for recall based on the target database table, and recalls the target database table from the database based on the structured query statement. The method converts the query text into a more professional and easy-to-understand domain terminology for the intelligent analysis platform through synonym conversion and feature value dimension filtering, greatly improving the ability of the intelligent analysis platform to map the query text to the professional database field, thereby improving the accuracy of the intelligent analysis platform to generate SQL statements, and ultimately improving the integrity and accuracy of the recalled database table.

[0019] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present application, but do not constitute a limitation on the embodiments of the present application. In the accompanying drawings: Figure 1 A schematic diagram of a process flow of a database table recall method according to an embodiment of the present application is shown; Figure 2 The internal structure diagram of a computer device according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the specific implementation methods described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0022] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or suggesting their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0023] Figure 1 The following schematically shows a flow chart of a method for recalling a database table according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a database table recall method, which can be applied to an intelligent analysis platform. The method includes the following steps.

[0024] Step 101: Obtain the query text submitted by the user.

[0025] In the embodiments of the present application, it should be noted that the intelligent analysis platform may refer to ChatBi, which is a platform that can be based on natural language processing and machine learning technologies, and through these technologies enable computers to understand human language and generate SQL statements based on user input, thereby realizing intelligent dialogue between users and database systems. The query text may refer to the query input by the user, where the query is the query condition entered by the user in the search engine, usually referring to keywords or query text. Specifically, the processor can obtain the query text submitted by the user through the search engine provided to the user by ChatBi.

[0026] It should be noted that after obtaining the query text, you can further determine whether the query text needs to be rewritten. For example, in some cases, the text sentence input by the user has reversed logic or typos. The query text and database professional field description can be placed in the big language model through the big language model, and appropriate prompt words can be written. The big language model can be called to rewrite the query text to improve the accuracy of subsequent field recall and table recall.

[0027] Step 102: perform word segmentation processing on the query text to obtain a plurality of initial word segments.

[0028] In the embodiments of the present application, it should be noted that word segmentation may refer to the process of dividing a text sequence into individual "words" or "tokens". This is a basic step in natural language processing. Almost all natural language processing tasks require word segmentation of the text. The purpose of word segmentation is to divide the text sequence into smaller units, such as words, phrases, punctuation marks, etc., for subsequent processing and analysis. In the present technical solution, when performing word segmentation on the query text, the constructed word segmentation model can be trained in the basic corpus of the industry to which the query text belongs, and the trained word segmentation model is used as a word segmentation tool to parse and segment the query text. The trained word segmentation model includes a word segmentation vocabulary and a stop word library. The word segmentation rules may include the following two: First, if the query text contains a word in the word segmentation lexicon, it will be matched; Second, if word A appears in the query text as a prefix of word B, word B is prioritized as the word segmentation to match with the word segmentation vocabulary. If word B is not matched, word A is used as the word segmentation to match with the word segmentation vocabulary.

[0029] Among them, the priority of the first one is lower than that of the second one. The stop word library may include meaningless words such as 'ah' and 'ya'. When a stop word appears in the query text, the word will not be included in the final word segmentation result, which is generally used for filtering. After obtaining the query text, the query text can be segmented based on the above word segmentation rules to obtain multiple initial word segments.

[0030] Step 103, perform synonym replacement on multiple initial word segments to obtain first standard word segments that meet the standard expressions.

[0031] In the embodiment of the present application, it should be noted that the query text input by the user may have colloquial and unprofessional expressions. In this case, ChatBi lacks a deep understanding of the user input content, and there are still deficiencies in the extraction and understanding of professional vocabulary in the query text, resulting in a low accuracy rate of the sql statements generated by ChatBi, thus causing deficiencies in the integrity and accuracy of the data recalled based on the sql statements. Therefore, in order to improve the accuracy rate of recalling database tables in the database when automatically generating sql statements subsequently, this technical solution adds a synonym replacement method. By performing synonym replacement on multiple initial word segments, the initial word segments are replaced with first standard word segments that meet the standard expressions. This method can greatly improve ChatBi's ability to map query text to professional database fields, thereby improving the accuracy rate of generating sql statements.

[0032] In the embodiment of the present application, performing synonym replacement processing on multiple initial word segments to obtain first standard word segments that meet the standard expressions includes: for each initial word segment, based on a preset synonym library, dividing the multiple initial word segments into first initial word segments with matching synonym pairs and second initial word segments without matching synonym pairs, where the preset synonym library includes multiple synonym pairs, and each synonym pair includes a synonym set composed of multiple mutually synonymous word segments and a standard word segment corresponding to the synonym set; for each first initial word segment, replacing each first initial word segment with the standard word segment included in the synonym pair that matches each first initial word segment; determining whether each second initial word segment has an implicit synonym relationship with any synonym pair; for any second initial word segment, in the case of determining that the second initial word segment has an implicit synonym relationship with any synonym pair, determining the synonym pair as the synonym pair that matches the second initial word segment, and replacing the second initial word segment with the standard word segment included in the synonym pair; determining the replaced first initial word segments and the replaced second initial word segments as first standard word segments that meet the standard expressions.

[0033] In this embodiment, it should be noted that in this technical solution, a synonym library is constructed by collecting vocabulary in each professional field. The synonym library includes a number of synonym pairs, and each synonym pair includes a synonym set consisting of multiple synonymous word segments and a standard word segment corresponding to the synonym set. For example, a synonym pair is {User_Query {1~i} ,Profession},User_Query {1~i} is a synonym set consisting of i mutually synonymous word segments, Profession is a synonym set User_Query {1~i} Therefore, by matching each initial word segment with all synonym pairs in the synonym library, multiple initial word segments can be divided into first initial word segments with matching synonym pairs and second initial word segments without matching synonym pairs. For each first initial word segment, each first initial word segment can be replaced with a standard word segment included in the synonym pair matching each first initial word segment.

[0034] It should be noted that, for each second initial word segment, although each second initial word segment does not have an obvious synonym pair matching it, there may still be some second initial word segments and synonym pairs in the synonym library that have implicit synonym relationships, wherein these implicit synonym relationships cannot be obtained by simple matching and require further feature analysis to obtain. Therefore, for each second initial word segment, further analysis can be performed to determine whether each second initial word segment has an implicit synonym relationship with any synonym pair. For any second initial word segment, if the second initial word segment has an implicit synonym relationship with any synonym pair, then the any synonym pair is determined as a synonym pair matching the second initial word segment, and the second initial word segment is replaced with the standard word segment included in the any synonym pair.

[0035] After all first initial word segments are replaced and second initial word segments that have an implicit synonymous relationship with the synonym pair are replaced, the replaced first initial word segments and the replaced second initial word segments are determined as first standard word segments that meet the standard expression.

[0036] In an embodiment of the present application, determining whether each second initial word segment has an implicit synonymy relationship with any synonym pair includes: performing text vector embedding processing on each second initial word segment and the standard word segment included in each synonym pair to obtain a first embedding vector corresponding to each standard word segment and a second embedding vector corresponding to each second initial word segment; for any second initial word segment, when a similarity value between the second embedding vector of the second initial word segment and the first embedding vector of any synonym pair is greater than a preset threshold, determining that an implicit synonymy relationship exists between the second initial word segment and the any synonym pair; for any second initial word segment, when a similarity value between the second embedding vector of the second initial word segment and the first embedding vector of any synonym pair is less than or equal to a preset threshold, determining that no implicit synonymy relationship exists between the second initial word segment and the any synonym pair.

[0037] In this embodiment, for each second initial word segment, further feature analysis can be used to determine whether each second initial word segment has an implicit synonym relationship with any synonym pair in the synonym library. Specifically, each second initial word segment and each standard word segment included in the synonym pair can be subjected to text vector embedding processing to obtain a first embedding vector corresponding to each standard word segment. pro and the second embedding vector Embed corresponding to each second initial word segment user Among them, text vector embedding processing can refer to a method of representing text information as a numerical vector, which involves representing words, sentences or other text units in the text as dense vectors of fixed length so that computers can understand and process them.

[0038] It should be noted that similarity matching can refer to measuring the similarity between two vectors. In this technical solution, the second embedding vector Embed generated by each second initial word segment can be user With each first embedding vector Embed pro Perform similarity matching to obtain each second embedding vector Embed user With each first embedding vectorEmbed pro For any second initial word segment, when the second embedding vector Embed of the second initial word segment user With any first embedding vector Embed pro When the similarity value between them is greater than the preset threshold Score, it can be considered that the second initial word segment is similar to the first embedding vector Embed pro There is an implicit synonym relationship between the corresponding synonym pairs. user With any first embedding vector Embed proWhen the similarity value between them is less than or equal to the preset threshold Score, it can be considered that the second initial word segment is similar to the first embedding vector Embed pro There is no implicit synonymy relationship between the corresponding synonym pairs. The preset threshold Score can be set according to actual needs.

[0039] In an embodiment of the present application, the recall method further includes: for any second initial word segment, when there is no implicit synonymy relationship between the second initial word segment and all synonym pairs, determining the second initial word segment as a first standard word segment that meets the standard expression.

[0040] In this embodiment, for any second initial word segment, the second embedding vector Embed of the second initial word segment is user With all the first embedding vectors Embed pro When the similarity values ​​between the first and second initial word segments are all less than or equal to the preset threshold Score, it can be considered that there is no implicit synonymy relationship between the second initial word segment and all synonym pairs. At this time, the second initial word segment can be determined as the first standard word segment that meets the standard expression.

[0041] Step 104: Filter the first standard word segment into a second standard word segment that meets the preset feature vector dimension.

[0042] In the embodiment of the present application, it should be noted that each word segment can be split into phrases of multiple feature vector dimensions. For example, taking a word segment "a cute cat" as an example, "a cute cat" can be split into phrases of multiple feature vector dimensions such as "a", "cute", "cat", "acute", "cute cat" and "a cute cat". The preset feature vector dimensions can be set according to actual needs. In order to further improve the accuracy of database table recall in the present technical solution, the first standard field can be further filtered by feature vector dimensions to filter the first standard word segment into a second standard word segment that meets the preset feature vector dimensions.

[0043] In the embodiment of the present application, filtering the first standard word segment into a second standard word segment that meets the preset feature vector dimension includes: filtering the first standard word segment into a second standard word segment that meets the preset feature vector dimension based on an n-gram algorithm.

[0044] In this embodiment, the n-gram algorithm may refer to an algorithm based on a statistical language model. The text content can be operated with a sliding window of size n by bytes to form a sequence of byte segments of length n. Each sequence of byte segments is called a gram. The occurrence frequencies of all grams are counted and filtered according to a preset threshold to form a list of key grams, which is the vector feature space of this text. Each gram in the list is a dimension of the feature vector. For example, taking a first standard word segment "a cute cat" as an example, after being processed by the n-gram algorithm, the 1-grams in "a cutecat" include "a", "cute", and "cat", the 2-grams include "a cute" and "cute cat", and the 3-gram includes "a cute cat". Taking the preset threshold, that is, the preset feature vector dimension, as 2 for example, the grams below 2-gram can be filtered, that is, the 1-grams including "a", "cute", and "cat" are filtered, and the 2-grams including "acute" and "cute cat", as well as the 3-gram including "a cute cat" are determined as the second standard word segments that meet the preset feature vector dimension. In the n-gram algorithm, the maximum value of the gram is the number n of segmented word segments, and then the words are concatenated in sequence to form phrases with word lengths from 1 to n. At the same time, in order to reduce the impact of meaningless phrases on the accuracy rate in the recall stage, a meaningless phrase filtering rule can also be introduced. For example, phrases starting with words such as '的' will be filtered out.

[0045] Step 105: Perform table field recall on the second standard word segments to determine the fields in the database corresponding to each second standard word segment and the database tables where the fields are located.

[0046] In the embodiment of the present application, it should be noted that recall may refer to selecting a few possible contents related to the user input from a large amount of information. After obtaining the second standard word segments, perform table field recall on the second standard word segments to determine the fields in the database corresponding to each second standard word segment and the database tables where the fields are located.

[0047] In an embodiment of the present application, table field recall is performed on the second standard word segment to determine the field corresponding to each second standard word segment in the database, and determining the database table where the field is located includes: for all database tables in the database, obtaining the second field included in each database table and the synonyms corresponding to each second field; vector embedding each second field and the synonyms corresponding to each second field, and storing the generated vector data in the vector database; determining the similarity score between each second standard word segment and each vector data in the vector database; for any second standard word segment, sorting each vector data in descending order based on the obtained similarity score, and determining the second field corresponding to the vector data whose similarity score is arranged within a preset numerical range as the field corresponding to the second standard word segment in the database, and determining the database table where the field is located.

[0048] In this embodiment, the second standard word segment is recalled for table fields to find out all the fields existing in the database table. Before the table field recall is performed, all the fields in the database need to be vectorized. Specifically, for all database tables in the database, firstly, the second field and the synonyms corresponding to each second field included in each database table are obtained, and a vector database that can be used to store vector data is built, and then the required embedding model for vector embedding is determined. Among them, the Embedding model is a technology widely used in machine learning and natural language processing, which aims to embed high-dimensional data such as text, pictures, videos, etc. into vectors to map them to low-dimensional space. Each second field and the synonyms corresponding to each second field can be vector-embedded through the selected embedding model, and the generated vector data can be stored in the constructed vector database. At this time, the second standard word segment is traversed in turn, and the similarity score between each second standard word segment and each vector data in the vector database is calculated. Specifically, the similarity score can be calculated by the following formula : , in, For any second standard segment, is any vector data.

[0049] It should be noted that the preset numerical range can be set according to actual needs. For example, it can be set by the top-k value method. The top-k value method is a data processing operation that aims to select the top k largest or smallest elements from a set of data. The k value can be set according to needs. In the present technical solution, the vector data arranged in the top k can be determined as vector data within the preset numerical range. After calculating the similarity score between each second standard word segment and each vector data, for any second standard word segment, each vector data can be sorted in descending order according to the size of the similarity score, and the second field corresponding to the vector data arranged in the top k similarity scores is determined as the field corresponding to the second standard word segment in the database. After obtaining the corresponding field, the database table where the field is located can be determined.

[0050] In an embodiment of the present application, the recall method also includes: determining a first field in the database for which a dimension value recall requirement exists, and obtaining the dimension value corresponding to each first field; constructing a dimension value query library for all first fields based on a dictionary tree, the dimension value query library including the correspondence between each first field and the corresponding dimension value; for each second standard word segment, inputting each second standard word segment into the dictionary tree to determine whether there is a dimension value corresponding to each second standard word segment in the dimension value query library based on the dictionary tree; for any second standard word segment, when it is determined that there is a dimension value corresponding to the second standard word segment in the dimension value query library, the first field corresponding to the corresponding dimension value is determined as the field corresponding to the second standard word segment in the database.

[0051] In this embodiment, it should be noted that it is necessary to consider that there are some second standard word segments including specific scene dimensions, such as involving a specific location, organization, business unit, etc. Therefore, when recalling table fields, it is necessary to identify the dimension values ​​of these word segments and use them as conditional filtering options when generating subsequent SQL statements. Specifically, first determine the first field in the database that has a dimension value recall requirement, that is, first determine the first field involving the specific scene dimension, and obtain the dimension value corresponding to each first field.

[0052] It should be noted that the dictionary tree may refer to the AC dictionary tree, which is a tree structure, also called a word lookup tree or a key tree. It is a variant of a hash tree and is mainly used to handle operations such as storage, query, and deletion of a large number of character strings. Through the AC dictionary tree, a dimension value query library for all first fields can be constructed, and the dimension value query library includes the correspondence between each first field and the corresponding dimension value. For each second standard word segment, each second standard word segment is input into the AC dictionary tree to determine whether there is a dimension value corresponding to each second standard word segment in the dimension value query library based on the AC dictionary tree. For any second standard word segment, when there is a dimension value corresponding to the second standard word segment in the dimension value query library, the first field corresponding to the corresponding dimension value can be determined as the field corresponding to the second standard word segment in the database.

[0053] It should be noted that when recalling table fields for the second standard word segments, it is also necessary to consider that there may be some second standard words involving specific digital dimensions such as time, number, email address, etc., and the dimension values ​​of these word segments also need to be identified. Specifically, table fields can be recalled through regular expressions.

[0054] Step 106: Determine the database table in which the number of correspondences between the second standard word segment and the field in the database meets a preset number threshold as the target database table to be recalled for the query text.

[0055] In an embodiment of the present application, after the second standard word segment is recalled for table fields, the fields corresponding to each second standard word segment in the database can be obtained, and the database table where the fields are located can be determined. After all second standard word segments are recalled, the number of correspondences in each database table can be counted, and the database table in which the number of correspondences between the second standard word segment and the field in the database meets the preset number threshold is determined as the target database table to be recalled for the query text. Among them, when a second standard word segment corresponds to any field in a database table once, it can be considered that the number of correspondences in the data table is +1, and the preset number threshold can be set according to actual needs, for example, it can include but is not limited to being set to the largest cumulative number. Therefore, at this time, the database table with the largest number of correspondences can be determined as the target database table to be recalled for the query text.

[0056] It should be noted that due to the existence of synonyms, some fields in the database may be recalled repeatedly. For example, word 'A' recalls database field 'X', and word 'C' also recalls database field 'X'. 'X' is recalled twice, and the results need to be deduplicated. At the same time, the fields recalled by dimension values ​​may be repeated with the fields recalled by table fields, so the results also need to be deduplicated.

[0057] Step 107: Generate a structured query statement for recall based on the target database table.

[0058] In the embodiment of the present application, it should be noted that the structured query statement may refer to an SQL statement, wherein the SQL statement is a special-purpose programming language used to access data and query, update and manage relational database systems. After obtaining the target database table to be recalled for the query text, ChatBi can generate an SQL statement for recall based on the target database table.

[0059] In an embodiment of the present application, the recall method also includes: after determining the target database table to be recalled for the query text, displaying the target database table to be recalled; after receiving a selection instruction triggered by the user, generating a structured query statement for recall based on the selection instruction, wherein the selection instruction is generated based on the database table selected by the user from the target database table to be recalled.

[0060] In this embodiment, it should be noted that in actual situations, there may be multiple database tables that have recalled the maximum number of table fields, that is, multiple database tables meet the preset number threshold. In this case, multiple target database tables can be displayed to the user, and the user determines the final selected database table. Therefore, after receiving the selection instruction triggered by the user, a SQL statement for recall is generated based on the selection instruction, wherein the selection instruction is generated based on the database table selected by the user from the target database tables to be recalled.

[0061] In an embodiment of the present application, the recall method also includes: after determining the target database table to be recalled for the query text, inputting the query text and the target database table into a decision model to filter a database table from the target database table based on the decision model; and generating a structured query statement for recall based on the filtered database table.

[0062] In this embodiment, it should be noted that in actual situations, there may be multiple database tables that recall the maximum number of table fields, that is, multiple database tables meet the preset number threshold. In this case, multiple target database tables and query texts can be uniformly input into the constructed decision model, and appropriate prompt words can be written. The decision model can judge and select a database table, and generate a SQL statement for recall based on the selected database table.

[0063] In an embodiment of the present application, the recall method also includes: defining a weight value for each field in each database table; after determining the target database table to be recalled for the query text, determining the total weight value of each database table based on the number of correspondences between the second standard word segment and the field in the database; and generating a structured query statement for recall based on the database table with the largest total weight value.

[0064] In this embodiment, it should be noted that in actual situations, there may be multiple database tables that have recalled the maximum number of table fields, that is, multiple database tables meet the preset number threshold. In this case, different weight values ​​can be set in advance for each field of each database table, so that the total weight value of each target database table is calculated, and the database table with the largest total weight value is used as the database table to be recalled in the end, and the SQL statement for recall is generated based on the database table with the largest total weight value.

[0065] Step 108: Retrieve the target database table from the database based on the structured query statement.

[0066] In the embodiment of the present application, after the SQL statement is generated, the target database table can be recalled from the database based on the SQL statement, and the recalled database table is displayed for user use.

[0067] The above technical solution converts the query text into more professional domain terms that are easier for the intelligent analysis platform to understand through synonym conversion and feature value dimension filtering, which greatly improves the ability of the intelligent analysis platform to map the query text to professional database fields, thereby improving the accuracy of the intelligent analysis platform in generating SQL statements, and ultimately improving the completeness and accuracy of the recalled database tables.

[0068] This technical solution adds two steps, question rewriting and synonym replacement, to the ChatBi intelligent dialogue process. The above two steps can convert the query input by the user into corresponding more professional domain terms through natural language processing technology. This approach greatly improves the ability of the ChatBi system to map user queries to professional database fields, thereby improving the accuracy of the large model to generate SQL statements. In the ChatBi intelligent dialogue process, word segmentation and n-gram short sentence generation steps are also added. The introduction of the above two steps can better improve the recall rate of table fields. In the ChatBi intelligent dialogue process, dimension value matching, field recall, and single table recall functions are also added. The above three functions can tell the large model which database fields, field dimension values, and database tables are needed when generating SQL statements. Thereby narrowing the reasoning scope of ChatBi and improving the accuracy of ChatBi's generated SQL statements.

[0069] The embodiment of the present application provides a database table recall device, including: a memory configured to store instructions; The processor is configured to call the instructions from the memory and implement the above-mentioned database table recall method when executing the instructions.

[0070] An embodiment of the present application provides a database table recall system, including the above-mentioned database table recall device.

[0071] An embodiment of the present application provides a storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned database table recall method is implemented.

[0072] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 2 As shown. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure) and a database (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02 and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store the recall method data of the database table. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, a method for recalling a database table is implemented.

[0073] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0074] An embodiment of the present application provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, a method for recalling a database table is implemented.

[0075] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing the program of the recall method steps of initializing a database table.

[0076] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0077] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0078] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0079] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0080] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0081] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash memory (flashRAM). The memory is an example of a computer-readable medium.

[0082] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0083] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0084] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A database table recall method, characterized in that: Applied to the intelligent analysis platform, the recall method includes: Get the query text submitted by the user; Performing word segmentation processing on the query text to obtain a plurality of initial word segments; Performing synonym replacement on the multiple initial word segments to obtain a first standard word segment that meets the standard expression; Filtering the first standard word segment into a second standard word segment that meets a preset feature vector dimension; Performing table field recall on the second standard word segment to determine the field corresponding to each second standard word segment in the database, and determining the database table where the field is located; Determine a database table in which the number of correspondences between the second standard word segment and the field in the database meets a preset number threshold as a target database table to be recalled for the query text; Generating a structured query statement for recall based on the target database table; The target database table is retrieved from the database based on the structured query statement.

2. The database table recall method according to claim 1, characterized in that: The performing synonym replacement processing on the multiple initial word segments to obtain a first standard word segment that meets the standard expression includes: For each initial word segment, the multiple initial word segments are divided into a first initial word segment having a matching synonym pair and a second initial word segment having no matching synonym pair based on a preset synonym library, wherein the preset synonym library includes a plurality of synonym pairs, each of which includes a synonym set consisting of a plurality of mutually synonymous word segments and a standard word segment corresponding to the synonym set; For each first initial word segment, replacing each first initial word segment with a standard word segment included in a synonym pair matching each first initial word segment; determining whether each second initial word segment has an implicit synonymy relationship with any synonym pair; For any second initial word segment, when it is determined that the second initial word segment has an implicit synonymy relationship with any synonym pair, the any synonym pair is determined as a synonym pair matching the second initial word segment, and the second initial word segment is replaced with a standard word segment included in the any synonym pair; The replaced first initial word segment and the replaced second initial word segment are determined as first standard word segments that meet the standard statement.

3. The database table recall method according to claim 2, characterized in that: The step of determining whether each second initial word segment has an implicit synonymy relationship with any synonym pair comprises: Performing text vector embedding processing on each second initial word segment and the standard word segment included in each synonym pair to obtain a first embedding vector corresponding to each standard word segment and a second embedding vector corresponding to each second initial word segment; For any second initial word segment, when the similarity value between the second embedding vector of the second initial word segment and the first embedding vector of any synonym pair is greater than a preset threshold, it is determined that there is an implicit synonym relationship between the second initial word segment and the any synonym pair; For any second initial word segment, when the similarity value between the second embedding vector of the second initial word segment and the first embedding vector of any synonym pair is less than or equal to the preset threshold, it is determined that there is no implicit synonymy relationship between the second initial word segment and the any synonym pair.

4. The database table recall method according to claim 2, characterized in that: The recall method further comprises: For any second initial word segment, when there is no implicit synonymy relationship between the second initial word segment and all synonym pairs, the second initial word segment is determined as a first standard word segment that meets the standard expression.

5. The database table recall method according to claim 1, characterized in that: The recall method further comprises: Determine the first field in the database for which a dimension value recall is required, and obtain the dimension value corresponding to each first field; Building a dimension value query library for all first fields based on the dictionary tree, wherein the dimension value query library includes a corresponding relationship between each first field and a corresponding dimension value; For each second standard word segment, input each second standard word segment into the dictionary tree to determine whether there is a dimension value corresponding to each second standard word segment in the dimension value query library based on the dictionary tree; For any second standard word segment, when it is determined that a dimension value corresponding to the second standard word segment exists in the dimension value query library, the first field corresponding to the corresponding dimension value is determined as the field corresponding to the second standard word segment in the database.

6. The database table recall method according to claim 1, characterized in that: The step of recalling the table fields of the second standard word segment to determine the fields in the database corresponding to each second standard word segment, and determining the database table where the fields are located includes: For all database tables in the database, obtaining the second field included in each database table and the synonyms corresponding to each second field; Performing vector embedding on each second field and a synonym corresponding to each second field, and storing the generated vector data in a vector database; Determine a similarity score between each second standard word segment and each vector data in the vector database; For any second standard word segment, each vector data is sorted in descending order based on the obtained similarity score, and the second field corresponding to the vector data whose similarity score is within a preset numerical range is determined as the field corresponding to the second standard word segment in the database, and the database table where the field is located is determined.

7. The database table recall method according to claim 1, characterized in that: The filtering of the first standard word segment into a second standard word segment conforming to a preset feature vector dimension comprises: The first standard word segment is filtered into a second standard word segment that meets a preset feature vector dimension based on an n-gram algorithm.

8. The database table recall method according to claim 1, characterized in that: The recall method further comprises: After determining the target database table to be recalled for the query text, displaying the target database table to be recalled; After receiving a selection instruction triggered by a user, the structured query statement for recall is generated based on the selection instruction, wherein the selection instruction is generated based on a database table selected by the user from the target database table to be recalled.

9. The database table recall method according to claim 1, characterized in that: The recall method further comprises: After determining the target database table to be recalled for the query text, inputting the query text and the target database table into a decision model to select a database table from the target database table based on the decision model; The structured query statement for recall is generated based on the screened database table.

10. The database table recall method according to claim 1, characterized in that: The recall method further comprises: For each field of each database table, define a weight value for each field; After determining the target database table to be recalled for the query text, determining a total weight value of each database table based on the number of correspondences between the second standard word segment and the field in the database; The structured query statement for recall is generated based on the database table with the largest total weight value.

11. A database table recall device, characterized in that: include: a memory configured to store instructions; A processor is configured to call the instruction from the memory and implement the database table recall method according to any one of claims 1 to 10 when executing the instruction.

12. A database table recall system, characterized in that: A recall device comprising a database table according to claim 11.

13. A machine-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor is configured to execute the database table recall method according to any one of claims 1 to 10.

Images