3468 results about "Table (database)" patented technology

The invention discloses a Text2SQL (Structured Query Language) implementation method and system, and relates to the field of data processing, and the method comprises the following steps: firstly, receiving a natural language query, and analyzing a query intention, field classification and a key entity through a planner; the searcher obtains domain knowledge, entity information, a database table structure and a historical query mode in a multi-path parallel mode based on the planning result; the generator constructs an SQL framework according to the retrieval result and generates an initial statement; the verifier carries out grammar, table field, authority and logic multi-dimensional verification on the SQL, and if the verification fails, iteration adjustment is carried out to generate logic; when the SQL is executed, the result is formatted and a natural language explanation containing query logic, a data source and a calculation method is generated if the SQL is executed successfully, and a diagnosis and error correction mechanism is started for correction and then rechecking is performed if the SQL is executed unsuccessfully. According to the method, through deep fusion of domain knowledge, whole-process verification error correction and interpretability enhancement, the accuracy, robustness and user interaction experience of SQL conversion in a professional scene are improved.

A database system generates a prompt for an LLM or other machine learning (ML) model to narrow the search space to highly relevant information about a database. A distinct instance of a classifier, a clustering algorithm, or a topic modeling model can be trained based on information from ML automation within the database system, respectively for each column or table in the database. Model instances can then be used during generative LLM inferencing to identify relevant sources of data to answer the user's question. Thus, the prompt generation combines ML automation and other ML models or an LLM for topic modeling and schema description.

The invention relates to the technical field of data processing, can be applied to business scenes of financial science and technology, medical health and the like, and discloses a database migration method, device, equipment and medium. And deploying a double-write control module of the application program layer to write the source database and the target database in parallel, and completing consistency verification through a data verification comparison task. And controlling a target database write-in channel according to a difference analysis result, closing a source database write-in channel in combination with a service takeover state, and completing a migration process from the source database to the target database. According to the method, full migration, application layer double writing, differential data verification and dynamic write-in channel control are combined, data consistency of the source database and the target database in the migration process is guaranteed, the source database and the target database are safely switched to after the service takeover capacity is verified, and the migration safety and continuity are improved.

The present disclosure relates to systems, non-transitory computer-readable media, and methods for linking a database schema to a natural language query. In particular, in some embodiments, the disclosed systems determine, from tables in a database schema, a subset of tables relevant to a natural language query by comparing embeddings for the tables in the database schema and embeddings for the natural language query. Additionally, in some implementations, the disclosed systems select, from a schema graph comprising nodes that represent the tables in the database schema, an additional table along a path between a pair of nodes representing a pair of tables from the subset of tables. Moreover, in some embodiments, the disclosed systems determine a set of relevant tables by appending the additional table to the subset of tables. Furthermore, in some implementations, the disclosed systems generate, from the set of relevant tables, a response for the natural language query.

The invention discloses a method and a system for realizing data synchronization between heterogeneous databases based on logs. The method comprises the following steps: acquiring connection information of a source database and a target database; querying a table structure of the source end database and generating an SQL statement to establish a data mapping rule; matching a corresponding rule model from a mapping rule model library; if the matching is not successful, the system abnormally quits and prompts the user to create a new mapping model and add the new mapping model to the mapping rule model base; converting a table structure of the source end database into a JSON character string based on a standard field type; an SQL statement required by the target database is output through the data mapping engine module; the message monitoring program reads the data in the temporary storage area and converts the data into data meeting the requirements of the target database; formatting the converted data into SQL statements supported by a target database; and sending the generated SQL statement to a target database to complete data migration. Accurate and controllable synchronization of the data and the model in a heterogeneous environment is realized.

A text-to-structured query language conversion method based on a large language model comprises the steps that database outline information is cut, and tables and columns needed by query of a non-target structured query language SQL are eliminated through correlation scoring and Steiner tree problem solving; screening context samples, predicting a similarity score from a text to a structured query language Text-to-SQL by using a cross encoder language model, and selecting a sample most related to a query problem; and injecting the cut database table and column information and the screened context sample into a cue word template, and generating an SQL query statement by using a large language model. The method can be used for optimizing the automatic conversion process from the natural language to the SQL and improving the conversion accuracy, and is used for providing a quick and convenient database retrieval mechanism for a non-SQL user and providing applications such as a data access mode in a wider form for mechanisms such as big language model retrieval enhancement generation RAG and the like.

Systems and methods for translating natural language to SQL on a custom enterprise data warehouse powered by Generative AI. With an embodiment of the present invention, a natural language question may be converted to a meaningful and accurate database query, e.g., SQL query, relevant to tables existing in an enterprise data warehouse. An embodiment of the present invention is directed to a comprehensive approach of transforming a natural language query to a focused SQL query using domain specific data models across firmwide metadata systems and data systems. In response to a user query, an embodiment of the present invention performs metadata analysis, targeted data retrieval and then SQL generation. An embodiment of the present invention may apply data warehousing standards and guidelines followed in the enterprise and provide a plug-and-play type architecture and solution that is scalable to large warehousing and other systems.

The invention provides a method for generating an SQL (Structured Query Language) by a natural language based on bidirectional mapping and semantic parsing, which relates to the technical field of database query and comprises the following steps of: extracting natural language query elements and packaging the natural language query elements into structured data, and establishing a mapping relationship from a query field to a service attribute and a physical data table by adopting a bidirectional Hash index technology; and automatically identifying multi-table association keys, performing semantic extension and compliance verification, generating an abstract syntax tree, performing processing according to user permission, and finally converting the abstract syntax tree into an SQL statement conforming to a target database syntax specification. According to the method, the accuracy and the efficiency of converting the natural language into the SQL are improved, and the flexibility and the safety of the system are enhanced.

The invention discloses a multi-table combined natural language query SQL (structured query language) generation method, which comprises the following steps of: 1, constructing a database meta-knowledge graph, and establishing a triple storage comprising a table structure, a primary and foreign key relationship and business description for each data table; 2, receiving a natural language query request, and calculating the topic relevancy between query semantics and each database table through a pre-trained topic matching model; 3, dynamically constructing a view, and logically associating the database tables of which the theme relevancy exceeds a threshold value to form a temporary view; 4, generating a context enhancement prompt, and combining the temporary view structure, the field semantic description of the view and the view content sample to form a structured prompt; 5, inputting the natural language query and the structured prompt into the large language model to generate candidate SQL statements; and step 6, executing verification and iterative optimization on the candidate SQL statements, verifying logic correctness through SQL execution plan analysis and result sampling, and triggering and prompting a reconstruction mechanism when detection is abnormal.

The invention provides a different types of database visualization table management method and system, and relates to the technical field of computers.The method comprises the steps that a database general operation system comprising a general operation interface and a dialect abstract base class is constructed, and a query performance evaluation system is established based on performance data; and predicting an optimal database connection mode through deep reinforcement learning and generating a multi-stage execution plan. Meanwhile, an anomaly feature knowledge base based on a graph neural network is constructed, and anomaly prediction and preventive processing are carried out based on the anomaly feature knowledge base. A multi-stage execution plan is executed after a database operation request is subjected to grammar adaptation, and a distributed transaction processor dynamically adjusts a transaction isolation level and a concurrency control parameter. And finally, according to the terminal and the data volume, selecting a proper data display mode to carry out visual presentation.

The invention provides a database adaptive migration method and system based on a credential environment, and relates to the technical field of data processing.The method comprises the steps that an intelligent migration model is used for analyzing the structure, including a table structure, index configuration, constraint conditions and data types, of a source database, data size distribution is calculated, and a source database structure feature result is generated; according to a source database structure feature result, automatically selecting and configuring a target database type, adjusting environment parameters of a target database, and generating target database configuration information; according to the structure of a source database and configuration information of a target database, a data mapping rule is automatically established by utilizing an intelligent migration model, and the data mapping rule comprises automatic correspondence of table names and field names, automatic conversion of data types, and an automatic migration scheme of indexes and constraints. According to the method, automatic and intelligent migration from the source database to the target database is realized through the intelligent migration model, the performance is automatically optimized, the migration report is generated, and the migration efficiency and quality are improved.

The embodiment of the invention provides a knowledge base construction method, a data processing method, equipment, a storage medium and a program product. The method comprises the following steps: generating table description information of a data table in database data and field description of the data table based on the database data and document data; identifying table relation information corresponding to the data table from historical query statements of the database data; therefore, the asset knowledge base can be constructed based on the table description information, the field description information and the table relation information, the table description information and the field description information are generated not only based on the database data, but also based on the document data, and the table relation information is recognized and obtained from the table description information and the field description information in combination with the historical query statements and the document data. Therefore, the information in the constructed asset knowledge base is rich and available, and the data quality of the asset knowledge base is improved.

The invention relates to a method for generating steel field TEXT2SQL based on LLM enhanced by RAG, which comprises the following steps: firstly, establishing connection with a steel database, and defining user semantic information and a database mode; then, obtaining user input and carrying out natural language processing; thirdly, guiding a large language model to generate a more accurate SQL query by utilizing the generated group of guiding words prompt; then, natural language query is converted into SQL query in combination with a large language model and an RAG technology; when a new user query is received, searching a historical query most similar to the new user query and a corresponding SQL query from the vector database; and finally, performing grammar correctness and logic rationality verification on the generated SQL statement, and outputting the SQL statement. According to the method, the problem that table names and field names are difficult to obtain when a large language model generates accurate SQL statements according to user description is solved. According to the method, the high-quality SQL query statement can be effectively generated, and the accuracy and the use efficiency of steel database query are improved.

The invention discloses a method and a device for adaptively generating a structural query statement by using a natural language. The method comprises the following steps: acquiring a natural language query statement input by a user; vectorizing the natural language query statement to obtain a first query statement and a data table and an SQL statement corresponding to the first query statement; rewriting the natural language query statement to obtain a second query statement; recalling a candidate data table and a candidate table field based on the second query statement and a preset to-be-queried data table; the first input data and the first sample data are input into the preset large language model, the query statement and the reasoning chain are structured, the natural language query statement input by the user is corrected, the difference between the natural language query statement and the database table structure is reduced, and the accuracy of data table recall is improved; by recalling the candidate data table, the reasoning difficulty is reduced, so that the accuracy of generating the structured query statement according to the natural language query statement is improved.

The invention discloses a context learning-based database query generation method and system and a storage medium, and the method comprises the steps: extracting keywords and phrases from a natural language query, retrieving values similar to the keywords in a database, and returning values most related to the keywords and corresponding columns thereof, retrieving description information related to the keywords and the phrases in a database directory; performing mode selection aiming at the most relevant value of each keyword and the corresponding column, namely screening out the column irrelevant to the natural language query, further selecting a table necessary for generating the SQL query, and finally reducing the mode to a minimum column set necessary for generating the SQL query; and correcting logic and grammar errors of the method model through the database mode, the natural language query, the generated candidate SQL query and the execution result thereof. According to the method, the efficient SQL query statement is generated by intelligently understanding the data context and the query requirement.

The invention provides an intelligent AI question answering method based on domain knowledge graph and table paraphrasing enhancement generation, which comprises the following steps: constructing a domain knowledge graph, and performing paraphrasing enhancement on a domain database table to obtain a table paraphrasing enhancement knowledge base; receiving a user question, and analyzing a keyword of the user question through a natural language processing technology; performing intention understanding analysis on the keyword based on a domain knowledge graph to obtain a knowledge graph retrieval result, the knowledge graph retrieval result at least comprising N pieces of similar path information matched with a user question in the knowledge graph; searching M reference tables in a table paraphrasing enhancement knowledge base based on the N pieces of similar path information; and generating an SQL query statement according to the N pieces of similar path information and the M reference tables by utilizing a language large model, and querying in a database based on the SQL query statement to obtain an answer to the user question.

The invention discloses a heterogeneous data conversion method and system based on a multi-modal large model, and aims to solve the problems of single-modal limitation, high migration cost and the like in traditional heterogeneous data processing. The method comprises the following steps: firstly, carrying out format adaptation and classification preprocessing on multi-source heterogeneous data such as images, texts, audios and videos; then, through a ViT, BERT and Transform cross attention mechanism, deep analysis and cross-modal semantic association of visual, text and audio features are realized, and visual-text-audio semantic mapping is established; and based on a dynamic prompt project, guiding the multi-modal large model to generate structured data according to a preset Schema zero sample. And then error correction and format normalization are carried out through a Drools rule engine to ensure that the data meet the business standard. And finally, supporting multi-format output of JSON / XML / database table and the like, and meanwhile, associating original data storage to realize data traceability. According to the method, the limitation of single-modal processing is broken through, and cross-modal data deep association is realized.

The invention discloses a self-adaptive SQL differential privacy noise adding method, and relates to the technical field of data protection. In order to accurately calculate and add noise of a proper magnitude in a complex SQL statement, the adopted scheme comprises the following steps: regularly collecting and updating metadata of a database, analyzing attribute features of a data table, and forming a data attribute feature library and a corresponding data sensitivity library; the SQL query request is rewritten into an abstract syntax tree and traversed, and an aggregation function, multi-table connection and a sub-query structure are recorded; key semantic information contained in the request is bound with the data sensitivity library, the sensitivity of the request is calculated, and a privacy budget is distributed in a self-adaptive mode; the privacy budget allowance is checked, when the allowance is sufficient, a data disturbance mechanism meeting-differential privacy is used for disturbing the query result, a disturbed result is obtained, and otherwise, the request is rejected. The method is used for accurately calculating the dynamic allocation privacy budget according to the characteristics of the complex SQL query.

The invention discloses a query statement conversion method and device based on a large model, equipment and a medium, and relates to the field of computer science, and the method comprises the steps: obtaining a natural language query statement sent by a user side, comparing the natural language query statement with a high-frequency problem in a target knowledge base, and obtaining a query statement conversion result; whether a high-frequency problem with the similarity with the natural language query statement exceeding a preset similarity threshold value exists in the target knowledge base or not is judged; if the high-frequency problem that the similarity with the natural language query statement exceeds a preset similarity threshold does not exist, determining a target database table corresponding to the natural language query statement based on the target knowledge graph, and extracting a creation statement corresponding to the target database table, so as to determine target information corresponding to the target database table according to the creation statement; and analyzing the target information by using a preset large model to generate an SQL statement corresponding to the natural language query statement. According to the method and the device, the natural language query statement can be efficiently and accurately converted into the structured SQL statement.

A method for extracting and displaying desired information from a set of tables includes storing a set of tables including information associated with a subsurface of the Earth; receiving a user query; selecting a table from the set of tables based on an embedding search performed for the user query, on a vector database of table-column questions of the set of tables; selecting one or more columns from the table based on a likelihood estimation performed in an embedding space, between (1) the user query and (2) a table summary and descriptions of columns for the selected table; determining one or more rows associated with the one or more columns; and displaying one or more answers in response to the user query.

The invention discloses an intelligent bid evaluation expert extraction and management system, and relates to the technical field of expert extraction systems. Comprising a data cleaning and compression module, a database logic and table relation design module, an expert information collection standard and automation module, an information real-time updating and principal management module, an extraction algorithm design module, a safety and authority management module and a real-time monitoring and auditing function module. Through a self-adaptive sparse auto-encoder and a dynamic clustering technology in the data cleaning and compression module, abnormal values in a bid evaluation expert database can be detected and cleaned, data compression is performed by adopting a self-adaptive dictionary coding algorithm, and the storage efficiency and query performance of the database are improved; based on a multi-modal information fusion and reinforcement learning method, accurate collection, dynamic updating and intelligent management of expert information are realized. The randomness and fairness of the bid evaluation expert extraction process are ensured by using a mode of combining deep reinforcement learning and a quantum random number generator in an extraction algorithm design module.

The invention relates to the technical field of semantic processing, in particular to a context-aware SQL (structured query language) generation method based on table field semantic enhancement, which comprises the following steps of: acquiring financial management and investment combination data of a user and mode information of a background database, constructing a mixed semantic index database, receiving natural language query input by the user, and generating a query result. Performing intention judgment on the query by utilizing the lightweight intention classification model, and if the intention is judged to be matched, calling a parameterized SQL template bound with the query intention and filling parameters to generate a final SQL query; if it is judged that the query intention is an exploration type intention, executing a two-stage SQL generation process, including generating an SQL skeleton according to the query intention, performing target type field recall based on the SQL skeleton, and filling the recalled field into the SQL skeleton to generate a final SQL query; and monitoring the execution performance of the final SQL query, and updating the query popularity metadata and the calculation cost metadata. According to the method, the SQL generation efficiency is improved through intelligent shunting and self-optimization.

The invention provides a data query method and system for converting a natural language into a database query language, and the method comprises the steps: analyzing the natural language input of a user through a multi-modal understanding agent on the basis of constructing a dynamic knowledge graph based on metadata, combining a historical session with a business term table, eliminating ambiguity, and generating a standardized Query, a retrieval routing agent selects a query strategy according to Query complexity, simple query directly matches a cache template, complex query traverses a knowledge graph, and related tables, fields and service constraints are returned; then, an expert committee agent generates an SQL (Structured Query Language) by adopting multi-stage collaboration, executes plan pre-evaluation, and selects a version with the highest comprehensive score; the test agent simulates and executes the SQL in the isolation environment, and verifies the grammar legality and the field permission; and finally, executing the detected SQL, and processing a result. According to the method, the accuracy of converting the natural language into the SQL (NL2SQL) in a complex database scene can be effectively improved.

A set of values can be selected from a plurality of fields of a table in a database. At least one adverb or adjective can be selected for the set of values. Join paths for values in the set of values can be determined. A structured query language pattern can be determined based, at least in part, on at least one value in the set of values, the at least one adverb or adjective for the set of values, and the join paths for the set of values. The structured query language pattern can be stored to first training data configured, at least in part, for use in machine learning to train a text-to-SQL model, the text-to-SQL model comprising a first artificial neural network and configured to convert first natural language text to a first structured query language query.

The invention discloses an SQL (Structured Query Language) statement generation method and equipment based on a natural language and a medium, and the method comprises the following steps: constructing an RAG knowledge base and a data table knowledge graph of a project, then obtaining a to-be-converted natural language of the project, and performing keyword retrieval on the natural language through the RAG knowledge base to obtain an association table set; performing graph traversal on the natural language through the data table knowledge graph to obtain an associated node set; packaging the screening condition of the natural language to be converted, the association table set and the association node set to generate a context knowledge base of the natural language; and finally, according to the context knowledge base and the semantic conversion model, obtaining an SQL statement of a natural language. The problem of low efficiency of multi-table association path discovery is solved by constructing a dynamic extensible database relationship graph; a unified retrieval enhancement mechanism for the RAG knowledge base, the extensible knowledge graph and the historical query features is established, and the problem that target context retrieval is low in efficiency is solved.

A computer system uses a machine-learned language model to generate an SQL query for a user query. The system receives a user query comprising a task for performing a database query. The system identifies an embedding for the user query to represent the user query. The system generates a prompt for input to a machine-learned language model, and the prompt specifies the user query, metadata associated with the identified data table and a request to generate one or more SQL statements for performing the database query on the data table. The system provides the prompt to a model serving system and receives an output generated that includes the requested SQL statements for performing the database query. The system presents a response to the user query using the received SQL statements.

The invention relates to the technical field of artificial intelligence, and particularly discloses a data query method and device based on a large language model and a medium. By training the large language model in advance, the large language model can be better guided to accurately generate the language expression meeting the user requirement in the database field table range, the understanding ability of the model for the field table in the specific database range is remarkably improved, meanwhile, training of internal data is avoided in the aspect of data query, and the user experience is improved. According to the method, the SQL statements are generated through a large model, data privacy of different scenes and different enterprises is effectively considered, a personalized answer mode suitable for the user can be assembled according to problems and context information of the user, the satisfaction degree of the user is improved, and interactivity and friendliness in the data query process are enhanced.

The invention belongs to the field of SQL generation, and particularly discloses an SQL generation method and system based on a large language model, a terminal and a medium, a natural language query statement and a target database mode are transmitted into the large language model, and N candidate tables most relevant to user query are selected and fed back by the large language model according to a first prompt word; selecting and feeding back a plurality of candidate data columns most relevant to the user query from each candidate table by the large language model according to the second cue word; retrieving an example question and answer pair similar to the user query from the vector database, wherein the example question and answer pair comprises an example question and an example SQL statement; generating a plurality of candidate SQL statements by the large language model according to the third cue word; and carrying out confidence coefficient filtering on the plurality of generated candidate SQL statements, and selecting a final SQL statement. According to the method, the natural language query statement is converted into the SQL statement through the large language model, the SQL statement generation efficiency and accuracy are improved, and then the usability and accuracy of database query are improved.

The invention belongs to the technical field of data processing, and discloses an Excel file dynamic analysis and verification method and system based on a rule template and a storage medium. Comprising the steps that a rule template is generated according to configuration information, and the rule template comprises one piece of table unit data and one or more pieces of column metadata; dynamically creating a database business data table for storing the analyzed Excel data based on the column metadata; reading the rule template, obtaining a corresponding form in the to-be-analyzed Excel file according to the table unit data, extracting Excel data in each row and each column in the form according to the column metadata, and storing the Excel data in a business data table; and querying a service data row in the database service data table, obtaining a verification rule according to the column metadata corresponding to the Excel data of each column in the service data row, and verifying the current Excel data according to the verification rule. According to the method, the adaptation capability of verification is enhanced, and meanwhile, customized development cost is not generated, so that the analysis and verification of the Excel are more flexible.

The invention discloses a database query generation method and system based on multi-selection optimization and a storage medium. The method comprises the steps that tables and columns related to natural language query are extracted from a database mode through multiple prompts; generating a plurality of candidate SQL queries by utilizing multiple prompts based on the extracted tables and columns related to the natural language queries and a selection strategy based on question similarity and mask question similarity; and performing screening according to the confidence score of each candidate SQL query, and selecting an optimal SQL query through a multi-selection method. According to the method and the device, the accuracy and the robustness of generating the SQL query are improved.