773 results about "Column (database)" patented technology

A database system generates a prompt for an LLM or other machine learning (ML) model to narrow the search space to highly relevant information about a database. A distinct instance of a classifier, a clustering algorithm, or a topic modeling model can be trained based on information from ML automation within the database system, respectively for each column or table in the database. Model instances can then be used during generative LLM inferencing to identify relevant sources of data to answer the user's question. Thus, the prompt generation combines ML automation and other ML models or an LLM for topic modeling and schema description.

The invention discloses a context learning-based database query generation method and system and a storage medium, and the method comprises the steps: extracting keywords and phrases from a natural language query, retrieving values similar to the keywords in a database, and returning values most related to the keywords and corresponding columns thereof, retrieving description information related to the keywords and the phrases in a database directory; performing mode selection aiming at the most relevant value of each keyword and the corresponding column, namely screening out the column irrelevant to the natural language query, further selecting a table necessary for generating the SQL query, and finally reducing the mode to a minimum column set necessary for generating the SQL query; and correcting logic and grammar errors of the method model through the database mode, the natural language query, the generated candidate SQL query and the execution result thereof. According to the method, the efficient SQL query statement is generated by intelligently understanding the data context and the query requirement.

A method for extracting and displaying desired information from a set of tables includes storing a set of tables including information associated with a subsurface of the Earth; receiving a user query; selecting a table from the set of tables based on an embedding search performed for the user query, on a vector database of table-column questions of the set of tables; selecting one or more columns from the table based on a likelihood estimation performed in an embedding space, between (1) the user query and (2) a table summary and descriptions of columns for the selected table; determining one or more rows associated with the one or more columns; and displaying one or more answers in response to the user query.

The invention belongs to the field of artificial intelligence, and relates to a query statement generation method, which comprises the following steps: receiving query information input by a user, and extracting query parameters of the query information; according to the query parameter, screening out context prompt information corresponding to the query parameter from a preset database; processing the query parameters and the context prompt information by using a pre-trained large language model to generate an initial query statement; analyzing the initial query statement to obtain an initial table name and an initial column name corresponding to the initial query statement, performing mode linking on the initial table name and the initial column name with a preset database, and generating prompt information after mode linking; and generating a target query statement by adopting a pre-trained large language model according to the prompt information after mode linking. In addition, the invention further relates to a block chain technology, and the generation result of the query statement and the like can be stored in a block chain. The accuracy of generating the query statement can be improved.

The invention belongs to the field of SQL generation, and particularly discloses an SQL generation method and system based on a large language model, a terminal and a medium, a natural language query statement and a target database mode are transmitted into the large language model, and N candidate tables most relevant to user query are selected and fed back by the large language model according to a first prompt word; selecting and feeding back a plurality of candidate data columns most relevant to the user query from each candidate table by the large language model according to the second cue word; retrieving an example question and answer pair similar to the user query from the vector database, wherein the example question and answer pair comprises an example question and an example SQL statement; generating a plurality of candidate SQL statements by the large language model according to the third cue word; and carrying out confidence coefficient filtering on the plurality of generated candidate SQL statements, and selecting a final SQL statement. According to the method, the natural language query statement is converted into the SQL statement through the large language model, the SQL statement generation efficiency and accuracy are improved, and then the usability and accuracy of database query are improved.

The invention discloses a database query generation method and system based on multi-selection optimization and a storage medium. The method comprises the steps that tables and columns related to natural language query are extracted from a database mode through multiple prompts; generating a plurality of candidate SQL queries by utilizing multiple prompts based on the extracted tables and columns related to the natural language queries and a selection strategy based on question similarity and mask question similarity; and performing screening according to the confidence score of each candidate SQL query, and selecting an optimal SQL query through a multi-selection method. According to the method and the device, the accuracy and the robustness of generating the SQL query are improved.

The invention discloses a natural language query method and system for a relational database, and the method comprises the steps: extracting a list structure from a local connection database, and generating a sub-database based on a natural language problem; inputting the sub-database mode and the natural language question into a screening cue word template to obtain a complete cue word, inputting the complete cue word into a large language model, and generating a simplified database mode after obtaining a table name and a column name; putting a natural language question into a prediction cue word template, obtaining a predicted SQL keyword skeleton, then converting the predicted SQL keyword skeleton into a prediction vector, screening out an example database, extracting question and answer pairs in the example database, extracting a keyword skeleton from an SQL statement of the question and answer pairs, and after extracting the keyword skeleton, converting the extracted keyword skeleton into a question and answer pair vector; calculating the similarity to obtain a similarity result and a corresponding prediction vector; the content is input into a cue word template, cue words are input into a large language model, the large language model generates SQL statements through a thinking chain, and natural language query is completed.

The invention provides a Text2SQL generation method and system driven by a large language model, and the method comprises the steps: recognizing key entities and relationships in a natural language question through the large language model, and carrying out the construction to obtain a query graph; constructing a hierarchical graph representation model based on the database mode of the target database; searching candidate sub-image sets which have the same structure as the query graph and are in semantic association with the query graph from the constructed graph representation model; and constructing a structured Prompt template based on the candidate sub-image set and the user question, generating an initial SQL statement, and outputting a final query result after grammar and semantic detection correction. According to the method, a traditional database is converted into the graph knowledge base, so that the SQL generation accuracy and performance are improved; guiding the large model to generate an SQL (Structured Query Language) meeting business requirements through knowledge graph storage and retrieval table and column information; and the accuracy of the SQL is continuously optimized by utilizing a self-adaptive feedback mechanism, so that the method is particularly suitable for a large-scale complex query scene, and self-improvement and customization of the model are realized.

The invention provides an SQL statement analysis method and device, equipment and a storage medium, and relates to the technical field of databases. According to the specific scheme, the method comprises the steps that SQL statements and metadata thereof are obtained, wherein the metadata comprises table names, column names and predicates of the SQL statements; target statistical information corresponding to the metadata is obtained from a preset statistical information base, and the target statistical information comprises table statistical information, column statistical information and index statistical information. And loading the statistical information to a test environment database to obtain a simulation database corresponding to the SQL statement. And based on the statistical information, determining an execution plan of the SQL statement, and executing the SQL statement in the simulation database according to the execution plan to obtain a statement analysis result. The method can effectively solve the problem of analysis distortion caused by the distribution difference of test and production environment data, and can obtain a more accurate SQL analysis result without complex data preparation.

The invention provides a knowledge graph construction method based on a low-code development platform, and belongs to the technical field of knowledge graph construction. The method comprises the following steps: counting and carding metadata and business entity structures in a low-code development platform, and formulating a mapping specification for mapping from a relational database to semantic data; obtaining a target relational database in the configuration file, extracting table data and column data in the target relational database, and converting the table data and the column data into entity class parameters and attribute class parameters in the knowledge graph based on a mapping specification; extracting an unstructured data source related to the target business from the low-code development platform, and converting the unstructured data source into structured data; and associating the structured data with the entity class parameters and the attribute class parameters, and storing the associated data in the knowledge graph. Automatic mapping and storage from the relational database to the knowledge graph are realized, so that the data utilization rate and the business intelligence level are improved.

The invention relates to a Text-to-SQL (Structured Query Language) complex query statement generation method and system based on a pre-trained large model. The method comprises the following steps: acquiring an input sequence comprising a user question and dictionary interpretation of table names and column names in a database; respectively inputting the user question and the dictionary explanation into a Sension-BERT model, obtaining Top-K explanations most related to the user question, and obtaining an enhanced input sequence according to the Top-K explanations most related to the user question; encoding the enhanced input sequence to generate an enhanced semantic vector, and constructing a hierarchical database mode pattern according to the enhanced semantic vector; inputting the hierarchical database mode graph into a hypergraph neural network model to update node features, and obtaining optimized node features; and obtaining a structured query statement according to the optimized node features. According to the method, complex association and multi-level query logic among multiple tables in a complex query scene can be better processed, and the performance in the complex query scene is improved.

A computer-implemented method and system for comparing and analyzing similarities between database tables. The method includes receiving a database instance containing at least two tables with multiple rows and columns, performing irreversible hashing operations on selected data within specified rows and columns of both tables, conducting statistical comparisons between corresponding columns to determine table similarities, and displaying the comparison results through a graphical user interface. The system employs data transformation through one-way hashing techniques and statistical analysis to evaluate structural and content similarities between database tables, enabling efficient comparison of structured data sets while maintaining data security through irreversible hashing.

The invention discloses a joint multi-task table semantic analysis method based on a pre-training model, and relates to the technical field of natural language processing and databases. According to the method, SQL statements are crawled from a specified website, SQL and tables are converted into natural language texts through a large language model, columns and tables are extracted to form positive and negative samples, experimental data is converted into a Spider data set format, and natural language problems and database mode tasks are completed through a cue word template and a few-sample frame; then, an MLNaT model of a 12-layer relation perception Transform architecture is constructed, statements and column names spliced according to a specific format are input, three tasks of mask language setting, column prediction and SQL generation are set, and pre-training is carried out; and finally, evaluating on a Spider data set according to an accurate set matching rate, and taking an RAT-SQL as a baseline model. It is verified that the MLNaT model is superior to the reference model in the aspects of column prediction and SQL generation.

Data structures and methods are described to provide a unique extendable group identifier for efficient aggregation query processing using multiple grouping keys. A method comprises retrieving a database query comprising an aggregate function of a selected column from a database table, grouped by a plurality of columns. The method further comprises maintaining a plurality of hash tables that map column values to grouping keys and identify a maximum grouping key. The method further comprises updating a plurality of bitmasks that define per-column grouping key bit positions, capable of storing the maximum grouping key, within a combined index value. The method further comprises allocating a result array sized according to the plurality of bitmasks. The method further comprises using the plurality of bitmasks and the plurality of hash tables to determine the combined index value to apply the aggregate function on the selected column for each row in the database table.

The embodiment of the invention provides a database table field index establishment method and device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the steps of obtaining a target query statement and statement execution plan information; splitting the target query statement to obtain statement key data; determining the total row number of the candidate database table based on the historical index database; extracting index use data from the candidate database table based on the statement execution plan information; determining a new field index from the candidate column indexes based on the candidate distinction degree of the candidate column indexes; verifying the new field index based on the statement execution plan information, the historical index database, the new region division degree of the new field index and the candidate region division degree to obtain an index verification result; and if the index verification result is that the index establishment verification is passed, performing field index establishment based on the new field index. According to the embodiment of the invention, the database table field index more suitable for the current query request can be automatically established, so that the data query efficiency is improved.

The invention relates to the technical field of natural language processing, and discloses an intelligent slot filling and mode linking method based on RAG and M-Schema, and the method comprises the steps: rapidly matching a preset FAQ knowledge base through an RAG-FAQ module, and extracting a predefined medical intention and a slot structure; if the matching fails, starting an NLU module to perform intention classification and slot position extraction; performing entity linking and normalization on the slot value by utilizing an RAG-medical field knowledge base context enhancement module, performing multi-dimensional verification in combination with M-Schema metadata of a target database, and outputting a matching column and a confidence score; aiming at uncertainty types such as medical term ambiguity and business rule conflict, a clarification problem is dynamically generated, user feedback is received, SQL query conforming to business rules is generated after slot position information is corrected, and finally the SQL query is transmitted to a medical database execution module to output an analysis result. According to the invention, the availability and accuracy of the general framework in the scene facing the hospital management field can be improved.

Techniques for priority-based masking policy selection in a database environment are described. Masking policies are defined and attached to columns of relational data for particular users or roles. The attachment of a masking policy to a column includes a user-specified priority value. When multiple policies could apply to a particular query, the conflict can be easily resolved and understood by use of the priority values, for example, by selecting a candidate policy having a highest priority value.

The invention discloses a method and a system for generating an SQL (Structured Query Language) query based on a natural language question. The method comprises the following steps: converting mode information of a target database and the natural language question into semantic vector representation; based on the semantic vector representation, a simplified database mode set related to SQL query is screened out through an attention mechanism; based on the natural language problem, the simplified database mode set and preset database constraint information, generating an SQL structural skeleton, and filling specific elements of the SQL structural skeleton to form a preliminary SQL query; and performing dynamic correction and verification on the initial SQL query by utilizing a large language model, and outputting a final SQL query. According to the method, association mining between user query and a database mode is effectively enhanced by utilizing a context-aware cross-encoder mechanism, and an implicit corresponding relation between a natural language problem and a database table / column can be more accurately identified and utilized.

The invention discloses a time sequence database architecture, a data writing method and electronic equipment, and the time sequence database architecture comprises a distributed storage module which enables data of the same equipment to be continuously distributed in a physical storage layer based on an equipment dimension fragmentation strategy; the independent index layer is decoupled from the data storage module and maintains a mapping relationship between the equipment identifier and the corresponding physical fragment through a metadata structure; the migration module is used for performing rolling updating on the hot data layer; and the query engine module is used for positioning the target physical fragment according to the equipment identifier and executing predicate push-down calculation on the compressed data. The data writing method comprises the following steps: receiving a data writing request, and analyzing a device identifier in the request; positioning a target physical fragment according to the equipment identifier in cooperation with a Hash algorithm; and adding the data to the column type storage file in the corresponding physical fragment according to a timestamp sequence, and synchronously updating the metadata mapping relationship of the independent index layer. According to the method, the data query performance and query efficiency can be improved, and the data storage cost is reduced.

A computerized method is provided for using large language models to integrate data from multiple sources including technical metadata (e.g. column and table names) with a dictionary of standard abbreviations employed in the metadata, representative data from the columns rows of the table, a business glossary of terms in the data, and representative queries used to interrogate the data, to create a non-technical or natural language description of business value of the table and its columns and data.

The invention provides a query optimization method and device based on OceanBase database data fragmentation and related products, and the method comprises the following steps: determining one or more columns as service keys based on a plurality of service tables, the service keys being used for connecting service data in the service tables; based on the service key, setting one or more columns as fragmentation keys, and combining the fragmentation keys with the service key so as to construct a fragmentation table in the OceanBase database; performing fragmentation processing on target data based on the fragmentation table to create a data fragmentation structure; based on the data fragmentation structure, constructing an extended SQL query statement carrying preset prompt optimization information SQL Hint; on the basis of the extended SQL query statement, a query optimizer of the OceanBase database selects a preset execution plan to query the service table so as to obtain a target result set, and the query performance is improved in combination with the parallel processing capacity of the database.

Methods, systems, and devices for data management are described. A method may include obtaining a new snapshot of a relational database of a source computing environment, comparing the new snapshot of the relational database to a prior snapshot of the relational database, updating, in response to the comparison indicating that data of a row of a relational database table changed between the prior snapshot and the new snapshot, a field of a second versioning control column of a corresponding row of a backup database table of the set of backup database tables to indicate a version of the new snapshot, and adding, in response to the comparison indicating that the data of the row of the relational database table changed, an additional row to the backup database table, where a field of a first versioning control column of the additional row includes an indication of the version of the new snapshot.

Some embodiments include information retrieval through query history insights by accessing query history of a first user, processing the query history of the first user using a first machine learning model to identify naming characteristics of the query history specific for the first user, and enriching a database comprising data associated with the first user with the identified naming characteristics of the query history. The system receives a new search query in natural language from the first user, processes the new search query in the natural language using a second machine learning model to identify embeddings within the new search query, identifies one or more recommended tables and corresponding columns, and causes display of the recommended tables and corresponding columns for each of the recommended tables by a user device of the first user.

The invention provides a data processing method, a server, a storage medium and a program product. The method relates to a computer technology, and comprises the following steps: in a multi-database and mass data table scene, in a global range of databases, according to description information of each database and description information of a table connection sub-graph of each database, roughly recalling related candidate databases; according to the natural language query, the description information of each data table in the candidate database and the description information of each table column, finely sorting the candidate database, determining a target database, a target data table and a target table column associated with the natural language query according to a fine sorting result, and combining rough recall and multi-level fine sorting to obtain a target database; according to the method, the user query is intelligently routed to the associated target database, target data table and target table column, the global data query requirement of the user is met, complex data treatment and scene isolation do not need to be depended on, the multi-database query process is simplified through uniform intelligent routing, and the operation convenience of multi-database query is improved.

The system obtains messages including multiple properties, such as a sender, a receiver, and content. The system creates a database to store the messages and presents the database in a user interface including a table including rows and columns. The database can be customized by including a custom column among the multiple columns. The system customizes the user interface by creating custom views of the database. The system receives an indication to create a view of the multiple messages, where the indication to create the view includes a criterion to select a subset of messages from the messages. The system creates the view including multiple layouts configured to visually provide important information associated with the subset of messages.

The invention discloses an HTAP database query optimization method and device and a storage medium. The query performance is remarkably improved by combining the advantages of row storage and column storage. Comprising the steps of analyzing SQL query to generate a logic query plan; evaluating the execution cost of each logic operator in different storage modes by using a mixed cost model, wherein hyper-parameters of the mixed cost model are optimized through Bayesian search; selectively adopting a dynamic programming algorithm or a greedy algorithm to generate a hybrid physical query plan with the minimum execution cost according to the table number; and executing the plan to obtain a query result. In addition, an MCTS algorithm is further utilized, the optimal column subset is selected in the preset memory budget limit based on the query historical record and loaded into the memory column storage, and the subsequent query performance is further optimized. The problem that row storage and column storage in an existing HTAP system cannot work cooperatively is solved, operator-level mixed execution is achieved, and meanwhile memory resource utilization is optimized.

The invention discloses a system and a method for supporting multi-version concurrency control in a column storage database, belongs to the technical field of databases, and aims to solve the technical problem of how to provide efficient and flexible concurrency control and data operation management. Comprising a multi-version management module for creating a transaction ID and a timestamp for each transaction and maintaining a version linked list for each data segment in a column storage database; the operation optimization module is used for performing data operation on the data in the column storage database based on the submitted transaction, performing operation according to a comparison result of a version number and a transaction timestamp when the data operation is executed, and storing the data before modification into a revocation buffer area; the transaction conflict processing module is used for detecting and processing transaction conflicts based on a preset conflict processing rule; the storage space management module divides the storage of the column storage database into table storage and local storage; and the version linked list management module is used for deleting the version information which is no longer referenced by any active transaction based on the transaction state and the version information.

The invention discloses a database performance optimization method based on partition management. The database performance optimization method comprises database storage optimization and key query table efficiency optimization. The database storage optimization comprises the steps of confirming the partition data volume and creating a corresponding partition large file table space; verifying whether the table can be partitioned; creating a temporary table which is the same as the original table, and creating partitions and partition names according to company column values; copying the data to a temporary table; copying the dependent object; partitioning the original table according to a list rule, and emptying data in the original table; synchronizing the data, the trigger and the index information in the temporary table to the partitioned original table; checking whether the partitioned data is consistent or not, verifying whether front-end query is influenced or not, checking index parameters, adopting the database optimization method, dividing the data into a plurality of partitions, mapping key large table data into different partition tables, reducing the data size of a single table, improving the efficiency and availability of the system and ensuring the stability and reliability of the system.

The invention provides an NL2SQL database table dynamic screening method and system based on multi-strategy fusion. The method comprises the steps that a preliminary candidate set is generated, a semantic encoder is used for encoding a user natural language problem and table and column information in a database into vectors, the semantic similarity is calculated, a preset number of tables are screened out from a whole database table based on the similarity, and a preliminary candidate table set is formed; performing deep analysis and reordering, performing multi-dimensional analysis on the tables in the preliminary candidate table set, and calculating a graph relationship compactness score and a content correlation score of each table; and performing fusion decision, inputting the semantic similarity score, the graph relationship compactness score and the content correlation score of each table into a fusion model to obtain a final comprehensive score of each table, and screening out a final correlation table set based on the final comprehensive score. The method has high accuracy and high recall rate, the generation speed and response efficiency of the NL2SQL system are improved, meanwhile, the robustness is high, and the practicability and generalization ability of the system are improved.

The invention discloses a row-level data permission authorization method and device, equipment and a medium, and relates to the technical field of computers. Comprising the following steps: registering basic data into a preset database; configuring a corresponding field management user with a field management authority for the preset service field; creating or updating a record in a user data permission table based on the user permission application to determine a permission relationship between the user identifier and the entity data; registering a service data table and a mapping relation between an entity column and an entity type in the service data table in a preset service field based on a user instruction of a field management user; and dynamically generating a permission filtering condition based on the query SQL and the user identifier so as to generate an SQL statement with permission filtering, and querying corresponding service data from the service data table based on the mapping relationship so as to generate a corresponding query result based on the service data. Therefore, the efficiency and flexibility of line-level data permission authorization can be improved on the premise of ensuring data security.