1072 results about "Column (database)" patented technology

A database system generates a prompt for an LLM or other machine learning (ML) model to narrow the search space to highly relevant information about a database. A distinct instance of a classifier, a clustering algorithm, or a topic modeling model can be trained based on information from ML automation within the database system, respectively for each column or table in the database. Model instances can then be used during generative LLM inferencing to identify relevant sources of data to answer the user's question. Thus, the prompt generation combines ML automation and other ML models or an LLM for topic modeling and schema description.

A text-to-structured query language conversion method based on a large language model comprises the steps that database outline information is cut, and tables and columns needed by query of a non-target structured query language SQL are eliminated through correlation scoring and Steiner tree problem solving; screening context samples, predicting a similarity score from a text to a structured query language Text-to-SQL by using a cross encoder language model, and selecting a sample most related to a query problem; and injecting the cut database table and column information and the screened context sample into a cue word template, and generating an SQL query statement by using a large language model. The method can be used for optimizing the automatic conversion process from the natural language to the SQL and improving the conversion accuracy, and is used for providing a quick and convenient database retrieval mechanism for a non-SQL user and providing applications such as a data access mode in a wider form for mechanisms such as big language model retrieval enhancement generation RAG and the like.

The embodiment of the invention provides a file resource reuse library management method and device based on a database, and efficient management of project resources is realized by constructing a distributed column family database cluster and dividing the distributed column family database cluster into a plurality of special column families. A multi-dimensional index mechanism is innovatively designed and comprises reverse indexing, reverse indexing and version linking, and flexible resource retrieval is supported. A distributed version lock and a transaction manager are introduced to process concurrent access, and a multi-user cooperation scene is intelligently processed through a conflict detector. And the system automatically processes resource import and tool chain configuration, so that the consistency and reliability of resource reuse are ensured. According to the method, the limitation of traditional file resource management is broken through, and an efficient resource reuse solution is provided for large-scale project development.

The invention discloses a context learning-based database query generation method and system and a storage medium, and the method comprises the steps: extracting keywords and phrases from a natural language query, retrieving values similar to the keywords in a database, and returning values most related to the keywords and corresponding columns thereof, retrieving description information related to the keywords and the phrases in a database directory; performing mode selection aiming at the most relevant value of each keyword and the corresponding column, namely screening out the column irrelevant to the natural language query, further selecting a table necessary for generating the SQL query, and finally reducing the mode to a minimum column set necessary for generating the SQL query; and correcting logic and grammar errors of the method model through the database mode, the natural language query, the generated candidate SQL query and the execution result thereof. According to the method, the efficient SQL query statement is generated by intelligently understanding the data context and the query requirement.

A method for extracting and displaying desired information from a set of tables includes storing a set of tables including information associated with a subsurface of the Earth; receiving a user query; selecting a table from the set of tables based on an embedding search performed for the user query, on a vector database of table-column questions of the set of tables; selecting one or more columns from the table based on a likelihood estimation performed in an embedding space, between (1) the user query and (2) a table summary and descriptions of columns for the selected table; determining one or more rows associated with the one or more columns; and displaying one or more answers in response to the user query.

The invention belongs to the field of artificial intelligence, and relates to a query statement generation method, which comprises the following steps: receiving query information input by a user, and extracting query parameters of the query information; according to the query parameter, screening out context prompt information corresponding to the query parameter from a preset database; processing the query parameters and the context prompt information by using a pre-trained large language model to generate an initial query statement; analyzing the initial query statement to obtain an initial table name and an initial column name corresponding to the initial query statement, performing mode linking on the initial table name and the initial column name with a preset database, and generating prompt information after mode linking; and generating a target query statement by adopting a pre-trained large language model according to the prompt information after mode linking. In addition, the invention further relates to a block chain technology, and the generation result of the query statement and the like can be stored in a block chain. The accuracy of generating the query statement can be improved.

The invention belongs to the field of SQL generation, and particularly discloses an SQL generation method and system based on a large language model, a terminal and a medium, a natural language query statement and a target database mode are transmitted into the large language model, and N candidate tables most relevant to user query are selected and fed back by the large language model according to a first prompt word; selecting and feeding back a plurality of candidate data columns most relevant to the user query from each candidate table by the large language model according to the second cue word; retrieving an example question and answer pair similar to the user query from the vector database, wherein the example question and answer pair comprises an example question and an example SQL statement; generating a plurality of candidate SQL statements by the large language model according to the third cue word; and carrying out confidence coefficient filtering on the plurality of generated candidate SQL statements, and selecting a final SQL statement. According to the method, the natural language query statement is converted into the SQL statement through the large language model, the SQL statement generation efficiency and accuracy are improved, and then the usability and accuracy of database query are improved.

The invention belongs to the technical field of data processing, and discloses an Excel file dynamic analysis and verification method and system based on a rule template and a storage medium. Comprising the steps that a rule template is generated according to configuration information, and the rule template comprises one piece of table unit data and one or more pieces of column metadata; dynamically creating a database business data table for storing the analyzed Excel data based on the column metadata; reading the rule template, obtaining a corresponding form in the to-be-analyzed Excel file according to the table unit data, extracting Excel data in each row and each column in the form according to the column metadata, and storing the Excel data in a business data table; and querying a service data row in the database service data table, obtaining a verification rule according to the column metadata corresponding to the Excel data of each column in the service data row, and verifying the current Excel data according to the verification rule. According to the method, the adaptation capability of verification is enhanced, and meanwhile, customized development cost is not generated, so that the analysis and verification of the Excel are more flexible.

The invention discloses a database-based data indexing and query optimization system and method, belongs to the technical field of databases, and aims to solve the technical problem of how to improve the data indexing efficiency and query performance of a database system. According to the technical scheme, the system comprises a query optimization module, an index construction module and a data storage module; the data storage module is used for storing data records in a database; data records are stored in a column mode, the data compression rate and query performance storage are improved, and disk I / O operation is reduced; the index construction module is used for constructing an index according to the data record in the data storage module, the index adopts a B + tree design, and the number of I / O operations during data transmission between a disk and a memory is reduced; and the query optimization module is used for receiving a query request and optimizing a query path according to the index constructed by the index construction module.

The invention discloses a database query generation method and system based on multi-selection optimization and a storage medium. The method comprises the steps that tables and columns related to natural language query are extracted from a database mode through multiple prompts; generating a plurality of candidate SQL queries by utilizing multiple prompts based on the extracted tables and columns related to the natural language queries and a selection strategy based on question similarity and mask question similarity; and performing screening according to the confidence score of each candidate SQL query, and selecting an optimal SQL query through a multi-selection method. According to the method and the device, the accuracy and the robustness of generating the SQL query are improved.

The invention discloses a natural language query method and system for a relational database, and the method comprises the steps: extracting a list structure from a local connection database, and generating a sub-database based on a natural language problem; inputting the sub-database mode and the natural language question into a screening cue word template to obtain a complete cue word, inputting the complete cue word into a large language model, and generating a simplified database mode after obtaining a table name and a column name; putting a natural language question into a prediction cue word template, obtaining a predicted SQL keyword skeleton, then converting the predicted SQL keyword skeleton into a prediction vector, screening out an example database, extracting question and answer pairs in the example database, extracting a keyword skeleton from an SQL statement of the question and answer pairs, and after extracting the keyword skeleton, converting the extracted keyword skeleton into a question and answer pair vector; calculating the similarity to obtain a similarity result and a corresponding prediction vector; the content is input into a cue word template, cue words are input into a large language model, the large language model generates SQL statements through a thinking chain, and natural language query is completed.

The invention provides a Text2SQL generation method and system driven by a large language model, and the method comprises the steps: recognizing key entities and relationships in a natural language question through the large language model, and carrying out the construction to obtain a query graph; constructing a hierarchical graph representation model based on the database mode of the target database; searching candidate sub-image sets which have the same structure as the query graph and are in semantic association with the query graph from the constructed graph representation model; and constructing a structured Prompt template based on the candidate sub-image set and the user question, generating an initial SQL statement, and outputting a final query result after grammar and semantic detection correction. According to the method, a traditional database is converted into the graph knowledge base, so that the SQL generation accuracy and performance are improved; guiding the large model to generate an SQL (Structured Query Language) meeting business requirements through knowledge graph storage and retrieval table and column information; and the accuracy of the SQL is continuously optimized by utilizing a self-adaptive feedback mechanism, so that the method is particularly suitable for a large-scale complex query scene, and self-improvement and customization of the model are realized.

The invention provides an SQL statement analysis method and device, equipment and a storage medium, and relates to the technical field of databases. According to the specific scheme, the method comprises the steps that SQL statements and metadata thereof are obtained, wherein the metadata comprises table names, column names and predicates of the SQL statements; target statistical information corresponding to the metadata is obtained from a preset statistical information base, and the target statistical information comprises table statistical information, column statistical information and index statistical information. And loading the statistical information to a test environment database to obtain a simulation database corresponding to the SQL statement. And based on the statistical information, determining an execution plan of the SQL statement, and executing the SQL statement in the simulation database according to the execution plan to obtain a statement analysis result. The method can effectively solve the problem of analysis distortion caused by the distribution difference of test and production environment data, and can obtain a more accurate SQL analysis result without complex data preparation.

The invention provides a knowledge graph construction method based on a low-code development platform, and belongs to the technical field of knowledge graph construction. The method comprises the following steps: counting and carding metadata and business entity structures in a low-code development platform, and formulating a mapping specification for mapping from a relational database to semantic data; obtaining a target relational database in the configuration file, extracting table data and column data in the target relational database, and converting the table data and the column data into entity class parameters and attribute class parameters in the knowledge graph based on a mapping specification; extracting an unstructured data source related to the target business from the low-code development platform, and converting the unstructured data source into structured data; and associating the structured data with the entity class parameters and the attribute class parameters, and storing the associated data in the knowledge graph. Automatic mapping and storage from the relational database to the knowledge graph are realized, so that the data utilization rate and the business intelligence level are improved.

The invention provides an index recommendation method and device, electronic equipment, a storage medium and a program product, and belongs to the technical field of data processing.The method comprises the steps that candidate composite indexes are determined based on the field distinction degree and the field occurrence frequency of multiple fields in a slow query statement; and determining an optimal recommendation index based on the execution cost of the candidate composite index. According to the index recommendation method and device, the electronic equipment, the storage medium and the program product provided by the invention, semantic features of two dimensions of field distinction degree and field occurrence frequency are fully utilized for an existing slow query statement with a poor index execution effect, and quantitative evaluation is performed through the execution cost of the candidate composite index, so that the efficiency of the slow query statement is improved. The advantages and disadvantages of the index are evaluated more comprehensively and objectively, the situation that the index created by depending on subjective experience does not accord with the actual situation is overcome, compared with a single-column index, the recommended composite index can better improve the statement query performance of a database, slow query is avoided, and better user experience is provided.

The invention relates to a Text-to-SQL (Structured Query Language) complex query statement generation method and system based on a pre-trained large model. The method comprises the following steps: acquiring an input sequence comprising a user question and dictionary interpretation of table names and column names in a database; respectively inputting the user question and the dictionary explanation into a Sension-BERT model, obtaining Top-K explanations most related to the user question, and obtaining an enhanced input sequence according to the Top-K explanations most related to the user question; encoding the enhanced input sequence to generate an enhanced semantic vector, and constructing a hierarchical database mode pattern according to the enhanced semantic vector; inputting the hierarchical database mode graph into a hypergraph neural network model to update node features, and obtaining optimized node features; and obtaining a structured query statement according to the optimized node features. According to the method, complex association and multi-level query logic among multiple tables in a complex query scene can be better processed, and the performance in the complex query scene is improved.

A method includes receiving instructions to perform order-dependent DataFrame operation on data. In response to receiving the instructions, the framework analyzes the instructions to identify the order-dependent DataFrame operation, and generates an executable query corresponding to the identified order-dependent DataFrame operation. The framework executes the generated executable query on the data stored in the first database in the cloud data platform, creates a row position column that generates for each row a unique integer identifier, reproducible across multiple sessions and / or queries for the same underlying data to make the data accessible via positional indexing, and assigns each row of the first database a unique row index value based on the row position column and the row position column order. The framework orders a result from performing the order-dependent DataFrame operation based on the unique row index value assigned to each row of the first database, and returns the ordered result to the user.

A computer-implemented method and system for comparing and analyzing similarities between database tables. The method includes receiving a database instance containing at least two tables with multiple rows and columns, performing irreversible hashing operations on selected data within specified rows and columns of both tables, conducting statistical comparisons between corresponding columns to determine table similarities, and displaying the comparison results through a graphical user interface. The system employs data transformation through one-way hashing techniques and statistical analysis to evaluate structural and content similarities between database tables, enabling efficient comparison of structured data sets while maintaining data security through irreversible hashing.

The invention discloses a joint multi-task table semantic analysis method based on a pre-training model, and relates to the technical field of natural language processing and databases. According to the method, SQL statements are crawled from a specified website, SQL and tables are converted into natural language texts through a large language model, columns and tables are extracted to form positive and negative samples, experimental data is converted into a Spider data set format, and natural language problems and database mode tasks are completed through a cue word template and a few-sample frame; then, an MLNaT model of a 12-layer relation perception Transform architecture is constructed, statements and column names spliced according to a specific format are input, three tasks of mask language setting, column prediction and SQL generation are set, and pre-training is carried out; and finally, evaluating on a Spider data set according to an accurate set matching rate, and taking an RAT-SQL as a baseline model. It is verified that the MLNaT model is superior to the reference model in the aspects of column prediction and SQL generation.

Data structures and methods are described to provide a unique extendable group identifier for efficient aggregation query processing using multiple grouping keys. A method comprises retrieving a database query comprising an aggregate function of a selected column from a database table, grouped by a plurality of columns. The method further comprises maintaining a plurality of hash tables that map column values to grouping keys and identify a maximum grouping key. The method further comprises updating a plurality of bitmasks that define per-column grouping key bit positions, capable of storing the maximum grouping key, within a combined index value. The method further comprises allocating a result array sized according to the plurality of bitmasks. The method further comprises using the plurality of bitmasks and the plurality of hash tables to determine the combined index value to apply the aggregate function on the selected column for each row in the database table.

The embodiment of the invention provides a database table field index establishment method and device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology. The method comprises the steps of obtaining a target query statement and statement execution plan information; splitting the target query statement to obtain statement key data; determining the total row number of the candidate database table based on the historical index database; extracting index use data from the candidate database table based on the statement execution plan information; determining a new field index from the candidate column indexes based on the candidate distinction degree of the candidate column indexes; verifying the new field index based on the statement execution plan information, the historical index database, the new region division degree of the new field index and the candidate region division degree to obtain an index verification result; and if the index verification result is that the index establishment verification is passed, performing field index establishment based on the new field index. According to the embodiment of the invention, the database table field index more suitable for the current query request can be automatically established, so that the data query efficiency is improved.

The invention relates to the technical field of natural language processing, and discloses an intelligent slot filling and mode linking method based on RAG and M-Schema, and the method comprises the steps: rapidly matching a preset FAQ knowledge base through an RAG-FAQ module, and extracting a predefined medical intention and a slot structure; if the matching fails, starting an NLU module to perform intention classification and slot position extraction; performing entity linking and normalization on the slot value by utilizing an RAG-medical field knowledge base context enhancement module, performing multi-dimensional verification in combination with M-Schema metadata of a target database, and outputting a matching column and a confidence score; aiming at uncertainty types such as medical term ambiguity and business rule conflict, a clarification problem is dynamically generated, user feedback is received, SQL query conforming to business rules is generated after slot position information is corrected, and finally the SQL query is transmitted to a medical database execution module to output an analysis result. According to the invention, the availability and accuracy of the general framework in the scene facing the hospital management field can be improved.

Techniques for priority-based masking policy selection in a database environment are described. Masking policies are defined and attached to columns of relational data for particular users or roles. The attachment of a masking policy to a column includes a user-specified priority value. When multiple policies could apply to a particular query, the conflict can be easily resolved and understood by use of the priority values, for example, by selecting a candidate policy having a highest priority value.

The invention discloses a method and a system for generating an SQL (Structured Query Language) query based on a natural language question. The method comprises the following steps: converting mode information of a target database and the natural language question into semantic vector representation; based on the semantic vector representation, a simplified database mode set related to SQL query is screened out through an attention mechanism; based on the natural language problem, the simplified database mode set and preset database constraint information, generating an SQL structural skeleton, and filling specific elements of the SQL structural skeleton to form a preliminary SQL query; and performing dynamic correction and verification on the initial SQL query by utilizing a large language model, and outputting a final SQL query. According to the method, association mining between user query and a database mode is effectively enhanced by utilizing a context-aware cross-encoder mechanism, and an implicit corresponding relation between a natural language problem and a database table / column can be more accurately identified and utilized.

The invention discloses a time sequence database architecture, a data writing method and electronic equipment, and the time sequence database architecture comprises a distributed storage module which enables data of the same equipment to be continuously distributed in a physical storage layer based on an equipment dimension fragmentation strategy; the independent index layer is decoupled from the data storage module and maintains a mapping relationship between the equipment identifier and the corresponding physical fragment through a metadata structure; the migration module is used for performing rolling updating on the hot data layer; and the query engine module is used for positioning the target physical fragment according to the equipment identifier and executing predicate push-down calculation on the compressed data. The data writing method comprises the following steps: receiving a data writing request, and analyzing a device identifier in the request; positioning a target physical fragment according to the equipment identifier in cooperation with a Hash algorithm; and adding the data to the column type storage file in the corresponding physical fragment according to a timestamp sequence, and synchronously updating the metadata mapping relationship of the independent index layer. According to the method, the data query performance and query efficiency can be improved, and the data storage cost is reduced.

A computerized method is provided for using large language models to integrate data from multiple sources including technical metadata (e.g. column and table names) with a dictionary of standard abbreviations employed in the metadata, representative data from the columns rows of the table, a business glossary of terms in the data, and representative queries used to interrogate the data, to create a non-technical or natural language description of business value of the table and its columns and data.

The invention provides a query optimization method and device based on OceanBase database data fragmentation and related products, and the method comprises the following steps: determining one or more columns as service keys based on a plurality of service tables, the service keys being used for connecting service data in the service tables; based on the service key, setting one or more columns as fragmentation keys, and combining the fragmentation keys with the service key so as to construct a fragmentation table in the OceanBase database; performing fragmentation processing on target data based on the fragmentation table to create a data fragmentation structure; based on the data fragmentation structure, constructing an extended SQL query statement carrying preset prompt optimization information SQL Hint; on the basis of the extended SQL query statement, a query optimizer of the OceanBase database selects a preset execution plan to query the service table so as to obtain a target result set, and the query performance is improved in combination with the parallel processing capacity of the database.

The subject technology provides techniques for enabling hidden database objects, which in an example are utilized for testing and verifying new database objects against existing workloads. Hidden database objects are a mechanism for bridging that gap by running user workloads on user data in advance of exposing the feature to users. This mechanism allows placing a database object as a hidden object nested beneath a user's visible object (e.g., table, column, view, and the like). Hidden database objects can be used to verify the functionality, parity, performance, and correctness of new unreleased features.

The subject technology receives a query, the query referencing a unified representation for structured type data and semi-structured type data, the unified representation being provided in storage and in memory during query processing, the unified representation comprising a set of structured type fields that include a set of semi-structured typed fields that enables type safety and enforcement for the set of structured type fields, and flexibility for the set of semi-structured typed fields in a same column, the unified representation in storage including type information for the semi-structured type data as part of the semi-structured type data, the unified representation being utilized for structured type data and semi-structured type data. The subject technology processes the query using the unified representation stored in the memory, the unified representation providing performance parity between structured type data and semi-structured type data.

Methods, systems, and devices for data management are described. A method may include obtaining a new snapshot of a relational database of a source computing environment, comparing the new snapshot of the relational database to a prior snapshot of the relational database, updating, in response to the comparison indicating that data of a row of a relational database table changed between the prior snapshot and the new snapshot, a field of a second versioning control column of a corresponding row of a backup database table of the set of backup database tables to indicate a version of the new snapshot, and adding, in response to the comparison indicating that the data of the row of the relational database table changed, an additional row to the backup database table, where a field of a first versioning control column of the additional row includes an indication of the version of the new snapshot.