296 results about "Column (database)" patented technology

The invention discloses a method and a system for generating an SQL (Structured Query Language) query based on a natural language question. The method comprises the following steps: converting mode information of a target database and the natural language question into semantic vector representation; based on the semantic vector representation, a simplified database mode set related to SQL query is screened out through an attention mechanism; based on the natural language problem, the simplified database mode set and preset database constraint information, generating an SQL structural skeleton, and filling specific elements of the SQL structural skeleton to form a preliminary SQL query; and performing dynamic correction and verification on the initial SQL query by utilizing a large language model, and outputting a final SQL query. According to the method, association mining between user query and a database mode is effectively enhanced by utilizing a context-aware cross-encoder mechanism, and an implicit corresponding relation between a natural language problem and a database table / column can be more accurately identified and utilized.

A computerized method is provided for using large language models to integrate data from multiple sources including technical metadata (e.g. column and table names) with a dictionary of standard abbreviations employed in the metadata, representative data from the columns rows of the table, a business glossary of terms in the data, and representative queries used to interrogate the data, to create a non-technical or natural language description of business value of the table and its columns and data.

Methods, systems, and devices for data management are described. A method may include obtaining a new snapshot of a relational database of a source computing environment, comparing the new snapshot of the relational database to a prior snapshot of the relational database, updating, in response to the comparison indicating that data of a row of a relational database table changed between the prior snapshot and the new snapshot, a field of a second versioning control column of a corresponding row of a backup database table of the set of backup database tables to indicate a version of the new snapshot, and adding, in response to the comparison indicating that the data of the row of the relational database table changed, an additional row to the backup database table, where a field of a first versioning control column of the additional row includes an indication of the version of the new snapshot.

The invention provides an NL2SQL database table dynamic screening method and system based on multi-strategy fusion. The method comprises the steps that a preliminary candidate set is generated, a semantic encoder is used for encoding a user natural language problem and table and column information in a database into vectors, the semantic similarity is calculated, a preset number of tables are screened out from a whole database table based on the similarity, and a preliminary candidate table set is formed; performing deep analysis and reordering, performing multi-dimensional analysis on the tables in the preliminary candidate table set, and calculating a graph relationship compactness score and a content correlation score of each table; and performing fusion decision, inputting the semantic similarity score, the graph relationship compactness score and the content correlation score of each table into a fusion model to obtain a final comprehensive score of each table, and screening out a final correlation table set based on the final comprehensive score. The method has high accuracy and high recall rate, the generation speed and response efficiency of the NL2SQL system are improved, meanwhile, the robustness is high, and the practicability and generalization ability of the system are improved.

The invention discloses a row-level data permission authorization method and device, equipment and a medium, and relates to the technical field of computers. Comprising the following steps: registering basic data into a preset database; configuring a corresponding field management user with a field management authority for the preset service field; creating or updating a record in a user data permission table based on the user permission application to determine a permission relationship between the user identifier and the entity data; registering a service data table and a mapping relation between an entity column and an entity type in the service data table in a preset service field based on a user instruction of a field management user; and dynamically generating a permission filtering condition based on the query SQL and the user identifier so as to generate an SQL statement with permission filtering, and querying corresponding service data from the service data table based on the mapping relationship so as to generate a corresponding query result based on the service data. Therefore, the efficiency and flexibility of line-level data permission authorization can be improved on the premise of ensuring data security.

The invention discloses a user ad-hoc query report realization method and device based on a large language model, and relates to the technical field of natural language processing. The method comprises the following steps: constructing an ad hoc query report system based on the large language model, wherein the ad hoc query report system comprises a semantic analysis module, an analysis engine module and a query statistics module; a natural language query statement input by a user is input into a semantic analysis module to be analyzed through a fine-tuned large language model, a structured query condition set and a structured query result index column set are output and input into an analysis engine module, and according to permission information of the current user, through pre-constructed index metadata and SQL semantic analysis rules, the query result index column set is analyzed, and the query result index column set is analyzed. Generating a query SQL statement; and inputting the query SQL statement into a query statistics module, connecting a target database to execute query operation to obtain a query result set, and performing data formatting processing on the query result set to obtain a final query result. By adopting the method, complex query requirements can be met.

A database system operates by: generating an initial query operator execution flow for a corresponding query for execution that includes an extend operation for execution serially before an aggregation operation indicating grouping performance of a corresponding aggregation by new column values of a new column generated via the extend operation via performance of a corresponding extend function of at least one input column; converting the initial query operator execution flow into an updated query operator execution flow semantically equivalent to the initial query operator execution flow based on: generating an updated aggregation operation to group by input column values of the at least one input column, and pushing the updated aggregation operation for execution serially before the extend operation; and executing the corresponding query by applying the updated query operator execution flow based on: executing the updated aggregation operation upon a plurality of rows to generate a plurality of aggregation values for a plurality of row groups via grouping by the at least one input column, and executing the extend operation after performing the updated aggregation operation by generating the new column from the at least one input column.

The invention discloses a Text-to-SQL (Structured Query Language) method based on a large language model. The method comprises a database mode linking module, a context information enriching module, a dynamic decomposition SQL (Structured Query Language) generation module, an execution feedback optimization module and a selection module based on self-consistency. Performing multi-path recall screening on related tables, columns and value fields in a database through a mode link; the mode information of the database is expressed through context information enrichment, key information is extracted, and a question is re-described; aiming at the problem of different complexity, different SQL generation schemes are designed, and a plurality of candidate SQL statements are generated; checking and correcting grammar and semantic errors existing in the SQL statement by executing feedback optimization; and through voting and a binary selection model, selecting the most accurate SQL query as a final SQL. The method can effectively solve the problems of enterprise data increase, difficulty in complex SQL generation and the like, improves the conversion accuracy and efficiency, enhances the adaptability to different database modes, and has important practical value.

The embodiment of the invention provides a data query method, a database system, equipment, a storage medium and a program product. The method comprises the following steps: when a query request is received, determining a first column table storage mode of a to-be-queried table according to a table name of the to-be-queried table carried in the query request; obtaining a column type storage file of the to-be-queried table; determining a second column storage table mode used when the column storage file is generated according to mode version information correspondingly recorded by the column storage file; obtaining first column data of the to-be-queried column from the column type storage file according to the second column table storage mode and the column name of the to-be-queried column carried in the query request; according to the first column table storage mode and the second column table storage mode, converting the first column data into second column data adaptive to the first column table storage mode; and determining a query result of the query request according to the second column of data. According to the technical scheme provided by the embodiment of the invention, the influence of database mode change on database service can be reduced.

A method, system, and computer program product are configured to: extract, from metadata of a table in a database, a table name abbreviation of the table and plural column name abbreviations of plural columns in the table; automatically generate a prompt for an artificial intelligence (AI) model, the prompt including the table name abbreviation, the plural column name abbreviations, and an instruction to expand the table name abbreviation and the plural column name abbreviations; provide the prompt to the AI model using a single call to the AI model; receive an output generated by the AI model based on the prompt; and process the output.

The invention provides a heterogeneous database query method, device and equipment based on large model driving. The method comprises the following steps: receiving a natural language query instruction of a user for a target service; performing semantic analysis on the query instruction by using a large language model to obtain a query keyword set, performing mapping processing on the keyword set and a heterogeneous database mode corresponding to a target service, and determining a condition column and a target column corresponding to the query instruction; determining a query task type corresponding to the query instruction according to a data type of a target column; constructing a semantic operator execution path according to the query task type; according to the keyword set and the condition column and the target column obtained through matching, corresponding semantic operators are called to execute the semantic operator execution path so as to obtain a query result corresponding to the query instruction, and by means of the technical scheme, the efficiency and reliability of heterogeneous database query can be improved.

Systems and methods are disclosed for converting natural language queries to a query instruction set for searching a data warehouse. To generate a query instruction set from a natural language query, a system iteratively uses a generative artificial intelligence (AI) model and database query tools to generate a query instruction set in a stepwise manner. The system and generative AI model do not require a priori knowledge of data table contents in the data warehouse, which may include sensitive information. In addition, the system does not require access to the data warehouse to generate the query instruction set. Instead, the system is implemented to use structure information from the data warehouse, including table lists (such as table names) and table format information (such as column names) of tables in the data warehouse, and the generative AI model is a generally trained model to generate the query instruction set.

The invention relates to the technical field of database query optimization and distributed system data processing, and discloses an intelligent routing optimization method and system for multi-table association query, and the method comprises the steps: a query topology analysis unit analyzes an object relation mapping configuration file to construct a query topological graph; the environment parameter detection unit collects real-time state data of the server cluster based on the query topological graph, wherein the real-time state data comprises a pre-estimated cache hit rate, associated sparseness and a target table data line number estimation value; the routing strategy decision-making unit compares the real-time state data with a preset threshold value, and selects a cache-driven step-by-step query strategy, a column-type compression transmission strategy or an asynchronous intermediate table unloading strategy; and the dynamic execution control unit executes data acquisition and assembly operation according to a selected strategy. According to the method, the problems of low database computing resource utilization rate and application server memory overflow caused by the adoption of a static strategy in a traditional framework are solved, and the system response speed and the service availability of multi-table association query are effectively improved.

The invention provides a method and a system for accurately converting a large-model natural language into an SQL (Structured Query Language) based on multi-modal fusion. The method comprises the following steps: S1, performing named entity recognition and dependency syntax analysis on a natural language text, performing vision-text dual-path processing on image data, and constructing a bipartite graph structure for table data so as to obtain a vision text composite feature and a table feature; s2, designing a modal attention controller and calculating the weight of each modal through a gating network; s3, constructing a knowledge graph, clearly recording all legal table names and column names in a database, and retrieving the knowledge graph in real time when a decoder generates the table names or the column names; s4, generating an SQL draft; s5, performing bidirectional verification correction on the SQL statement; and S6, outputting the legal SQL statement. According to the method, multi-modal information can be integrated, information loss and semantic deviation are reduced, illegal statements are avoided through grammar and semantic dual verification, and the overall reliability of SQL statement generation is improved.

The invention discloses a task alignment Text-to-SQL (Structured Query Language) method and system based on a code style thinking chain. The method comprises the following steps: constructing a three-layer structured knowledge base comprising mode-level knowledge, instance-level knowledge and example-level knowledge; generating an initial query based on the target database and the user query; according to the list information in the initial query, constructing an instruction sequence represented by a similar SQL code, and forming a code style thinking chain; based on user query and initial query, acquiring multi-dimensional knowledge from the three-layer structured knowledge base and constructing an enhanced prompt context; and integrating the code style thinking chain and the enhanced prompt context based on the prompt template, and generating an executable final query through a multi-stage task alignment mechanism and a final prompt. According to the method, the problems of incoherence and inaccuracy of logical reasoning, insufficient knowledge generalization ability and multi-type model illusion existing in cross-domain, multi-table connection and complex query in the prior art are solved.

A present invention embodiment processes user queries. Using a large language model (LLM), a first Structured Query Language (SQL) statement is generated based on a user query provided by a user. A plurality of entities are extracted, based on the first SQL statement, including columns, filter columns, and categorical filters. A semantic search is performed on an index of a plurality of databases using the extracted entities to determine a confidence score for each database of the plurality of databases. A database is selected based on the confidence score and a schema of the selected database is extracted. A second SQL statement is generated by the LLM based on the user query and the schema. The second SQL statement is executed on the selected database to generate query results that are presented to the user.

Methods and systems for retrieving data from at least one database based on a query input related to a clinical trial. A system processes a received query input related to a clinical trial using a pre-trained language model neural network. The neural network generates a structured representation of the query input. The system maps a first data field of the structured representation to a first column name and maps a second data field of the structured representation to a second column name. The system generates a database query based on (i) a database schema, (ii) the first column name, (iii) the second column name, (iv) data values associated with the first data field, and (v) data values associated with the second data field. The database query specifies an operation for joining data associated with the first column name with data associated with the second column name.

Embodiments herein extract information from documents (e.g., scanned images of documents) to fields of a database or other collection of fields. The location and contents of blocks of text within the document are detected and then applied to a trained model to map a subset of the text to a set of target fields, where the target fields include one or more sets of repeated fields (e.g., corresponding to rows of a table). This mapping is presented to a user, optionally superimposed on an image of the document, to facilitate the user providing corrective feedback to the mapping. The mapping can then be updated, and the model trained to exhibit improved accuracy, based on the corrective feedback. The corrective feedback can include indicating the extent of a table and / or rows or columns thereof, facilitating correction of large numbers of field mappings.

Methods and systems for retrieving data from at least one database based on a query input related to a clinical trial. A system processes a received query input related to a clinical trial using a pre-trained language model neural network. The neural network generates a structured representation of the query input. The system maps a first data field of the structured representation to a first column name and maps a second data field of the structured representation to a second column name. The system generates a database query based on (i) a database schema, (ii) the first column name, (iii) the second column name, (iv) data values associated with the first data field, and (v) data values associated with the second data field. The database query specifies an operation for joining data associated with the first column name with data associated with the second column name.

Embodiments of the present disclosure include techniques for encoding data locations in a database using bit maps. In one embodiment, a plurality of bit maps are generated for unique values in a column of a table. The bit maps may be ordered in columns, where columns correspond to unique values a particular table column. Bit maps for each comprise a first binary value when table value in a corresponding row matches a corresponding bit map unique value. Bit map values are set to a second binary value opposite the first binary value when the bit map unique value does not match the table row value for previous bit maps and don't care values for subsequent bit maps. In one embodiment, the bit maps are ordered by bit density.

A method (300) of storing semi-structured data (12U) includes receiving user data (12) comprising semi-structured user data from a user (10) of a query system (150). The method includes receiving an indication (14) that the semi-structured user data fails to include a fixed schema. In response, the method further includes parsing the semi-structured user data into a plurality of data paths (210) and extracting a data type (220) associated with each respective data path of the plurality of data paths. The method additionally includes storing the semi-structured user data as a row entry in a table (204) of a database in communication with the query system, wherein each column value associated with the row entry corresponds to a respective one of the plurality of data paths and the data type associated with the respective data path.

A database system operates by: executing a query against a relational database table indicating a predicate applied to geospatial data of a geospatial data column, based on: applying an inner predicate to internal level nodes when traversing a set of internal levels of the each tree-based index structure to identify a first subset of leaf nodes in a plurality of leaf nodes of the each tree-based index structure based on identifying internal nodes having internal node bounding boxes meeting the inner predicate; and applying a leaf predicate to only leaf nodes included in the first subset of leaf nodes of the tree-based index structure to identify a second subset of leaf nodes of the first subset of leaf nodes corresponding to only leaf nodes of the first subset of leaf nodes having corresponding leaf node bounding boxes meeting the leaf predicate; wherein a query resultant of the query is generated based on geospatial objects of the geospatial data column for ones of a plurality of rows of the relational database table indicated by the second subset of leaf nodes.

The invention provides a database index table generation method, a query method and electronic equipment, and relates to the technical field of databases. According to the specific scheme, the method comprises the steps that in response to a received database optimization instruction, a to-be-optimized database and a target query statement are determined based on the database optimization instruction; determining a target query field from the target query statement based on the execution plan information of the target query statement; identifying the complexity of a target data column which is stored in the to-be-optimized database and is related to the target query field, and determining an identification result; and generating an index table for querying data in the target data column by using an index generation strategy matched with the condition type of the target query condition corresponding to the target query field under the condition that the recognition result represents that the complexity of the target data column meets the index generation condition.

Systems and methods receive, from client application(s), request(s) to access data from platforms, translate the request(s) to API calls, the translating utilizing access protocols of each of the respective platforms. The data are retrieved and aggregated from data sources of each of the platforms, the data being aggregated into a standardized format. The systems and methods ascertain which data fields of the aggregated and standardized data include information relevant for analysis, the ascertaining being based on predefined data categories. Accuracy of relevant data located in the data fields that include the information relevant for analysis is verified and the relevant data are incorporated, based on accuracy of the relevant data being verified, into data columns for display via a centralized graphical user interface dashboard. Display, via the centralized graphical user interface dashboard, of one or more graphical elements incorporating the relevant data is initiated.

The invention discloses a database row and column synchronization method and device, equipment and a medium. The method comprises the steps that data updating information of a first structure database is obtained and analyzed, at least one first updating data set of a first structure is obtained, structure conversion is conducted on all the first updating data sets, and a second updating data set of a second structure is obtained; adding an updating time sequence identifier for each second updating data group; and inserting each second update data group and the corresponding update time sequence identifier into a second structure database, and querying data by the second structure database according to the update time sequence identifier. The embodiment of the invention can improve the row and column synchronization efficiency of the database and improve the accuracy of data synchronization.

This application provides a data sorting method, a database engine, and a storage medium. In this application, for scenarios involving multi-column sorting, the data rows to be sorted are first grouped according to the clustering column. Then, the sorting is performed between and within each data group according to the sorting column, using the same sorting order. The sorting result obtained in this way retains the clustering characteristics of the clustered columns while essentially ensuring the global order of the data, which is beneficial for improving the computational performance related to the clustered columns. Especially when querying based on the sorting result, it ensures that the performance of queries by data range is largely unaffected, while significantly improving the performance of queries by data grouping.

The invention provides a distributed database data access method and related equipment, and relates to the technical field of computers. The method comprises the following steps: acquiring a data access request initiated by a user, wherein the data access request is a request based on an SQL statement; analyzing the SQL statement to obtain a to-be-accessed data table; determining a permission label of the to-be-accessed data table, wherein the permission label comprises at least one of a table permission label, a column permission label and a row permission label; performing permission verification on the user based on the permission label of the to-be-accessed data table; and if the permission verification is passed, executing the SQL statement to query the distributed database to obtain a data access result, and returning the data access result to the user. The authority control granularity can be refined, an authority strategy does not need to be independently configured for the data table, and the operation and maintenance difficulty is reduced.

The presently disclosed subject matter includes a system and method of managing computer memory of a column-oriented database, which enables to reduce the tradeoff between eviction and recycling of intermediate query results and thus retain much of the recycling capability at a considerably lower computer memory cost. The computer memory comprises one or more intermediates, each intermediate being generated by execution of a respective instruction, the instruction being part of a given query. An hierarchical data-structure is generated and stored in the computer memory, wherein different nodes in the hierarchical data-structure represent different respective intermediates stored in the computer memory. Responsive to a delete command, the at least one intermediate is deleted from the computer memory, while a respective node representing the at least one intermediate is maintained in the hierarchical data-structure.

A database management system generates a cache entry system view of a database cache. Also, a new column is generated for the cache entry system view, where the new column is an ENTRY_HASH column to identify each entry of the database cache. In an example, the database management system detects a request to remove a given entry of the database cache, where the request includes a given ENTRY_HASH value to locate the given entry of the database cache. In response to receiving the request, the database management system identifies the given entry of the database cache based on the given ENTRY_HASH value. Then, the database management system removes the given entry of the database cache. Also, the database management system notifies a cache manager that the given entry has been removed.

The application provides a cross-source synchronization table index creation method and device, equipment, storage medium and program product, which can be applied to the technical field of big data. The method comprises the following steps: receiving a query statement for obtaining data from a data warehouse; extracting a query field set and a condition field set from the query statement; obtaining data from the data warehouse according to the query statement, and synchronizing the obtained data to a target table in an online database, wherein the target table comprises data columns corresponding to the query field set; merging the condition field set into a candidate index field set of the target table; calculating a dispersion index of each field in the candidate index field set based on the condition value distribution corresponding to each field in the candidate index field set; selecting a target field set from the candidate index field set based on the dispersion index; and creating an index for each field in the target field set on the target table.