Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

508 results about "Query plan" patented technology

A query plan (or query execution plan) is an ordered set of steps used to access data in a SQL relational database management system. This is a specific case of the relational model concept of access plans.

Service proxy method and system based on Dores front-end node

The invention provides a Dores front-end node-based service proxy method and system, and relates to the technical field of database access request processing, and the method comprises the steps: collecting front-end node performance indexes, extracting node performance characteristics through employing multi-head recursive attention and causal convolution solution, extracting inter-node correlation characteristics through combining graph attention convolution, and obtaining a database access request; the method comprises the following steps: constructing a node state vector, optimizing a multi-layer connection pool parameter by utilizing an online learning algorithm, establishing a node incidence matrix based on comparative learning, constructing a node performance probability graph evolution model through variational reasoning and Monte Carlo sampling, predicting a node performance change trend, and receiving a database access request; and generating a query plan by adopting an edge attention message passing mechanism, carrying out resource evaluation in combination with performance trend prediction, carrying out scheduling decision making by utilizing multi-agent reinforcement learning and Monte Carlo tree search, carrying out fault detection and migration according to node performance deviation, and updating a probability graph evolution model.
Owner:北京科杰科技有限公司

Self-adaptive query optimizer of time sequence database

The invention discloses a self-adaptive query optimizer of a time sequence database, which comprises a query mode analysis and classification module used for analyzing and classifying query modes so as to identify common query modes and possible optimization points; the self-adaptive optimization strategy generation module is used for automatically generating and adjusting a query optimization strategy based on the query mode analysis result; the machine learning and prediction module is used for predicting future query mode changes and optimizing the future query mode changes in advance; the real-time optimization and execution module is used for applying the generated optimization strategy to a database in real time and monitoring the effect of the optimization strategy; and the user feedback and custom optimization module is used for feeding back an optimization effect and providing custom optimization parameters and strategies. According to the method, the performance stability and the query efficiency of the time sequence database are effectively improved by dynamically adjusting the query plan, optimizing resource management and utilizing continuous optimization driven by machine learning.
Owner:成都虚谷伟业科技有限公司

System and method of privacy-preservation in caching of graph query execution plans

A method is provided for caching a query submitted to a graph data storage system, including receiving a query from a query caller directed to the graph data storage system, and examining the query, via a cache suitability determination engine, to determine if the query is suitable for caching under a first graph query caching policy based on the query caller being a trusted caller and / or the query scenario being a trusted query scenario. If suitable, generating a query execution plan for the query via a graph query plan generating engine and examining the query plan to determine if the query plan is suitable for caching under the first policy. If not suitable, caching the query under a second or a third graph caching policy based on the geographic location of a memory system on which the query plan is stored, and providing the query execution plan.
Owner:MICROSOFT TECHNOLOGY LICENSING LLC

Operator unloading method and system of vectorization execution engine based on DPU heterogeneous architecture

The invention provides an operator unloading method and system of a vectorization execution engine based on a DPU heterogeneous architecture, the DPU heterogeneous architecture comprises a CPU and a DPU network card, and the DPU heterogeneous architecture is applied to an open source distributed computing system; the open source distributed computing system performs grammar analysis on a query language input by a user and adds physical execution stage operation to obtain a physical execution plan, and the Spark plug-in intercepts the physical execution plan and converts the physical execution plan into a query plan based on a set rule; transmitting to a vectorization engine execution path and calling a vectorization execution engine to carry out data processing on a plurality of operators in the query plan; a performance function of a DPU heterogeneous calculation version in a function system of the vectorization execution engines is called, calculation resource consumption of operators capable of being processed by the multiple vectorization execution engines in the query plan is calculated, the operators higher than a preset standard are unloaded to a DPU network card from an original CPU, and the data processing performance of the query task is improved through operator unloading.
Owner:YUSUR TECH CO LTD

System and Method for Input Data Query Processing

A computer-based system and corresponding computer-implemented method process input data queries in a manner enabling advanced functionality for data analytics. The method transforms a query plan tree into a query strategy tree. The query plan tree is constructed from an input data query associated with a computation workload. The method compiles the query strategy tree into dataflow graph(s) (DFG(s)). The method transmits the DFG(s) for execution via a virtual platform. The method monitors execution of the DFG(s). The method outputs, based on a result of the execution monitored, a response to the input data query. The result is received from the virtual platform and represents at least one computational result of processing the computation workload by the virtual platform. The system and method enable rapid and efficient retrieval and analysis of data stored in data lakes and other data storage systems, in response to user queries.
Owner:DATAPELAGO INC

Auxiliary query optimizer providing improved query performance

Techniques and solutions are provided for improved query optimization, including for sub-portions of a query plan. A query is submitted to an inline query optimizer and at least one auxiliary query optimizer. The query is optimized by the inline query optimizer and the auxiliary query optimizer. A query processor can evaluate costs associated with query plans produced by the inline query optimizer and the auxiliary query optimizer and select a plan for execution that is most performant.
Owner:SAP SE

Interpretable database query optimization method and system

The invention discloses an interpretable database query optimization method and system, and relates to the field of database query optimizing.The method comprises the steps that a random forest model is constructed based on a genetic programming algorithm; obtaining a to-be-queried SQL statement; generating a prompt set based on a scanning method and a connection method, and generating a query plan set for each to-be-queried SQL statement according to the prompt set; encoding each query plan in the query plan set to obtain a feature vector set; inputting the feature vector set into a random forest model, and outputting a judgment rule set, feature importance ranking and a prediction execution time set; determining the query plan corresponding to the minimum prediction execution time in the prediction execution time set as an optimal query plan; based on the optimal query plan, a database query engine executes a query. According to the method, the database is queried through the determined optimal query plan, so that the execution efficiency and credibility of query can be improved.
Owner:HANGZHOU DIANZI UNIV

Dynamic selection of database data topologies for performing queries

Dynamic selection of topologies for database data may be performed for performing queries. Different possible query plans for different assignments of topologies are determined for a query. One of the query plans is selected according to a cost-based model that assigns costs to different uses of the different topologies. The query is then performed using the selected query plan.
Owner:AMAZON TECH INC

Relational subtree matching for improved query performance

The present disclosure provides techniques and solutions for improved query optimization. A query plan is received, and at least a portion of the query plan is identified to be analyzed for logically equivalent query plans. A signature is generated for the at least a portion of the query plan. One or more query plans are identified that have signature that matches the signature of the at least a portion of the query plan, but where such query plans are logically equivalent, but not identical, to the at least a portion of the query plan. A query plan of the one or more query plans is substituted in the query plan for the at least a portion of the query plan.
Owner:SAP SE

Query plan construction method and apparatus, electronic device and storage medium

The present application discloses a query plan construction method and apparatus, an electronic device and a storage medium. The method of embodiments of the present application comprises: acquiring a data query expression, the data query expression comprising a keyword; obtaining basic sub-expressions in the data query expression and an association relationship between the basic sub-expressions according to the keyword; expanding the basic sub-expressions to obtain equivalent sub-expressions of the basic sub-expressions, wherein an equivalent sub-expression and a corresponding basic sub-expression have an equivalent expression meaning; from the basic sub-expressions and the equivalent sub-expressions of the basic sub-expressions, determining sub-expressions to be constructed; and constructing a data query plan by means of the sub-expressions to be constructed and the association relationship. In the present application, the data query plan can be constructed according to the association relationship by a plurality of sub-expressions to be constructed, so that the plurality of sub-expressions to be constructed do not conflict with each other during construction of the data query plan, thereby facilitating the construction of the data query plan, and improving the query performance.
Owner:TENCENT TECHNOLOGY (SHENZHEN) CO LTD

Computing resource configuration method and device of FPGA (Field Programmable Gate Array) equipment and FPGA equipment

The embodiment of the invention provides a computing resource configuration method and device of FPGA equipment and the FPGA equipment. The method comprises the steps that a heterogeneous query execution plan is generated according to a query request of a user, and the heterogeneous query plan comprises heterogeneous operators and non-heterogeneous operators; and determining a first number of resource blocks required to be configured for executing the heterogeneous operator based on the calculation throughput for processing a non-heterogeneous operator adjacent to the heterogeneous operator and the calculation throughput which can be provided by a single resource block in a programmable array logic FPGA (Field Programmable Gate Array) device. Through the embodiment of the invention, the problems that the throughput of the FPGA equipment is not matched with the calculation throughput of the CPU and the resources of the FPGA equipment cannot be reasonably utilized in the current dynamic configuration scheme in the related technology are solved, and the effect of ensuring that the resources of the FPGA equipment are reasonably utilized in the query acceleration process of the database is achieved.
Owner:ZTE CORP

Query performance prediction using multiple experts

A future workload may be predicted for a database system using analysis of queries submitted for execution. A feature vector for a query may be determined according to a query plan for the query. If the feature vector has not been previously seen, or has not been sufficiently seen, by the database system, a machine learning inference may be used to predict performance characteristics of the query, the machine learning system trained using previous feature vectors and performance characteristics of executed queries. If the feature vector has been sufficiently seen previously by the database system, a history of performance characteristics of previous queries with similar or the same feature vector may be used to predict the performance characteristics. The predictions may then be used to configure or reconfigure processing cluster(s) of the database system.
Owner:AMAZON TECH INC

Spring Batch and large model-based data synchronization method and device

The invention provides a data synchronization method and device based on Spring Batch and a large model. Synchronously initializing a large model micro service and a Spring Batch framework, and establishing a model service hot standby mechanism; obtaining data source features through a metadata scanning engine driven by a large model, and generating an adaptive synchronization strategy; optimizing a query plan based on the synchronization strategy, dynamically controlling data fragmentation and executing streaming preprocessing and semantic mapping conversion; a target table structure is adapted, a three-level conflict detection mechanism is adopted to eliminate data conflicts, and meanwhile transaction processing and batch writing logic are optimized; the data quality is verified through three-level verification, performance parameters are dynamically adjusted and optimized, and fault-tolerant recovery and resource elastic expansion and contraction are achieved; metadata are synchronized, an execution report is generated, knowledge base self-learning is completed based on task full-link indexes, and a data consanguinity map and a reusable processing case library are constructed. According to the scheme, diversified and dynamic enterprise data synchronization requirements can be met.
Owner:SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD

Database query plan optimization method and system

The invention discloses a database query plan optimization method and system, and belongs to the technical field of databases and artificial intelligence. AI model training: training the model by using the preprocessed historical query data according to a specific model architecture, a training mode and resource and time arrangement, and dynamically adjusting the model according to the change of the database; query analysis: evaluating the complexity of the query and the possibly involved data volume, and allocating a complexity level and a data access volume range for the query; query plan generation: generating and evaluating a plurality of candidate query plans based on prediction and query analysis results of the AI model, and selecting an optimal query plan as a final execution scheme; and performing query execution and monitoring. According to the method, the query execution time can be greatly shortened, and the database response speed is increased; system resources are utilized more effectively; dynamic changes of the database can be adapted, and the query plan is automatically updated according to structure and data distribution adjustment.
Owner:上海沄熹科技有限公司

Breaking change detection for federated systems

Systems and methods provide for managing changes to federated systems. In accordance with some aspects, a change to a federated system is received. A historical query is accessed from a query store storing information regarding queries received by the federated system. A query plan evaluation is performed for the historical query based on the change to the federated system, and an action is caused to be performed for the change based on the query plan evaluation.
Owner:EBAY INC

Communication network multi-source big data analysis method based on distributed database collaboration

The invention discloses a communication network multi-source big data analysis method based on distributed database collaboration, and relates to the technical field of communication data analysis, and the method comprises the steps: 1, generating a session contract number, a table snapshot number, a flow site set and a transaction time point at the start of a session, registering a collaborative metadata table, judging semantic compatibility, and setting a state; step 2, pushing down three anchors to a lake-warehouse, a message bus and a transaction library according to the serial number, performing freezing reading according to the anchors to obtain a lake-warehouse segment, a streaming segment and a transaction segment, performing alignment extraction on a joint feature set in the step 3, generating output through a deterministic operator chain, and storing the extracted joint feature set in a database; the method comprises the following steps: step 1, setting a session contract number, a field source and an operator relationship, reading an anchor code as a blood relationship code and writing the blood relationship code into a result table, and step 2, resetting three anchors and multiplexing a query plan during reproduction verification and cross-domain migration, and integrating blood relationship equivalence, result equivalence, freezing consistency, registration difference and migration fingerprints to realize session-level consistent reading, recalculation and auditing.
Owner:毕思博

Heterogeneous data source query method, system and equipment based on Dores database and medium

The invention relates to the technical field of data processing, and particularly provides a heterogeneous data source query method, system and device based on a Dores database and a medium, and the method comprises the following steps: obtaining a query text submitted by a user, and analyzing query parameters of the query text; obtaining metadata of each data source, and generating a plurality of candidate global logic plans based on the metadata and the query parameters; screening a global optimal query plan from the plurality of candidate global logic plans by using a cost model; and converting the global optimal query plan into a federated query statement supported by Dores, and executing the federated query statement by using a Dores federated query function to obtain target data. According to the method, a plurality of candidate global logic plans are automatically generated based on heterogeneous data source metadata and query parameters, and the optimal plan is accurately screened by using the cost model, so that the execution efficiency of cross-source query is remarkably improved, and low-efficiency scanning and network transmission bottlenecks are avoided.
Owner:INSPUR YUNZHOU (SHANDONG) IND INTERNET CO LTD

HTAP database query optimization method and device and storage medium

The invention discloses an HTAP database query optimization method and device and a storage medium. The query performance is remarkably improved by combining the advantages of row storage and column storage. Comprising the steps of analyzing SQL query to generate a logic query plan; evaluating the execution cost of each logic operator in different storage modes by using a mixed cost model, wherein hyper-parameters of the mixed cost model are optimized through Bayesian search; selectively adopting a dynamic programming algorithm or a greedy algorithm to generate a hybrid physical query plan with the minimum execution cost according to the table number; and executing the plan to obtain a query result. In addition, an MCTS algorithm is further utilized, the optimal column subset is selected in the preset memory budget limit based on the query historical record and loaded into the memory column storage, and the subsequent query performance is further optimized. The problem that row storage and column storage in an existing HTAP system cannot work cooperatively is solved, operator-level mixed execution is achieved, and meanwhile memory resource utilization is optimized.
Owner:TSINGHUA UNIVERSITY

Persisting database query plans that include dynamic conditions

The present disclosure involves systems, software, and computer implemented methods for managing database query plans with dynamic conditions. One example method includes receiving a request to process a query based on at least one dynamic condition. A serialized abstract query plan is identified, deserialized, and converted into a query optimizer tree. A condition provider procedure is invoked to generate a first filter string for each view in the query optimizer tree. Comparisons are made to determine matches between first filter strings and corresponding second filter string stored in the query optimizer tree. In response to determining that a first filter string does not match the corresponding second filter string, query plan compilation is triggered. In response to determining that each filter string matches each corresponding second filter string, a query plan is generated based on the converted query optimizer tree and the query is executed using the query plan.
Owner:SAP SE

Method and apparatus for detecting scheduling deadlock in data query, and device

Discloses are a method for detecting a scheduling deadlock in data query. A query plan tree generated by a query statement includes a common temporary table production operator node, a common temporary table consumption operator node, and a hash join operator node. At least one right table chain in the tree is determined. Operator nodes in the right table chain are traversed from bottom to top starting from the leaf node in the right table chain, and the other operator nodes in the tree are traversed, to analyze an execution order relationship between the operator nodes traversed in the tree. When it is found based on the execution order relationship that there are two identical common temporary table consumption operator nodes in the tree that have scheduling dependency, and their upstream operators are the same common temporary table production operator, it is determined that there is a scheduling deadlock.
Owner:BEIJING VOLCANO ENGINE TECH CO LTD

Extensible system, method and equipment for multi-language data analysis and medium

The invention discloses a multi-language data analysis-oriented extensible system, method, equipment and medium, and relates to the technical field of multi-language analysis, the multi-language data analysis-oriented extensible system comprises a grammar analysis module used for analyzing a query language and generating a language-specific syntax tree structure after analysis; the grammar adaptation module is used for converting a language-specific grammar tree structure into an abstract grammar tree AST in a uniform format; the abstract syntax tree processing module injects a permission control strategy, a field desensitization rule and an alarm exception expression into the AST; the business processing module is used for executing business-related rule check and label injection; the query plan generation module is used for constructing a logic query path and a corresponding distributed execution scheme; and the task scheduling module issues the distributed execution scheme to the corresponding execution node. According to the method, by constructing a unified multi-language grammar analysis framework and a standardized abstract syntax tree AST representation mechanism, compatibility and fusion of multiple query languages such as SQL, SPL and natural language are achieved.
Owner:YUNNAN POWER GRID CO LTD

Database query optimization method and device, equipment and medium

The invention relates to the technical field of data query, in particular to a database query optimization method and device, equipment and a medium. The method comprises the following steps: acquiring a query plan corresponding to a query statement, applying available resources for executing the query plan by the query statement in the process of executing the query plan, if the application fails, acquiring N execution paths corresponding to continuing to execute the query plan, N being an integer greater than zero, evaluating the execution cost of the query plan under each execution path, the execution cost of each execution path is obtained, the execution path corresponding to the minimum execution cost value is selected as the optimal execution path, and the query plan is executed according to the optimal execution path. According to the method and the device, when resources are insufficient, the execution cost of the query statement under different execution paths is evaluated, the execution path with the minimum execution cost is determined as the optimal path, the query statement is executed by using the optimal path, and the execution efficiency is improved.
Owner:SHENZHEN INST OF COMPUTING SCI

Using worst-case predicate selectivity for database query plans

The present disclosure involves systems, software, and computer implemented methods for using worst-case predicate selectivity for database query plans. One example method includes identifying a query that includes an initial set of parameter values for a set of predicates. An order of predicate evaluation is determined for a query plan by determining a worst-case selectivity for each predicate and ordering the predicates based on the worst-case selectivities. A second instance of the query is received that includes a second set of parameter values. The second instance of the query is executed, to generate a query result, using the query plan and the second set of parameters by evaluating the set of predicates based on the second set of parameters and the order of predicate evaluation determined based on the worst-case selectivities. The query result is provided in response to the second instance of the query.
Owner:SAP SE

Log query method and device, medium, electronic equipment and program product

The invention relates to a log query method and device, a medium, electronic equipment and a program product. The log query performance can be improved. The method comprises the following steps: receiving a log query request, wherein the log query request is used for requesting to query target log content within a target time range; respectively searching a target time index and a target log storage file in a target time index file corresponding to the target time range from a time index file and a log storage file constructed based on logs, and searching a target query range corresponding to the target time index from the target log storage file; constructing a query plan according to the target query range; and according to the query plan, controlling the working node to concurrently search a data segment index matched with a data block specified by the allocated query subtask from a data segment index file corresponding to the target log storage file, and querying target log content based on a target data block corresponding to the data segment index.
Owner:BEIJING VOLCANO ENGINE TECH CO LTD

Data query method and device for distributed database, equipment, storage medium and program product

The invention discloses a data query method, device and equipment for a distributed database, a storage medium and a program product, and the method comprises the steps: building a query plan evaluation model based on the data transmission time delay between every two working nodes in a distributed cluster, and the computing power resource and data storage layout information of each working node; under the condition that a query request is received, calling a query plan evaluation model, respectively evaluating a plurality of query plans generated based on the query request, and determining a target query plan from the plurality of query plans based on an evaluation result; determining at least one target working node of the to-be-queried data in the distributed cluster; and on the basis of a preset push-down strategy, pushing down each query operator contained in the target query plan to each target working node where the to-be-queried data is located, so that each target working node executes each query operator on the basis of the target query plan to obtain a data query result. By adopting the method, the data query efficiency of the distributed database can be improved.
Owner:CHINA SOUTHERN POWER GRID COMPANY

Driving type data analysis method and system based on AI

The invention belongs to the field of artificial intelligence and big data analysis, and particularly relates to an AI-based drive type data analysis method and system.The method comprises the steps that S1, on the basis of natural language query input by a user, candidate SQL and / or DSL statements are generated through a pre-trained big language model; s2, generating a parallel query plan and optimizing the parallel query plan based on the candidate SQL and / or DSL statements and the metadata knowledge graph; s3, extracting authorization data; s4, fusing the extracted data to generate an analysis feature wide table, and identifying key features, abnormal modes and text abstracts in the analysis feature wide table; s5, executing controlled retrieval in the business knowledge graph based on the key features and the abnormal mode, and generating a security evidence sub-graph; s6, inputting the text abstract of the analysis feature wide table and the security evidence sub-graph into an AI inference engine, and outputting a structured report; and S7, displaying the structured report. The problems of high technical threshold and weak safety and interpretability in a data analysis process are solved.
Owner:重庆富民银行股份有限公司

Query block boundary identification

The subject technology receives a query. The subject technology generates a set of query blocks based on parsing the query. The subject technology stores query block metadata for each query block from the set of query blocks. The subject technology restores a set of logical query block boundaries. The subject technology performs a hash-based query block matching. The subject technology generates, after performing the hash-based query block matching, a final query plan. In an example, if a (same) query is executed again the same logical query blocks would produce the same metadata (e.g., identifier, name and hash). This metadata can be used to match logical query blocks between multiple executions of the same query.
Owner:SNOWFLAKE INC

Federal query method and device oriented to multiple data sources, equipment and medium

The invention provides a multi-data-source-oriented federal query method and device, equipment and a medium, and the method comprises the steps: obtaining a data analysis request of a user, converting a data query rule into a relational algebraic node format according to configuration item information and metadata information, obtaining at least one target query plan, and sending the at least one target query plan to a server; generating a query request corresponding to the data source according to the data query parameter of the at least one target query plan to query a corresponding underlying database to obtain target data, receiving the target data of each data source, preprocessing each target data, and performing statistical analysis according to the preprocessed target data to obtain a statistical analysis result; the preprocessing comprises data fusion and alignment; according to the method, the isomerism of different underlying databases can be shielded, the query process oriented to multiple data sources is simplified, the problems of complex cross-data-source analysis process and low efficiency can be effectively solved, and subsequent statistical analysis requirements can be supported by fusing and aligning the queried target data.
Owner:CISDI INFORMATION TECH CO LTD

Supply chain finance statistical data downloading method

The invention discloses a statistical data downloading method for supply chain finance, which relates to the technical field of finance and comprises an acceptance permission judgment module, a query resource control module and a desensitization packaging evidence storage module. The acceptance permission judgment module is used for uniformly accessing the statistical application, completing identity verification, main body range binding, index range binding, time range binding and area range binding, and generating an acceptance bill and a judgment bill; the query resource control module is used for generating a query plan under the index aperture mapping dictionary, the partition information and the resource quota, binding a concurrent quota, a throttling quota and a priority queue, and outputting an executable instruction stream; and the desensitization packaging evidence storage module is used for executing differential desensitization according to the main body type and the sensitivity level, performing aggregation calculation and deduplication execution, completing result set packaging, digital watermarking and downloading delivery, generating a downloading certificate and writing a certificate abstract and an evidence index into an evidence storage chain. The method has the characteristic of high consistency.
Owner:JIANGSU VOCATIONAL COLLEGE OF BUSINESS

Pipeline bypass for certain query plan operators

In some implementations, there is provided pipeline bypassing of certain operators. In some implementations, a method includes generating a query plan including at least one pipeline of operators; determining whether the at least one pipeline of operators includes a first operator that requires a complete result set as an input and further includes a second operator that supports providing the complete result set using a state identified by a state reference; and in response to determining the at least one pipeline of operators includes the first operator that requires the complete result set as the input and further includes the second operator that supports providing the complete result set using the state identified by the state reference, bypassing in the query plan pipelining between the first operator and the second operator.
Owner:SAP SE