534 results about "Query optimization" patented technology

A system and methodology for parallel query optimization using semantic-based partitioning is described. In one embodiment, for example, in a database system comprising a database storing data in database tables, a method is described for improving query performance by dynamically partitioning the data, the method comprises steps of: receiving a query requesting data from the database; generating a plurality of subplans for executing the query, each subplan including one or more operators for performing relational operations; adding operators for partitioning data and performing a given relational operation in parallel to at least some of the plurality of subplans; and building a plan for execution of the query based, at least in part, upon selecting subplans having favorable execution costs

More efficient querying of a multi-tenant database using dynamic tuning of database indices. A layer of meta-data associates data items with tenants, e.g., via tags, and the meta-data is used to optimize searches by channeling processing resources during a query to only those pieces of data bearing the relevant tenant's unique tag.

A relational database warehouse system with query optimization capabilities is described that allows for speedy identification of sets of records of interest from amongst tens of millions of records. The records of interest may include complex derived attributes, generated, at least in part, by aggregating data from a plurality of records in base data tables. In various embodiments, the query optimization capabilities allow the database warehouse system to identify conditions under which normal query execution may be replaced by one or more optimized execution methods, including, for example, eliminating unnecessary inner join operations on base data tables specified by a query, re-ordering the execution of group-by operations and left-outer join operations to greatly reduce the size of join tables produced while processing a query, and / or consolidating a set of segmentation queries for execution in one pass over the records of the database.

Humans identify location and directions based on visual cues, such as landmarks, rather than a precise coordinate grid. A database of landmarks can be used to determine the location of a user who can merely describe their location in the same manner they would to any human contact. The location of the user can be determined based on the landmarks described in the input, and the areas from which all of those landmarks are visible. Various databases can be used, including text-based and three-dimensional. Similarly, various mechanisms can be used to determine the user's location, including hierarchical approaches, query optimization approaches, and visibility percentage approaches. Ultimately, the user's location can be determined without the need for expensive, dedicated infrastructure, without compromising the user's security, and without subjecting the user to system outages.

For a database query that defines a plurality of separate snowflake schemas, a query optimizer computes separately for each of the snowflake schemas a logical access plan for obtaining from that schema's tables a respective record set that includes the data requested from those tables by that query. The query optimizer also computes a logical access plan for obtaining the query's results from the record sets in which execution of the logical access plans thus computed will result.

In embodiments, methods and systems for improving a query in a database system are provided. These method and system embodiments can enable greater contextual knowledge about the types and use of data in tables underlying a relational database to be employed to improve query efficiency. By employing contextual information, embodiments can provide improved queries and / or make recommendations to a query optimizer of a database system to improve its operation based upon knowledge of the data and / or application gathered. Embodiments can be useful in improving query performance in multi-tenant database systems.

A database query optimizer constructs a graph comprising nodes, relations, and expressions. The query optimizer then constructs execution plans for sub-parts of the graph. The combination of execution plans make up the overall execution plan for the query. The execution plan information is appended to the graph itself, allowing changing an execution plan in one portion of the graph without necessarily changing execution plans in other portions of the graph. By representing a query using the graph of the preferred embodiments that includes execution plan information, the query optimizer is able to evaluate the execution plans of different options quickly and efficiently, thereby enhancing the performance of the query optimizer.

A method for evaluating a user query on a relational database having records stored therein, a workload made up of a set of queries that have been executed on the database, and a query optimizer that generates a query execution plan for the user query. Each query plan includes a plurality of intermediate query plan components that verify a subset of records from the database meeting query criteria. The method accesses the query plan and a set of stored intermediate statistics for records verified by query components, such as histograms that summarize the cardinality of the records that verify the query component. The method forms a transformed query plan based on the selected intermediate statistics (possibly by rewriting the query plan) and estimates the cardinality of the transformed query plan to arrive at a more accurate cardinality estimate for the query. If additional intermediate statistics are necessary, a pool of intermediate statistics may be generated based on the queries in the workload by evaluating the benefit of a given statistic over the workload and adding intermediate statistics to the pool that provide relatively great benefit.

Database system query optimizers use several techniques such as histograms and sampling to estimate the result sizes of operators and sub-plans (operator trees) and the number of distinct values in their outputs. Instead of estimates, the invention uses the exact actual values of the result sizes and the number of distinct values in the outputs of sub-plans encountered by the optimizer. This is achieved by optimizing the query in phases. In each phase, newly encountered sub-plans are recorded for which result size and / or distinct value estimates are required. These sub-plans are executed at the end of the phase to determine their actual result sizes and the actual number of distinct values in their outputs. In subsequent phases, the optimizer uses these actual values when it encounters the same sub-plan again.

A system and method for optimizing a database query with improved performance enhancements is herein disclosed. The database query consists of one or more logical expressions. Through the repeated application of one or more rules, the logical expressions are transformed into execution plans. The query optimizer partitions the database query into one or more subproblems with each subproblem consisting of one or more logical expressions. A plan is obtained for each subproblem with the plan for the database query including the plans for each subproblem. The query optimizer is cost-based and uses rules including transformation and implementation rules that are used to perform transformations on the logical expressions in a subproblem in order to produce a plan. The rules are classified into context-free and context-sensitive in order to avoid generating duplicate expressions. Context-free rules are applied once for each logical expression and context-sensitive rules are applied once for each logical expression for a particular optimization goal. In a preferred embodiment, the query optimizer performs several optimization passes over the database query in order to obtain an optimal plan. For each pass, if no optimal plan exists for the requested optimization goal, existing plans having the same optimization goal are utilized with each input reoptimized for a more cost effective plan.

System and methodology for automatic tuning of database query optimizer is described. In one embodiment, in a database system having an optimizer for selecting a query plan for executing a database query, a method of the present invention is described for automatically tuning query performance to prevent query performance regression that may occur during upgrade of the database system from a prior version to a new version, the method comprises steps of: in response to receiving a given database query for execution, specifying a query plan generated by the prior version's optimizer as a baseline best plan for executing the given database query; generating at least one new query plan using the new version's optimizer; learning performance for each new query plan generated by recording corresponding query execution metrics; if a given new query plan is observed to have better performance than the best plan previously specified, specifying that given new query plan to be the best plan for executing the given database query; if a given new query plan is observed to have worse performance than the best plan previously specified, specifying that given new query plan to be a bad plan to be avoided in the future; and automatically tuning future execution of the given database query by using the query plan that the system learned was the best plan.

A system for automating data partitioning in a parallel database includes plural nodes connected in parallel. Each node includes a database server and two databases connected thereto. Each database server includes a query optimizer. Moreover, a partitioning advisor communicates with the database server and the query optimizer. The query optimizer and the partitioning advisor include a program for recommending and evaluating data table partitions that are useful for processing a workload of query statements. The data table partitions are recommended and evaluated without requiring the data tables to be physically repartitioned.

A method for sharing window-based joins includes slicing window states of a join operator into smaller window slices, forming a chain of sliced window joins from the smaller window slices, and reducing by pipelining a number of the sliced window joins. The method further includes pushing selections down into chain of sliced window joins for computation sharing among queries with different window sizes. The chain buildup of the sliced window joins includes finding a chain of the sliced window joins with respect to one of memory usage or processing usage.

The subject invention relates to systems and methods that automatically create, update, and use statistics over complex objects within a database query processor. A system is provided that facilitates employment of statistics in connection with database optimization. The system includes a component that receives information relating to database performance and an optimization component that automatically identifies, from the information, a minimal set of statistics to employ in a query. A loader dynamically loads and employs the set of statistics during a query optimization process. The optimization component can employ the statistics over computed columns, whereby the columns can be computed from persisted and non-persisted data.

To maintain the integrity and operability of enterprise database systems, queries intended for a particular database engine are intercepted, preferably by a separate processing system, prior to being acted upon by the database engine. The query is evaluated to determine its projected impact on available database system resources. The variables evaluated include user rights, administrator-imposed limits to searches, the amount of presently-available system resources, the cost of query and the like. If the query's potential impact does not surpass a predetermined threshold, it is submitted to the search engine. If the query is too resource-intensive or costly, the query may be (i) optimized by a query-optimization system, (ii) stored until more system resources are available, or (iii) rejected.

A “Query Optimizer” provides a cost estimation metric referred to as “Maximum Accumulated Overload” (MAO). MAO is approximately equivalent to maximum system latency in a data stream management system (DSMS). Consequently, MAO is directly relevant for use in optimizing latencies in real-time streaming applications running multiple continuous queries (CQs) over high data-rate event sources. In various embodiments, the Query Optimizer computes MAO given knowledge of original operator statistics, including “operator selectivity” and “cycles / event” in combination with an expected event arrival workload. Beyond use in query optimization to minimize worst-case latency, MAO is useful for addressing problems including admission control, system provisioning, user latency reporting, operator placements (in a multi-node environment), etc. In addition, MAO, as a surrogate for worst-case latency, is generally applicable beyond streaming systems, to any queue-based workflow system with control over the scheduling strategy.

An optimizer / normalizer is used to generate optimized intermediate language representation of an input query, such as an XML input query. A method of optimization of an input query in intermediate language form includes receiving the input query, examining the nodes in a left-depth first manner to identify code patterns and node types which are subjects for optimization, tagging the identified code patterns until the intermediate language representation of the input query has been examined in its entirety, searching from the top of the intermediate language representation for tagged code patterns, and adjusting the tagged code patterns with improved code patterns to form an optimal representation for an input query. The input to the optimizer / normalizer is assumed to be an input query transformed into an intermediate language representation containing code patterns and nodes, each node having a respective node type.

The invention relates to the field of heterogeneous information retrieval, in particular to an intelligent retrieval and analyzing method based on domain ontology and information mining and a visual analyzing system comprising the method. The system mainly comprises a field data acquisition subsystem, a corpus resource processing subsystem, an information mining subsystem and a visual analyzing subsystem, wherein the field data acquisition subsystem is used for acquiring data by network capturing and local uploading, the field data acquisition subsystem is used for pre-processing field related data, the field data acquisition subsystem is used for analyzing and mining related information in corpus, and the visual analyzing subsystem is used for dynamically displaying and counting and analyzing retrieval results. Concepts in a domain ontology base and mutual relations of the concepts are fully used by the system for heterogeneous information mining and visual analysis, requirements of users can be correctly understood to automatically cluster hierarchical structural information of a certain field so as to support the users to inquire key words, phrases and simple sentences and optimize retrieval results, relevant concepts and extension concepts can be found by ontological reasoning to support graphic display preview of each information meaning in the inquiry results, and the professional-field information retrieval performance can be remarkably improved to realize dynamic information display.

An improved method, system, and computer program product is disclosed that provides a hybrid approach to optimization which allows different subsets of data accessed by a query to be optimized with different access paths and execution approaches. Transformations may be performed to re-write the query, which restructures the query in a way that facilitates the hybrid optimization process. Multiple transformations may be interleaved to produce an efficient re-written query.

Improved query optimizer using implied predicates. The system facilitates allowing a query optimizing component to introduce into a query, extra predicates that facilitate the following: render the same results as the original query; are used as dictated by rules passed to the optimizing component—these rules specify whether the new predicate is an equivalence (that will substitute the old predicate) or an implication (in which the old predicate is preserved); are considered as cost-based alternatives, and discarded if not useful; are tied to index utilization; and can consider both standard and multi-valued indices.

The subject invention leverages threshold-based strategies applied to relational data to facilitate in determining an optimal execution plan for top-k selection queries. These strategies utilize a given query and relational data metadata to identify possible execution plans. This allows alternatives to scanning techniques to be considered in order to further enhance the overall efficiency of the optimal execution plan. A query optimizer can prune, for example, the alternative execution plans during enumeration of the plan space and / or during cost evaluations of the possible alternative execution plans. A cost model for the query optimizer can utilize a cost function based on an approximation of the number of iterations required to complete a threshold-based strategy.

Methods, systems and computer program products for detecting anomalies for a database system are provided. A method may include extracting workload features from a query optimizer based on a query workload and generating feature models for the extracted workload features. The method may also include extracting instance features from the query optimizer based on a query instance. Instance feature values may be obtained. The method may further include applying a query instance to the workload feature models to produce a prediction value for each workload feature. Anomalies may be reported based on a comparison of each instance feature value with a corresponding prediction value. A system for detecting anomalies for a database system may include a query optimizer, a feature modeler and an anomaly detector.

In accordance with embodiments, there are provided mechanisms and methods for query optimization in a database system. These mechanisms and methods for query optimization in a database system can enable embodiments to optimize OR expression filters referencing different logical tables. The ability of embodiments to optimize OR expression filters referencing different logical tables can enable optimization that is dynamic and specific to the particular tenant for whom the query is run and improve the performance and efficiency of the database system in response to query requests.

An autonomic tool that supervises the collection and maintenance of database statistics for query optimization by transparently deciding what statistics to gather, when and in what detail to gather them. Feedback from data-driven statistics collection is simultaneously combined with feedback from query-driven learning-based statistics collection, to better process both rapidly changing data and data that is queried frequently. The invention monitors table activity and decides if the data in a table has changed sufficiently to require a refresh of invalid statistics. The invention determines if the invalidity is due to correlation between purportedly independent data, outdated statistics, or statistics that have too few frequent values. Tables and column groups are ranked in order of statistical invalidity, and a limited computational budget is prioritized by ranking subsequent gathering of improved statistics. Multiple tables can have their statistics refreshed over time, and the maintenance effort is concentrated on the most important tables.

An optimizer / normalizer is used to generate optimized intermediate language representation of an input query, such as an XML input query. A method of optimization of an input query in intermediate language form includes receiving the input query, examining the nodes in a left-depth first manner to identify code patterns and node types which are subjects for optimization, tagging the identified code patterns until the intermediate language representation of the input query has been examined in its entirety, searching from the top of the intermediate language representation for tagged code patterns, and adjusting the tagged code patterns with improved code patterns to form an optimal representation for an input query. The input to the optimizer / normalizer is assumed to be an input query transformed into an intermediate language representation containing code patterns and nodes, each node having a respective node type.

A system and method for creation, modification and / or use of plan guide(s) is provided. Plan guide(s) can be employed to optimize the performance of query(ies) when it is not possible and / or desirable to change the text of the query directly. Plan guides can be useful, for example, when a small subset of queries in a database application deployed from a third-party vendor is not performing as expected. Plan guides influence optimization of queries by attaching query hints to them. In accordance with an aspect of the subject invention, a query plan execution system is provided. The system includes an execution component, a query optimizer, and, a plan cache. Plan guides and other associated metadata are stored in a plan guide metadata store.

A database monitor tracks performance statistics and information about the execution of different SQL statements. A query optimizer benefits from these statistics when generating an access plan. In particular, the query optimizer, upon receiving an SQL statement, searches the records of the database monitor for similar SQL statements that have previously been executed. As part of determining the best access plan for the current SQL statement, the query optimizer considers the information retrieved from the database monitor. In this way, the access plan that is generated can automatically be tuned based on empirical performance evidence.

A system and method for avoiding accessing a remote database twice, once during query execution and again to retrieve objects identified by the query, when an application requires objects on which to operate. During query plan generation in response to a request for database objects from an application, a query optimizer inserts cache operators into the candidate plans, and then a cost-benefit analysis is undertaken to identify the best plan. The best plan is then used to execute the query, with the cache operators causing objects identified during query execution to be cached locally to the requesting application as the query is being executed, thereby avoiding requiring the application to access the database after query execution.