308 results about "Query plan" patented technology

A method is provided for caching a query submitted to a graph data storage system, including receiving a query from a query caller directed to the graph data storage system, and examining the query, via a cache suitability determination engine, to determine if the query is suitable for caching under a first graph query caching policy based on the query caller being a trusted caller and / or the query scenario being a trusted query scenario. If suitable, generating a query execution plan for the query via a graph query plan generating engine and examining the query plan to determine if the query plan is suitable for caching under the first policy. If not suitable, caching the query under a second or a third graph caching policy based on the geographic location of a memory system on which the query plan is stored, and providing the query execution plan.

A computer-based system and corresponding computer-implemented method process input data queries in a manner enabling advanced functionality for data analytics. The method transforms a query plan tree into a query strategy tree. The query plan tree is constructed from an input data query associated with a computation workload. The method compiles the query strategy tree into dataflow graph(s) (DFG(s)). The method transmits the DFG(s) for execution via a virtual platform. The method monitors execution of the DFG(s). The method outputs, based on a result of the execution monitored, a response to the input data query. The result is received from the virtual platform and represents at least one computational result of processing the computation workload by the virtual platform. The system and method enable rapid and efficient retrieval and analysis of data stored in data lakes and other data storage systems, in response to user queries.

Dynamic selection of topologies for database data may be performed for performing queries. Different possible query plans for different assignments of topologies are determined for a query. One of the query plans is selected according to a cost-based model that assigns costs to different uses of the different topologies. The query is then performed using the selected query plan.

The invention provides a data synchronization method and device based on Spring Batch and a large model. Synchronously initializing a large model micro service and a Spring Batch framework, and establishing a model service hot standby mechanism; obtaining data source features through a metadata scanning engine driven by a large model, and generating an adaptive synchronization strategy; optimizing a query plan based on the synchronization strategy, dynamically controlling data fragmentation and executing streaming preprocessing and semantic mapping conversion; a target table structure is adapted, a three-level conflict detection mechanism is adopted to eliminate data conflicts, and meanwhile transaction processing and batch writing logic are optimized; the data quality is verified through three-level verification, performance parameters are dynamically adjusted and optimized, and fault-tolerant recovery and resource elastic expansion and contraction are achieved; metadata are synchronized, an execution report is generated, knowledge base self-learning is completed based on task full-link indexes, and a data consanguinity map and a reusable processing case library are constructed. According to the scheme, diversified and dynamic enterprise data synchronization requirements can be met.

The invention discloses a communication network multi-source big data analysis method based on distributed database collaboration, and relates to the technical field of communication data analysis, and the method comprises the steps: 1, generating a session contract number, a table snapshot number, a flow site set and a transaction time point at the start of a session, registering a collaborative metadata table, judging semantic compatibility, and setting a state; step 2, pushing down three anchors to a lake-warehouse, a message bus and a transaction library according to the serial number, performing freezing reading according to the anchors to obtain a lake-warehouse segment, a streaming segment and a transaction segment, performing alignment extraction on a joint feature set in the step 3, generating output through a deterministic operator chain, and storing the extracted joint feature set in a database; the method comprises the following steps: step 1, setting a session contract number, a field source and an operator relationship, reading an anchor code as a blood relationship code and writing the blood relationship code into a result table, and step 2, resetting three anchors and multiplexing a query plan during reproduction verification and cross-domain migration, and integrating blood relationship equivalence, result equivalence, freezing consistency, registration difference and migration fingerprints to realize session-level consistent reading, recalculation and auditing.

The invention relates to the technical field of data processing, and particularly provides a heterogeneous data source query method, system and device based on a Dores database and a medium, and the method comprises the following steps: obtaining a query text submitted by a user, and analyzing query parameters of the query text; obtaining metadata of each data source, and generating a plurality of candidate global logic plans based on the metadata and the query parameters; screening a global optimal query plan from the plurality of candidate global logic plans by using a cost model; and converting the global optimal query plan into a federated query statement supported by Dores, and executing the federated query statement by using a Dores federated query function to obtain target data. According to the method, a plurality of candidate global logic plans are automatically generated based on heterogeneous data source metadata and query parameters, and the optimal plan is accurately screened by using the cost model, so that the execution efficiency of cross-source query is remarkably improved, and low-efficiency scanning and network transmission bottlenecks are avoided.

The invention discloses an HTAP database query optimization method and device and a storage medium. The query performance is remarkably improved by combining the advantages of row storage and column storage. Comprising the steps of analyzing SQL query to generate a logic query plan; evaluating the execution cost of each logic operator in different storage modes by using a mixed cost model, wherein hyper-parameters of the mixed cost model are optimized through Bayesian search; selectively adopting a dynamic programming algorithm or a greedy algorithm to generate a hybrid physical query plan with the minimum execution cost according to the table number; and executing the plan to obtain a query result. In addition, an MCTS algorithm is further utilized, the optimal column subset is selected in the preset memory budget limit based on the query historical record and loaded into the memory column storage, and the subsequent query performance is further optimized. The problem that row storage and column storage in an existing HTAP system cannot work cooperatively is solved, operator-level mixed execution is achieved, and meanwhile memory resource utilization is optimized.

The present disclosure involves systems, software, and computer implemented methods for managing database query plans with dynamic conditions. One example method includes receiving a request to process a query based on at least one dynamic condition. A serialized abstract query plan is identified, deserialized, and converted into a query optimizer tree. A condition provider procedure is invoked to generate a first filter string for each view in the query optimizer tree. Comparisons are made to determine matches between first filter strings and corresponding second filter string stored in the query optimizer tree. In response to determining that a first filter string does not match the corresponding second filter string, query plan compilation is triggered. In response to determining that each filter string matches each corresponding second filter string, a query plan is generated based on the converted query optimizer tree and the query is executed using the query plan.

The invention discloses a multi-language data analysis-oriented extensible system, method, equipment and medium, and relates to the technical field of multi-language analysis, the multi-language data analysis-oriented extensible system comprises a grammar analysis module used for analyzing a query language and generating a language-specific syntax tree structure after analysis; the grammar adaptation module is used for converting a language-specific grammar tree structure into an abstract grammar tree AST in a uniform format; the abstract syntax tree processing module injects a permission control strategy, a field desensitization rule and an alarm exception expression into the AST; the business processing module is used for executing business-related rule check and label injection; the query plan generation module is used for constructing a logic query path and a corresponding distributed execution scheme; and the task scheduling module issues the distributed execution scheme to the corresponding execution node. According to the method, by constructing a unified multi-language grammar analysis framework and a standardized abstract syntax tree AST representation mechanism, compatibility and fusion of multiple query languages such as SQL, SPL and natural language are achieved.

The present disclosure involves systems, software, and computer implemented methods for using worst-case predicate selectivity for database query plans. One example method includes identifying a query that includes an initial set of parameter values for a set of predicates. An order of predicate evaluation is determined for a query plan by determining a worst-case selectivity for each predicate and ordering the predicates based on the worst-case selectivities. A second instance of the query is received that includes a second set of parameter values. The second instance of the query is executed, to generate a query result, using the query plan and the second set of parameters by evaluating the set of predicates based on the second set of parameters and the order of predicate evaluation determined based on the worst-case selectivities. The query result is provided in response to the second instance of the query.

The invention discloses a data query method, device and equipment for a distributed database, a storage medium and a program product, and the method comprises the steps: building a query plan evaluation model based on the data transmission time delay between every two working nodes in a distributed cluster, and the computing power resource and data storage layout information of each working node; under the condition that a query request is received, calling a query plan evaluation model, respectively evaluating a plurality of query plans generated based on the query request, and determining a target query plan from the plurality of query plans based on an evaluation result; determining at least one target working node of the to-be-queried data in the distributed cluster; and on the basis of a preset push-down strategy, pushing down each query operator contained in the target query plan to each target working node where the to-be-queried data is located, so that each target working node executes each query operator on the basis of the target query plan to obtain a data query result. By adopting the method, the data query efficiency of the distributed database can be improved.

The invention belongs to the field of artificial intelligence and big data analysis, and particularly relates to an AI-based drive type data analysis method and system.The method comprises the steps that S1, on the basis of natural language query input by a user, candidate SQL and / or DSL statements are generated through a pre-trained big language model; s2, generating a parallel query plan and optimizing the parallel query plan based on the candidate SQL and / or DSL statements and the metadata knowledge graph; s3, extracting authorization data; s4, fusing the extracted data to generate an analysis feature wide table, and identifying key features, abnormal modes and text abstracts in the analysis feature wide table; s5, executing controlled retrieval in the business knowledge graph based on the key features and the abnormal mode, and generating a security evidence sub-graph; s6, inputting the text abstract of the analysis feature wide table and the security evidence sub-graph into an AI inference engine, and outputting a structured report; and S7, displaying the structured report. The problems of high technical threshold and weak safety and interpretability in a data analysis process are solved.

The subject technology receives a query. The subject technology generates a set of query blocks based on parsing the query. The subject technology stores query block metadata for each query block from the set of query blocks. The subject technology restores a set of logical query block boundaries. The subject technology performs a hash-based query block matching. The subject technology generates, after performing the hash-based query block matching, a final query plan. In an example, if a (same) query is executed again the same logical query blocks would produce the same metadata (e.g., identifier, name and hash). This metadata can be used to match logical query blocks between multiple executions of the same query.

The invention provides a multi-data-source-oriented federal query method and device, equipment and a medium, and the method comprises the steps: obtaining a data analysis request of a user, converting a data query rule into a relational algebraic node format according to configuration item information and metadata information, obtaining at least one target query plan, and sending the at least one target query plan to a server; generating a query request corresponding to the data source according to the data query parameter of the at least one target query plan to query a corresponding underlying database to obtain target data, receiving the target data of each data source, preprocessing each target data, and performing statistical analysis according to the preprocessed target data to obtain a statistical analysis result; the preprocessing comprises data fusion and alignment; according to the method, the isomerism of different underlying databases can be shielded, the query process oriented to multiple data sources is simplified, the problems of complex cross-data-source analysis process and low efficiency can be effectively solved, and subsequent statistical analysis requirements can be supported by fusing and aligning the queried target data.

The invention discloses a statistical data downloading method for supply chain finance, which relates to the technical field of finance and comprises an acceptance permission judgment module, a query resource control module and a desensitization packaging evidence storage module. The acceptance permission judgment module is used for uniformly accessing the statistical application, completing identity verification, main body range binding, index range binding, time range binding and area range binding, and generating an acceptance bill and a judgment bill; the query resource control module is used for generating a query plan under the index aperture mapping dictionary, the partition information and the resource quota, binding a concurrent quota, a throttling quota and a priority queue, and outputting an executable instruction stream; and the desensitization packaging evidence storage module is used for executing differential desensitization according to the main body type and the sensitivity level, performing aggregation calculation and deduplication execution, completing result set packaging, digital watermarking and downloading delivery, generating a downloading certificate and writing a certificate abstract and an evidence index into an evidence storage chain. The method has the characteristic of high consistency.

A system may include a plurality of processing nodes in communication with a storage device configured to store a plurality of data. The processing nodes may receive a query on at least a portion of the data and may generate a query plan in natural language format. The processing nodes may generate a large language model (“LLM”) input based on the natural language format of the query plan and may execute an LLM on the LLM input. The processing nodes may generate, in response to execution of the LLM, a plurality of recommended actions to perform to improve the query plan. The processing nodes may receive input to execute at least one of the plurality of recommended actions and may alter the query plan in accordance with the at least one of the plurality of recommended actions. A method and computer-readable medium are also disclosed.

The present disclosure involves systems, software, and computer implemented methods for managing database query plans with dynamic conditions. One example method includes receiving a request to process a query based on a dynamic condition. In response to determining that a plan cache does not include a cached query plan for the query, query plan compilation is triggered to generate a query plan. In response to determining that a cached query plan exists, for each query view, a condition provider procedure is invoked to generate a first filter string. First filter strings are compared to corresponding second filter strings in the cached plan. In response to determining that a first filter string does not match the corresponding second filter string, query plan compilation is triggered. In response to determining that each filter string matches each corresponding second filter string, the database query is executed using the cached query plan.

The invention discloses a data query method and device, electronic equipment, a storage medium and a program product, and relates to the field of a data processing technology, an artificial intelligence technology, a large model technology and a large language model.The method comprises the steps that in response to a query instruction of a first query problem, the first query problem is displayed on an interaction page, triggering processing of the first query problem according to a query plan, wherein the query plan comprises a plurality of query steps; displaying the first processing result of each query step on an interaction page; and in response to a modification instruction for the first processing result of the target query step, triggering re-query for the first query problem, and obtaining and displaying a second processing result of each query step. According to the method, the query result of each query step can be intuitively displayed, the inaccurate query step can be intuitively understood, the query step is timely modified and re-queried, and the query efficiency is improved.

A method for verifying a user agent includes transmitting, by a core node, to a user agent, a predefined set of user requests. For each user request in the predefined set of user requests, the core node receives an embedding generated based on the user request, thereby receiving a plurality of embeddings based on the predefined set of user requests. The core node determines whether the plurality of generated embeddings satisfy a similarity criterion. The core node transmits, to the user agent, a task to generate a query plan and receives a query plan generated by the user agent for processing a user request identified in the task. The core node determines whether the query plan satisfies a query plan criterion, thereby producing query plan adequacy output. The core node determines whether to approve the user agent based on the similarity output and the query plan adequacy output.

In some embodiments, there is provided generating a logical query plan for a query that includes a single fact table and multiple reducer tables; optimizing the logical query plan for query execution; and executing the optimized logical query plan.

The invention relates to the technical field of computers, and discloses a data processing method and device, electronic equipment, a storage medium and a program product.The method comprises the steps that a query plan tree of a database is obtained, and the query plan tree comprises at least one plan node; aiming at each plan node in the query plan tree, coding node attributes of the plan nodes to obtain initial codes of the plan nodes; coding the plan nodes based on hierarchical positions of the plan nodes in the query plan tree and connectivity between the plan nodes in the query plan tree to obtain position codes of the plan nodes; the initial codes and the position codes of the plan nodes are fused to obtain coding results of the plan nodes so as to obtain target coding results of the query plan tree, and the target coding results are used for adjusting configuration parameters of a database. The encoding problem of database query can be solved.

IO scheduling method and device operating the same, are provided. A device connected to a host device through a CXL protocol, the device comprises a device memory which stores a query plan table and query plan status information synchronized with the host device and a device processor which processes a task according to the query plan table and updates query plan status information when receiving an operation command from the host.

The invention relates to the technical field of data storage, in particular to a remote visual monitoring shelter data management system based on a cloud platform, and the system comprises the steps: obtaining shelter multi-dimensional monitoring data, and automatically marking context metadata including domain function attributes, business key levels, task IDs and task stages; constructing a dynamic data portrait according to the metadata, routing the data to logic storage hierarchies with different costs and performances through a rule engine, and executing life cycle management; constructing a task ID global secondary index, and constructing a data fingerprint and fusion event index for a key event and a context data set thereof; and providing a unified query interface, and executing a cross-hierarchy federated query plan by using hierarchical storage information and a fusion index. The overall efficiency of shelter data management can be effectively improved.

The invention discloses a data query method and device based on a natural language. The method comprises the following steps: receiving natural language input of a user; performing semantic analysis on the natural language input, and determining a user intention and a plurality of semantic elements; based on the user intention, target query operators corresponding to the semantic elements respectively are determined in a query operator sequence, and the query operator sequence comprises a plurality of query operators arranged according to the execution sequence; generating a query plan based on the target query operators, the semantic elements corresponding to the target query operators and the execution sequence of the multiple target query operators; a query statement is generated based on the query plan and target database object identifiers corresponding to the semantic elements respectively, and the target database object identifiers comprise at least one of table names and field names; and performing data query based on the query statement. According to the embodiment of the invention, the accuracy and the interpretability of the query statement can be improved, so that the accuracy and the reliability of the query result are improved.

Examples detect equivalent subexpressions within a computational workload. Examples include converting a query plan tree associated with a first subexpression into a matrix. The first subexpression is a portion of a database query from the computational workload. Each node in the query plan tree is represented as a row of the matrix. The matrix is converted into a first vector. The first subexpression is determined to be equivalent to a second subexpression by comparing the first vector to a second vector associated with the second subexpression. The comparison includes computing a distance between the first and second vectors that is lower than a distance threshold. The computational workload is modified, based on the determining, to perform the first subexpression and exclude performance of the second subexpression as duplicative.

The invention discloses a cardinal number estimation method based on a bidirectional long-short term memory network and ensemble learning, and aims to solve the limitation of a traditional cardinal number estimation method and an existing learning type method in accuracy and efficiency. According to the method, the structure, connection, operation and filtering condition information of the query plan tree is efficiently extracted through the four sub-encoders, the extracted feature sequence is compressed by using the bidirectional long-short-term memory network, the context dependency relationship between the nodes is effectively captured, and the model learning difficulty is reduced. Besides, a Bayesian neural network is introduced to be combined with an active learning strategy to screen and construct a plurality of high-value training data subsets, and a robust integrated model is trained on the basis to be used for cardinality estimation. According to the method, the cardinality estimation accuracy is remarkably improved on multiple data sets, and when complex query and multi-table connection scenes are processed, the method shows better comprehensive performance compared with other methods.

A method of incorporating a bloom filter (BF) into cost-based bottom-up query optimization includes: receiving a query including a request to join at least two data tables that include a first data table and a second data table; generating, in response to the request, at least two query plans that includes a first query plan including subplans to: create the BF based on the second data table, apply the BF to the first data table for filtering data of the first data table before the first data table is joined with another data table, and join the first data table with the another data table; submitting the at least two query plans to a cost-based bottom-up optimizer to obtain a target query plan; and providing, as a response to the received query, a set of data that is retrieved from a data retrieval system by executing the target query plan.

The invention relates to the technical field of terminals, and provides a text processing method, device and equipment and a computer readable storage medium, and the method comprises the following steps: obtaining a to-be-queried text; on the basis of the to-be-queried text, query operation information is generated, and the query operation information represents a query plan when the to-be-queried text is queried; determining a plurality of to-be-queried sub-texts based on the query operation information; when the text generation model is adopted to perform the first query processing on the plurality of to-be-queried sub-texts, the at least two to-be-queried sub-texts in the plurality of to-be-queried sub-texts are processed in parallel, so that the time of performing the first query processing on the plurality of to-be-queried sub-texts by the text generation model is shortened; therefore, the time for generating the plurality of first sub-query results is shortened, and the efficiency for generating the plurality of first sub-query results is improved.

A distributed database identifies classifications of risk associated with stages of a query plan. The distributed database generates an execution plan in which incompatible risk classifications are assigned to separate stages of an execution plan that is derived from the query plan. The stages are assigned to computing nodes for execution based, at least in part, on the risk classifications. A result for the query is generated based on execution of the stages on the assigned computing nodes.