403 results about "Query optimization" patented technology

The invention relates to the field of internet attack detection, in particular to a data security event real-time monitoring method and system, and the method comprises data collection, multi-modal fusion, threat detection, causal reasoning, dynamic response and feedback optimization. Compared with the traditional security analysis which depends on isolated data dimension or simple rule association, is difficult to capture a cross-data-source complex attack mode, and is faced with the limitations of low calculation efficiency, slow link traceability, storage access bottleneck and the like in mass data association analysis, the scheme integrates multi-source heterogeneous data into a dynamic association network through graph structure modeling, so that the security analysis efficiency is improved. Hidden association and behavior patterns among users, equipment and IPs are deeply mined by utilizing a GNN framework, the suspicious degree among entities can be accurately quantified, and hidden attack chains can be identified; and meanwhile, a hybrid storage architecture and a query optimization technology are adopted, so that a security analyst can backtrack a complex attack path in a second level while breaking through the bottleneck of large-scale graph data access performance, and the threat hunting efficiency and the high-level attack traceability are remarkably improved.

The invention provides a traffic comprehensive query optimization method and system applied to intelligent human-computer interaction, and the method comprises the steps: obtaining a traffic query request input by a target user in an interaction scene, carrying out the multi-modal feature analysis of the traffic query request, generating a semantic understanding feature and a scene correlation feature, and obtaining a semantic understanding feature and a scene correlation feature based on a preset language large model; performing dynamic interaction feature fusion processing on the semantic understanding feature and the scene association feature to generate a multi-dimensional intention feature, and performing service path optimization on the traffic query request of the target user according to the multi-dimensional intention feature to generate a comprehensive response strategy; and converting the comprehensive response strategy into interactive feedback information, dynamically adapting to equipment parameters in the real-time environment association information, and executing feedback operation to complete optimized response of the traffic query request. According to the invention, the response real-time performance, decision reasonability and execution reliability of the intelligent traffic query service can be improved.

According to the artificial intelligence-based data analysis system and method provided by the invention, a set of novel data analysis system architecture is designed, and key technical components such as natural language processing, a large language model, database query optimization, multi-modal visualization and a semantic knowledge graph are fused; data transmission between modules is realized through unified intermediate data objects such as a structured query intention, an analytic tree and a structured query language template object, and asynchronous collaboration is realized through event driving and a message queue mechanism, so that a non-professional user can input an analysis request through a natural language; semantic analysis, structured query language query statement generation, data query and result visualization presentation are automatically completed, the method can be widely applied to data analysis scenes in the industries of government affairs, traffic, finance, education and the like, the data use efficiency is improved, the technical threshold is reduced, and the digital decision-making ability is enhanced.

A system (100) for AI-assisted query optimization and self-optimizing data pipelines, comprising: a query analysis and classification module configured to monitor, log, and categorize incoming queries based on execution complexity, type, and frequency; an AI-based query execution optimizer that uses machine learning models to analyze historical query performance and recommend dynamically optimized execution plans; a self-optimizing data pipeline module configured to monitor and adapt ETL workflows by modifying transformation logic, workload balancing, and execution sequences; an adaptive indexing and storage management module configured to dynamically restructure indexing strategies, including predictive indexing and partitioning, to improve query performance; an intelligent caching and pre-fetching module configured to anticipate frequently used queries and pre-load relevant data to reduce redundant computations; a workload balancing and resource optimization module configured to dynamically distribute query loads among computing resources based on the constraints of the real-time system (100); an anomaly detection and performance monitoring module configured to continuously track query execution metrics, identify inefficiencies, and mitigate performance bottlenecks using AI-based predictive analytics; a reinforcement learning-based query optimization module configured to iteratively refine query execution strategies based on real-time feedback and evolving data properties; a multi-cloud and cross-platform optimization engine configured to ensure seamless operation across heterogeneous database management systems and cloud platforms.

The invention relates to the technical field of distributed agent collaboration and graph database query optimization, in particular to a graph query processing method and system based on multi-agent collaboration, and the method comprises the following steps: reasoning a coordination agent to analyze the semantics and intention of user query by using a large language model; a structured multi-agent execution plan is output, and an entity search agent or a relation analysis agent needing to be called is explicitly called; the agent coordinator receives the multi-agent execution plan and selects agents according to task types and task allocation; obtaining a distributed intermediate result set; starting a result aggregation mechanism, and performing typed processing, priority ranking and deduplication merging on the distributed intermediate result set to obtain an arrangement result; and outputting an answer for answering the initial question of the user. The system comprises an agent execution plan output module, a multi-task execution module and an arrangement result reasoning module. According to the method, efficient, extensible and fault-tolerant graph query processing is realized.

Latency, response times, and efficiency improvements for data querying are provided herein, particularly in the context of querying large database systems and data tables from disparate data sources. There are provided systems and methods for automatic query and data retrieval optimization through procedural generation of data tables from query patterns. A service provider may utilize different computing services for query processing and data retrieval for different applications and services used by internal and / or external users. Instead of querying large database systems and numerous data tables, pre-aggregated data tables may instead be used and searched by procedurally generating such tables based on precomputation rules and query patterns. Once patterns have been identified in queries, corresponding data may be aggregated from data sources in a pre-aggregated data table. Query optimization rules may then be used to have these data tables queried in place of their original sources.

The invention provides a household appliance knowledge question-answering method and system based on retrieval enhancement generation. The method comprises the following steps: acquiring household appliance field multi-modal data from a multi-format document library; extracting text information, table information and chart information in the multi-modal data; performing domain term injection processing on the extracted information, and constructing a packet domain enhancement index; receiving a natural language question input by a user; the natural language problem is analyzed through a query optimizer, and semantic retrieval and keyword retrieval are executed in parallel; carrying out fusion processing on the semantic retrieval result and the keyword retrieval result; selecting matched document fragments by adopting a relevancy sorting algorithm; inputting the matched document fragments into a large language model to generate candidate answers; verifying the compliance and traceability of the candidate answers through a credibility evaluation module; outputting a final answer with a reference source; and storing the high-frequency questions and the final answers into a cache library to solve the problems that the answer accuracy of a knowledge question-answering system is reduced and the response efficiency is limited.

A database query optimization method, comprising: acquiring a query statement, the query statement comprising a connection operation and a group aggregation operation; determining that a connection key and a group key in the query statement are the same, wherein an aggregation function in the query statement can be split, and an input parameter of the aggregation function comes from the same side as a connection condition in the query statement; creating a fusion operator, the fusion operator being used for completing the connection operation and the group aggregation operation; when a cost for executing the fusion operator is less than or equal to a target cost, on the basis of the fusion operator, generating an optimal execution plan related to the fusion operator, wherein the target cost is a cost for executing a connection operator and a group aggregation operator; and performing the operations step by step on the basis of an operation sequence in the optimal execution plan, so as to obtain a query result set. Therefore, when the query statement meets corresponding features, the connection operation and the group aggregation operation can be fused into one operator, thereby reducing the processing flow of an intermediate result, accelerating the execution of the query statement, and improving the query execution performance.

The database query optimization method comprises the steps that a query statement is obtained, and the query statement comprises a connection operation and a grouping aggregation operation; determining that a connection key and a grouping key in the query statement are the same, an aggregation function in the query statement is detachable, and input parameters of the aggregation function are from the same side of a connection condition in the query statement; creating a fusion operator, wherein the fusion operator is used for completing a connection operation and a grouping aggregation operation; under the condition that the cost of executing the fusion operator is smaller than or equal to a target cost, an optimal execution plan related to the fusion operator is generated according to the fusion operator, and the target cost is the cost of executing the connection operator and the grouping aggregation operator; and executing operations step by step according to the operation sequence in the optimal execution plan to obtain a query result set. Therefore, when the query statement meets the corresponding characteristics, the connection and grouping aggregation operations can be fused into one operator, so that the processing flow of an intermediate result can be reduced, the execution of the query statement is accelerated, and the execution performance of the query is improved.

The invention discloses a retrieval enhancement generation method and device of an electric power system, a medium and equipment, and the method comprises the steps: constructing an electric power knowledge graph according to an original corpus of the electric power system, and constructing a structure index and a semantic index according to the electric power knowledge graph; performing query optimization on the original query content input by the user to obtain professional query content; performing neighbor sampling on the electric power knowledge graph according to the professional query content to obtain a query related sub-graph; according to the professional query content, the query related sub-graph, the structure index and the semantic index, performing mixed retrieval of semantics and maps to obtain a query retrieval result; and performing enhanced generation on the query retrieval result based on an attribution mechanism to obtain a user query answer. According to the technical scheme, the credibility of the generated result can be improved, meanwhile, the retrieval efficiency is improved, the query threshold is lowered, and the interpretability of the generated result is improved.

The invention provides an intelligent routing optimization method and system for multi-table association query, and relates to the technical field of routing optimization, and the method comprises the steps: extracting static features and dynamic features in a query request to construct a multi-dimensional feature vector; and calculating association degrees among the tables based on the static characteristics and grouping, and predicting a system resource state by using a time sequence prediction model. And inputting the grouping result and the resource state sequence into a two-channel neural network model, respectively processing by using a graph neural network and a recurrent neural network, and fusing features through an attention mechanism. And finally, the decision-making layer network outputs a query optimization scheme including a distributed execution plan, a data fragmentation strategy and a resource allocation scheme. In the execution process, the performance is monitored in real time, and the optimization scheme is adjusted according to the state information. In addition, an incremental training set is constructed by utilizing execution historical information, the model is updated online, the model adaptability is improved, and closed-loop optimization is formed, so that the query efficiency is remarkably improved, and the system resource consumption is reduced.

The invention discloses a dynamic business data updating and querying method and system, and particularly relates to the technical field of information processing, and the method comprises the following steps: S1, real-time data acquisition and preprocessing; s2, updating the dynamic data; s3, intelligent query optimization; s4, result feedback and visualization are carried out; and S5, performing anomaly detection and self-repairing. According to the method, the real-time processing capability and query efficiency of massive heterogeneous data are remarkably improved, the distributed sensor network is combined with lightweight encryption and a differential privacy mechanism to realize localized cleaning and feature extraction of multi-source data, the load of a central server is effectively reduced, and the risk of sensitive information leakage is avoided; an incremental synchronization mechanism depends on a message queue and Hash chain log technology to ensure the atomicity and traceability of data updating in a high-concurrency scene, and by combining a dynamically generated space-time multi-dimensional index and an adaptive weight distribution algorithm, the system can intelligently analyze query semantics and optimize path selection, and the response time of complex query is greatly shortened.

The invention belongs to the field of query optimization, and particularly relates to a fragmentation storage and query optimization method and system for a high-concurrency database, and the method comprises the steps: analyzing business features through a preset decision tree model, and selecting an optimal fragmentation key, constructing a fragmentation rule engine and initializing a database in combination with hash, range and list fragmentation rules and a fragmentation splitting-merging strategy; generating a query abstract syntax tree by using an ANTLR4 analyzer, and detecting whether a preset Cube is hit or not to directly return a result; if not, dynamically routing to a target fragment list based on a fragment key field, load balancing and a failover strategy, accelerating data retrieval in combination with a high-frequency index, and combining fragment-level results through a T-TopK algorithm; according to the method, the self-adaptive matching of the fragmentation strategy and the service requirement, the calculation push-down of the query process and the result optimization are realized, and the low delay and the high availability in a high-concurrency scene are ensured.

An adaptive random access system and method with learned query optimization for compacted data files that enhances random access performance through machine learning and pattern recognition. The system incorporates a query pattern learning module that analyzes historical access patterns and user behavior to build statistical models of data usage. An adaptive estimator module improves location estimation accuracy by incorporating learned patterns rather than relying solely on mathematical calculations. A predictive boundary detector uses learned codeword patterns to more accurately identify boundaries in compacted data, reducing misalignment errors. An intelligent search engine coordinates optimization strategies including context-aware search string parsing and encoding strategy selection based on learned performance data. A dynamic codebook optimizer reorganizes sourceblock layout based on access frequencies and co-occurrence patterns to improve retrieval speed. An enhanced search cache implements predictive caching algorithms that anticipate user queries and proactively load relevant data.

The invention provides a query optimization method and device based on OceanBase database data fragmentation and related products, and the method comprises the following steps: determining one or more columns as service keys based on a plurality of service tables, the service keys being used for connecting service data in the service tables; based on the service key, setting one or more columns as fragmentation keys, and combining the fragmentation keys with the service key so as to construct a fragmentation table in the OceanBase database; performing fragmentation processing on target data based on the fragmentation table to create a data fragmentation structure; based on the data fragmentation structure, constructing an extended SQL query statement carrying preset prompt optimization information SQL Hint; on the basis of the extended SQL query statement, a query optimizer of the OceanBase database selects a preset execution plan to query the service table so as to obtain a target result set, and the query performance is improved in combination with the parallel processing capacity of the database.

The invention discloses a remote sensing target detection large model construction method based on context feature deep learning, and the method comprises the following steps: S1, dividing an original data set according to a strategy, forming a training set, a verification set and a test set, and guaranteeing that each subset has representativeness in sample number, scene diversity and target distribution; s2, constructing a remote sensing target detection large model by adopting a deep learning method based on context features, and fusing multi-level features and introducing a query optimization mechanism; and S3, performing model testing and performance evaluation, selecting optimal model parameters, performing comprehensive verification on the test set, and evaluating the comprehensive performance of the model in the aspects of detection precision, speed and robustness. According to the method, the pre-trained DINOv2 backbone network, the context feature enhancement module and the intensive supervision strategy are introduced, so that rich context information can be captured in a multi-scale and complex scene, and the detection precision is greatly improved.

The invention discloses a batch information retrieval and structured output method and system, and belongs to the technical field of big data analysis and information security, batch information retrieval and structured output are realized based on multi-source heterogeneous file analysis, and the method specifically comprises the steps of resource index construction and data processing, integrating a single resource and an integration type resource into a search engine through configuration by customizing configuration; field mapping information is stored in a configuration database, and whether the fields are mapping of batch query or not is achieved; dynamic analysis engine and element extraction: dynamically identifying the format type of an uploaded file through a file header feature code, and extracting key element information including an identity card number and a mobile phone number by adopting a streaming analysis and hybrid classification model; performing distributed batch retrieval and query optimization; and performing structured output and safety compliance control. According to the method, the fusion processing efficiency and security of multi-source data in scenes such as public security investigation and financial anti-fraud can be remarkably improved.

The invention discloses an HTAP database query optimization method and device and a storage medium. The query performance is remarkably improved by combining the advantages of row storage and column storage. Comprising the steps of analyzing SQL query to generate a logic query plan; evaluating the execution cost of each logic operator in different storage modes by using a mixed cost model, wherein hyper-parameters of the mixed cost model are optimized through Bayesian search; selectively adopting a dynamic programming algorithm or a greedy algorithm to generate a hybrid physical query plan with the minimum execution cost according to the table number; and executing the plan to obtain a query result. In addition, an MCTS algorithm is further utilized, the optimal column subset is selected in the preset memory budget limit based on the query historical record and loaded into the memory column storage, and the subsequent query performance is further optimized. The problem that row storage and column storage in an existing HTAP system cannot work cooperatively is solved, operator-level mixed execution is achieved, and meanwhile memory resource utilization is optimized.

The invention relates to the technical field of database optimization, and discloses a multi-table connection sequence query optimization method, device and equipment, and the method comprises the steps: carrying out the feature extraction of a received database query statement, obtaining a query feature representation, and enabling the database query statement to comprise the data query of a plurality of tables; inputting the query feature representation into a pre-trained target network model, and predicting execution time of different connection sequences in a database query statement to obtain an execution time degree; and on the basis of a Monte Carlo tree, taking the query feature representation and the execution time degree as reward functions, and optimizing the connection sequence of the database query statements to obtain a target connection sequence. According to the method, feature extraction, model prediction and an optimization process based on the Monte Carlo tree are integrated, the execution time and complexity are comprehensively considered, the optimal connection sequence is selected, the limitation that a traditional method is only based on a cost model is overcome, and more accurate connection sequence selection is achieved.

The invention provides an intelligent tour guide man-machine conversation method and system based on knowledge graph retrieval, and relates to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: collecting relevant data of scenic spots in a preset range, carrying out structured processing on all the relevant data of the scenic spots, and constructing a knowledge graph in the tour guide field; establishing indexes for entities and relationships in the knowledge graph, and introducing an adaptive query optimization mechanism; selecting a pre-trained language model and performing LoRA fine tuning on the pre-trained language model to obtain a fine tuning language model; a user initiates a dialogue request; the dialogue request accesses the fine tuning language model to obtain a preliminary answer result, and accesses the knowledge graph to obtain knowledge graph information; and matching and fusing the preliminary answer result and the knowledge graph information, and outputting a corrected answer result. According to the invention, knowledge can be effectively fused between the tour guide explanation field and the general field, and the accuracy and comprehensiveness of answers are improved.

The present disclosure involves systems, software, and computer implemented methods for managing database query plans with dynamic conditions. One example method includes receiving a request to process a query based on at least one dynamic condition. A serialized abstract query plan is identified, deserialized, and converted into a query optimizer tree. A condition provider procedure is invoked to generate a first filter string for each view in the query optimizer tree. Comparisons are made to determine matches between first filter strings and corresponding second filter string stored in the query optimizer tree. In response to determining that a first filter string does not match the corresponding second filter string, query plan compilation is triggered. In response to determining that each filter string matches each corresponding second filter string, a query plan is generated based on the converted query optimizer tree and the query is executed using the query plan.

The invention provides a medical data query optimization method and system based on semantic understanding, and relates to the technical field of medical data query, and the method comprises the steps: firstly obtaining a user medical data query request, a historical query record set and a current medical scene description; performing semantic intention scene deconstruction and historical association mapping on the query request to generate a deconstruction result; constructing a semantic path dynamic adaptation model based on the deconstruction result, and generating a model parameter set; calling the initial query path, performing path reconstruction according to the model output adaptation relation, and generating an optimized query path; and executing query operation to obtain an initial result, collecting user feedback for semantic calibration, generating an optimization result, feeding back the optimization result to the user terminal, and associating related information to a historical record set. According to the invention, the accuracy and efficiency of medical data query are improved.

The invention relates to the technical field of sales data management, and discloses an electronic component online sales data management and maintenance system, which comprises a data acquisition and processing module, a user demand matching module, a data verification module, an inventory optimization module and the like. The data acquisition and processing module acquires multi-dimensional sales data, and performs natural language processing and dynamic knowledge graph construction, processing and data association; the user demand matching module generates an initial matching result based on a collaborative filtering algorithm and semantic clustering; the data verification module verifies the data by using a Hash algorithm and a block chain network; and the inventory optimization module optimizes the inventory according to the timeliness weight factor and a reinforcement learning algorithm. In addition, the system is further provided with an abnormal transaction detection module, a multi-source data integration module and an interactive query optimization module which are respectively used for detecting abnormal transactions, integrating multi-source data and optimizing query. The system effectively solves the problem of electronic component online sales data management, and improves sales efficiency and user experience.

A hypergraph workload manager in a server is configured for failure tolerant and explainable state machine driven hypergraph execution. The hypergraph executor comprises a query optimizer, a hypergraph enlister, a pipeline analyzer, and a state machine generator. The query optimizer translates a user query into a query operator graph. The hypergraph enlister enlists the query operator graph into a hypergraph containing a set of query operator graphs representative of already submitted user queries. The enlistment is configured to join query operator graphs where it makes sense to optimize query executions. Updates to the hypergraph based on the enlistment results in a set of disconnected graphs. The pipeline analyzer performs an analysis of all operators of all queries in the hypergraph to find an optimal sequencing of execution. The state machine generator is configured to generate a hierarchical state machine for all operators of a disconnected graph of the hypergraph.

The invention relates to the technical field of big data storage, and discloses a multi-level data storage and mixed query method and system, a storage medium and equipment, and the method comprises the steps: collecting original buried point data, carrying out real-time stream processing, storing the processed real-time data in a hot storage layer, carrying out timing offline batch processing on historical buried point data, and storing the processed real-time data in a hot storage layer; converting into an optimized storage format and storing in an object storage system; an extended metadata management system is constructed, and structure information, partition information and access frequency information of data in the object storage system are automatically extracted and managed; executing query, and reducing the data scanning amount through a query optimization strategy; automatically migrating data among different storage layers through a dynamic cold and hot data layering mechanism; a unified query interface is provided, a query request is distributed to a corresponding real-time processing system or an offline query system based on an intelligent routing strategy, and a query result is cached and returned, so that high-performance and low-cost storage and second-level query of data are realized.

The invention provides a visual form construction method, which comprises the following steps: receiving user form operation data, carrying out business logic calculation, state change control and inter-service coordination operation on the user form operation data based on domain-driven design, and generating domain events, state data and service governance data. Domain events and state data are recorded to an event storage system, a query optimization view is constructed, event logs and query view data are generated, and dual optimization of data persistence and query efficiency is ensured. According to a preset security policy rule, real-time security monitoring is performed on the service governance data, a system health state index and security audit data are generated, and a system-level observability and security protection mechanism is established. And performing interface rendering and synthesis based on the event log, the query view data, the system health state index and the security audit data to generate a user interactive form interface. Through fusion processing of multi-source data, deep integration of interface rendering and security policies is realized.

The invention relates to the technical field of data analysis, in particular to a multi-mode heterogeneous data query method which comprises the following steps: receiving a query statement sent by a user side, performing lexical analysis on the query statement through an analyzer, and performing semantic check on the analyzed query statement to obtain an initial query statement; sending the initial query statement to a query optimizer to generate a corresponding query syntax tree; after the query syntax tree is sent to a Data-speed engine for prompt word engineering processing, indexing is carried out in combination with an index vector library to obtain an index result, and an execution plan is intelligently predicted based on the index result; inputting the intelligent prediction execution plan into a large language model system, outputting an execution code corresponding to a current execution task and operators of an available instruction set corresponding to different execution tasks in real time, and introducing an entity identifier into an attention mechanism of the large language model system for supervision and fine adjustment; and calling an operator corresponding to the execution code through a Data-speed JIT generator, and generating an execution result based on the operator.

The invention relates to a database technology, in particular to a query optimization method, a storage medium and equipment. The query optimization method comprises the following steps: obtaining a to-be-processed query statement; judging whether the to-be-processed query statement contains a window function or not; and if yes, converting the window function into transverse connection, and generating an optimized query statement based on the transverse connection. According to the query optimization method, the window function in the query statement is converted into the transverse connection, so that the window function in the query statement is eliminated, the corresponding grouping and sorting operation is removed, the problem of high performance loss caused by excessive unnecessary grouping and sorting operation due to execution of the window function is avoided, and the query efficiency is improved. And the execution efficiency of the query statement is improved, so that the performance of the database is improved.

The invention discloses an Internet of Things full data persistent storage system and method based on multi-stage time granularity. The system comprises a data acquisition module, a multi-stage storage management module, a dynamic index module, a compression engine module and a query engine module. The data acquisition module is used for receiving equipment data in real time, the multi-level storage management module is used for dynamically dividing time windows and executing a cold and hot data migration strategy, the dynamic index module is used for constructing a joint index of time dimensions and equipment IDs, and the compression engine module is used for selecting an optimal compression algorithm according to data features and transmitting the optimal compression algorithm to the storage management module. And the query engine module is used for analyzing a query condition, positioning a data block through an index and optimizing an IO reading sequence. Compared with a traditional time sequence database, the storage cost is reduced through a dynamic time granularity division and hierarchical compression algorithm; and on the basis of query optimization of multi-level indexes, the query efficiency in the complex time range is improved by several times, and the query response time in the complex time range is reduced from the minute level to the sub-second level.

The invention provides a design method and system for efficiently querying a bank data warehouse based on an MPP architecture, and the method comprises the steps: S1, collecting and cleaning data from a plurality of bank business systems, and loading the data to a Greenplum distributed data warehouse; s2, performing hierarchical modeling in the warehouse, constructing a customer center wide table, and storing data in each node in a balanced manner by adopting partition and hash distribution strategies; s3, designing a batch processing task scheduling process, and improving the batch processing efficiency by utilizing parallel execution; s4, optimizing configuration indexes and materialized views according to the queries, and accelerating query response by utilizing parallel computing of the MPP architecture; and S5, monitoring a system load and dynamically and elastically scaling cluster resources to deal with service peak and valley demands. By means of the steps, high-performance batch processing and real-time query support of EB-level bank data are achieved on the GreenplumMPP cluster, the data processing efficiency and the query response speed are remarkably improved, and good expansibility is achieved so as to meet the service development requirement.