495 results about "Query optimization" patented technology

The invention provides a streaming query semantic map adaptive enhancement method and system based on cognitive computing, and relates to the technical field of semantic maps, comprising the steps of computing node affinity through a map attention mechanism, identifying knowledge clusters by using a community discovery algorithm, performing knowledge deduction by taking the knowledge clusters as a basic unit, and predicting and complementing potential semantic relationships. And after multi-agent reinforcement learning optimization verification, outputting an enhanced semantic map and a credibility score table. And deploying the enhanced atlas to an edge network, carrying out parallel semantic understanding and query optimization suggestion generation, integrating optimization suggestions through federal learning, updating the central cognitive model, and distributing the central cognitive model to the edge network. According to the method, the semantic map is enhanced through cognitive calculation, so that more accurate semantic understanding and more efficient query optimization are realized.

The invention provides a natural language SQL (Structured Query Language) conversion method based on a data platform and a large language model, which relates to the technical field of data processing and comprises the following steps of: performing vectorization storage on metadata, blood relationship, indexes and historical query information, performing intention judgment and vector matching on a user query request, and determining a query condition based on a matching result. An association path is constructed according to data consanguinity, and an optimal SQL statement is generated by utilizing a historical query optimization table connection sequence and combining index matching. According to the method, intelligent conversion from the natural language to the efficient SQL can be realized, the query efficiency is improved, and the use threshold of a user is reduced.

The invention provides an unstructured database federated learning collaboration method and system based on swarm intelligence, and relates to the technical field of structured data, and the method comprises the steps: training a query pattern recognition sub-model, a path planning sub-model and a resource allocation sub-model through federated learning, and constructing a swarm intelligence collaboration network. After a user query request is analyzed by the query mode recognition sub-model, candidate query paths are generated and optimized by fusing an ant colony algorithm and a particle swarm algorithm, and a query execution scheme is generated. And the resource allocation sub-model performs resource scheduling accordingly. And generating a performance optimization target for updating local model parameters of each node, and fusing the updated parameters through a security aggregation algorithm to guide the next query optimization. According to the method, the query efficiency of the unstructured database is improved, and the resource consumption is reduced.

The invention relates to the field of internet attack detection, in particular to a data security event real-time monitoring method and system, and the method comprises data collection, multi-modal fusion, threat detection, causal reasoning, dynamic response and feedback optimization. Compared with the traditional security analysis which depends on isolated data dimension or simple rule association, is difficult to capture a cross-data-source complex attack mode, and is faced with the limitations of low calculation efficiency, slow link traceability, storage access bottleneck and the like in mass data association analysis, the scheme integrates multi-source heterogeneous data into a dynamic association network through graph structure modeling, so that the security analysis efficiency is improved. Hidden association and behavior patterns among users, equipment and IPs are deeply mined by utilizing a GNN framework, the suspicious degree among entities can be accurately quantified, and hidden attack chains can be identified; and meanwhile, a hybrid storage architecture and a query optimization technology are adopted, so that a security analyst can backtrack a complex attack path in a second level while breaking through the bottleneck of large-scale graph data access performance, and the threat hunting efficiency and the high-level attack traceability are remarkably improved.

The invention provides a traffic comprehensive query optimization method and system applied to intelligent human-computer interaction, and the method comprises the steps: obtaining a traffic query request input by a target user in an interaction scene, carrying out the multi-modal feature analysis of the traffic query request, generating a semantic understanding feature and a scene correlation feature, and obtaining a semantic understanding feature and a scene correlation feature based on a preset language large model; performing dynamic interaction feature fusion processing on the semantic understanding feature and the scene association feature to generate a multi-dimensional intention feature, and performing service path optimization on the traffic query request of the target user according to the multi-dimensional intention feature to generate a comprehensive response strategy; and converting the comprehensive response strategy into interactive feedback information, dynamically adapting to equipment parameters in the real-time environment association information, and executing feedback operation to complete optimized response of the traffic query request. According to the invention, the response real-time performance, decision reasonability and execution reliability of the intelligent traffic query service can be improved.

According to the artificial intelligence-based data analysis system and method provided by the invention, a set of novel data analysis system architecture is designed, and key technical components such as natural language processing, a large language model, database query optimization, multi-modal visualization and a semantic knowledge graph are fused; data transmission between modules is realized through unified intermediate data objects such as a structured query intention, an analytic tree and a structured query language template object, and asynchronous collaboration is realized through event driving and a message queue mechanism, so that a non-professional user can input an analysis request through a natural language; semantic analysis, structured query language query statement generation, data query and result visualization presentation are automatically completed, the method can be widely applied to data analysis scenes in the industries of government affairs, traffic, finance, education and the like, the data use efficiency is improved, the technical threshold is reduced, and the digital decision-making ability is enhanced.

The invention discloses an RAG performance improvement method based on process optimization. The method comprises the following steps: S1, preprocessing text data; s2, performing hierarchical text segmentation, obtaining a document, storing the text by utilizing a database after the document is segmented, converting the text into vectors, and storing the vectors into a vector knowledge base; s3, querying and rewriting to obtain a rewritten question; s4, combining parent document search with mixed retrieval, and retrieving related contexts for the rewritten questions; s5, performing rearrangement and denoising to obtain a final context; and S6, generating an answer, and sending the final context into a large language model for understanding questions and answers. According to the method, the links of text preprocessing, text partitioning, text embedding, query optimization, retrieval, retrieval post-processing, generation and the like are sequentially optimized according to the data flowing direction, the semantic retrieval precision is improved, and the performance of RAG in the aspects of text retrieval precision, answer generation quality, user interaction and the like can be remarkably improved; and the flexibility and the practicability of the system are enhanced, so that the system is more excellent in practical application.

The invention relates to the technical field of intelligent data processing and artificial intelligence in the electric power industry, and provides an electric power semantic large model adaptive data query and dynamic response method and system.The method comprises the steps that electric power industry natural language query and related images are obtained, and after feature extraction is conducted through a large language model and a deep learning model, the images are obtained; the knowledge graph and the visual question and answer graph neural network are utilized, features of different modes are unified into the same representation in a bidirectional fusion mode, fused features are obtained, after intelligent error correction and natural language query optimization are conducted through the hypergraph neural network and multiple rounds of interaction, an extended query statement is obtained through hypergraph reasoning, and a query result is obtained. Converting the extended query statement into a structured query statement through an autoregression model to obtain a query result; and performing dynamic permission adaptation, sensitive data protection and compliance and security check on a query result to realize dynamic response. And the query accuracy and efficiency of the power industry are greatly improved.

A system (100) for AI-assisted query optimization and self-optimizing data pipelines, comprising: a query analysis and classification module configured to monitor, log, and categorize incoming queries based on execution complexity, type, and frequency; an AI-based query execution optimizer that uses machine learning models to analyze historical query performance and recommend dynamically optimized execution plans; a self-optimizing data pipeline module configured to monitor and adapt ETL workflows by modifying transformation logic, workload balancing, and execution sequences; an adaptive indexing and storage management module configured to dynamically restructure indexing strategies, including predictive indexing and partitioning, to improve query performance; an intelligent caching and pre-fetching module configured to anticipate frequently used queries and pre-load relevant data to reduce redundant computations; a workload balancing and resource optimization module configured to dynamically distribute query loads among computing resources based on the constraints of the real-time system (100); an anomaly detection and performance monitoring module configured to continuously track query execution metrics, identify inefficiencies, and mitigate performance bottlenecks using AI-based predictive analytics; a reinforcement learning-based query optimization module configured to iteratively refine query execution strategies based on real-time feedback and evolving data properties; a multi-cloud and cross-platform optimization engine configured to ensure seamless operation across heterogeneous database management systems and cloud platforms.

The invention relates to the technical field of distributed agent collaboration and graph database query optimization, in particular to a graph query processing method and system based on multi-agent collaboration, and the method comprises the following steps: reasoning a coordination agent to analyze the semantics and intention of user query by using a large language model; a structured multi-agent execution plan is output, and an entity search agent or a relation analysis agent needing to be called is explicitly called; the agent coordinator receives the multi-agent execution plan and selects agents according to task types and task allocation; obtaining a distributed intermediate result set; starting a result aggregation mechanism, and performing typed processing, priority ranking and deduplication merging on the distributed intermediate result set to obtain an arrangement result; and outputting an answer for answering the initial question of the user. The system comprises an agent execution plan output module, a multi-task execution module and an arrangement result reasoning module. According to the method, efficient, extensible and fault-tolerant graph query processing is realized.

Latency, response times, and efficiency improvements for data querying are provided herein, particularly in the context of querying large database systems and data tables from disparate data sources. There are provided systems and methods for automatic query and data retrieval optimization through procedural generation of data tables from query patterns. A service provider may utilize different computing services for query processing and data retrieval for different applications and services used by internal and / or external users. Instead of querying large database systems and numerous data tables, pre-aggregated data tables may instead be used and searched by procedurally generating such tables based on precomputation rules and query patterns. Once patterns have been identified in queries, corresponding data may be aggregated from data sources in a pre-aggregated data table. Query optimization rules may then be used to have these data tables queried in place of their original sources.

The invention provides a household appliance knowledge question-answering method and system based on retrieval enhancement generation. The method comprises the following steps: acquiring household appliance field multi-modal data from a multi-format document library; extracting text information, table information and chart information in the multi-modal data; performing domain term injection processing on the extracted information, and constructing a packet domain enhancement index; receiving a natural language question input by a user; the natural language problem is analyzed through a query optimizer, and semantic retrieval and keyword retrieval are executed in parallel; carrying out fusion processing on the semantic retrieval result and the keyword retrieval result; selecting matched document fragments by adopting a relevancy sorting algorithm; inputting the matched document fragments into a large language model to generate candidate answers; verifying the compliance and traceability of the candidate answers through a credibility evaluation module; outputting a final answer with a reference source; and storing the high-frequency questions and the final answers into a cache library to solve the problems that the answer accuracy of a knowledge question-answering system is reduced and the response efficiency is limited.

A database query optimization method, comprising: acquiring a query statement, the query statement comprising a connection operation and a group aggregation operation; determining that a connection key and a group key in the query statement are the same, wherein an aggregation function in the query statement can be split, and an input parameter of the aggregation function comes from the same side as a connection condition in the query statement; creating a fusion operator, the fusion operator being used for completing the connection operation and the group aggregation operation; when a cost for executing the fusion operator is less than or equal to a target cost, on the basis of the fusion operator, generating an optimal execution plan related to the fusion operator, wherein the target cost is a cost for executing a connection operator and a group aggregation operator; and performing the operations step by step on the basis of an operation sequence in the optimal execution plan, so as to obtain a query result set. Therefore, when the query statement meets corresponding features, the connection operation and the group aggregation operation can be fused into one operator, thereby reducing the processing flow of an intermediate result, accelerating the execution of the query statement, and improving the query execution performance.

The database query optimization method comprises the steps that a query statement is obtained, and the query statement comprises a connection operation and a grouping aggregation operation; determining that a connection key and a grouping key in the query statement are the same, an aggregation function in the query statement is detachable, and input parameters of the aggregation function are from the same side of a connection condition in the query statement; creating a fusion operator, wherein the fusion operator is used for completing a connection operation and a grouping aggregation operation; under the condition that the cost of executing the fusion operator is smaller than or equal to a target cost, an optimal execution plan related to the fusion operator is generated according to the fusion operator, and the target cost is the cost of executing the connection operator and the grouping aggregation operator; and executing operations step by step according to the operation sequence in the optimal execution plan to obtain a query result set. Therefore, when the query statement meets the corresponding characteristics, the connection and grouping aggregation operations can be fused into one operator, so that the processing flow of an intermediate result can be reduced, the execution of the query statement is accelerated, and the execution performance of the query is improved.

The invention relates to the technical field of information management, in particular to a data query result sorting method and system fusing a large model and incremental learning. The method comprises the following steps: constructing a preliminary query model by utilizing a pre-trained large model, and generating a high-quality preliminary result; then inserting a low-correlation confrontation sample, increasing the diversity of results, and collecting user feedback; based on historical query behaviors, training a user query satisfaction model, and modeling satisfaction from multiple angles; satisfaction prediction is introduced into query optimization, strategies are dynamically adjusted for queries with low satisfaction, and incremental query results are generated and stored in a cache; fusing the preliminary result and the increment result, removing low-correlation items, and dynamically updating a query list; according to the method, the stability is kept while the data quality is improved, the dynamic optimization balance of the query result is realized, and the user experience is continuously improved.

Techniques and solutions are provided for improved query optimization, including for sub-portions of a query plan. A query is submitted to an inline query optimizer and at least one auxiliary query optimizer. The query is optimized by the inline query optimizer and the auxiliary query optimizer. A query processor can evaluate costs associated with query plans produced by the inline query optimizer and the auxiliary query optimizer and select a plan for execution that is most performant.

The invention discloses a retrieval enhancement generation method and device of an electric power system, a medium and equipment, and the method comprises the steps: constructing an electric power knowledge graph according to an original corpus of the electric power system, and constructing a structure index and a semantic index according to the electric power knowledge graph; performing query optimization on the original query content input by the user to obtain professional query content; performing neighbor sampling on the electric power knowledge graph according to the professional query content to obtain a query related sub-graph; according to the professional query content, the query related sub-graph, the structure index and the semantic index, performing mixed retrieval of semantics and maps to obtain a query retrieval result; and performing enhanced generation on the query retrieval result based on an attribution mechanism to obtain a user query answer. According to the technical scheme, the credibility of the generated result can be improved, meanwhile, the retrieval efficiency is improved, the query threshold is lowered, and the interpretability of the generated result is improved.

The invention provides an intelligent routing optimization method and system for multi-table association query, and relates to the technical field of routing optimization, and the method comprises the steps: extracting static features and dynamic features in a query request to construct a multi-dimensional feature vector; and calculating association degrees among the tables based on the static characteristics and grouping, and predicting a system resource state by using a time sequence prediction model. And inputting the grouping result and the resource state sequence into a two-channel neural network model, respectively processing by using a graph neural network and a recurrent neural network, and fusing features through an attention mechanism. And finally, the decision-making layer network outputs a query optimization scheme including a distributed execution plan, a data fragmentation strategy and a resource allocation scheme. In the execution process, the performance is monitored in real time, and the optimization scheme is adjusted according to the state information. In addition, an incremental training set is constructed by utilizing execution historical information, the model is updated online, the model adaptability is improved, and closed-loop optimization is formed, so that the query efficiency is remarkably improved, and the system resource consumption is reduced.

The invention discloses a dynamic business data updating and querying method and system, and particularly relates to the technical field of information processing, and the method comprises the following steps: S1, real-time data acquisition and preprocessing; s2, updating the dynamic data; s3, intelligent query optimization; s4, result feedback and visualization are carried out; and S5, performing anomaly detection and self-repairing. According to the method, the real-time processing capability and query efficiency of massive heterogeneous data are remarkably improved, the distributed sensor network is combined with lightweight encryption and a differential privacy mechanism to realize localized cleaning and feature extraction of multi-source data, the load of a central server is effectively reduced, and the risk of sensitive information leakage is avoided; an incremental synchronization mechanism depends on a message queue and Hash chain log technology to ensure the atomicity and traceability of data updating in a high-concurrency scene, and by combining a dynamically generated space-time multi-dimensional index and an adaptive weight distribution algorithm, the system can intelligently analyze query semantics and optimize path selection, and the response time of complex query is greatly shortened.

The invention belongs to the field of query optimization, and particularly relates to a fragmentation storage and query optimization method and system for a high-concurrency database, and the method comprises the steps: analyzing business features through a preset decision tree model, and selecting an optimal fragmentation key, constructing a fragmentation rule engine and initializing a database in combination with hash, range and list fragmentation rules and a fragmentation splitting-merging strategy; generating a query abstract syntax tree by using an ANTLR4 analyzer, and detecting whether a preset Cube is hit or not to directly return a result; if not, dynamically routing to a target fragment list based on a fragment key field, load balancing and a failover strategy, accelerating data retrieval in combination with a high-frequency index, and combining fragment-level results through a T-TopK algorithm; according to the method, the self-adaptive matching of the fragmentation strategy and the service requirement, the calculation push-down of the query process and the result optimization are realized, and the low delay and the high availability in a high-concurrency scene are ensured.

The present disclosure provides techniques and solutions for improved query optimization. A query plan is received, and at least a portion of the query plan is identified to be analyzed for logically equivalent query plans. A signature is generated for the at least a portion of the query plan. One or more query plans are identified that have signature that matches the signature of the at least a portion of the query plan, but where such query plans are logically equivalent, but not identical, to the at least a portion of the query plan. A query plan of the one or more query plans is substituted in the query plan for the at least a portion of the query plan.

An adaptive random access system and method with learned query optimization for compacted data files that enhances random access performance through machine learning and pattern recognition. The system incorporates a query pattern learning module that analyzes historical access patterns and user behavior to build statistical models of data usage. An adaptive estimator module improves location estimation accuracy by incorporating learned patterns rather than relying solely on mathematical calculations. A predictive boundary detector uses learned codeword patterns to more accurately identify boundaries in compacted data, reducing misalignment errors. An intelligent search engine coordinates optimization strategies including context-aware search string parsing and encoding strategy selection based on learned performance data. A dynamic codebook optimizer reorganizes sourceblock layout based on access frequencies and co-occurrence patterns to improve retrieval speed. An enhanced search cache implements predictive caching algorithms that anticipate user queries and proactively load relevant data.

A method for query optimization may include capturing a proposed database query input into a user interface. The method may further include providing the proposed database query to a machine-learning model. The machine-learning model may have been trained, using one or more gathered and / or simulated sets of query execution overhead data and one or more gathered and / or simulated sets of database queries, to determine a potential execution overhead of a database query and output a query execution score. The method may further include outputting, by the machine-learning model, the query execution score based on the proposed database query. The method may further include determining that the query execution score exceeds a query execution score threshold. The method may further include triggering a corrective action based on the query execution score exceeding the query execution score threshold.

The invention provides a query optimization method and device based on OceanBase database data fragmentation and related products, and the method comprises the following steps: determining one or more columns as service keys based on a plurality of service tables, the service keys being used for connecting service data in the service tables; based on the service key, setting one or more columns as fragmentation keys, and combining the fragmentation keys with the service key so as to construct a fragmentation table in the OceanBase database; performing fragmentation processing on target data based on the fragmentation table to create a data fragmentation structure; based on the data fragmentation structure, constructing an extended SQL query statement carrying preset prompt optimization information SQL Hint; on the basis of the extended SQL query statement, a query optimizer of the OceanBase database selects a preset execution plan to query the service table so as to obtain a target result set, and the query performance is improved in combination with the parallel processing capacity of the database.

The invention discloses a remote sensing target detection large model construction method based on context feature deep learning, and the method comprises the following steps: S1, dividing an original data set according to a strategy, forming a training set, a verification set and a test set, and guaranteeing that each subset has representativeness in sample number, scene diversity and target distribution; s2, constructing a remote sensing target detection large model by adopting a deep learning method based on context features, and fusing multi-level features and introducing a query optimization mechanism; and S3, performing model testing and performance evaluation, selecting optimal model parameters, performing comprehensive verification on the test set, and evaluating the comprehensive performance of the model in the aspects of detection precision, speed and robustness. According to the method, the pre-trained DINOv2 backbone network, the context feature enhancement module and the intensive supervision strategy are introduced, so that rich context information can be captured in a multi-scale and complex scene, and the detection precision is greatly improved.

The invention discloses a batch information retrieval and structured output method and system, and belongs to the technical field of big data analysis and information security, batch information retrieval and structured output are realized based on multi-source heterogeneous file analysis, and the method specifically comprises the steps of resource index construction and data processing, integrating a single resource and an integration type resource into a search engine through configuration by customizing configuration; field mapping information is stored in a configuration database, and whether the fields are mapping of batch query or not is achieved; dynamic analysis engine and element extraction: dynamically identifying the format type of an uploaded file through a file header feature code, and extracting key element information including an identity card number and a mobile phone number by adopting a streaming analysis and hybrid classification model; performing distributed batch retrieval and query optimization; and performing structured output and safety compliance control. According to the method, the fusion processing efficiency and security of multi-source data in scenes such as public security investigation and financial anti-fraud can be remarkably improved.

The invention discloses an HTAP database query optimization method and device and a storage medium. The query performance is remarkably improved by combining the advantages of row storage and column storage. Comprising the steps of analyzing SQL query to generate a logic query plan; evaluating the execution cost of each logic operator in different storage modes by using a mixed cost model, wherein hyper-parameters of the mixed cost model are optimized through Bayesian search; selectively adopting a dynamic programming algorithm or a greedy algorithm to generate a hybrid physical query plan with the minimum execution cost according to the table number; and executing the plan to obtain a query result. In addition, an MCTS algorithm is further utilized, the optimal column subset is selected in the preset memory budget limit based on the query historical record and loaded into the memory column storage, and the subsequent query performance is further optimized. The problem that row storage and column storage in an existing HTAP system cannot work cooperatively is solved, operator-level mixed execution is achieved, and meanwhile memory resource utilization is optimized.

The invention relates to the technical field of database optimization, and discloses a multi-table connection sequence query optimization method, device and equipment, and the method comprises the steps: carrying out the feature extraction of a received database query statement, obtaining a query feature representation, and enabling the database query statement to comprise the data query of a plurality of tables; inputting the query feature representation into a pre-trained target network model, and predicting execution time of different connection sequences in a database query statement to obtain an execution time degree; and on the basis of a Monte Carlo tree, taking the query feature representation and the execution time degree as reward functions, and optimizing the connection sequence of the database query statements to obtain a target connection sequence. According to the method, feature extraction, model prediction and an optimization process based on the Monte Carlo tree are integrated, the execution time and complexity are comprehensively considered, the optimal connection sequence is selected, the limitation that a traditional method is only based on a cost model is overcome, and more accurate connection sequence selection is achieved.

The invention provides an intelligent tour guide man-machine conversation method and system based on knowledge graph retrieval, and relates to the technical field of artificial intelligence and natural language processing. The method comprises the following steps: collecting relevant data of scenic spots in a preset range, carrying out structured processing on all the relevant data of the scenic spots, and constructing a knowledge graph in the tour guide field; establishing indexes for entities and relationships in the knowledge graph, and introducing an adaptive query optimization mechanism; selecting a pre-trained language model and performing LoRA fine tuning on the pre-trained language model to obtain a fine tuning language model; a user initiates a dialogue request; the dialogue request accesses the fine tuning language model to obtain a preliminary answer result, and accesses the knowledge graph to obtain knowledge graph information; and matching and fusing the preliminary answer result and the knowledge graph information, and outputting a corrected answer result. According to the invention, knowledge can be effectively fused between the tour guide explanation field and the general field, and the accuracy and comprehensiveness of answers are improved.

The present disclosure involves systems, software, and computer implemented methods for managing database query plans with dynamic conditions. One example method includes receiving a request to process a query based on at least one dynamic condition. A serialized abstract query plan is identified, deserialized, and converted into a query optimizer tree. A condition provider procedure is invoked to generate a first filter string for each view in the query optimizer tree. Comparisons are made to determine matches between first filter strings and corresponding second filter string stored in the query optimizer tree. In response to determining that a first filter string does not match the corresponding second filter string, query plan compilation is triggered. In response to determining that each filter string matches each corresponding second filter string, a query plan is generated based on the converted query optimizer tree and the query is executed using the query plan.