234 results about "Query optimization" patented technology

According to the artificial intelligence-based data analysis system and method provided by the invention, a set of novel data analysis system architecture is designed, and key technical components such as natural language processing, a large language model, database query optimization, multi-modal visualization and a semantic knowledge graph are fused; data transmission between modules is realized through unified intermediate data objects such as a structured query intention, an analytic tree and a structured query language template object, and asynchronous collaboration is realized through event driving and a message queue mechanism, so that a non-professional user can input an analysis request through a natural language; semantic analysis, structured query language query statement generation, data query and result visualization presentation are automatically completed, the method can be widely applied to data analysis scenes in the industries of government affairs, traffic, finance, education and the like, the data use efficiency is improved, the technical threshold is reduced, and the digital decision-making ability is enhanced.

The invention relates to the technical field of distributed agent collaboration and graph database query optimization, in particular to a graph query processing method and system based on multi-agent collaboration, and the method comprises the following steps: reasoning a coordination agent to analyze the semantics and intention of user query by using a large language model; a structured multi-agent execution plan is output, and an entity search agent or a relation analysis agent needing to be called is explicitly called; the agent coordinator receives the multi-agent execution plan and selects agents according to task types and task allocation; obtaining a distributed intermediate result set; starting a result aggregation mechanism, and performing typed processing, priority ranking and deduplication merging on the distributed intermediate result set to obtain an arrangement result; and outputting an answer for answering the initial question of the user. The system comprises an agent execution plan output module, a multi-task execution module and an arrangement result reasoning module. According to the method, efficient, extensible and fault-tolerant graph query processing is realized.

The invention provides a household appliance knowledge question-answering method and system based on retrieval enhancement generation. The method comprises the following steps: acquiring household appliance field multi-modal data from a multi-format document library; extracting text information, table information and chart information in the multi-modal data; performing domain term injection processing on the extracted information, and constructing a packet domain enhancement index; receiving a natural language question input by a user; the natural language problem is analyzed through a query optimizer, and semantic retrieval and keyword retrieval are executed in parallel; carrying out fusion processing on the semantic retrieval result and the keyword retrieval result; selecting matched document fragments by adopting a relevancy sorting algorithm; inputting the matched document fragments into a large language model to generate candidate answers; verifying the compliance and traceability of the candidate answers through a credibility evaluation module; outputting a final answer with a reference source; and storing the high-frequency questions and the final answers into a cache library to solve the problems that the answer accuracy of a knowledge question-answering system is reduced and the response efficiency is limited.

An adaptive random access system and method with learned query optimization for compacted data files that enhances random access performance through machine learning and pattern recognition. The system incorporates a query pattern learning module that analyzes historical access patterns and user behavior to build statistical models of data usage. An adaptive estimator module improves location estimation accuracy by incorporating learned patterns rather than relying solely on mathematical calculations. A predictive boundary detector uses learned codeword patterns to more accurately identify boundaries in compacted data, reducing misalignment errors. An intelligent search engine coordinates optimization strategies including context-aware search string parsing and encoding strategy selection based on learned performance data. A dynamic codebook optimizer reorganizes sourceblock layout based on access frequencies and co-occurrence patterns to improve retrieval speed. An enhanced search cache implements predictive caching algorithms that anticipate user queries and proactively load relevant data.

The invention provides a medical data query optimization method and system based on semantic understanding, and relates to the technical field of medical data query, and the method comprises the steps: firstly obtaining a user medical data query request, a historical query record set and a current medical scene description; performing semantic intention scene deconstruction and historical association mapping on the query request to generate a deconstruction result; constructing a semantic path dynamic adaptation model based on the deconstruction result, and generating a model parameter set; calling the initial query path, performing path reconstruction according to the model output adaptation relation, and generating an optimized query path; and executing query operation to obtain an initial result, collecting user feedback for semantic calibration, generating an optimization result, feeding back the optimization result to the user terminal, and associating related information to a historical record set. According to the invention, the accuracy and efficiency of medical data query are improved.

A hypergraph workload manager in a server is configured for failure tolerant and explainable state machine driven hypergraph execution. The hypergraph executor comprises a query optimizer, a hypergraph enlister, a pipeline analyzer, and a state machine generator. The query optimizer translates a user query into a query operator graph. The hypergraph enlister enlists the query operator graph into a hypergraph containing a set of query operator graphs representative of already submitted user queries. The enlistment is configured to join query operator graphs where it makes sense to optimize query executions. Updates to the hypergraph based on the enlistment results in a set of disconnected graphs. The pipeline analyzer performs an analysis of all operators of all queries in the hypergraph to find an optimal sequencing of execution. The state machine generator is configured to generate a hierarchical state machine for all operators of a disconnected graph of the hypergraph.

The invention relates to the technical field of big data storage, and discloses a multi-level data storage and mixed query method and system, a storage medium and equipment, and the method comprises the steps: collecting original buried point data, carrying out real-time stream processing, storing the processed real-time data in a hot storage layer, carrying out timing offline batch processing on historical buried point data, and storing the processed real-time data in a hot storage layer; converting into an optimized storage format and storing in an object storage system; an extended metadata management system is constructed, and structure information, partition information and access frequency information of data in the object storage system are automatically extracted and managed; executing query, and reducing the data scanning amount through a query optimization strategy; automatically migrating data among different storage layers through a dynamic cold and hot data layering mechanism; a unified query interface is provided, a query request is distributed to a corresponding real-time processing system or an offline query system based on an intelligent routing strategy, and a query result is cached and returned, so that high-performance and low-cost storage and second-level query of data are realized.

The invention provides a visual form construction method, which comprises the following steps: receiving user form operation data, carrying out business logic calculation, state change control and inter-service coordination operation on the user form operation data based on domain-driven design, and generating domain events, state data and service governance data. Domain events and state data are recorded to an event storage system, a query optimization view is constructed, event logs and query view data are generated, and dual optimization of data persistence and query efficiency is ensured. According to a preset security policy rule, real-time security monitoring is performed on the service governance data, a system health state index and security audit data are generated, and a system-level observability and security protection mechanism is established. And performing interface rendering and synthesis based on the event log, the query view data, the system health state index and the security audit data to generate a user interactive form interface. Through fusion processing of multi-source data, deep integration of interface rendering and security policies is realized.

The invention discloses a news detection method and system based on a dual-model collaborative optimization framework, and a medium. The method comprises the following steps: screening news materials and news descriptions from a news material library to construct a training data set; respectively inputting the training data set into a student model and a teacher network model to obtain a first image feature and a second image feature, and aligning the first image feature to a space of the second image feature to obtain a third image feature, inputting the third image feature into a decoder of a teacher network model to perform target query optimization so as to obtain a fourth image feature, taking a prediction score obtained by the teacher network model as a soft label, supervising training of a student model until a preset condition is met, and determining a trained target detection model; and inputting the news material obtained in real time into the target detection model, and determining the news category and the sensitive element, so that the news detection accuracy and efficiency can be improved.

The invention discloses a natural resource big data off-line comprehensive analysis, annotation and proof system and method, and belongs to the technical field of natural resource informatization and geographic information processing. The system comprises the following components: a spatial data fusion module, which is used for realizing unified management and efficient processing of heterogeneous spatial data; the spatial index optimization module is used for realizing quick retrieval and high-performance query optimization of spatial data; the intelligent annotation and arrangement module is used for realizing low-code rapid configuration and business process automatic arrangement of different natural resource investigation scenes; the off-line analysis module is used for realizing buffer area analysis, space superposition and terrain analysis in an off-line environment; the real-time visualization module is used for realizing multi-level map display, track recording, vector element editing and multimedia evidence collection; and the data quality management module is used for realizing automatic quality inspection of geometry, topology and attributes and statistical analysis, version control and export management of annotation results.

The invention provides a query optimization method, electronic equipment, a storage medium and a program product. The method includes. Based on the left deep tree, the optimization problem of the table connection sequence in the database is converted into a secondary unconstrained binary optimization QUBO problem; the relationship in the left deep tree is a table in the database; solving a QUBO problem through a quantum solving method to obtain a connection relation of the tables in the left deep tree; and determining an optimized table connection sequence based on the connection relationship of the tables in the left deep tree.

The invention discloses a column storage-based big data compression and query optimization method, which relates to the technical field of big data compression and query optimization, and comprises the following steps of: performing sliding window division on original data of column storage to obtain a plurality of data windows; calculating a feature vector based on the value distribution characteristics in the multi-data window, wherein the feature vector is used for representing a local statistical rule of the data; performing dynamic merging according to the similarity of the feature vectors of the adjacent data windows to form a compressed block; selecting a corresponding compression coding mode based on the data type and the distribution characteristic of the compression block, and generating a metadata index containing a minimum value, a maximum value, a coding type and a compression size; after receiving a query request, mapping a predicate condition to a compressed coding space, and executing non-decompressed filtering judgment on a compressed block by utilizing the metadata index and a compressed coding type; and determining a decompression range based on a filtering result of the compressed blocks, and only performing local decompression and data reduction on the possibly hit compressed blocks.

The invention discloses a semantic-driven intelligent query method, system and device based on a security event library and a medium, and the method comprises the steps: improving the query accuracy through deep semantic understanding and cross-modal correlation analysis; potential threat association difficult to recognize can be found through multi-modal semantic association and threat graph construction; the query habit and professional domain knowledge of the user can be automatically learned in combination with a query optimization mechanism driven by reinforcement learning, personalized customization of a query strategy is realized, the manual adjustment frequency is reduced, and the man-machine cooperation efficiency is improved; on the whole, based on a dynamic optimization mechanism of a plug-in architecture and a large vertical model, the system can quickly adapt to changes of a network security environment of a power system, the response speed to novel threats is increased, and the adaptability of the system is enhanced.

Techniques and solutions are provided for implementing query constraints. A keyword in a query language is provided that indicates the presence of a constraint. During query execution, if the constraint is not satisfied, the query can be terminated / caused to fail. In some cases, the keyword is introduced into a query by a query optimizer. In a specific example, the keyword is introduced as part of optimizing a query where at least some query operations are performed using a federated database system. The keyword indicating the constraint can be included in a query language statement and sent to the federated database system for execution. If the constraint is not satisfied, the federated database system can send a failure notification to the main database system.

In some examples, a seed query is received. A set of candidate queries is retrieved from a database based on matching the seed query with each candidate query. Based on the seed query, a ranking model is used to assign a relevance score to each candidate query. The relevance score indicates suitability of the candidate query as a follow-up query to the seed query (“next query” suitability). At least one candidate query is outputted based on the relevance scores. For example, candidate queries may be ordered or filtered based on their relevancy scores. In some implementations, an action is instigated automatically or semi-automatically based on a model-generated response to a candidate query. In some implementations, generative model calls are instigated on the seed query and a next query selected from the candidate next queries.

The invention provides a self-adaptive cardinality estimation method, system and device and a storage medium, and belongs to the field of database query optimization, and the method comprises the steps: constructing original connection sequence data and an optimal table sequence of a table based on a disclosed benchmark test data set as a data set, and converting the data in the data set into an original feature vector; the original feature vectors are input into a plurality of Transform models with different hyper-parameter settings for model training, a linear transformation module is included in front of each Transform model, and a plurality of cardinal number prediction results are obtained through training; and selecting a model with a minimum error as an optimal model according to a cardinality prediction result to obtain all original query feature vectors and corresponding optimal models, and learning a mapping relationship between the query features and the optimal models by utilizing a classifier. Obtaining a new SQL query input classifier, and distributing an optimal model based on the index data set; and processing the feature vector by using the optimal model to obtain a cardinality estimation result. The defect of single model query is changed, and the accuracy of a query result is improved based on feature classification query.

The invention relates to the technical field of distributed database query optimization, in particular to a distributed database query dynamic optimization method and device and electronic equipment.The distributed database query dynamic optimization method includes the steps that a multi-dimensional feature unified coding framework is constructed, an SQL logic structure, a fragment topology state and a historical load rule are fused into a three-dimensional tensor, and the three-dimensional tensor is obtained; and a dynamic routing decision mechanism is formed by combining the multi-scale feature extraction capability of the convolutional neural network. According to the scheme, the problem of dynamic response missing caused by feature splitting of a traditional method is effectively solved, the execution efficiency of complex query is remarkably improved by quantifying a fragment weight matrix to optimize a node selection strategy, the utilization rate of calculation and network resources is improved, and the adaptability to a long-period load rule is enhanced; and full-link optimization from feature coding to intelligent decision making is realized.

The invention relates to a distributed cognitive map construction and dynamic updating system and method. The distributed cognitive map construction and dynamic updating system comprises a map construction module, an incremental updating module, a distributed consistency module and a query optimization module. The graph construction module is used for converting multi-source and multi-mode heterogeneous fashion data into a unified and structured knowledge graph; the incremental updating module is used for realizing dynamic maintenance and real-time performance of the knowledge graph; the distributed consistency module is used for ensuring high consistency, high availability and change traceability of knowledge graph data among a plurality of nodes; and the query optimization module is used for providing efficient and flexible knowledge graph query capability and carrying out rapid retrieval and complex analysis on fashion knowledge. By means of the design, high-availability, high-consistency and semantic-rich fashion cognitive map construction, maintenance and updating are achieved.

The invention discloses a quantum method and device for processing database multi-query optimization and a storage medium. The quantum method comprises the steps that database multi-query optimization problem expression is converted into a secondary unconstrained binary optimization problem; running a quantum circuit on a quantum computer, and calculating an energy value corresponding to Hamiltonian according to a test state prepared in a quantum processor; summing the energy values on a classical computer, and optimizing parameters of the quantum circuit through a classical optimizer according to a summing result; and iteratively executing the steps of quantum circuit operation, energy calculation and parameter optimization until an energy value converges to a stable value, obtaining an optimal solution of a multi-query optimization problem, and solving the quadratic unconstrained binary optimization model in a quantum-classical hybrid framework by the variable component sub-feature solver algorithm. According to the method, the quantum computer is introduced into the database optimizer, the multi-query optimization problem is solved by using a variable component subscript solicitation solver algorithm, and a problem modeling strategy is improved.

The invention relates to the technical field of data processing, and discloses a data online modeling and data management method for dimension modeling. The method comprises the following steps: performing dimension characteristic baseline acquisition on a multi-source heterogeneous data stream to generate a dimension attribute characteristic set; performing dimension hierarchical modeling based on the feature set, and outputting a result containing an atomic layer, an aggregation layer and an application layer; calculating a dimension correlation degree according to the topological relation of the service entities, and generating a correlation constraint parameter; detecting dynamic modeling conflicts based on the parameters, and outputting a conflict dimension identification set; distributing a query direction optimization weight according to the identifier set to obtain a dimension query optimization strategy; incremental data loading is controlled based on a strategy, and an incremental loading sequence divided according to time windows is generated; performing closed-loop consistency verification on the sequence, and outputting a verification result set containing data integrity and association consistency indexes; recording a modeling state snapshot according to the result set, and generating a snapshot with a version mark; and analyzing the abnormal backtracking path based on the snapshot, and outputting a dimension data recovery path instruction.

The application relates to the technical field of metadata query optimization, and discloses a metadata storage query efficiency monitoring optimization method, which comprises the following steps: step one, obtaining historical access data of metadata, matching the metadata with a preset data feature model according to the historical access data, and obtaining a feature model corresponding to the metadata; judging the cold and hot degree of the metadata according to the feature model and data type information of the metadata; step two, setting a corresponding storage strategy for each metadata according to the cold and hot degree; step three, in the metadata query process, the response data and performance indexes are monitored in real time, the real-time monitoring result is compared with a monitoring threshold corresponding to the cold and hot degree of the metadata, and directional optimization is carried out according to the comparison result. According to the preset data feature model, the access trend of the metadata is judged, and a relatively accurate reference basis can be provided for the cold and hot division of the metadata.

The application discloses a kind of blockchain data traceability query optimization methods, by introducing the method of cache optimization, utilize cache to reduce the number of disk IO in traceability, improve the efficiency of traceability search, while multi-level cache structure is designed, the problem that cache hit rate is not high under the condition that the memory resource of full node is limited is solved, that is, while giving consideration to the consumption of memory resource in the improvement of cache hit rate, reduce the burden of full node.A kind of blockchain data traceability query optimization method first by the node in network to full node of blockchain initiates traceability query request, then in full node query cache, finally carries out consistency check to full node, realizes blockchain data traceability query.The application improves the traceability query efficiency of blockchain system, query credibility, with certain practicality.

The invention discloses a standard SQL statement automatic generation and execution acceleration method and system, and the method comprises the following steps: carrying out the multi-dimensional association definition in response to an operation instruction based on a plurality of data sets presented by a front-end page of query equipment, and forming a corresponding association sub-query statement; the associated sub-query statement is an embedded part for realizing a query requirement in a standard SQL statement; and when query equipment executes the associated sub-query statements, executing sub-query optimization by adopting a standard sub-query mode, an existence detection sub-query mode and a set comparison sub-query mode based on sub-query result types, generating an optimal SQL query statement, and executing the optimal SQL query statement. According to the method, data association is performed on the front-end page according to the data range selected by the user to generate the query statement, and automatic performance optimization is performed on the query statement, so that the data query efficiency is improved. The problems of high threshold and low efficiency of traditional SQL development are solved, and meanwhile intelligent improvement of query performance is achieved.

The application belongs to the field of connection cardinality estimation, and provides a method and system for adaptive multi-table connection cardinality estimation. The method for adaptive multi-table connection cardinality estimation comprises receiving an input query statement and analyzing the contained relationship and the connection relationship of multi-table connection of the query statement; the analysis result of the query statement is subjected to feature vectorization, and then the cardinality of the query statement is predicted based on a cardinality estimation model; the query statement is converted into a database query prompt, a query optimizer is operated through the prompt, the query statement is executed, and a real cardinality value is obtained; the feature vector corresponding to the analysis result of the query statement and the real cardinality value constitute a training sample, and the cardinality estimation model is retrained to learn the correlation of multi-table joint query, which is used for estimation of the next connection cardinality.

The invention provides a retrieval enhancement generation method and device, electronic equipment and a storage medium, and relates to the technical field of data processing.The retrieval enhancement generation method comprises the steps that an original document is analyzed to generate a document object containing a title, a text and metadata, a problem type topic is generated based on the text, and a key entity is extracted; titles, key entities and problem-type topics are combined to generate embedded vectors for associated storage, intention recognition and query optimization are performed during user query, and responses are dynamically generated through similarity retrieval in combination with historical sessions, so that the problems that in the prior art, document semantic processing is rough, only simple segmentation vectorization is performed, and semantic focusing representation is lacked can be solved; the problems of low retrieval precision and context association fracture caused by key entity and context decoupling, session memory mechanism deficiency and retrieval strategy solidification are solved.

The invention discloses a data query optimization method and device, an equipment storage medium and a product, and is applied to the technical field of data processing. Index definition configuration information is acquired, data associated with at least one index is extracted from a target data set, processing is performed according to the index definition configuration information, and a multi-dimensional index result set is generated; obtaining a plurality of historical query statements executed on the multi-dimensional index result set; the method comprises the following steps: performing query feature analysis on a historical query statement to determine initial field information for creating an index; determining index field information based on the initial field information and a preset index auditing strategy; on the basis of the index field information, one or more indexes are created for the multi-dimensional index result set, and therefore the data query efficiency can be improved.

The invention provides a Text2SQL multi-table query optimization method and device based on a large model, equipment and a medium, and relates to the technical field of databases, the method comprises the following steps: receiving a to-be-processed original data query request, and rewriting the original data query request to obtain a standard data query request; determining a target matching result corresponding to the standard data query request based on the standard data query request and the field semantic information associated with the database table through a semantic matching large model; based on a minimum necessary table principle and the target matching result, determining a query source table corresponding to the standard data query request; and constructing a target executable SQL statement corresponding to the standard data query request according to the standard data query request and the query source table corresponding to the standard data query request. According to the method, the accuracy of data matching can be remarkably improved, and the complexity of SQL query is reduced.

The invention discloses an intelligent data service system based on a dynamic data source and an implementation method thereof. The system comprises a dynamic data source management module, an intelligent query engine, a filter chain processing system, a cache management system and an access point controller. The dynamic data source management module is responsible for dynamic loading, monitoring and management of data sources and supports dynamic switching and load balancing of the data sources. The intelligent query engine provides a unified query interface and supports query optimization of various data source types. The filter chain processing system realizes data processing and expansion before and after query, and supports dynamic script loading. The cache management system provides intelligent cache and performance optimization and supports dynamic generation of cache keys and flexible configuration of cache strategies, and the access point controller provides RESTfulAPI interface service and supports REST request processing of a client. According to the invention, dynamic management, intelligent routing and high-performance data query service of multiple data sources are realized.

The invention provides a direction grouping index-based mask rule check rapid query method and device, a medium, a program product and a terminal, and the method comprises the steps: obtaining mask pattern data after optical proximity correction, and carrying out the feature extraction of the mask pattern data, so as to obtain a feature line segment set; performing direction grouping processing on all line segments in the feature line segment set to obtain line segment subsets in different directions; adopting a bucket dividing strategy to construct a corresponding bucket dividing index structure for each line segment subset; and executing fast parallel line segment query of mask rule check based on the bucket index structure constructed by each line segment subset. By means of the direction grouping and bucket indexing technology, the indexing method for parallel line segment query optimization is provided, the efficiency of parallel line segment rule verification in MRC checking is remarkably improved, the use of a complex R-Tree structure can be avoided, memory occupation is reduced, and the indexing method is suitable for large-scale mask data of an advanced process.

The application discloses a news detection method and system based on a double-model collaborative optimization framework, and a medium, which comprises the following steps: screening news materials and news descriptions from a news material library to construct a training data set; inputting the training data set into a student model and a teacher network model respectively to obtain first image features and second image features, aligning the first image features to the space of the second image features to obtain third image features, inputting the third image features into a decoder of the teacher network model to perform target query optimization to obtain fourth image features, and taking the prediction score obtained by the teacher network model as a soft label to supervise the training of the student model until a preset condition is met to determine a trained target detection model; and inputting real-time acquired news materials into the target detection model to determine a news category and sensitive elements, so that the application can improve the accuracy and efficiency of news detection.