15 results about "Materialized view" patented technology

The multi-tenant system includes one or more hardware processors that obtain a query that includes statements defining one or more operations to be performed on a subset of constituent records stored in a source database. The hardware processors parse the query to decipher a type of the one or more operations, the subset of the constituent records referred to, and one or more other conditions of the operations. The hardware processors obtain the subset of the constituent records from the parsed query from the source database. The hardware processors execute the one or more operations, to generate a materialized view that denormalizes the data. The hardware processors detect any changes in the subset of the constituent records, synchronize the materialized view with the any changes, store the synchronized materialized view within a data platform, and transmit the synchronized materialized view to a target database.

The application discloses a kind of multidimensional data management methods based on data analysis, it is related to data storage management technical field, including obtaining data record to be written, and the dimension level mark corresponding to the multidimensional data storage structure of the data record to be written is parsed;Based on the dimension level mark and the aggregation level corresponding to each materialized view in the preset materialized view set, determine the level offset of the data record to be written relative to each materialized view;According to the level offset, determine the associated update times of at least one target materialized view in the materialized view set triggered in the writing process for the data record to be written;The application is by parsing the level offset between the dimension level mark of data to be written and the aggregation level of materialized view, the associated update cost caused by writing operation is quantified before writing and is judged, guarantee the query response stability of target materialized view.

This invention relates to the field of database technology and discloses a dynamic data management method oriented towards business complexity. This method parses business logs to construct a real-time business interaction topology graph, calculates the structural entanglement degree representing logical coupling and the access discrete entropy representing the degree of hotspot dispersion, and generates a business state vector. It uses a time series model to predict future states, employs a recursive cost function considering lock blocking diffusion and resource contention effects to calculate execution costs, and generates policy instructions when the net utility meets a lag threshold. Responding to the instructions, it executes dynamic materialized views or transient vertical splitting operations including hot / cold separation, and performs nonlinear adaptive synchronous flow control based on real-time load. This invention quantifies business complexity through graph theory features and combines a predictive decision-making mechanism to achieve proactive data architecture optimization, effectively solving the performance degradation problem in highly coupled and hotspot scenarios.

The application relates to a blockchain cryptocurrency fund flow searching method and a control system. The method comprises the following steps: acquiring transaction detail data information, preprocessing the transaction detail data information, and generating a materialized view; presetting a search condition of a query address, and acquiring the query address from the materialized view according to the search condition; traversing the query address, obtaining a first result graph, saving a target address in the first result graph, and searching from the target address in sequence through breadth-first search and depth-first search to obtain a target path list. The breadth-first search and the depth-first search are combined, the searching efficiency is effectively improved, the requirement of fast searching a feature address is met, the data searching speed and precision can reach an optimal level, and the calculation pressure of a server is reduced.

Embodiments described herein are directed to generating and returning materialized views for queries (or subexpressions thereof) having a particular relationship with each other. For instance, machine learning-based techniques may be utilized to identify query subexpressions that have at least one of a semantically equivalent relationship or a containment relationship with each other. Responsive to identifying such relationship(s), a materialized view may be generated for the identified subexpressions. When a query is subsequently received, machine learning-based techniques may be utilized to determine whether a subexpression of the query possesses at least one of a semantically equivalent relationship or a containment relationship with another subexpression for which a materialized view has been generated. Responsive to determining that such a subexpression of the query possesses one or more of such relationships, the materialized view generated for the other subexpression is returned.

This specification discloses a method for querying graph data. The method identifies a common query across various query loads, performs a query using this common query in a graph database instance, and stores the results as a materialized view of the graph database instance. When a query statement to be executed is received, if the query statement contains the common query, the query result obtained by executing the query using the query statement in the graph database instance is determined based on the stored materialized view. Through this method, when querying using a query statement to be executed, at least a portion of the query results can be determined based on the pre-stored materialized view, without needing to fully execute the query process corresponding to the query statement on the graph database instance, thus effectively improving query speed.

This application relates to a method, apparatus, electronic device, and storage medium for generating materialized views. The method involves: filtering target query patterns from database query logs; generating candidate materialized view creation statements corresponding to the target query patterns; creating candidate materialized views based on these statements in a resource-isolated sandbox environment and quantitatively evaluating the candidate materialized views; deciding whether to approve and deploy the candidate materialized views; and conducting a health assessment of deployed materialized views, performing corresponding optimization operations based on the assessment results. This achieves automated and precise selection of high-value optimization targets, reducing reliance on senior experts. Quantitative evaluation of candidate materialized views ensures that deployed materialized views provide clear benefits. Furthermore, multi-dimensional health assessments of deployed materialized views automatically identify and eliminate inefficient materialized views, enabling continuous and automated optimization and recycling of computing and storage resources.

The application provides a lightweight real-time data synchronization and processing method and system for a virtual wafer factory, the method comprising: performing full-amount extraction and synchronization of historical data of a source-end database through a DataX tool; performing real-time monitoring and capturing of incremental change data of the source-end database through a CDC tool, and sending the incremental change data to a Kafka message queue in the form of a message; consuming the incremental change data from the Kafka message queue through a NiFi data flow tool; writing data into a TimeScaleDB time series database; using an aggregation engine built in the TimeScaleDB to perform continuous aggregation operation on the written data, generate and maintain a materialized view, and the materialized view is used for direct query and calling by a business system, so as to realize storage-computing fusion of data storage and aggregation calculation. The application can solve the problems of long existing big data ETL architecture tool chain, large resource consumption and high operation and maintenance cost, and realize lightweight operation of data cleaning and aggregation calculation.

The application discloses a data query method and related products, the method comprises the following steps: obtaining index metadata through an index mode; the index mode comprises a visual configuration index mode or a formula definition index mode; based on the association table information in the index metadata, an associated wide table is constructed; based on the associated wide table, the upstream table data is updated synchronously to obtain associated data; based on the index metadata and the associated data, a materialized view is constructed; the materialized view is constructed based on the index metadata, the associated data and a pre-computed result; the pre-computed result is obtained by pre-computing based on the index metadata and the associated data; a query request is matched with the materialized view, and a pre-computed result matched with the query request is determined as a final result. The application realizes an end-to-end closed loop from index definition, data preparation to accelerated query, effectively solves the problems of poor flexibility, low performance, high maintenance cost and the like in a traditional scheme.

The application discloses a kind of multidimensional data management methods based on data analysis, it is related to data storage management technical field, including obtaining data record to be written, and the dimension level mark corresponding to the multidimensional data storage structure of the data record to be written is parsed;Based on the dimension level mark and the aggregation level corresponding to each materialized view in the preset materialized view set, determine the level offset of the data record to be written relative to each materialized view;According to the level offset, determine the associated update times of at least one target materialized view in the materialized view set triggered in the writing process for the data record to be written;The application is by parsing the level offset between the dimension level mark of data to be written and the aggregation level of materialized view, the associated update cost caused by writing operation is quantified before writing and is judged, guarantee the query response stability of target materialized view.

The present application relates to the technical field of data warehouse index calculation, and proposes an index calculation method and device based on a hierarchical index system, comprising: acquiring external data sources collected to an ODS layer, connecting fact tables and dimension tables into wide tables and saving to a DWD layer; based on the DWD layer, constructing atomic indexes, derivative indexes and composite indexes, integrating to form index aggregation; performing index aggregation through a data acquisition task, loading the calculation result to a target table, constructing an index API to return a query result; estimating the performance quantitative value of the index aggregation, when exceeding an optimization decision threshold, tracing back to the upstream to obtain dependent indexes of each level; based on the dependent indexes, constructing corresponding materialized views, and using database transparent rewriting technology to redirect the query request to the materialized view. The present application improves the execution efficiency and system response performance of index calculation, and realizes intelligent, automatic and high-performance calculation processing of a large-scale index system.

Materialized views may be optimally selected for materialized view creation or refresh. Materialized views for creation or refresh may be identified. A subset of the materialized may be determined according to an optimization technique that selects the subset of the materialized views according to a maximized performance benefit for maintaining the subset of materialized views based on a workload of the database system caused by a set of previously received queries. Individual ones of the subset of materialized views to create or refresh may be selected and performed according to a performance benefit-based order of the subset of the materialized views.

The application discloses a kind of physical view incremental refreshing method, device, equipment, medium and product, it is related to database technical field.The method comprises: dividing base table, obtains at least two sub-tables of shard;Log temporary table and log main table of physical view are created for the sub-table of shard;In data shard, shard local physical view is created, and incremental refreshing control information is registered;According to incremental refreshing point, the log temporary table and the log main table determine incremental change set;According to the base table involved in the shard local physical view and the incremental change set, the shard local physical view is refreshed, obtains the new shard local physical view or intermediate result set of the sub-table of shard;At least two new shard local physical views or intermediate result sets of sub-table of shard are merged, and the target physical view of the base table is obtained.

This invention relates to the field of internet advertising technology and provides an intelligent decision-making system for advertising placement based on real-time stream computing. The system includes: a heterogeneous stream computing engine module; a cold-start state space prediction module; a stream complex event detection module; and a real-time online analysis and storage cluster module. This module employs a large-scale parallel processing architecture and a primary key update model to achieve second-level writing of business metrics and prediction results and accelerated materialized view queries. By constructing a full-link real-time stream closed-loop architecture, it achieves millisecond-level response from user behavior perception to automatic execution of placement decisions. By introducing a dynamic prediction model and a complex event detection mechanism, the system effectively solves the problems of decision blind spots and nonlinear risk identification in the early stages of placement, significantly enhancing the predictability and accuracy of placement strategies.