16 results about "Online analytical processing" patented technology

The invention provides predicate evaluation based on CPU utilization of a shared storage device. A hybrid transactional / analytic processing (HTAP) database includes an online transactional processing (OLTP) engine, an online analytic processing (OLAP) engine provided separately from the OLTP engine, and a shared storage that holds data for a plurality of tenants. The shared storage device is configured to receive a request from the OLAP engine for predicate evaluation for data stored on the shared storage device. The shared storage device is further configured to determine that the CPU utilization of the shared storage device exceeds a predicate rejection threshold. Upon determining that the CPU utilization of the shared storage device exceeds a predicate rejection threshold, the shared storage device is further configured to inhibit evaluation of at least a portion of the predicate.

A hybrid transactional / analytical processing (HTAP) database is provided, which includes an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage holding data. The shared storage includes an OLTP storage holding data in row format for responding to queries of the OLTP engine, and an OLAP storage holding data in column format for responding to queries of the OLAP engine. The OLAP storage includes a delta store configured to record changes made to the OLTP storage in an update, and a base store configured to implement the update from the delta store on data in persistent storage in a flush. The HTAP database further includes a table metadata service configured to generate and perform operations on table metadata of the data that is updated in the flush.

The invention relates to the field of big data processing, in particular to an online analysis instrument drift compensation method based on big data analysis, and the method comprises the steps: collecting an original measurement value of an online analysis instrument in real time, and determining a time index of a current sampling moment relative to a blowback recovery starting point; based on the time index, obtaining a pre-calibrated historical average deficit value and a characteristic slope reference value; calculating a working condition matching coefficient and a real-time recovery slope of an original measurement value; constructing a response rate deviation factor representing the difference between the actual rate and the theoretical rate; and carrying out dynamic weighting on the historical average deficit value based on the response rate deviation factor, and superposing a weighting result to the original measurement value to obtain an output concentration value. By introducing the point-to-point constraint of the rate dimension, the phase dislocation in the recovery process is effectively eliminated, and the depth self-adaptive compensation under the full-scale complex working condition is realized.

This invention provides a pre-computation-based query method and system, including constructing a hierarchical index based on information from a pre-computation table, filtering a candidate set of pre-computation tables that meet preset conditions using the index, and updating the index information corresponding to the pre-computation table when a user creates, updates, or deletes it. Based on the candidate set of pre-computation tables, the system performs custom matching and rewriting of the query SQL statement using the current query engine. It also dynamically schedules pre-computation tasks based on historical workload to determine the initialization and updating times of the pre-computation table. This invention optimizes the query engine based on pre-computation and uses a space-for-time tradeoff to optimize the performance of online analytical processing tasks under a unified architecture.

The invention provides intelligent query routing in a hybrid database. The present disclosure also provides a hybrid transaction / analytic processing (HTAP) database. The HTAP database includes a shared storage device that holds data, an online analytical processing (OLAP) engine, and an online transactional processing (OLTP) engine provided separately from the OLAP engine. The OLTP engine includes a query router configured to route a query received by the OLTP engine to the OLAP engine or the OLTP engine for execution.

The invention discloses a streaming loading tool creating method, a data importing method, a data importing device and data importing equipment, which are applied to the technical field of the Internet and comprise the following steps of: obtaining messages of an open source message queue, based on the message of the open source message queue, generating a structured query language statement used for creating a library table in an online analysis processing database by calling a large language model; automatically creating a library table based on the structured query language statement for creating the library table to obtain a target library table; based on the information of the target library table, calling a large language model to generate a structured query language statement for creating a streaming loading tool; and automatically creating the streaming loading tool based on the structured query language statement for creating the streaming loading tool. According to the method and the device, large-scale creation of the streaming loading tool can be automatically and intelligently completed on the basis of the structured query language statement for creating the streaming loading tool, so that the creation efficiency of the streaming loading tool is improved.

The invention provides a project cost data management system, and relates to the technical field of project cost, the project cost data management system comprises a dynamic association engine, a cost index multi-dimensional model module and a distributed block chain storage module, the dynamic association engine is used for establishing an association relationship among a project quantity list, a quota library, a real-time market price and design change data; a cost result is automatically generated and a cost deviation is updated in real time based on the association relationship; the cost index multi-dimensional model module is used for constructing a cost index multi-dimensional model based on a data warehouse and an online analysis processing technology; according to the method, a four-dimensional binding model of the bill of quantity-quota library-real-time market price-design change is constructed through the dynamic association engine, automatic data association is realized by presetting association rules, and manual matching and input are not needed; the cost is updated in real time and the deviation is calculated when any data changes, compared with traditional manual operation, the efficiency is higher, the cost deviation rate is smaller, and the cost risk caused by manual errors is reduced.

A hybrid transactional / analytical processing (HTAP) database is provided, which includes an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage holding data. The shared storage includes an OLTP storage holding data in row format for responding to queries of the OLTP engine, and an OLAP storage holding data in column format for responding to queries of the OLAP engine. The OLAP storage includes a delta store configured to record changes made to the OLTP storage in an update, and a base store configured to implement the update from the delta store on data in persistent storage in a flush. The HTAP database further includes a table metadata service configured to generate and perform operations on table metadata of the data that is updated in the flush.

Embodiments of the present application disclose a power supply detection method and device, a storage medium and a computer device. According to the power supply characteristic data, a first data is obtained, the first data is stored in a first message queue to obtain a second data, the second data is processed by a stream computing component to obtain a third data, the third data is stored in a second message queue to obtain a fourth data, the fourth data is connected to a columnar storage database, and a materialized view component is created, the fourth data is stored in a first table engine by the materialized view component and is deduplicated to obtain a fifth data, the fifth data is analyzed in real time according to a query request, and an analysis result is displayed. The data is processed in real time by stream computing, the processed data is stored in the columnar storage database, the data is analyzed in real time by online analytical processing, and real-time detection is realized, and the detection efficiency of power supply abnormalities is improved.

This disclosure describes a Hybrid Transactional and Analytical Processing (HTAP) technology. An HTAP system includes: a first processing engine configured to perform online transaction processing, a second processing engine configured to perform online analytical processing, and a storage device communicating with the first and second processing engines. The first processing engine, the second processing engine, and the storage device may be modularized and configured to be decoupled from each other. The system may be configured to have the first processing engine capture data in real time, organize data in a first component of the storage device according to a first format for use by the first processing engine, propagate data to a second component of the storage device system, and organize data in a second format in the second component of the storage device for use by the second processing engine.

The application discloses a technology consultation entity portrait online analysis processing analysis and device, the method comprises the following steps: obtaining an entity attribute label based on a foreground attribute label and a background attribute label; wherein the background attribute label is obtained based on processing of entity original data in a technology consultation scene; corresponding data label conversion is performed according to the entity attribute label and a preset data type conversion relationship table, the converted data label is stored to obtain a data storage result; the preset user rule is converted to JSON format data based on the data storage result, the JSON format data is parsed to generate a corresponding SQL statement query result; data rendering is performed based on the SQL statement query result, and entity portrait analysis in the technology consultation scene is completed according to the rendering result. The application eliminates the differences between technology consultation entity attributes, realizes flexible data query and millisecond-level response, and promotes the realization of industrial digitization and data value.

The invention provides a data analysis processing method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of data processing. The analysis processing method comprises the steps that original data are accessed from a data source according to a message queue, and the original data comprise real-time data and offline data; performing real-time analysis processing on the real-time data according to a stream processing technology; and performing off-line analysis processing on the off-line data according to an on-line analytical processing (OLAP) technology and the materialized view. The problems of low resource utilization rate, high operation and maintenance complexity, high cost, high data delay and data storage limitation in related technologies are at least solved. The method is suitable for data analysis processing scenes.

This application provides a method, apparatus, and medium for processing CellTrace data. The method involves acquiring CellTrace data; converting the CellTrace data to a target format data object; extracting information from the target format data object to obtain valid information data; and obtaining personalized indicator data based on the valid information data. This personalized indicator data is used for online analysis processing in an end-to-end analysis system. By processing CellTrace data, this application enables it to effectively support end-to-end analysis systems, thereby improving mobile network quality and ultimately enhancing the user experience for mobile network users.

The invention relates to the technical field of power marketing data processing, in particular to a user payment behavior division method based on DBSCAN-CTOA optimized LDA, and the method comprises the steps: collecting power user data to form a user payment original data set, constructing a data cube through hybrid online analysis processing, mining key features of user payment behaviors, and dividing the user payment behaviors into a payment data set; a user payment feature data set is formed, a density clustering algorithm is adopted to generate a noiseless data cluster for the feature data set, a chaos tornado optimization algorithm is utilized to perform global optimization on payment feature projection dimensions and payment data regularization parameters of linear discriminant analysis, and an optimal parameter combination adaptive to payment data distribution is generated. And inputting the noiseless data cluster and the optimized parameters into a linear discriminant analysis model, calculating an intra-class divergence matrix and an inter-class divergence matrix to solve an optimal projection direction, realizing classification of user payment behaviors, and solving the problem that an existing user payment management system lacks an effective data induction and analysis method.

A hybrid transactional / analytical processing (HTAP) database includes an online transaction processing (OLTP) engine, an online analytical processing (OLAP) engine provided separately from the OLTP engine, and a shared storage holding data of multiple tenants. The shared storage is configured to receive a request for a first scan from a first tenant and a request for a second scan from a second tenant, execute the first scan, and after a predetermined first period of time has elapsed without the first scan finishing, pause the first scan and execute the second scan.

The application discloses a real-time interactive extreme climate disaster event correlation mining method, which is from the perspective of knowledge aggregation, integrates the spatio-temporal data cube technology into the interactive mining process of the spatio-temporal correlation mode, and breaks through the key bottleneck that the complex mining constraints and the real-time interactive performance cannot be considered in the human-computer interaction. The method breaks through the limitation of the traditional Cube dimension orthogonal independence, unifies the three types of complex constraints, i.e. the spatio-temporal boundary randomness, the spatio-temporal coupling and the multi-dimensional rules, into the multi-dimensional query view in the data cube, thereby supporting the view control ability of the analyst based on the multi-dimensional and multi-scale Cube and the spatial online analytical processing operation, realizing the visualization exploration of the spatio-temporal non-stationary, multi-scale effect and spatio-temporal differentiation characteristics of the extreme climate disaster event correlation mode, overcoming the problem that the spatio-temporal correlation knowledge cannot be distributed and aggregated in theory, and establishing a multi-granularity spatio-temporal correlation mode distributed aggregation mechanism of the pre-aggregated knowledge query and the online mining cooperation.