993 results about "Datawarehouses" patented technology

Improved method of and apparatus for aggregating data elements in multidimensional databases (MDDB). In one aspect of the present invention, the apparatus is realized in the form of a high-performance stand-alone (i.e. external) aggregation server which can be plugged-into conventional OLAP systems to achieve significant improments in system performance. In accordance with the principles of the present invention, the stand-alone aggregation server contains a scalable MDDB and a high-performance aggregation engine that are integrated into the modular architecture of the aggregation server. The stand-alone aggregation server of the present invention can uniformly distribute data elements among a plurality of processors, for balanced loading and processing, and therefore is highly scalable. The stand-alone aggregation server of the present invention can be used to realize (i) an improved MDDB for supporting on-line analytical processing (OLAP) operations, (ii) an improved Internet URL Directory for supporting on-line information searching operations by Web-enabled client machines, as well as (iii) diverse types of MDDB-based systems for supporting real-time control of processes in response to complex states of information reflected in the MDDB. In another aspect of the present invention, the apparatus is integrated within a database management system (DBMS). The improved DBMS can be used to realize achieving a significant increase in system performance (e.g. deceased access / search time), user flexibility and ease of use. The improved DBMS system of the present invention can be used to realize an improved Data Warehouse for supporting on-line analytical processing (OLAP) operations or to realize an improved informational database system, operational database system, or the like.

Improved method of and apparatus for joining and aggregating data elements integrated within a relational database management system (RDBMS) using a non-relational multi-dimensional data structure (MDD). The improved RDBMS system of the present invention can be used to realize achieving a significant increase in system performance (e.g. [deceased] decreased access / search time), user flexibility and ease of use. The improved RDBMS system of the present invention can be used to realize an improved Data Warehouse for supporting on-line analytical processing (OLAP) operations or to realize an improved informational database system or the like.

An Internet-based consumer product brand marketing communication system which enables manufacturers, retailers and their respective agents, and consumers to carryout product-related functions along the demand side of the retail chain. A central RDBMS (i.e. data warehouse) stores a central database of consumer product related information resource files (IRFs) each IRF being indexed with a Universal Product Number (UPN) assigned thereto by the manufacturer of the consumer product, or its designated agent, which relates to the consumer product. A first subsystem enables a manufacturer's marketing, brand and / or product managers to create and manage a local RDBMS containing UPN-indexed IRFs related to the consumer products of the manufacturer being offered for sale in both physical and / or electronic marketplaces, and periodically transport the local RDBMS of UPN-indexed IRFs to the central RDBMS by electronic data transport techniques. The local RDBMS of UPN-indexed IRFs is selected by the manufacturer's marketing, brand and / or product managers so as to create a desired brand image for each the consumer product of the manufacturer. A second subsystem enables consumers to access one or more UPN-indexed IRFs in the central RDBMS, to request and obtain information about a manufacturer's consumer product so as to make informed / educated purchases along the demand side of the retail chain. A third subsystem enables manufacturers and their advertising and marketing agents to access one or more UPN-indexed IRFs in the central RDBMS to display consumer product advertisements to consumers, at or near the point of purchase or sale within both physical and / or electronic retail shopping environments so as to project the desired brand image to consumers. A fourth subsystem enables retailers and their marketing and promotional agents to access one or more UPN-indexed IRFs in the central RDBMS, to promote consumer products to consumers, at or near the point of purchase or sale within both physical and / or electronic retail shopping environments so as to promote the sale of such products in inventory.

A system and method for automatically reconfiguring a computing environment comprises a consumption analysis server, a placement server, a deployment server in communication with a set of virtual machine monitors and a data warehouse in communication with a set of data collection agents, and a database. The consumption analysis server operates on measured resource utilization data in the data warehouse to yield a set of resource consumptions, available capacities and host and virtual machine configurations from the computing environment. The deployment server continuously monitors an event triggering condition and when the triggering condition is met, the placement server assigns a set of target virtual machines to a target set of hosts in a new placement and the deployment server implements the new placement through communication with the set of virtual machine monitors. The placement server right-sizes the virtual machines and the target set of hosts.

According to the present invention there is provided a method for establishing a data warehouse capable from a plurality of source databases including at least one relational database and at least one object-oriented database, comprising the steps of: integrating the schema of said plurality of source databases into a global schema, including resolving semantic conflicts between said source databases, and establishing a frame metadata model for describing data stored in said local databases, said frame metadata model including means for describing any constraints developed during schema integration and further including means for describing relationships between data stored in local object-oriented databases.

The present invention provides an automated data warehousing and OLAP cube building process. The invention allows a person who is not a database query language expert to build validated data warehouses, and OLAP cubes based on such data warehouses.

Provided is a day-ahead load reduction system based on a customer baseline load for inducing a user to efficiently manage energy consumption by applying an incentive (user compensation according to load reduction) to achieve load reduction and load decentralization. The day-ahead load reduction system based on a customer baseline load operates in connection with a provider terminal and a user terminal through a network to induce a reduction in the load of a user and includes an AMI / AMR translator collecting load profile data of the user in real time, converting the load profile data and storing the load profile data in a meter data warehouse; a meter data management system monitoring and analyzing the load profile data stored in the meter data warehouse in real time; a demand response operation system managing the demand of the user by using the load profile data and performing overall management, analysis and verification of a day-ahead load reduction event; a customer energy management system operating in connection with the demand response operation system and providing information on the load to the user through the user terminal in real time to allow the user to control the load; and an account system operating in connection with the demand response operation system and the customer energy management system, calculating an incentive for the day-ahead load reduction event and notifying a provider and the user of the incentive through the provider terminal and the user terminal.

The invention discloses a Kubernetes based capacitor cloud configuration and an interaction method among modules of cloud configuration. The Kubernetes based capacitor cloud configuration comprises a processor, and a mirror image construction module, a data warehouse module, a load balancing module, a service discovery module and a container monitoring module connected with the processor; the mirror image construction module provides mirror image file making, storing and distributing services; the data warehouse module stores and processes data in a database of a cluster; the load balancing module carries out load balancing on calculating nodes in the Kubernetes cluster; the service discovery module obtains information of dynamic change of the calculating nodes in the Kubernetes cluster; and the container monitoring module collects and displays information of motion state of the calculating nodes in the Kubernetes cluster. The cloud configuration and the interaction method can be applied to container systems which need high availability and expansibility conveniently.

A system to load data in a data warehouse includes reception of a plurality of records, determination, for each of the plurality of records, of values representing differences between a record and each other of the plurality of records, identification of at least two of the plurality records as duplicates based on a determined value representing a difference between the two records, and storage of the two records in the data warehouse in association with a same identifier. Determination of the values may include determination, for each of a first plurality of data fields of the record, of a first value representing a difference between data specified in the data field and data specified in a respective one of a second plurality of data fields of one of the other of the plurality of records, determination, for each of the second plurality of data fields, of a second value representing a difference between data specified in the data field and data specified in a respective one of the first plurality of data fields, and determination of a third value representing a difference between the record and the one of the other of the plurality of records based on the determined first and second values.

A computer system that supports and analyzes Internet and / or electronic commerce data includes at least one user station transmitting a user request, and at least one Internet server, connectable to the user station, and providing the user station access to external data sources. The computer system also includes at least one data collection and analysis system. The data collection and analysis system includes at least one analysis module performing statistical and analytical analysis of the Internet and / or electronic commerce data for use by a user in performing one or more business decisions, and at least one Internet and / or electronic commerce data warehouse which is responsively connected to the analysis module and separated from external systems. The data warehouse receives the Internet and / or electronic commerce data, and provides access to the Internet and / or electronic commerce data by the user. The data collection and analysis system also includes at least one data mapping module which searches, collects and transforms the Internet and / or electronic commerce data, and transmits the same to the data warehouse for storage.

An improved method of and apparatus for aggregating data including a scalable multi-dimesional database (MDDB) and a high-performance aggregation engine that are integrated into a database management system (DBMS). The improved DBMS can be used to realize achieving a significant increase in system performance (e.g. deceased access / search time), user flexibility and ease of use. The improved DBMS system of the present invention can be used to realize an improved Data Warehouse for supporting on-line analytical processing (OLAP) operations or to realize an improved informational database system, operational database system, or the like.

A system and method for reconfiguring a computing environment comprising a consumption analysis server, a placement server and a data warehouse in communication with a set of data collection agents and a database. The consumption analysis server operates on measured resource utilization data to yield a set of resource consumptions in a set of regularized time blocks in a set of sample periods, and, to group regularized time blocks across a set of sample periods to form a set of interval groups. The placement server assigns a set of target virtual machines to the target set of hosts in a new placement and scores the new placement in an effort to meet a threshold score based on an objective function of resource capacity headroom. In one aspect, the scoring relies on percentile analysis of resource consumption in the interval groups. The new placement is implemented in the computing environment.

An ETL / EAI data warehouse management system and method for processing data by dynamically distributing the computational load across a cluster network of distributed servers using a master node and multiple servant nodes, where each of the servant nodes owns all of its resources independently of the other nodes.

Improved method of and apparatus for joining and aggregating data elements integrated within a relational database management system (RDBMS) using a non-relational multi-dimensional data structure (MDD). The improved RDBMS system of the present invention can be used to realize achieving a significant increase in system performance (e.g. deceased access / search time), user flexibility and ease of use. The improved RDBMS system of the present invention can be used to realize an improved Data Warehouse for supporting on-line analytical processing (OLAP) operations or to realize an improved informational database system or the like.

A system and method for reconfiguring a computing environment comprising a consumption analysis server, a placement server, an infrastructure management client and a data warehouse in communication with a set of data collection agents and a database. The consumption analysis server operates on measured resource utilization data to yield a set of resource consumptions in regularized time blocks, collects host and virtual machine configurations from the computing environment and determines available capacity for a set of target hosts. The placement server assigns a set of target virtual machines to the target set of hosts in a new placement. In one mode of operation the new placement is nearly optimal. In another mode of operation, the new placement is “good enough” to achieve a threshold score based on an objective function of resource capacity headroom. The new placement is implemented in the computing environment.

A data warehouse solution system comprises a metadata model, a user interface and an engine. The metadata model has an information needs model including metadata regarding information needs for building reports by users, and a data information model including metadata describing data that is available for building reports. The user interface has a customer user interface for presenting the information needs model to the users for report generation, and a modeling user interface for presenting the data information model to the users for manipulating data warehouse objects. The engine has a report management service unit for providing report management service using the information needs model, and a data management service unit for providing data management service including generation of a data warehouse using the data information model.

Techniques for transforming records prior to loading the records into a data warehouse in an efficient manner. In one embodiment, instead of using lookup transformations, a database operation such as an outer join operation is used to transform records stored in a fact staging table to add foreign keys to the records corresponding to the dimension tables that are associated with a fact table in a data warehouse to which the records are to be loaded. Since the outer join operation is memory intensive, a determination is made as to which dimension tables from the multiple dimension tables associated with the fact table are suitable, from a performance perspective, for the outer join operation. Only those suitable dimension tables are outer joined to the fact staging table to add the foreign keys for the outer joined dimension tables to the fact staging table records. Lookup transformations may be used to add foreign keys to the fact staging table records for the non-suitable dimension tables. The transformed fact records with the foreign keys may then be loaded into the fact table in the data warehouse.

Techniques for maintaining data quality of transformed data generated using an Extract-Transform-Load (ETL) process and stored in at least one data warehouse, the method comprising generating a quality metric for each of a plurality of units of the transformed data with reference to at least one data quality measurement rule, the quality metric for each unit of the transformed data representing a validity measure defined by the corresponding data quality measurement rule; and generating a report organizing the quality metrics for selected units of the transformed data.

A system for identifying hidden connections between non-sentiment phrases. The system comprises a network interface enabling an access to one or more data sources; a data warehouse storage for at least storing a plurality of phrases including sentiment phrases and non-sentiment phrases; an analysis unit for identifying hidden connections between non-sentiment phrases based on at least one proximity rule and for generating at least an association between at least two non-sentiment phrases having a hidden connection and a sentiment phrase, wherein an association between the at least two non-sentiment phrases having the hidden connection and the corresponding sentiment phrase is a term taxonomy.

The invention discloses a telecom operator mass data processing method based on a Hadoop platform. The telecom operator mass data processing method based on the Hadoop platform comprises the following steps that firstly a Sqoop tool is used for extracting data in an original data system into a Hadoop local server; then tables are created in Hive and a Hive script is written according to the data model and operation requirements of an operator data warehouse; then the Hive script is executed to convert the source data and load the data into an objective table of the Hive; finally Hive query languages or a MapReduce program is written according to requirements to carry out query analysis on the data in the objective table. According to the telecom operator mass data processing method based on the Hadoop platform, all kinds of tools are sufficiently used in the Hadoop platform on the foundation that operational requirements are met to achieve telecom operator mass data processing, and working efficiency is greatly improved.

The current invention discloses methods for transforming a set of relations into multidimensional data cubes. A syntheses process is disclosed that dynamically and with minimal user input eliminates ambiguities when populating a data cube by introducing table-like virtual relations. The methods are generic and applicable to many data warehouse designs. The methods support relational OLAP for a wider variety of data and structures than possible using current relational implementation schemas.

In one embodiment, a method an apparatus for managing a system comprising at least one operational data store and a data warehouse that is associated with at least some of the data in the operational data set, comprising automatically updating the schema of the data warehouse to reflect a change to the schema of the at least one operational data store. In a further embodiment, a method and apparatus for requiring that a designer of a database system provide metadata that defines changes to be made to a data warehouse in response to a modification of an area in an operational store. In another embodiment, a method and apparatus for detecting schema changes to an operational store, so that appropriate action may be taken in a data warehouse. In a further embodiment, a method and apparatus for implementing changes to a data warehouse schema, comprising the execution of one or more lower level calls to appropriate APIs to modify the schemas of both a relational database and an OLAP database.

The invention provides a subsystem of a TDS dynamic data warehouse, which comprises a data warehouse detailing subsystem used for reprocessing the transaction data provided by ETL and storing the transaction data in the form of the transaction data, the renewal is frequently executed, so that the consistency or synchronization between the warehouse and the data of a transaction system is retained, so as to achieve real time, the complexity and the working capacity of the data are processed through simplifying the frequent renewal link, so that the load of the data processing is backwards transferred to a multi-dimensional data warehouse, the system structure is simplified, and the level of the system equipment is reduced. The subsystem of the TDS dynamic data warehouse further comprises an accessing subsystem used for excavating the access of terminals to the detailed warehouse; and a processing subsystem used for processing the generation and storage process of managing and dispatching the batch processing multi-dimensional data. The invention provides an entire TDS dynamic data warehouse subsystem, so that the requirements of terminal users excavated through the data analysis for the detailed and real-time data are met, and the TDS dynamic data warehouse subsystem is particularly suitable for the occasion on which the moderate-scale dynamic data warehouse needs to be independently established.

Methods, systems, and articles of manufacture for constructing and populating data marts with dimensional data models from a set of data repositories that contain factual and association information about a set of related assets are disclosed. An intermediate data warehouse is generated to process the facts and associations for each asset. Using the intermediate warehouse, one or more data marts are generated with fact tables, dimensions, and hierarchies to fully model the information available for each asset.

A computerized maintenance management system for vehicles comprises a data warehouse, comprising a communications link for receiving data relating to vehicle maintenance, a storage component for storing received data, and a modeling component comprising modeling algorithms stored within the data warehouse. The modeling component is thus able to process the data in real-time within the data warehouse, and outputs a predictive model with recommended maintenance and / or replacement schedules that is always based on the most current available data.

A system for generating a model for tracking a changing dimension feature of data in a business model includes a memory for storing the business model and a computer readable code for modeling the changes in the changing dimension feature and a processor for executing the computer readable code to perform method steps including capturing the changing dimension feature of the business model, transforming the changing dimension feature into a data warehouse model corresponding the business model, identifying changing dimensions and attributes in the changing dimension feature according to the data warehouse model, and generating a run-time deployable component for tracking the changing dimension feature based on the identified changing dimensions and attributes.

The invention provides a data processing method and data processing device applied to a data warehouse. The data processing method includes: when a first data base is operated, rectifying records of a mirror database in the first database in the data warehouse according to an incremental log corresponding to the operation and then storing the rectified records in a key / value database; fetching latest entries from the key / value database, additionally storing the entries in a first data table of the data warehouse and enabling the first data table to include historical versions of the records of the mirror database; searching data in the first data table. Therefore, timeliness of the data in the data warehouse can be improved.

Embodiments of the invention disclose a data query method and apparatus, and a computer readable storage medium. The method comprises the steps of obtaining a data query request submitted by a client; if the data query request carries a view name of a target view, performing reconstruction processing on the data query request to obtain a target query statement of at least one data table corresponding to the target view; using the metadata information in the data warehouse to analyze the target query statement to determine a query engine and a query rule, wherein the query engine is a distributed computing engine or a local computing engine; obtaining a data query result of the data query request from a target database in a data warehouse through a query engine and a query rule, sending the data query result to the client side, wherein the data warehouse comprises one or two of offline data and real-time data. By means of the data query method and device, synchronous query processing of offline data and real-time data can be achieved, the query difficulty of complex data is lowered, and the data query efficiency is effectively improved.