90 results about "Parallel database" patented technology

An ultra-shared-nothing parallel database system includes at least one master node and multiple slave nodes. A database consisting of at least one fact table and multiple dimension tables is partitioned and distributed across the slave nodes of the database system so that queries are processed in parallel without requiring the transfer of data between the slave nodes. The fact table and a first dimension table of the database are partitioned across the slave nodes. The other dimension tables of the database are duplicated on each of the slave nodes and at least one of these other dimension tables is partitioned across the slave nodes.

A system for automating data partitioning in a parallel database includes plural nodes connected in parallel. Each node includes a database server and two databases connected thereto. Each database server includes a query optimizer. Moreover, a partitioning advisor communicates with the database server and the query optimizer. The query optimizer and the partitioning advisor include a program for recommending and evaluating data table partitions that are useful for processing a workload of query statements. The data table partitions are recommended and evaluated without requiring the data tables to be physically repartitioned.

The invention discloses a data partitioning method for a distributed parallel database system. The method comprises the following steps of: establishing a fact table and a dimension table according to the constructed distributed parallel database system; inserting records of the dimension table and the fact table on different nodes according to a partitioning rule; copying the records of the dimension table to the nodes of the fact table; and deleting and updating the data. When a data set or data stream is imported or inserted into the distributed database system in a partitioning way, the relation between tables defined by a database schema can be met on each node, particularly the primary key-foreign key restrictive condition, so the data on each node has local completeness of the data. For the query processing on the connection between the tables by using the primary key-foreign key restrictive condition, the data of each node has the local completeness on the query, so dynamic repartitioning of data between the nodes is not needed; and thus the method has the advantages of preventing time-consuming network transmission of the data, shortening the query response time and improving the query efficiency.

Methods, systems, and apparatuses for improving performance of parallel database query plans are described. An exchange operator is positioned in a query tree. A child operator of the exchange operator is parallelized into a plurality of parallel child operators, each of the parallel child operators coupled to the exchange operator in a respective branch of a plurality of parallel branches of the query tree. An output of each of the plurality of parallel child operators may be buffered at the exchange operator. Furthermore, child operators of the plurality of parallel child operators may also be parallelized. Query plans of any form and containing any number of operators may be parallelized in this manner. Any number of parallel branches may be used, independent of the number of operators in the original plan. The parallelized query plans achieve effective load balancing across all branches.

A parallel processing architecture for a relational database management system (RDBMS) that supports both a process model operating system and a thread model operating system. The RDBMS is implemented as a shared nothing, single database image utilizing Parallel Database Extensions (PDEs) that insulate the RDBMS from the specifics of the operating system and that provide the necessary techniques for accessing common memory segments.

A database processing system including a plurality of partitioned databases. Data processing is performed with pieces of information processing apparatus associated with each of the partitioned databases respectively. In response to a query, a status table indicating availability of each information processing apparatus is read from the storage. Of the pieces of information processing apparatus for processing the received query, at least a serviceable one is determined as a process request destination. A process request corresponding to the query is transmitted to the information processing apparatus determined as the process request destination. The process request is received through a communication unit, and data on the database are consequently processed. A processing result is transmitted to a transmitting source through the communication unit.

Embodiments are directed to determining optimal partition configurations for distributed database data and to implementing parallel query optimization memo data structure to improve partition configuration cost estimation efficiency. In an embodiment, a computer system accesses a portion of database data and various database queries for a given database. The computer system determines, based on the accessed database data and database queries, a partition configuration search space which includes multiple feasible partition configurations for the database data and a workload of queries expected to be executed on that data. The computer system performs a branch and bound search in the partition configuration search space to determine which data partitioning path has the lowest partitioning cost. The branch and bound search is performed according to branch and bound search policies. The computer system also outputs the partition configuration with the determined lowest partitioning cost.

A cluster node within a cluster of a highly parallel database system includes at least one processing unit that runs a set of first tier threads and a set of second tier threads, a storage disk drive, and a networking interface. When a first tier thread receives a task, it divides the task into a set of subtasks. The first tier thread also assigns the set of subtasks between a subset of the set of second tier threads for execution. Each second tier thread within the subset processes the one or more subtasks it is assigned to. When the task is a work, the subtasks are work units. When the task is a work unit, the subtasks are subwork units.

The invention discloses an implementation method for operator reuse in a parallel database, comprising the following steps of: step 1, generating a serial query plan for query through a normal query planning method, wherein the query plan is a binary tree structure; step 2, executing the query plane by scanning from top to bottom, searching materialized reusable operators, changing the query plane structure, and changing thread level materialized operators into global reusable materialized operators; step 3, parallelizing the query plan changed in the step 2, and generating a plan forest for parallel execution of a plurality of threads; step 4, executing global reusable operator combination on the plan forest generated in the step 3, and generating a directed graph plan for the materialized reusable operators capable of being executed by the plurality of threads in parallel; step 5, executing own plan part in the directed graph by each thread in parallel, wherein the thread which executes the global reusable operator firstly is called a main thread, the main thread locks the global reusable operator and truly executes the operator and the plan of the operator, and other threads wait; step 6, unlocking the global reusable operator by the main thread after execution, wherein other threads start to read data from the global reusable operator and continue to execute own plan tree;and step 7, releasing the materialized data of the operator by the main thread after all the plans read the data of the global reusable operator.

The invention discloses a data query method and device for a parallel database. The method comprises the steps of: respectively performing target data grouping aggregation on a target data table according to corresponding associated fields between the target data table and other data tables on each database node; respectively performing data re-partitioning on corresponding grouping aggregation results and corresponding other data tables in a Hash manner according to the corresponding associated fields on each database node; collecting data re-partitioning results of the grouping aggregation results and data re-partitioning results of the other data tables on each database node into a target database node; and performing target data connecting aggregation on the data re-partitioning results of the grouping aggregation results and the data re-partitioning results of the other data tables on the target database node. According to the method provided by the invention, data aggregation query can be realized, meanwhile, the parallelism of the query is improved, the resource utilization rate of a cluster is increased, the network cost is reduced, and the query performance is improved.

A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes a query optimizer for optimizing data queries. The optimizer estimates the column cardinality of a set of rows based on estimated column cardinalities of disjoint subsets of the set of rows. For a particular column, the actual column cardinality of the set of rows is the sum of the actual column cardinalities of the two subsets of rows. The optimizer creates two respective Bloom filters from the two subsets, and then combines them to create a combined Bloom filter using logical OR operations. The actual column cardinality of the set of rows is estimated using a computation from the combined Bloom filter.

The present invention extends to methods, systems, and computer program products for formulating global statistics for parallel databases. In general, embodiments of the invention merge (combine) information in multiple compute node level histograms to create a global histogram for a table that is distributed across a number of compute nodes. Merging can include aligning histogram step boundaries across the compute node histograms. Merging can include aggregating histogram step-level information, such as, for example, equality rows and average range rows (or alternately equality rows, range rows, and distinct range rows), across the compute node histograms into a single global step. Merging can account for distinct values that do not appear at one or more compute nodes as well as distinct values that are counted at multiple compute nodes. A resulting global histogram can be coalesced to reduce the step count.

To guarantee that contents of an update by a transaction in a parallel database management system. A database management system includes a replica database management unit that manages the replica database, records synchronous information at a timing at which one of the transaction generated is valid in every database management unit and other transactions are invalid in every database management unit, extracts update information and the synchronous information for creating the replica database from the update logs, and causes the replica database management unit to import the update information of each transaction that has become valid before the synchronous information was recorded.

The invention provides a method and a device for querying a double-transcript parallel database, and belongs to the technical field of databases. The method comprises the following steps: obtaining a query request and data storage unit information; forming a plurality of execution plans according to the query request and the data storage unit information; calculating resource occupation rates of the execution plans according to the resource utilization rates of execution nodes and the estimated data transmission quantity of the execution nodes in the execution plans; selecting one execution plan from the execution plans according to the resource occupation rate; and querying data according to the selected execution plan. The method provided by the invention is used for calculating the resource occupation rates of the execution plans according to the resource utilization rates of the execution nodes and the estimated data transmission quantity of the execution nodes in the execution plans and selecting one execution plan from the execution plans according to the resource occupation rate, and since the resource occupation condition influences the data transfer time consumption and the data query efficiency, the data transfer time consumption of the execution plan in the final query is short, and the data query efficiency is high.

The invention discloses a distributed parallel database system based on an Infiniband network and a data processing method. The system comprises a scheduling cluster, a data cluster and a management cluster. Each cluster is composed of at least two data nodes, and the data nodes are connected through the Infiniband network. The data processing method comprises a data distribution method, a data loading method, a data query method and a data de-duplication method. Aiming at the current situation that the number of distributed parallel processing database cluster node servers is large, the characteristics of high bandwidth, low delay and low memory of an Infiniband network are fully utilized, and a mode of applying the network to data is designed, including database system configuration, data loading and storage, data query, data calculation and the like. The method is high in universality, breaks through network bandwidth bottleneck, storage space and calculation delay limitation of a current database system, guarantees high availability of the system, and provides high-performance service for users.

A method and system for assigning a user default role in a remote database of a database system is disclosed. The method and system comprises the steps of activating a default role for the remote database and utilizing the activated default role to access data within the remote database. Accordingly, a system and method is provided that allows a user to access a remote database via a default role. The system and method only requires that default role information be stored in a current role database structure and be accessible by a user. In so doing, a user can easily access information in the remote database through the default role. Therefore, this system is compatible and easily implemented utilizing existing parallel database systems.