High-speed importing method of cluster data in data base
A data import and database technology, applied in the database field, can solve the problems of limiting the efficiency of data import, balancing the high-speed import of data and processing and computing, affecting the overall performance of the database cluster, etc., so as to improve the import performance and meet the effect of high-speed data import.
Active Publication Date: 2014-03-26
天津神舟通用数据技术有限公司
View PDF2 Cites 1 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Therefore, when performing high-speed and continuous import of data, the control node will inevitably become a bottleneck due to a large amount of CPU, memory, and network resource overhead, thereby affecting the overall performance of the database cluster.
[0006] 2. Server performance limits data import efficiency
[0008] A database cluster with multiple control nodes can assign data import to multiple control nodes for parallel processing, but this allocation method can only alleviate the resource contention in the data import process to a certain extent, but cannot fundamentally solve the data problem. Performance issues with high-speed import
[0009] To sum up, the method of completing all operations by the control node cannot achieve a balance between high-speed data import and processing calculations
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0026] Embodiments of the present invention are described in further detail below in conjunction with the accompanying drawings:
[0027] A method for high-speed import of database cluster data, which is as follows image 3 Implemented on the system framework shown. In this system framework, the data import module is separated from the control node in the database cluster and the controller module is added to form an independent data import driver component. The data import driver component can exist independently or be embedded in the JDBC of the database cluster. , ODBC, OLEDB and other standard database access interfaces are provided to users. The core of the data import driver component is the data import module and the controller module. The data import module includes data preparation, data slice calculation, and data cache sending module; the controller includes an interactive communication module and a monitoring thread module. The interactive communication module is ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention relates to a high-speed importing method of cluster data in a data base. The method is characterized by including the following steps: step one, a data import driving component builds connection with an interaction module of a database cluster and obtains connection information of each database node; step two, the data import driving component obtains pattern information and divided storage information of a target table to be connected with the related database node through the database cluster connection built in the step one; and the data import driving component performs data import to the related database node, and step three, an interaction control module collets data import results and updates related statistical information of the target table. According to the data import driving component, control nodes are librated, and the data import is achieved under the condition of least interference of the database cluster; and simultaneously, an independent data import mode is used, so that parallel import is achieved conveniently by addition of clients, the import performance is promoted, and requirements of enterprises for rapid data import are satisfied.
Description
technical field [0001] The invention belongs to the technical field of databases, in particular to a method for high-speed importing of database cluster data. Background technique [0002] With the advent of the information age, data has exploded. In the face of massive data, enterprises have higher and higher requirements for the processing power and computing intensity of database systems. Therefore, database clusters have gradually replaced single-server databases and become the first choice for enterprises to deal with massive data. A database cluster combines multiple database servers to form a cluster through a high-speed network, which can achieve better overall performance than a single large server. The mass of data not only refers to the large scale of data, but also reflects the rapid growth of data. Therefore, enterprises require that database clusters not only have efficient data computing and processing capabilities, but also have high-speed data import capab...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 李晓鹏冯柯蒋志勇李阳刘荣蒋旭王洋洋赵雪静王效忠关刚饶路
Owner 天津神舟通用数据技术有限公司