Server cluster data synchronization method and device
A server cluster and data synchronization technology, applied in the Internet field, can solve problems such as poor timeliness, inability to support real-time ETL, and the inability of ETL processing solutions to run stably, and achieve good scalability.
Inactive Publication Date: 2020-06-09
AISPEECH CO LTD
View PDF0 Cites 2 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0004] During the practice of this application, the inventors of the present application found that the current related technologies have at least the following defects: 1) The ETL capability is poor under the large amount of data, but under the background of big data and the Internet of Things, the data volume of small and medium-sized enterprises exceeds 100 million It is also very common, especially because it cannot support real-time ETL, resulting in poor timeliness
2) The server cluster mode is generally used in large data volume scenarios, but the current ETL processing scheme cannot run stably in the cluster
[0005] For the above problems, there is currently no better solution in the industry
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0017] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.
[0018] It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.
[0019] This application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules inc...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a server cluster data synchronization method and device. The method comprises the steps that a server node reads configuration information according to a data source in pre-configured ETL configuration data and reads a source data set from at least one heterogeneous data source, and the ETL configuration data further comprises a data conversion rule and target data output configuration information; for each piece of source data in the source data set, the corresponding source data is converted into data with multiple target data formats according to a data conversion rule to obtain a target data set, the multiple target data formats includes a data format suitable for at least one heterogeneous data source; and for the target data set corresponding to each piece ofsource data in the source data set, each piece of target data in the corresponding target data set is output to each corresponding data source based on the target data output configuration information. In this way, the data synchronization process is flexibly achieved in the mode that component configuration is conducted in the server nodes, and good expansibility is achieved.
Description
technical field [0001] The present application belongs to the technical field of the Internet, and in particular relates to a server cluster data synchronization method and device. Background technique [0002] With the development of enterprise information construction, more and more enterprises have established numerous information systems to help enterprises process and manage internal and external business. However, with the increase of information systems, the information systems that work in isolation have caused a large amount of redundant data and duplication of work by business personnel. [0003] Data integration technology came into being, and ETL (Extraction-Transformation-Loading, data extraction-transformation-loading) is an important part of data integration technology. Through ETL, the data of the business system can be extracted, cleaned and transformed, and then loaded into the data warehouse to integrate the scattered, messy, heterogeneous, and non-unifor...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/25G06F16/27
CPCG06F16/254G06F16/27
Inventor 赵伟
Owner AISPEECH CO LTD