Data synchronization method of heterogeneous database platform

Abstract

The application discloses a data synchronization method of a heterogeneous database platform, and comprises the following steps: step one, frame data is acquired from the heterogeneous database platform according to the same data structure, and the frame data needing to be updated is identified, and the identified frame data is data to be synchronized; step two, the network cluster inside a platform where the current service is located is listened to, real-time deployment identification of a data plane agent is acquired from the network cluster, and the data range to be synchronized is calculated according to the implementation deployment identification; step three, a plurality of logical modules perform logical operation on the identified data range to be synchronized; the data synchronization method of the heterogeneous database platform avoids multiple synchronization of modified data between the client and the server through steps of acquiring synchronization data, setting a synchronization range, generating a data list, confirming the synchronization range, generating a synchronization command and the like, and improves the data synchronization efficiency of the heterogeneous database platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data synchronization, in particular to a data synchronization method of a heterogeneous database platform. BACKGROUND

[0002] It is known that a heterogeneous database system is a collection of related database systems, which can realize data sharing and transparent access. Several database systems exist before joining the heterogeneous database system and have their own database management systems. Each component of the heterogeneous database has its own autonomy. While realizing data sharing, each database system still has its own application characteristics, integrity control and security control. The goal of the heterogeneous database system is to combine and share data information resources, hardware device resources and human resources between different databases. One of the key points is to establish a global data model or global view based on the local database model. This global model is particularly important for establishing high-level decision support systems. Large institutions have branches in many places, and each branch database has its own information data. Decision makers generally only care about macro information described by the global model. The description of the global model of the heterogeneous database based on data warehouse technology is a good solution. Data warehouse can collect information from multiple databases in the heterogeneous database system and establish a unified global model. The collected data also supports access to historical data. Users query decision support through the unified data interface provided by the data warehouse. In addition, database conversion tools can realize data model conversion between different database systems. The problem that needs to be further studied is that if the database conversion is combined with data definition mode conversion and data conversion, it may cause multiple copies of the same data set in the heterogeneous database system, so a new access control mechanism needs to be introduced.

[0003] The existing heterogeneous database platform needs to be synchronized multiple times when the actual needs of the user change, which will cause a large amount of repeated data transmission in the database, and when the user modifies the same data multiple times according to actual needs, the modified data will be synchronized multiple times between the client and the server, thereby wasting network resources and making the data synchronization efficiency of the heterogeneous database platform to be improved. Therefore, we propose a data synchronization method of a heterogeneous database platform. SUMMARY

[0004] The purpose of the present application is to provide a data synchronization method of a heterogeneous database platform to solve the problems raised in the background art.

[0005] In order to achieve the above object, the present application provides the following technical scheme: a data synchronization method of a heterogeneous database platform, comprising the following steps:

[0006] Step one, frame data is obtained from the heterogeneous database platform according to the same data structure, and the frame data to be updated is identified, and the identified frame data is the data to be synchronized;

[0007] Step two, the network cluster in which the current service is located is monitored, and the real-time deployment identifier of the data plane agent is obtained from the network cluster, and the data range to be synchronized is calculated according to the implementation deployment identifier;

[0008] Step three, a plurality of logic modules perform logical operation on the identified data range to be synchronized, and then generate a list to the data end where the logic module is located;

[0009] Step four, according to the data range, a plurality of related data plane agents are issued data, and a data model is used to describe the calling and proxy relationship of the data through a tag selector;

[0010] Step five, when the system cannot calculate the data range to be synchronized in the heterogeneous database platform, synchronize all data in the network cluster where the current service is located;

[0011] Step six, the calculation unit of the related data plane agent is obtained in the data model where the current service is located, and the range of the data to be synchronized is calculated through the calculation unit, and the calculated data range to be synchronized is issued; the calculation unit completes the operation through a logic circuit composed of extremely small electronic components of the electronic circuit, and processes binary high and low level signals;

[0012] Step seven, according to the confirmation result of the data synchronization range, the data to be synchronized and all data in the kernel file system buffer are written into the storage medium respectively through the standard system call function of the operating system, and the command of data synchronization is generated;

[0013] Step eight, according to the data synchronization instruction, the data to be synchronized and all data are sequentially synchronized into the backup database of the heterogeneous database platform; the backup database is an incremental backup, which can use a new image of the entire database, which is smaller than the initial full backup, and only contains the changed database since the last full backup.

[0014] Preferably, in step one, the frame data is composed of a frame header, a data part and a frame tail, wherein the frame header and the frame tail contain necessary control information including synchronization information; the data part mainly includes data information required to be transmitted and stored in the daily operation process of the heterogeneous database platform.

[0015] Preferably, in step two, the data plane agent can enable one frame data to read and write another frame data inside the heterogeneous database platform; in addition, the cluster communication system is a mobile communication system for group dispatching command communication, has available channels that can be shared by all users of the system, has an automatic channel selection function, and is a multi-purpose, high-efficiency wireless dispatching communication system that shares resources, shares costs, shares channel equipment and services;

[0016] Preferably, in step three, the logic module can divide the internal system of the heterogeneous database platform into multiple logic units, and the multiple logic units can respectively realize function determination and operation; and the logic unit is a network port that enters the system network architecture, through which a user can access network resources and realize communication between one programmer and another programmer.

[0017] Preferably, in step four, the data call can define the data format and also call the parameters and types of service data; the data model describes the static characteristics, dynamic behavior and constraint conditions of the system, provides an abstract framework for information representation and operation of the database system, and the content described by the data model includes three parts of data structure, data operation and data constraint.

[0018] Preferably, in step five, the data in all network authorities where the current network is located can realize information sharing, so that the to-be-synchronized data maintains integrity and uniformity.

[0019] Preferably, in step seven, the kernel file buffer area is used to temporarily store file data during reading and writing, and a certain space is reserved in the memory area, so that the reading frequency of the data storage medium is reduced.

[0020] Preferably, in step seven, the storage medium is the carrier of data storage, and the amount of information stored can meet the writing of the to-be-synchronized data.

[0021] Compared with the prior art, the beneficial effects of the present application are:

[0022] The data synchronization method of the heterogeneous database platform can confirm the to-be-synchronized data before entering the backup database through the steps of acquiring the synchronization data, the synchronization range, generating the data list, confirming the synchronization range, and generating the synchronization command, and then generating the synchronization command after confirmation, thereby avoiding multiple synchronization of the modified data between the client and the server, reducing the waste of network resources, and improving the data synchronization efficiency of the heterogeneous database platform. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The step flow chart of the data synchronization method of the heterogeneous database platform of the present application; DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work are within the protection scope of the present application.

[0025] Please refer to Figure 1 The present application provides a technical solution: a data synchronization method of a heterogeneous database platform, comprising the following steps:

[0026] Step one, frame data is obtained from the heterogeneous database platform according to the same data structure, and the frame data to be updated is identified, and the identified frame data is the data to be synchronized; the frame data is composed of a frame header, a data part and a frame tail, wherein the frame header and the frame tail contain necessary control information including synchronization information; the data part mainly includes data information required to be transmitted and stored in the daily operation process of the heterogeneous database platform;

[0027] Step two, the network cluster inside the platform where the current service is located is listened to, and the real-time deployment identifier of the data plane agent is obtained from the network cluster inside, and then the data range to be synchronized is calculated according to the implementation deployment identifier; the data plane agent can enable one frame data in the heterogeneous database platform to read and write another frame data; in addition, the cluster communication system is a mobile communication system for group dispatching command communication, which has a shared channel for all users of the system, has an automatic channel selection function, is a shared resource, shares the cost, and is a multi-purpose and high-efficiency wireless dispatching communication system of shared channel equipment and service;

[0028] Step three, a plurality of logic modules perform logical operation on the identified data range to be synchronized, and then generate a list to the data end where the logic module is located; the logic module can divide the internal system of the heterogeneous database platform into a plurality of logic units, and the plurality of logic units can respectively realize function determination and operation; and the logic unit is a network port of the system network architecture, through which a user can access network resources and realize communication between one programmer and another programmer;

[0029] Step four, according to the data range to a plurality of related data surface agent issued data, and then use the data model through the label selector to describe the data call and agent relationship; data call can define the data format, and also can call the service data parameters and types; data model from the description of the static characteristics, dynamic behavior and constraint conditions for the system, for the information representation and operation of database system provides an abstract framework, the content described by the data model respectively includes data structure, data operation and data constraint three parts;

[0030] Step five, when the system cannot calculate the range of the data to be synchronized in the heterogeneous database platform, synchronize all the data in the current network cluster; all the data in the current network cluster can realize information sharing, so that the data to be synchronized can maintain integrity and uniformity;

[0031] Step six, obtain the computing unit of the related data surface agent in the current service data model, and then calculate the range of the data to be synchronized through the computing unit, and issue the calculated data to be synchronized; the computing unit completes the operation through the logic circuit composed of extremely small electronic components, and processes binary high and low level signals;

[0032] Step seven, according to the confirmation result of the data synchronization range, write the data to be synchronized and all the data in the kernel file system buffer of the operating system into the storage medium respectively through the standard system call function of the operating system, and generate the data synchronization command; the kernel file buffer is a certain space reserved in the memory area for temporarily storing file data during reading and writing, so as to reduce the reading frequency of the data storage medium; the storage medium is the carrier of data storage, and the amount of information stored can meet the writing of the data to be synchronized;

[0033] Step eight, according to the data synchronization instruction, synchronize the data to be synchronized and all the data to the backup database of the heterogeneous database platform in turn; wherein, the backup database is an incremental backup, which can use a new image of the entire database, which is smaller than the initial full backup, and only contains the changed database since the last full backup.

[0034] In summary: the data synchronization method of the heterogeneous database platform can confirm the data to be synchronized before entering the backup database by setting the steps of obtaining the synchronization data, the synchronization range, generating the data list, confirming the synchronization range, generating the synchronization command, etc., and generating the synchronization command after confirmation, so as to avoid the multiple synchronization of the modified data between the client and the server, reduce the waste of network resources, and improve the data synchronization efficiency of the heterogeneous database platform.

[0035] The portions of the application not described herein are the same as or can be implemented using the prior art. Although embodiments of the application have been illustrated and described, it will be clear to those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the application. The scope of the application is limited only by the claims and the equivalents thereof.

Claims

1. A data synchronization method of a heterogeneous database platform, characterized by, The method comprises the following steps: Step one, frame data is obtained from a heterogeneous database platform according to the same data structure, and frame data that needs to be updated is identified, and the identified frame data is data to be synchronized; Step two, the network cluster in which the current service is located is monitored, and the real-time deployment identifier of the data plane agent is obtained from the network cluster, and the data range to be synchronized is calculated according to the implementation deployment identifier; Step three, a plurality of logic modules perform logical operation on the identified data range to be synchronized, and then generate a list to the data end where the logic module is located; Step four, a plurality of related data plane agents are issued data according to the data range, and a data model is used to describe the calling and proxy relationship of the data through a tag selector; Step five, when the system cannot calculate the data range to be synchronized in the heterogeneous database platform, all data in the current network cluster is synchronized; Step six, the calculation unit of the related data plane agent is obtained in the data model where the current service is located, and the range of the data to be synchronized is calculated through the calculation unit, and the calculated data range to be synchronized is issued; Step seven, according to the confirmation result of the data synchronization range, the kernel file system buffer of the data to be synchronized and all data are written into the storage medium respectively through the standard system call function of the operating system to generate the command of data synchronization; Step eight, the data to be synchronized and all data are synchronized to the backup database of the heterogeneous database platform in sequence according to the data synchronization instruction.

2. The data synchronization method of the heterogeneous database platform according to claim 1, characterized in that: In step one, the frame data is composed of a frame header, a data part and a frame tail, wherein the frame header and the frame tail contain necessary control information including synchronization information; the data part mainly includes data information required to be transmitted and stored in the daily operation process of the heterogeneous database platform.

3. The data synchronization method of the heterogeneous database platform according to claim 1, characterized in that: In step two, the data plane agent can enable one frame data in the heterogeneous database platform to read and write another frame data; the cluster communication system is a mobile communication system for group dispatching command communication, which has a shared channel for all users of the system, and has an automatic channel selection function, which is a shared resource, cost sharing, shared channel equipment and service of multi-purpose and high-efficiency wireless dispatching communication system.

4. The data synchronization method of a heterogeneous database platform according to claim 1, characterized in that: In step three, the logic module can divide the internal system of the heterogeneous database platform into a plurality of logic units, and the plurality of logic units can respectively realize function determination and operation; the logic unit is a network port into the system network architecture, through which the user can access network resources.

5. The data synchronization method of a heterogeneous database platform according to claim 1, characterized in that: In step four, the data calling can define the data format, and also can call the parameters and types of service data; the data model describes the static characteristics, dynamic behavior and constraint conditions of the system, and provides an abstract framework for information representation and operation of the database system, and the content described by the data model includes three parts of data structure, data operation and data constraint.

6. The data synchronization method of a heterogeneous database platform according to claim 1, characterized in that: In step five, the data in all network clusters where the current service is located can realize information sharing, so that the data to be synchronized maintains integrity and uniformity.

7. The data synchronization method of a heterogeneous database platform according to claim 1, characterized in that: In step seven, the kernel file buffer is used to temporarily store file data during reading and writing, and a certain space is reserved in the memory area, so as to reduce the reading times of the data storage medium.

8. The data synchronization method of a heterogeneous database platform according to claim 7, characterized in that: In step seven, the storage medium is the carrier of data storage, and the amount of stored information can meet the writing of the data to be synchronized.

Images