A method and system for processing logical replication of a Caché database to heterogeneous data

Abstract

The application discloses a processing method and system for Caché database logical replication to heterogeneous data, and the method comprises the following steps: extracting field and index information of a persistent class from a Caché database class definition class by using an ODBC interface; obtaining full-amount business data corresponding to the field of the persistent class based on the field and index information of the persistent class; acquiring DML incremental data of the persistent class change stored in a journal log of the Caché database; and sending the full-amount business data and the DML incremental data to a server where a target-end heterogeneous database is located. The application can effectively replicate data in the Caché database to other data systems in real time, so as to realize fast sharing of data.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a method and system for processing Caché database logical replication to heterogeneous data. Background Art

[0002] Real-time sharing of medical data is crucial for epidemic prevention and control, medical treatment, and scientific research. It is a core requirement for improving health system efficiency, responding to emergencies, and promoting medical research. However, current technologies are inflexible in addressing these challenges, as medical information systems face heterogeneous data formats, increasing real-time requirements, and compatibility with non-open-source databases.

[0003] As a non-open-source database, Cache databases, particularly in the medical field, face a series of technical challenges in real-time sharing of medical data due to their unique storage structure, complex journal format, and limited public information. To date, no domestic technology exists that can effectively parse Cache database journals and perform logical analysis based on them to achieve real-time incremental synchronization of medical data. Summary of the Invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the present invention proposes a processing method for logically replicating a Caché database to heterogeneous data. This method obtains dictionary information of persistent classes in the Caché database and acquires the full data in the Caché database through an ODBC interface. Furthermore, it logically analyzes the Caché database journal to obtain incremental data, thereby replicating the data in the Caché database to other data systems in real time.

[0006] Another object of the present invention is to provide another method for processing Caché database logical replication to heterogeneous data.

[0007] A third object of the present invention is to provide a processing system for logically replicating a Caché database to heterogeneous data.

[0008] A fourth object of the present invention is to provide another processing system for logical replication of Caché database to heterogeneous data.

[0009] To achieve the above objectives, the present invention provides a method for processing Caché database logical replication to heterogeneous data, which is applied to the source end and includes:

[0010] Use the ODBC interface to extract the fields and index information of the persistent class from the Caché database class definition class;

[0011] Obtaining full business data of corresponding fields of the persistent class based on the fields and index information of the persistent class;

[0012] Get the incremental DML data of persistent class changes stored in the Caché database journal;

[0013] The full business data and the DML incremental data are sent to the server where the target heterogeneous database is located to complete real-time data sharing.

[0014] The processing method for logical replication of a Caché database to heterogeneous data according to the embodiment of the present invention may also have the following additional technical features:

[0015] In one embodiment of the present invention, after obtaining the full amount of service data, the method further includes:

[0016] Analyzing the full amount of business data to convert the full amount of business data into a full amount of binary files according to the full amount of data analysis result;

[0017] If the data volume of the full binary file is greater than a first preset threshold, splitting the full binary file into multiple full sub-files;

[0018] The serial numbers corresponding to the plurality of full quantum files are sequentially sorted, and the plurality of full quantum files are stored in a local disk according to the first serial number sorting result.

[0019] In one embodiment of the present invention, after obtaining the DML incremental data, the method further includes:

[0020] Analyzing the DML incremental data to convert the DML incremental data into an incremental binary file according to the incremental data analysis result;

[0021] If the data volume of the incremental binary file is greater than a second preset threshold, splitting the incremental binary file into multiple incremental sub-files;

[0022] The numbers corresponding to the plurality of incremental sub-files are sequentially sorted, and the plurality of incremental sub-files are stored in a local disk according to the second number sorting result.

[0023] In one embodiment of the present invention, sending the full business data and the DML incremental data to a server where a target heterogeneous database is located includes:

[0024] Sending the numbered multiple full quantum files to the server where the target heterogeneous database is located through the network according to the first numbering sorting result; and

[0025] After the sending of the multiple full sub-files is completed, the multiple incremental sub-files are sent to the server where the target end heterogeneous database is located according to the second number sorting result.

[0026] To achieve the above-mentioned object, the present invention further provides a method for processing Caché database logical replication to heterogeneous data, which is applied to a target end and includes:

[0027] Receive the full business data and DML incremental data sent by the source end and store them on the disk of the server where the target heterogeneous database is located;

[0028] According to the format requirements of the target heterogeneous database, the full business data and DML incremental data are sequentially loaded using the ODBC interface or the API interface of the target heterogeneous database to complete real-time data sharing.

[0029] To achieve the above-mentioned object, the present invention further provides a system for processing Caché database logical replication to heterogeneous data, comprising:

[0030] Dictionary module, used to extract persistent class fields and index information from Caché database class definition classes using the ODBC interface;

[0031] A full module is used to obtain the full business data of the corresponding fields of the persistent class based on the fields and index information of the persistent class;

[0032] The incremental module is used to obtain the incremental DML data of persistent class changes stored in the Caché database journal;

[0033] The sending module is used to send the full business data and the DML incremental data to the server where the target heterogeneous database is located to achieve real-time data sharing.

[0034] To achieve the above-mentioned object, the present invention further provides a system for processing Caché database logical replication to heterogeneous data, comprising:

[0035] The receiving module is used to receive the full business data and DML incremental data sent by the source end and store them on the disk of the server where the target heterogeneous database is located;

[0036] The loading module is used to sequentially load the full business data and DML incremental data using the ODBC interface or the API interface of the target heterogeneous database according to the format requirements of the target heterogeneous database, thereby completing real-time data sharing.

[0037] The processing method and system for logically replicating a Caché database to heterogeneous data in embodiments of the present invention obtains dictionary information of persistent classes in the Caché database and the full data in the Caché database through an ODBC interface, and logically analyzes the Caché database journal to obtain incremental data. This allows real-time replication of data from the Caché database to other data systems, effectively enabling data stream sharing among multiple databases.

[0038] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0040] Figure 1 is a flowchart of a method for processing Caché database logical replication to heterogeneous data according to an embodiment of the present invention;

[0041] Figure 2 is a flowchart of another method for processing Caché database logical replication to heterogeneous data according to an embodiment of the present invention;

[0042] Figure 3 is a structural diagram of a processing system for logical replication of a Caché database to heterogeneous data according to an embodiment of the present invention;

[0043] Figure 4 is a structural diagram of another processing system for logical replication of Caché databases to heterogeneous data according to an embodiment of the present invention;

[0044] Figure 5 4 is a structural diagram of another processing system for logical replication of a Caché database to heterogeneous data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0046] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0047] The following describes a method and system for processing Cache database logical replication to heterogeneous data, proposed according to an embodiment of the present invention, with reference to the accompanying drawings.

[0048] Figure 1 FIG. 1 is a flowchart of a method for processing Caché database logical replication to heterogeneous data according to an embodiment of the present invention. Figure 1 As shown, applied to the source end, the method includes:

[0049] S1, using the ODBC interface to extract the fields and index information of the persistent class from the Caché database class definition class.

[0050] Specifically, the present invention uses the ODBC interface to obtain the fields and index information of the persistent class from the Caché database class definition class.

[0051] As you can understand, the ODBC interface, or Open Database Connectivity, was created to facilitate data sharing between heterogeneous databases. It has become a key component of the Windows Open System Architecture (WOSA) and a database access interface standard for the Windows environment. ODBC provides a unified interface for accessing heterogeneous databases, allowing applications to access data managed by different DBMSs using SQL as the data access standard. It enables applications to directly manipulate data in the database, eliminating the need for changes to the database. ODBC can be used to access database files on various computers, even non-database objects such as Excel spreadsheets and ASCII data files.

[0052] S2: Based on the fields and index information of the persistent class, obtain the full business data of the corresponding fields of the persistent class.

[0053] Specifically, based on the acquired persistent class fields and index information, the full business data of the corresponding fields of the persistent class is obtained from the Caché database, analyzed and converted into binary files and saved to the local disk. Subsequent incremental data is then copied based on this data.

[0054] It can be understood that the present invention uses the ODBC interface to obtain the full business data of the corresponding fields of the persistent class based on the structure, fields and index information of the persistent class, analyzes and converts it into binary files and saves it to the local disk. When a single file is too large, it will be split and saved by number.

[0055] Exemplarily, after obtaining the full amount of business data, the present invention analyzes the full amount of business data to convert the full amount of business data into a full amount of binary file according to the full amount of data analysis result;

[0056] If the data volume of the full binary file is greater than a first preset threshold, splitting the full binary file into multiple full sub-files;

[0057] The serial numbers corresponding to the plurality of full quantum files are sequentially sorted, and the plurality of full quantum files are stored in a local disk according to a first serial number sorting result.

[0058] S3, obtains the DML incremental data of persistent class changes stored in the Caché database journal.

[0059] Specifically, the present invention parses the journal log of the Caché database, obtains the DML incremental data of the persistent class changes stored in the journal log, analyzes and converts it into a binary file, and stores it in a local disk for subsequent use.

[0060] It is understandable that after the above-mentioned full business data processing is completed, the journal log files of the Caché database are continuously read and parsed to obtain the DML incremental data of the persistent class changes stored in the journal logs, analyze and convert them into binary files and save them to the local disk. If a single file is too large, it will be split and saved by number.

[0061] Specifically, after obtaining the DML incremental data, the present invention analyzes the DML incremental data to convert the DML incremental data into an incremental binary file according to the incremental data analysis result;

[0062] If the data volume of the incremental binary file is greater than a second preset threshold, splitting the incremental binary file into multiple incremental sub-files;

[0063] The numbers corresponding to the plurality of incremental sub-files are sequentially sorted, and the plurality of incremental sub-files are stored in a local disk according to the second number sorting result.

[0064] Through the data conversion and file splitting of the embodiments of the present invention, even large amounts of data can be effectively managed, avoiding the problem of a single file being too large.

[0065] S4 sends the full business data and DML incremental data to the server where the target heterogeneous database is located to achieve real-time data sharing.

[0066] Specifically, the present invention sends the full data file obtained above to the receiving module of the target end through the network according to the number, and continues to send the completed incremental data files to the receiving module of the target end in the order of the numbers after the full data file is sent.

[0067] In an embodiment of the present invention, multiple numbered full-quantum files are sent to a server where a target heterogeneous database is located via a network according to a first numbering sorting result. Furthermore, after the multiple full-quantum files are sent, multiple incremental sub-files are sent to the server where the target heterogeneous database is located according to a second numbering sorting result, thereby replicating data in a Caché database to other data systems (such as Kafka, Hadoop, Oracle, MySQL, and PostgreSQL) in real time to achieve data sharing.

[0068] According to an embodiment of the present invention, a processing method for logically replicating a Caché database to heterogeneous data utilizes an ODBC interface to extract fields and index information of a persistent class from a Caché database class definition class; based on the fields and index information of the persistent class, full business data for the corresponding fields of the persistent class is obtained; incremental DML data of persistent class changes stored in the Caché database journal is obtained; and the full business data and incremental DML data are sent to a server where a target heterogeneous database is located to achieve real-time data sharing, thereby effectively realizing the forwarding and reception of data streams between databases.

[0069] In order to implement the above embodiment, Figure 2 As shown, this embodiment also provides another method for processing Caché database logical replication to heterogeneous data, which is applied to the target end and includes:

[0070] S101: Receive the full business data and DML incremental data sent by the source end and store them on the disk of the server where the target heterogeneous database is located;

[0071] S102 , according to the format requirements of the target heterogeneous database, use the ODBC interface or the API interface of the target heterogeneous database to sequentially load the full binary file and the DML incremental data respectively, to complete real-time data sharing.

[0072] It can be understood that, after the source side determines the persistent class's fields and index information from the Caché database class definition using the ODBC interface, it retrieves the full business data for the corresponding fields from the Caché database based on the retrieved persistent class's fields and index information. This data is analyzed and converted into binary files and saved to the local disk, upon which subsequent incremental data replication is performed. The Caché database's journal is parsed to obtain the incremental DML data for persistent class changes stored in the journal, which is then analyzed and converted into binary files. The target heterogeneous database server receives the full data file and, after that, sequentially receives the incremental data files.

[0073] Furthermore, after receiving the full data file and the incremental data file, they are stored on the disk of the server where the target heterogeneous database is located; according to the format requirements of the target heterogeneous database, the ODBC interface or the API interface of the target heterogeneous database is used to sequentially load the received full files and incremental files to complete real-time data sharing.

[0074] Specifically, the present invention uses the ODBC interface or the API interface provided by the target heterogeneous database to load the received full data files in numerical order according to the format requirements of the target heterogeneous database such as field type and field length, and then continues to load the received incremental data files in numerical order after completion to complete real-time data sharing.

[0075] According to the processing method for logical replication of a Caché database to heterogeneous data in an embodiment of the present invention, the full business data and DML incremental data sent by the source end are received and stored on the disk of the server where the target heterogeneous database is located; according to the format requirements of the target heterogeneous database, the full business data and DML incremental data are sequentially loaded using the ODBC interface or the API interface of the target heterogeneous database to achieve real-time data sharing, which can effectively realize the forwarding and reception of data streams between databases.

[0076] In order to implement the above embodiment, Figure 3 As shown, this embodiment also provides another processing system 10 for logical replication of Caché databases to heterogeneous data, including:

[0077] The dictionary module 100 is used to extract the fields and index information of the persistent class from the Caché database class definition class using the ODBC interface;

[0078] The full module 200 is used to obtain the full business data of the corresponding fields of the persistent class based on the fields and index information of the persistent class;

[0079] An increment module 300 is configured to acquire DML increment data of persistent class changes stored in a journal log of a Caché database;

[0080] A sending module 400 is configured to send the full-amount service data and the DML increment data to a server where a target-end heterogeneous database is located, so as to complete real-time sharing of data.

[0081] Further, the full-amount module 300 is further configured to:

[0082] analyze the full-amount service data to convert the full-amount service data into a full-amount binary file according to a full-amount data analysis result;

[0083] if a data amount of the full-amount binary file is greater than a first preset threshold, split the full-amount binary file into a plurality of full-amount sub-files;

[0084] sequentially sort numbers corresponding to the plurality of full-amount sub-files, and store the plurality of full-amount sub-files in a local disk according to a first number sorting result.

[0085] Further, the increment module 300 is further configured to:

[0086] analyze the DML increment data to convert the DML increment data into an increment binary file according to an increment data analysis result;

[0087] if a data amount of the increment binary file is greater than a second preset threshold, split the increment binary file into a plurality of increment sub-files;

[0088] sequentially sort numbers corresponding to the plurality of increment sub-files, and store the plurality of increment sub-files in a local disk according to a second number sorting result.

[0089] Further, the sending module 400 is further configured to:

[0090] send the plurality of full-amount sub-files with the numbers according to the first number sorting result to the server where the target-end heterogeneous database is located through a network; and

[0091] after the plurality of full-amount sub-files are sent, send the plurality of increment sub-files to the server where the target-end heterogeneous database is located according to the second number sorting result.

[0092] The processing system for logical replication of Caché database to heterogeneous data according to the embodiment of the present application comprises a dictionary module, an all-amount module, an incremental module and a sending module.

[0093] In order to realize the above-mentioned embodiment, as shown in the figure, Figure 4 The present embodiment further provides another processing system 20 for logical replication of Caché database to heterogeneous data, which comprises:

[0094] The receiving module 30 is configured to receive the all-amount business data and the DML incremental data issued by the source end and store them in the disk of the server where the target end heterogeneous database is located.

[0095] The loading module 40 is configured to sequentially load the all-amount business data and the DML incremental data according to the format requirement of the target end heterogeneous database by using the ODBC interface or the API interface of the target end heterogeneous database, so as to realize real-time sharing of data.

[0096] The processing system for logical replication of Caché database to heterogeneous data according to the embodiment of the present application comprises a receiving module and a loading module.

[0097] In order to realize the above-mentioned embodiment, as shown in the figure, Figure 5 The present embodiment further provides another processing system for logical replication of Caché database to heterogeneous data, which comprises:

[0098] The source end dictionary module is configured to acquire the field and index information of the persistent class from the class definition class of the Caché database by using the ODBC interface.

[0099] The source end all-amount module is configured to acquire the all-amount business data of the corresponding field of the persistent class from the Caché database according to the acquired field and index information of the persistent class, analyze and convert the all-amount business data into a binary file and save the binary file to the local disk, and then perform subsequent replication of incremental data on the basis.

[0100] The source-side incremental module parses the Caché database journal logs, obtains the DML incremental data of persistent class changes stored in the journal logs, analyzes and converts it into binary files, and stores them on the local disk for subsequent transmission by the sending module.

[0101] The source-side sending module sends the full data file to the receiving module of the server where the target-side heterogeneous database is located, and then sends the incremental data files sequentially after it is completed;

[0102] The target-side receiving module stores the received full and incremental data files on the disk of the server where the target-side heterogeneous database is located.

[0103] The target-side loading module uses the ODBC interface or the API interface of the target-side heterogeneous database to sequentially load the received full files and incremental files according to the format requirements of the target-side heterogeneous database, thereby completing real-time data sharing.

[0104] According to an embodiment of the present invention, a processing system for logically replicating a Caché database to heterogeneous data obtains dictionary information of persistent classes in the Caché database through an ODBC interface, acquires full data in the Caché database, and logically analyzes the Caché database journal to obtain incremental data, thereby replicating data in the Caché database to other data systems in real time.

[0105] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0106] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

Claims

1. A method for processing Caché database logical replication to heterogeneous data, characterized in that: Applied to a source end in a medical information system, the method includes: Use the ODBC interface to extract the fields and index information of the persistent class from the Caché database class definition class; Based on the fields and index information of the persistent class, obtaining full business data of the corresponding fields of the persistent class from the Caché database, converting the full business data into a full binary file, determining whether the data volume of the full binary file is greater than a first preset threshold, and if so, splitting the full binary file into multiple full quantum files, sequentially sorting the files according to the numbers corresponding to each full quantum file, and storing the numbered full quantum files on a local disk according to the sorting result; Obtaining DML incremental data of persistent class changes stored in the journal log of the Caché database, converting the DML incremental data into an incremental binary file, and determining whether the data volume of the incremental binary file is greater than a second preset threshold. If so, splitting the incremental binary file into multiple incremental sub-files, sorting the sub-files sequentially according to the numbers corresponding to each incremental sub-file, and storing the multiple incremental sub-files on the local disk according to the sorting result; According to the sorting results of the multiple full quantum files and the multiple incremental sub-files, the multiple full quantum files and the multiple incremental sub-files are sequentially and continuously sent to the disk of the server where the target heterogeneous database is located, so that the target heterogeneous database uses the ODBC interface or the API interface of the target heterogeneous database according to the format requirements to sequentially and continuously load the received full files and incremental files respectively, thereby completing real-time sharing of data streams.

2. The method according to claim 1, characterized in that Sending the full business data and the DML incremental data to the server where the target heterogeneous database is located includes: Sending the numbered full quantum files to the disk of the server where the target heterogeneous database is located through the network according to the sorting results of the multiple full quantum files; and After the sending of the plurality of full quantum subfiles is completed, the plurality of incremental subfiles are sent to the disk of the server where the target heterogeneous database is located according to the sorting result of the plurality of incremental subfiles.

3. A method for processing Caché database logical replication to heterogeneous data, characterized in that: Applied to a target end in a medical information system, the method includes: Receiving the multiple full quantum files and multiple incremental sub-files as described in claim 1 or 2 sent by the source end, and storing them on the disk of the server where the heterogeneous database on the target end is located; According to the format requirements of the target heterogeneous database, the full business data and DML incremental data are sequentially loaded using the ODBC interface or the API interface of the target heterogeneous database to complete real-time sharing of data streams.

4. A processing system for logical replication of a Caché database to heterogeneous data, characterized in that: Applied to a source end in a medical information system, the system comprising: Dictionary module, used to extract persistent class fields and index information from Caché database class definition classes using the ODBC interface; a full module, configured to obtain, from the Caché database, full business data of the corresponding fields of the persistent class based on the fields and index information of the persistent class, convert the full business data into a full binary file, determine whether the data volume of the full binary file is greater than a first preset threshold, and if so, split the full binary file into multiple full sub-files, sort the files sequentially according to the numbers corresponding to each full sub-file, and store the numbered full sub-files on a local disk according to the sorting results; an incremental module for obtaining DML incremental data of persistent class changes stored in the journal log of the Caché database, converting the DML incremental data into an incremental binary file, determining whether the data volume of the incremental binary file is greater than a second preset threshold, and if so, splitting the incremental binary file into multiple incremental sub-files, sorting the sub-files sequentially according to the numbers corresponding to each incremental sub-file, and storing the multiple incremental sub-files on the local disk based on the sorting result; The sending module is used to sequentially and continuously send the multiple full quantum files and the multiple incremental sub-files to the disk of the server where the target heterogeneous database is located according to the sorting results of the multiple full quantum files and the multiple incremental sub-files, so that the target heterogeneous database uses the ODBC interface or the API interface of the target heterogeneous database according to the format requirements to sequentially and continuously load the received full files and incremental files respectively, thereby completing real-time sharing of data streams.

5. The system according to claim 4, characterized in that The sending module is further used for: Sending the numbered full quantum files to the disk of the server where the target heterogeneous database is located through the network according to the sorting results of the multiple full quantum files; and After the sending of the plurality of full quantum subfiles is completed, the plurality of incremental subfiles are sent to the disk of the server where the target heterogeneous database is located according to the sorting result of the plurality of incremental subfiles.

6. A processing system for logical replication of a Caché database to heterogeneous data, characterized in that: Applied to a target end in a medical information system, the system comprising: A receiving module, configured to receive the multiple full quantum files and multiple incremental sub-files as described in claim 1 or 2 sent by the source end, and store them on the disk of the server where the heterogeneous database on the target end is located; The loading module is used to sequentially load the full business data and DML incremental data using the ODBC interface or the API interface of the target heterogeneous database according to the format requirements of the target heterogeneous database, thereby completing real-time sharing of data streams.

Images