Data synchronization method between heterogeneous databases for database localization substitution

Abstract

The embodiments of the present application relate to the field of data synchronization technology, and in particular to a data synchronization method between heterogeneous databases for database domestication replacement, including: determining the source database of the information system and selecting a new domestic database; exporting the table structure in the source database as a table creation statement compiled in the database common language, and executing it in the new domestic database so that the two have the same table structure; judging whether the source database can be shut down; if it can be shut down, shut down the source database, write a first application, and use the first application to complete data synchronization; if it cannot be shut down, write a second application, use the second application, synchronize the data at and before the first time point according to the shutdown method, and synchronize the data after the first time point by extracting logs. Through this method, data synchronization between heterogeneous databases can be achieved efficiently, quickly, accurately, and at low cost.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of data synchronization technology, and in particular, to a method for synchronizing data between heterogeneous databases for localization replacement of databases. Background Art

[0002] With the acceleration of digital transformation and the rapid development of information technology, the degree of digitalization in various industries is getting higher and higher, and the information systems that carry digital business are also experiencing diversified development. With this comes the construction, upgrading, and transformation of information systems. In these tasks, databases and database software that provide data computing, storage, processing and other services are an object that cannot be ignored.

[0003] During the work of domestic database replacement, microservice transformation and upgrading, and architecture optimization, the original data in the source database needs to be preserved and synchronized (migrated) to the new domestic database so as to continue to provide data services and meet various business requirements.

[0004] Most of the data synchronization (data migration) methods proposed so far are for isomorphic databases, which have the same data format and can be directly synchronized. However, when replacing databases with local ones, technicians found that there was a large heterogeneous gap between foreign and domestic databases, and data synchronization could not be performed directly. The only way was to rely on technicians to organize and reconstruct the data in the source database (old database) and then store it in the new domestic database, which was time-consuming, labor-intensive, and not smart enough.

[0005] The Chinese invention patent with application number 202210208292.2 discloses a data migration method and system for domestic substitution. It establishes a connection with the source database through an API interface (Application Programming Interface), calls the metadata interface to obtain a result set, processes the result set to obtain information about each table in the source database, and then extracts the data information from the source data table and stores it in a JSON (JavaScript Object Notation) file. The data is converted according to the data type conversion method, and a table creation SQL (Structured Query Language) statement is generated. The connection is made to the destination database, the table structure is created, and the data information in the JSON file is parsed. Finally, data is inserted into the created new table to complete the data migration, thereby enabling fast, correct, and efficient smooth migration of database structure, content, and its system, which not only improves data management efficiency but also ensures data security.

[0006] However, the essence of this method is to shut down the source database, and then achieve data migration by calling the API interface, converting data types, creating tables, and inserting data, without considering the business needs of the information system. If the source database cannot be shut down, this method cannot be used for data migration, which will not meet the needs of domestically replacing databases deployed with critical businesses. In addition, when the amount of data is very large, this method requires a lot of time to extract data from the source database, which is costly. Summary of the invention

[0007] In view of this, the embodiments of the present application propose a data synchronization method between heterogeneous databases for domestic database replacement, aiming to achieve data synchronization between heterogeneous databases efficiently, quickly, accurately and at low cost. When the source database cannot be shut down, additional data synchronization can also be achieved, which is conducive to the domestic database replacement of the entire industry.

[0008] In order to achieve the above-mentioned purpose, an embodiment of the present application provides a data synchronization method between heterogeneous databases for database domestication replacement, including: determining a source database of an information system, and selecting a new domestic database according to the business needs of the information system, where the source database and the new domestic database are heterogeneous; exporting the table structure in the source database as a table creation statement compiled in a database common language, and executing it in the new domestic database, so that the new domestic database has the same table structure as the source database; judging whether the source database can be shut down according to the business needs, business characteristics and source database characteristics of the information system; if the source database can be shut down, shutting down the source database, writing a first application, and using the first application The data in each table in the source database is uniformly exported as data insertion statements written in the database common language, and executed in the new domestic database in sequence to complete the data synchronization between the source database and the new domestic database; if the source database cannot be shut down, a second application is written, and the second application is used to export the data at the first time point and before the first time point in each table in the source database based on the set first time point as data insertion statements written in the database common language, and executed in the new domestic database in sequence, and after the first time point, the log of the server where the source database is located is extracted, and based on the log of the server where the source database is located, additional synchronization of the data after the first time point is performed.

[0009] In order to achieve the above-mentioned purpose, the embodiment of the present application also provides a data synchronization system between heterogeneous databases for domestic database replacement, including: a database selection module, which is used to determine the source database of the information system, and select a new domestic database according to the business needs of the information system, and the source database and the new domestic database are heterogeneous; a table structure synchronization module, which is used to uniformly export the table structure in the source database into table creation statements compiled in the database common language, and execute them in the new domestic database, so that the new domestic database has the same table structure as the source database; an outage judgment module, which is used to judge whether the source database can be shut down according to the business needs, business characteristics and source database characteristics of the information system; an outage data synchronization module, which is used to shut down the source database when the source database can be shut down, and write A first application is used to uniformly export the data in each table in the source database into data insert statements written in the database common language, which are executed in turn in the new domestic database to complete data synchronization between the source database and the new domestic database; a non-stop data synchronization module is used to write a second application when the source database cannot be shut down, and the second application is used to uniformly export the data at the first time point and before the first time point in each table in the source database based on a set first time point as data insert statements written in the database common language, which are executed in turn in the new domestic database, and after the first time point, the log of the server where the source database is located is extracted, and based on the log of the server where the source database is located, additional synchronization of the data after the first time point is performed.

[0010] In order to achieve the above-mentioned purpose, an embodiment of the present application also provides a server, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data synchronization method between heterogeneous databases for database domestication replacement as described above.

[0011] In order to achieve the above-mentioned purpose, an embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the data synchronization method between heterogeneous databases for database domestication replacement as described above.

[0012] The data synchronization method between heterogeneous databases for database localization replacement proposed in the embodiment of the present application is generally divided into two parts for data synchronization between heterogeneous databases. First, the table structure between the two databases is synchronized, and then the data in the source database is synchronized to the new domestic database. When performing data synchronization, the source database is judged to be out of service according to the business needs, business characteristics and source database characteristics of the information system. In the case where the source database can be shut down, a first application is written, and the data in each table in the source database is uniformly exported as a data insertion statement compiled in the database common language by the first application, and is executed in the new domestic database in sequence to complete the data synchronization between the source database and the new domestic database. In the case where the source database cannot be shut down, a second application is written, and the second application is used to synchronize the existing data in the manner of shutdown synchronization, and then the data that has not arrived is synchronized by extracting logs and performing log analysis. This method can realize data synchronization between heterogeneous databases efficiently, quickly, accurately and at low cost. Even if the source database cannot be shut down, data can be synchronized by extracting logs and performing log analysis, which is helpful for the localization replacement of databases in the entire industry and provides an effective means for data security and network security.

[0013] Optionally, after a first time point, the log of the server where the source database is located is extracted, and based on the log of the server where the source database is located, additional synchronization of the data after the first time point is performed, including: after the first time point, creating several sub-processes, different sub-processes are responsible for additional synchronization of data in different tables of the source database, and each sub-process is executed concurrently; using a second application to extract the log of the server where the source database is located, performing log analysis on the log of the server where the source database is located, extracting operation statements for the source database, and separating operation statements after the first time point; converting the operation statements after the first time point into database operation statements compiled in a common database language, and executing them in a new domestic database to perform additional synchronization of the data after the first time point; comparing the tables and data between the source database and the new domestic database, if the two are consistent, confirming that the data synchronization is completed, otherwise, setting a second time point, and performing a new round of additional synchronization until the tables and data between the source database and the new domestic database are consistent.

[0014] Optionally, if the amount of data in the source database is greater than a preset processing threshold, several first applications and several second applications are written to perform high-concurrency data synchronization; wherein different first applications and different second applications are deployed on different servers, or different first applications and different second applications are deployed on the same server but use different port numbers to bind different network cards of the server, thereby maximizing the utilization of network input and output and maximizing the utilization of server hardware resources.

[0015] Optionally, judging whether the source database can be shut down is performed based on the business requirements, business characteristics and source database characteristics of the information system, including: determining a business requirement score based on the business requirements of the information system, where the business requirements of the information system are transactional business requirements or analytical business requirements; determining a business characteristic score based on the business characteristics of the information system, where the business characteristics of the information system include processing volume, processing frequency, processing time, praise rate, complaint rate and queuing time; determining a source database characteristic score based on the source database characteristics, where the source database characteristics are described by performance indicators, capacity indicators, availability indicators, security indicators and efficiency indicators; determining a shutdown score based on the business requirement score, business characteristic score and source database characteristic score, where if the shutdown score is lower than a preset shutdown threshold, shutdown is not allowed, and if the shutdown score is not lower than the preset shutdown threshold, shutdown is allowed.

[0016] Optionally, based on the business requirement score, the business feature score and the source database feature score, the outage score is determined by the following formula:

[0017] ;

[0018] ;

[0019] ;

[0020] ;

[0021] in, Indicates the outage score, represents the business need score, represents the business feature score, represents the source database feature score, , and They represent the preset first outage weight coefficient, the second outage weight coefficient and the third outage weight coefficient respectively. When the business demand of the information system is a transactional business demand, , Indicates the score of transactional business requirements. When the business requirements of the information system are analytical business requirements, , , Indicates the analytical business needs score, Indicates the handling volume score. Indicates the frequency score. Indicates the time spent on processing. Indicates the praise rate score, represents the complaint rate score, represents the queue time score, , , , , and respectively represent a preset first service characteristic weight coefficient, a second service characteristic weight coefficient, a third service characteristic weight coefficient, a fourth service characteristic weight coefficient, a fifth service characteristic weight coefficient and a sixth service characteristic weight coefficient, represents the performance index score, represents the capacity index score, represents the usability index score, represents the safety index score, represents the efficiency index score, , , , and They respectively represent the preset first source database characteristic weight coefficient, the second source database characteristic weight coefficient, the third source database characteristic weight coefficient, the fourth source database characteristic weight coefficient and the fifth source database characteristic weight coefficient.

[0022] Optionally, the first application and the second application are written in Java language.

[0023] Optionally, the business requirements of the information system are divided into transactional business requirements and analytical business requirements; transactional business requirements include system login services, archive storage services, remote fee control services, remote recharge services, terminal login services, terminal online rate query services, and fee control recharge status query services; for transactional business requirements, it is necessary to select a transactional domestic database as a new domestic database, and the transactional domestic databases include GaussDB database, OceanBase database, DAMO database, KingBase database, and TDSQL database; analytical business requirements include historical services, historical query services, collection success rate query services, substation line loss query services, metering anomaly monitoring services, collection completeness rate query services, distribution transformer load analysis services, and external real-time publishing services; for analytical business requirements, it is necessary to select an analytical domestic database as a new domestic database, and the analytical domestic databases include DWS database, GBase database, and TDSQL-A database. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related technologies, the drawings required for use in the embodiments of the present application or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 It is a flow chart of a data synchronization method between heterogeneous databases for database localization replacement provided in one embodiment of the present application;

[0026] Figure 2 It is a structural diagram of a data synchronization system between heterogeneous databases for database localization replacement provided in another embodiment of the present application;

[0027] Figure 3 It is a structural diagram of a server provided in another embodiment of the present application. DETAILED DESCRIPTION

[0028] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented. The division of the following embodiments is for the convenience of description, and the specific implementation of the present application should not be construed as any limitation, and the various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0029] In order to solve the technical problem that the currently proposed data synchronization between heterogeneous databases does not take into account the business needs of the information system, an embodiment of the present application proposes a data synchronization method between heterogeneous databases for database localization replacement, which is applied to the server. The implementation details of the data synchronization method between heterogeneous databases for database localization replacement proposed in this embodiment are specifically described below. The following content is only the implementation details provided for the convenience of understanding and is not necessary for the implementation of this solution.

[0030] The specific process of the data synchronization method between heterogeneous databases for database localization substitution proposed in this embodiment can be as follows: Figure 1 As shown, including:

[0031] Step 101, determine the source database of the information system, and select a new domestic database according to the business requirements of the information system, the source database and the new domestic database are heterogeneous.

[0032] In the specific implementation, when the server replaces the database of the information system with a local one, it first needs to determine the source database of the information system and select a new local database based on the business needs of the information system. It is worth noting that due to technical barriers and other reasons, the source database and the new local database are heterogeneous.

[0033] In one example, the business requirements of an information system are divided into two categories: transactional business requirements and analytical business requirements.

[0034] Transactional business requirements include system login services, file storage services, remote fee control services, remote recharge services, terminal login services, terminal online rate query services and fee control and recharge status query services.

[0035] For transactional business needs, you need to choose a transactional domestic database as the new domestic database. The available transactional domestic databases include GaussDB database, OceanBase database, DAMO database, KingBase database and TDSQL database.

[0036] Analytical business needs include historical business, historical query business, collection success rate query business, substation line loss query business, metering anomaly monitoring business, collection completeness rate query business, distribution transformer load analysis business and external real-time publishing business.

[0037] For analytical business needs, it is necessary to select an analytical domestic database as a new domestic database. The available analytical domestic databases include DWS database, GBase database and TDSQL-A database.

[0038] Step 102, export the table structure in the source database into a table creation statement compiled in the database common language, and execute it in the new domestic database, so that the new domestic database has the same table structure as the source database.

[0039] In the specific implementation, after the server selects a new domestic database, it needs to synchronize the table structure first, that is, export the table structure in the source database into a table creation statement compiled in the database common language, and then execute it in the new domestic database, so that the new domestic database has the same table structure as the source database.

[0040] In one example, the common database language is generally SQL (Structured Query Language), and can also be Python, Java, etc.

[0041] Step 103, judging whether the source database can be shut down according to the business requirements, business characteristics and source database characteristics of the information system, if so, executing step 104, otherwise, executing step 105.

[0042] In the specific implementation, after the server completes the synchronization of the table structure, it can enter the data synchronization stage. Data synchronization is divided into two types: data synchronization with shutdown and data synchronization without shutdown, which depends on whether the source database can be shut down. The server needs to determine whether the source database can be shut down based on the business requirements, business characteristics and source database characteristics of the information system.

[0043] In one example, the server first determines the business requirement score based on the business requirements of the information system. The business requirements of the information system are transactional business requirements or analytical business requirements. For analytical business requirements, they are important but not urgent, and generally can be stopped, while transactional business requirements are important and urgent, and need to provide services to users continuously, and generally cannot be stopped.

[0044] Subsequently, the server determines the business characteristic score based on the business characteristics of the information system, where the business characteristics of the information system include handling volume, handling frequency, handling time, favorable rate, complaint rate and queue time. Generally speaking, the source database corresponding to the business characteristics of low handling volume, low handling frequency, short handling time, high favorable rate, low complaint rate and short queue time can be shut down.

[0045] After that, the server determines the source database characteristic score based on the source database characteristics, where the source database characteristics are described by performance indicators, capacity indicators, availability indicators, security indicators, and efficiency indicators. The worse these indicators are, the more likely the outage will fail, so generally no outage will occur.

[0046] Finally, the server determines the outage score of the source database based on the business demand score, the business feature score and the source database feature score. If the outage score is lower than the preset outage threshold, the outage is not allowed. If the outage score is not lower than the preset outage threshold, the outage is allowed. The preset outage threshold can be set by a person skilled in the art according to actual needs.

[0047] It is worth noting that the business characteristics of the information system have additional impacts on whether to suspend operations. If the volume of processing is too large, the frequency of processing is too high, the processing time is too long, the praise rate is too low, the complaint rate is too high, and the waiting time is too long, then suspension will not be allowed.

[0048] In one example, the server determines the outage score based on the business requirement score, the business feature score, and the source database feature score, which can be implemented by the following formula:

[0049] ;

[0050] ;

[0051] ;

[0052] ;

[0053] in, Indicates the outage score, Indicates the business need score, represents the business feature score, represents the source database feature score, , and They represent the preset first outage weight coefficient, the second outage weight coefficient and the third outage weight coefficient respectively. When the business demand of the information system is a transactional business demand, , Indicates the score of transactional business requirements. When the business requirements of the information system are analytical business requirements, , , Indicates the analytical business needs score, Indicates the handling volume score. Indicates the frequency score. Indicates the time spent on processing. Indicates the praise rate score, represents the complaint rate score, represents the queue time score, , , , , and respectively represent a preset first service characteristic weight coefficient, a second service characteristic weight coefficient, a third service characteristic weight coefficient, a fourth service characteristic weight coefficient, a fifth service characteristic weight coefficient and a sixth service characteristic weight coefficient, represents the performance index score, represents the capacity index score, represents the usability index score, represents the safety index score, represents the efficiency index score, , , , and They respectively represent the preset first source database characteristic weight coefficient, the second source database characteristic weight coefficient, the third source database characteristic weight coefficient, the fourth source database characteristic weight coefficient and the fifth source database characteristic weight coefficient.

[0054] Step 104, shut down the source database, write a first application, use the first application to uniformly export the data in each table in the source database into data insertion statements compiled in the database common language, and execute them in the new domestic database in turn to complete data synchronization between the source database and the new domestic database.

[0055] In a specific implementation, after the server determines that the source database can be shut down, it immediately shuts down the source database, writes a first application (which can be written automatically by the server or manually by a technician), and uses the first application to uniformly export the data in each table in the source database into data insertion statements written in the database common language, and executes them in the new domestic database in turn to complete data synchronization between the source database and the new domestic database.

[0056] In one example, the first application is written using the Java language.

[0057] In one example, the database universal language is selected as SQL language.

[0058] In one example, if the amount of data in the source database is greater than a preset processing threshold, it is necessary to write several first applications to perform high-concurrency data synchronization, thereby improving the speed and efficiency of data synchronization. Among them, the preset processing threshold can be set by a technician in this field according to actual needs. Different first applications are deployed on different servers to prevent conflicts and interferences, or different first applications are deployed on the same server, but different port numbers are used to bind different network cards of the server, so that while avoiding conflicts and interferences, the maximum utilization of network input and output and the maximum utilization of server hardware resources are achieved.

[0059] Step 105, write a second application, and use the second application to export the data at the first time point and before the first time point in each table in the source database as data insert statements compiled in the common language of the database, based on the set first time point, and execute them in the new domestic database in sequence. After the first time point, extract the log of the server where the source database is located, and based on the log of the server where the source database is located, perform additional synchronization of the data after the first time point.

[0060] In the specific implementation, after the server determines that the source database cannot be shut down, it writes a second application (which can be automatically written by the server or manually written by a technician). Using the second application, based on the set first time point, the data at the first time point and before the first time point in each table in the source database are uniformly exported as data insertion statements compiled in the database common language, and are executed in turn in the new domestic database, that is, the data at the first time point and before the first time point are synchronized according to the shutdown method. After the first time point, the server extracts the log of the server where the source database is located, analyzes it based on the log of the server where the source database is located, and performs additional synchronization of the data after the first time point.

[0061] In one example, after a first time point, the server creates several sub-processes, and different sub-processes are responsible for appending and synchronizing data in different tables of the source database, and each sub-process is executed concurrently, thereby effectively improving the speed and efficiency of data synchronization. The server uses a second application to extract the log of the server where the source database is located, performs log analysis on the log of the server where the source database is located, extracts the operation statements for the source database, and separates the operation statements after the first time point, converts the operation statements after the first time point into database operation statements compiled in the database common language, and executes them in the new domestic database to perform appending and synchronization of data after the first time point. After completing an appending synchronization of data, the server needs to compare the tables and data between the source database and the new domestic database. If the tables and data between the source database and the new domestic database are consistent, then confirm that the data synchronization is completed. Otherwise, set a second time point and perform a new round of appending and synchronization until the tables and data between the source database and the new domestic database are consistent.

[0062] In one example, the second application is written in Java.

[0063] In one example, the database universal language is selected as SQL language.

[0064] In one example, if the amount of data in the source database is greater than a preset processing threshold, it is necessary to write several second applications to perform high-concurrency data synchronization, thereby improving the speed and efficiency of data synchronization. The preset processing threshold can be set by a technician in this field according to actual needs. Different second applications are deployed on different servers to prevent conflicts and interferences, or different second applications are deployed on the same server, but different port numbers are used to bind different network cards of the server, so that while avoiding conflicts and interferences, the maximum utilization of network input and output and the maximum utilization of server hardware resources are achieved.

[0065] In this embodiment, data synchronization between heterogeneous databases is divided into two parts. First, the table structure between the two databases is synchronized, and then the data in the source database is synchronized to the new domestic database. When performing data synchronization, the source database is judged to be out of service according to the business needs, business characteristics and source database characteristics of the information system. In the case where the source database can be shut down, a first application is written, and the data in each table in the source database is uniformly exported as a data insertion statement compiled in the database common language by the first application, and is executed in the new domestic database in sequence to complete the data synchronization between the source database and the new domestic database. In the case where the source database cannot be shut down, a second application is written, and the second application is used to synchronize the existing data in the way of shutdown synchronization, and then the data that has not arrived is synchronized by extracting logs and analyzing logs. This method can realize data synchronization between heterogeneous databases efficiently, quickly, accurately and at low cost. Even if the source database cannot be shut down, data can be synchronized by extracting logs and performing log analysis, which is helpful for the localization of databases in the entire industry and provides an effective means for data security and network security.

[0066] The step division of the above methods is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application; adding insignificant modifications to the algorithm or process or introducing insignificant designs without changing the core design of the algorithm and process are all within the scope of protection of this application.

[0067] Another embodiment of the present application proposes a data synchronization system between heterogeneous databases for domestic database replacement. The details of the data synchronization system between heterogeneous databases for domestic database replacement proposed in this embodiment are described in detail below. The following content is only the implementation details provided for easy understanding and is not necessary for the implementation of this example.

[0068] Figure 2 This is a schematic diagram of a data synchronization system structure between heterogeneous databases for domestic database replacement proposed in this embodiment, including: a database selection module 201, a table structure synchronization module 202, an outage judgment module 203, an outage data synchronization module 204 and a non-outage data synchronization module 205.

[0069] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by this application, but this does not mean that there are no other units in this embodiment.

[0070] It is not difficult to find that this embodiment is a system embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiments.

[0071] Another embodiment of the present application proposes a server, whose specific structure can be as follows: Figure 3 As shown, it includes: at least one processor 301; and a memory 302 that is communicatively connected to the at least one processor 301; wherein the memory 302 stores instructions that can be executed by the at least one processor 301, and the instructions are executed by the at least one processor 301 so that the at least one processor 301 can execute a data synchronization method between heterogeneous databases for database localization replacement described in the above-mentioned method embodiments.

[0072] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators and power management circuits together, which are all well known in the art, so this article will not describe them further. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the processor.

[0073] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0074] Another embodiment of the present application proposes a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement a data synchronization method between heterogeneous databases for database domestication replacement as described in the above method embodiments.

[0075] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, ROM (Read-Only Memory), RAM (Random Access Memory), disk or optical disk and other media that can store program codes.

[0076] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A data synchronization method between heterogeneous databases for database localization substitution, characterized in that: include: Determine the source database of the information system and select a new domestic database based on the business needs of the information system. The source database and the new domestic database are heterogeneous. Export the table structure in the source database as a table creation statement in the database common language, and execute it in the new domestic database, so that the new domestic database has the same table structure as the source database; Determine whether the source database can be shut down based on the business requirements, business characteristics, and source database characteristics of the information system; If the source database can be shut down, shut down the source database, write a first application program, and use the first application program to uniformly export the data in each table in the source database into data insertion statements compiled in a database common language, and execute them in the new domestic database in sequence to complete data synchronization between the source database and the new domestic database; If the source database cannot be shut down, a second application is written. The second application is used to export the data at the first time point and before the first time point in each table in the source database as a data insert statement compiled in the database common language, based on the set first time point. The statements are executed in the new domestic database in sequence. After the first time point, the log of the server where the source database is located is extracted, and the data after the first time point is appended and synchronized based on the log of the server where the source database is located. Determine whether the source database can be shut down based on the business requirements, business characteristics, and source database characteristics of the information system, including: Determine the business requirement score based on the business requirements of the information system. The business requirements of the information system are transactional business requirements or analytical business requirements. Determine the business characteristic score based on the business characteristics of the information system. The business characteristics of the information system include the processing volume, processing frequency, processing time, favorable comment rate, complaint rate and queuing time; Determine the source database characteristic score based on the source database characteristics. The source database characteristics are described by performance indicators, capacity indicators, availability indicators, security indicators, and efficiency indicators. Determine the outage score based on the business requirement score, business feature score, and source database feature score. If the outage score is lower than the preset outage threshold, outage is not allowed. If the outage score is not lower than the preset outage threshold, outage is allowed. The outage score is determined based on the business requirement score, business feature score, and source database feature score, using the following formula: ； ； ； ； in, Indicates the outage score, represents the business need score, represents the business feature score, represents the source database feature score, , and They represent the preset first outage weight coefficient, the second outage weight coefficient and the third outage weight coefficient respectively. When the business demand of the information system is a transactional business demand, , Indicates the score of transactional business requirements. When the business requirements of the information system are analytical business requirements, , , Indicates the analytical business needs score, Indicates the handling volume score. Indicates the frequency score. Indicates the time spent on processing. Indicates the praise rate score, represents the complaint rate score, represents the queue time score, , , , , and respectively represent a preset first service characteristic weight coefficient, a second service characteristic weight coefficient, a third service characteristic weight coefficient, a fourth service characteristic weight coefficient, a fifth service characteristic weight coefficient and a sixth service characteristic weight coefficient, represents the performance index score, represents the capacity index score, represents the usability index score, represents the safety index score, represents the efficiency index score, , , , and They respectively represent the preset first source database characteristic weight coefficient, the second source database characteristic weight coefficient, the third source database characteristic weight coefficient, the fourth source database characteristic weight coefficient and the fifth source database characteristic weight coefficient.

2. The data synchronization method between heterogeneous databases for database localization substitution according to claim 1 is characterized in that: After the first time point, extract the log of the server where the source database is located, and perform additional synchronization of the data after the first time point based on the log of the server where the source database is located, including: After the first time point, several sub-processes are created. Different sub-processes are responsible for appending and synchronizing data in different tables of the source database. Each sub-process is executed concurrently. Extracting logs of the server where the source database is located by using the second application program, performing log analysis on the logs of the server where the source database is located, extracting operation statements on the source database, and separating operation statements after the first time point; Convert the operation statements after the first time point into database operation statements compiled in the database common language, and execute them in the new domestic database to perform additional synchronization of the data after the first time point; Compare the tables and data between the source database and the new domestic database. If the two are consistent, confirm that the data synchronization is completed. Otherwise, set a second time point and perform a new round of additional synchronization until the tables and data between the source database and the new domestic database are consistent.

3. The data synchronization method between heterogeneous databases for database localization substitution according to claim 2 is characterized in that: If the amount of data in the source database is greater than a preset processing threshold, several first applications and several second applications are written to perform high-concurrency data synchronization; wherein different first applications and different second applications are deployed on different servers, or different first applications and different second applications are deployed on the same server but use different port numbers to bind different network cards of the server, thereby maximizing the utilization of network input and output and maximizing the utilization of server hardware resources.

4. The data synchronization method between heterogeneous databases for database localization substitution according to any one of claims 1 to 3, characterized in that: The first application and the second application are written in Java language.

5. The data synchronization method between heterogeneous databases for database localization substitution according to any one of claims 1 to 3, characterized in that: The business requirements of information systems are divided into transactional business requirements and analytical business requirements; Transactional business requirements include system login, file storage, remote fee control, remote recharge, terminal login, terminal online rate query, and fee control and recharge status query. For transactional business needs, you need to choose a transactional domestic database as the new domestic database. Transactional domestic databases include GaussDB, OceanBase, DAMO, KingBase and TDSQL. Analytical business requirements include historical business, historical query business, data collection success rate query business, substation line loss query business, metering anomaly monitoring business, data collection completeness rate query business, distribution transformer load analysis business and external real-time publishing business; For analytical business needs, it is necessary to select analytical domestic databases as new domestic databases. Analytical domestic databases include DWS database, GBase database and TDSQL-A database.

6. A data synchronization system between heterogeneous databases for database localization substitution, characterized in that: include: The database selection module is used to determine the source database of the information system and select a new domestic database based on the business needs of the information system. The source database and the new domestic database are heterogeneous. The table structure synchronization module is used to export the table structure in the source database into table creation statements written in the database common language and execute them in the new domestic database, so that the new domestic database has the same table structure as the source database; The outage determination module is used to determine whether the source database can be shut down according to the business requirements, business characteristics and source database characteristics of the information system; The shutdown data synchronization module is used to shut down the source database when the source database can be shut down, write a first application program, and use the first application program to uniformly export the data in each table in the source database into data insertion statements compiled in a database common language, and execute them in the new domestic database in turn to complete data synchronization between the source database and the new domestic database; The non-stop data synchronization module is used to write a second application program when the source database cannot be shut down. The second application program is used to export the data at the first time point and before the first time point in each table in the source database as a data insertion statement compiled in the database common language based on the set first time point, and execute them in the new domestic database in sequence. After the first time point, the log of the server where the source database is located is extracted, and the data after the first time point is appended and synchronized based on the log of the server where the source database is located. Determine whether the source database can be shut down based on the business requirements, business characteristics, and source database characteristics of the information system, including: Determine the business requirement score based on the business requirements of the information system. The business requirements of the information system are transactional business requirements or analytical business requirements. Determine the business characteristic score based on the business characteristics of the information system. The business characteristics of the information system include the processing volume, processing frequency, processing time, favorable comment rate, complaint rate and queuing time; Determine the source database characteristic score based on the source database characteristics. The source database characteristics are described by performance indicators, capacity indicators, availability indicators, security indicators, and efficiency indicators. Determine the outage score based on the business requirement score, business feature score, and source database feature score. If the outage score is lower than the preset outage threshold, outage is not allowed. If the outage score is not lower than the preset outage threshold, outage is allowed. The outage score is determined based on the business requirement score, business feature score, and source database feature score, using the following formula: ； ； ； ； in, Indicates the outage score, represents the business need score, represents the business feature score, represents the source database feature score, , and They represent the preset first outage weight coefficient, the second outage weight coefficient and the third outage weight coefficient respectively. When the business demand of the information system is a transactional business demand, , Indicates the score of transactional business requirements. When the business requirements of the information system are analytical business requirements, , , Indicates the analytical business needs score, Indicates the handling volume score. Indicates the frequency score. Indicates the time spent on processing. Indicates the praise rate score, represents the complaint rate score, represents the queue time score, , , , , and respectively represent a preset first service characteristic weight coefficient, a second service characteristic weight coefficient, a third service characteristic weight coefficient, a fourth service characteristic weight coefficient, a fifth service characteristic weight coefficient and a sixth service characteristic weight coefficient, represents the performance index score, represents the capacity index score, represents the usability index score, represents the safety index score, represents the efficiency index score, , , , and They respectively represent the preset first source database characteristic weight coefficient, the second source database characteristic weight coefficient, the third source database characteristic weight coefficient, the fourth source database characteristic weight coefficient and the fifth source database characteristic weight coefficient.

7. A server, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data synchronization method between heterogeneous databases for database localization replacement as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it is possible to implement the data synchronization method between heterogeneous databases for database localization substitution as described in any one of claims 1 to 5.

Images