Method and apparatus for processing operation of heterogeneous database, electronic device and storage medium

Abstract

The application provides an operation processing method and device of a heterogeneous database, electronic equipment and a storage medium. The method comprises the following steps: in response to a database operation request of an application system, calling an entity unified parent class by using a database entity class; calling a double-write parent class by using the entity unified parent class, so that the double-write parent class respectively calls a first access instance for performing a database operation on a first database and a second access instance for performing a database operation on a second database, the first database and the second database being heterogeneous databases; if the database operation on the first database is successful, continuing to perform the database operation on the second database; and if the database operation on the first database fails, not performing the database operation on the second database. The technical effect of effectively maintaining the double-write data synchronization between the heterogeneous databases and enabling the application system to quickly switch the databases can be achieved.

Description

Technical Field

[0001] This application relates to databases and big data technologies, and in particular to a method, apparatus, electronic device, and storage medium for operating and processing heterogeneous databases. Background Technology

[0002] To enhance technological self-reliance in the financial sector, several domestically developed databases have emerged as alternatives to foreign products such as Oracle, ensuring the stable operation of financial network systems.

[0003] In some domestic application scenarios, there are situations where heterogeneous databases need to be switched. For example, when using openGauss database to replace Oracle database, in order to cope with emergency switchbacks due to various reasons, it is necessary to perform dual-write operations on heterogeneous databases to maintain dual-write data synchronization between heterogeneous databases, so that the application system can quickly switch back to Oracle database when needed. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for processing heterogeneous databases, which solves the problem in the prior art that dual-write operations on heterogeneous databases cannot be effectively performed, resulting in the inability of application systems to quickly switch databases. It achieves the technical effect of effectively maintaining dual-write data synchronization between heterogeneous databases, enabling application systems to quickly switch databases.

[0005] On the one hand, this application provides a method for operating and processing heterogeneous databases, the method comprising:

[0006] In response to database operation requests from the application system, the database entity class calls the unified parent class of entities. The database entity class corresponds one-to-one with the database table, and the unified parent class of entities is abstracted from the same fields contained in all database tables.

[0007] The entity unified parent class calls the dual-write parent class so that the dual-write parent class respectively calls the first access instance for performing database operations on the first database and the second access instance for performing database operations on the second database, wherein the first database and the second database are heterogeneous databases;

[0008] If the database operation on the first database is successful, the database operation on the second database continues; if the database operation on the first database fails, the database operation on the second database is no longer performed.

[0009] Furthermore, before calling the unified parent class of entities using the database entity class in response to a database operation request from the application system, the method further includes:

[0010] Create a first data source configuration file for the first database and a second data source configuration file for the second database;

[0011] Create a first data source subclass corresponding to the first data source configuration file and a second data source subclass corresponding to the second data source configuration file, wherein the first data source subclass and the second data source subclass are used to manage the data source of the first database and the data source of the second database, respectively;

[0012] Based on the database language mapping configuration file, the first access instance and the second access instance are created respectively according to the data source of the first database and the data source of the second database.

[0013] Furthermore, based on the database language mapping configuration file, and according to the data source of the first database and the data source of the second database, the first access instance and the second access instance are created respectively, including:

[0014] Based on the database language mapping configuration file, a database language session factory class for the first database is created according to the data source of the first database, and a database language session factory class for the second database is created according to the data source of the second database.

[0015] The first access instance is created based on the database language session factory class of the first database; and the second access instance is created based on the database language session factory class of the second database.

[0016] Furthermore, create a first data source configuration file for the first database and a second data source configuration file for the second database, including:

[0017] A database connection pool is used to establish a connection between the first database and the second database, and configuration files for the first data source and the second data source are created.

[0018] The first data source configuration file and the second data source configuration file are respectively configured with connection pool parameters of the database connection pool. The connection pool parameters include the following information: database connection information, database connection driver type, connection pool size information, and connection recycling information.

[0019] Furthermore, the method also includes:

[0020] In the database language mapping configuration file, the correspondence between database language SQL statements and the tag attributes of the database language mapping configuration file is configured in advance, wherein the tag attributes of the database language mapping configuration file include namespaces and unique identifiers;

[0021] Modify the syntax of the SQL statement to a syntax format compatible with both the first database and the second database;

[0022] Determine the input parameter data type and output parameter data type of the database language mapping configuration file, wherein the input parameter data type is used to specify the data type of the query conditions of the SQL statement, and the output parameter data type is used to specify the data type of the query result of the SQL statement.

[0023] Furthermore, the method also includes:

[0024] Based on the database language mapping configuration file, it is determined whether to perform the database operation on both the first database and the second database. The database operation includes at least one of the following: add, delete, modify, and query; and

[0025] Based on the database language mapping configuration file, determine whether to enable rollback operation for the first database when database operation on the second database fails.

[0026] Furthermore, the method also includes:

[0027] When a database operation on the second database fails, the failed data is recorded in the failure log.

[0028] When the application system is detected to be idle, a compensation synchronization operation is performed on the second database based on the failure log.

[0029] Furthermore, the step of performing compensation synchronization operations on the second database based on the failure log includes:

[0030] According to the recording order of the failed operation data in the failure log, the failed operation data is parsed one by one to resynchronize the failed operation data to the second database;

[0031] At the end of the synchronization, the read position of the failure log is recorded so that subsequent compensation synchronization operations can be performed based on the read position.

[0032] Furthermore, the method also includes:

[0033] In response to the database rollback request of the application system, the data source of the first database is configured to read the configuration file of the second data source, and the data source of the second database is configured to read the configuration file of the first data source.

[0034] Restart the application system.

[0035] Furthermore, the entity class, the unified entity parent class, and the dual-write parent class are all stored in the server's web container, and there is a progressive calling relationship between the entity class, the unified entity parent class, and the dual-write parent class.

[0036] On the other hand, this application provides an operating processing apparatus for heterogeneous databases, the apparatus comprising:

[0037] The first calling module is used to respond to database operation requests from the application system by calling the unified parent class of entities using the database entity class. The database entity class corresponds one-to-one with the database table, and the unified parent class of entities is abstracted from the same fields contained in all database tables.

[0038] The second calling module is used to call the dual-write parent class using the unified parent class of the entity, so that the dual-write parent class respectively calls the first access instance for performing database operations on the first database and the second access instance for performing database operations on the second database, wherein the first database and the second database are heterogeneous databases;

[0039] The operation processing module is configured to continue executing the database operation on the second database if the database operation on the first database is successful, and to stop executing the database operation on the second database if the database operation on the first database fails.

[0040] On the other hand, this application provides an electronic device, including: a processor and a memory connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement any of the methods described above.

[0041] On the other hand, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement any of the methods described above.

[0042] On the other hand, this application provides a computer program product, including a computer program that, when executed by a processor, implements any of the methods described above.

[0043] The heterogeneous database operation processing method provided in this application, in response to database operation requests from application systems, uses a database entity class to call a unified entity parent class; the unified entity parent class then calls a dual-write parent class, so that the dual-write parent class respectively calls a first access instance for performing database operations on a first database and a second access instance for performing database operations on a second database, where the first and second databases are heterogeneous databases; if the database operation on the first database is successful, the database operation on the second database continues; if the database operation on the first database fails, the database operation on the second database is no longer executed. This solves the problem in existing technologies where dual-write operations on heterogeneous databases cannot be effectively executed, causing application systems to be unable to quickly switch databases, and achieves the technical effect of effectively maintaining dual-write data synchronization between heterogeneous databases, enabling application systems to quickly switch databases. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] Figure 1 A flowchart illustrating an operation processing method for a heterogeneous database provided in an embodiment of this application;

[0046] Figure 2 A flowchart illustrating an operation processing method for a heterogeneous database provided in an embodiment of this application;

[0047] Figure 3 This is a configuration flowchart of a heterogeneous database operation processing method provided in an embodiment of this application;

[0048] Figure 4 A flowchart illustrating an operation processing method for a heterogeneous database provided in an embodiment of this application;

[0049] Figure 5 A flowchart illustrating an operation processing method for a heterogeneous database provided in an embodiment of this application;

[0050] Figure 6 A structural block diagram of a heterogeneous database operation processing device provided in an embodiment of this application;

[0051] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0052] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0053] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0054] First, let me explain the terms used in this application:

[0055] Low-frequency trading systems are trading systems with lower processing power compared to high-frequency trading systems. High-frequency trading systems generally refer to high-concurrency trading systems, which process tens to thousands of transactions per second. Low-frequency trading systems have a processing power that is one to two orders of magnitude lower, processing tens of transactions per minute or hundreds of transactions per hour.

[0056] Data source: A device or raw medium that provides the required data. In this article, it refers to the database used by the database application.

[0057] Oracle Database: A relational distributed database.

[0058] openGauss database: an enterprise-grade open-source relational database.

[0059] Java Database Connectivity (JDBC) is a Java API that specifies how client programs access databases. It consists of a set of classes and interfaces written in Java, providing methods such as querying and updating data in the database.

[0060] Database synchronization: Synchronizing data from the source database to the target database. The main synchronization methods include log-based (Oracle redo, MySQL binlog), timestamp-based, trigger-based, snapshot-based, and offline batch processing-based.

[0061] Dual write: When an application system writes to one database, it also writes to another database simultaneously. Here, "write" refers to the add, delete, and update operations during database operations.

[0062] POJOs: also known as Plain Ordinary Java Objects, represent a collection of data.

[0063] Extensible Markup Language (XML) is a subset of Standard Generalized Markup Language (SGML) that can be used to mark up data and define data types. It is a source language that allows users to define their own markup language.

[0064] SQL (Structured Query Language) is a database language with multiple functions such as data manipulation and data definition. This language has interactive features and can provide great convenience to users. Database management systems should make full use of SQL to improve the working quality and efficiency of computer application systems.

[0065] A web container is a service application. Each port on a server hosts a program that provides the corresponding service and handles requests from clients. Examples include Tomcat in Java, and IIS or PWS in ASP. A server can have multiple containers.

[0066] MyBatis is a persistence layer framework that supports regular SQL queries, stored procedures, and advanced mappings.

[0067] Druid is an open-source, high-performance database connection pool that supports all JDBC-compatible databases.

[0068] Existing database synchronization solutions each have their own advantages and disadvantages, as well as their own applicable scenarios. For example, some are paid tools, some require intrusion into the database server, and some cannot achieve real-time performance, etc.

[0069] Traditional dual-write solutions involve significant code intrusion into the application system, and for heterogeneous databases, the corresponding SQL statements need to be rewritten for different databases, resulting in substantial modifications and high costs.

[0070] The heterogeneous database operation and processing method provided in this application aims to solve the above-mentioned technical problems of the prior art. The technical solution of this application and how it solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0071] Figure 1This is a flowchart illustrating a heterogeneous database operation processing method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0072] S101 responds to the database operation request from the application system by calling the unified parent class of the entity using the database entity class.

[0073] S102, the above-mentioned entity unified parent class calls the dual-write parent class, so that the above-mentioned dual-write parent class respectively calls the first access instance for performing database operations on the first database and the second access instance for performing database operations on the second database, wherein the above-mentioned first database and the above-mentioned second database are heterogeneous databases.

[0074] S103, if the database operation on the first database is successful, the database operation on the second database continues; if the database operation on the first database fails, the database operation on the second database is no longer executed.

[0075] Optionally, the aforementioned application system can be a low-frequency trading system, used to process transactions with lower concurrency processing capabilities than high-frequency trading systems, such as financial transactions.

[0076] In one example, the database entity classes mentioned above correspond one-to-one with the database tables in the database. The unified parent class of the entities is abstracted from the same fields contained in all database tables. For example, each database table contains a timestamp field and a version number field, so the unified parent class of entities contains these two attributes. Then each entity class inherits from this unified parent class of entities, so there is no need to write these two attributes again.

[0077] Optionally, in one example, since openGauss database is used instead of Oracle database in some domestic application scenarios, the first database can be openGauss database and the second database can be Oracle database; similarly, if Oracle database is needed to replace openGauss database, then the second database can be openGauss database and the first database can be Oracle database.

[0078] The applicable scenarios for the embodiments of this application are data synchronization between an openGauss database and an Oracle database, or data synchronization between an Oracle database and an openGauss database.

[0079] Optionally, the Oracle database mentioned above is a relational distributed database, and the openGauss database is an enterprise-grade open-source relational database. In this embodiment, the first and second databases can also be other databases besides those mentioned above, as long as they are heterogeneous.

[0080] Optionally, the database operation in this embodiment is a dual-write operation, that is, when the application system writes to one database in a heterogeneous database, it also writes to another database at the same time. The "write" here includes, but is not limited to, performing operations such as adding, deleting, querying, and modifying the database.

[0081] In one example, taking the first database as an openGauss database and the second database as an Oracle database as an example, the heterogeneous database operation processing method provided in this application embodiment can be based on the fact that the openGauss database and the Oracle database have been synchronized. The heterogeneous database dual write operation performed can be completed in the case of a low-frequency trading system in this application embodiment, since the application system in this application embodiment is a low-frequency trading system. Therefore, this application embodiment can complete the data synchronization between the openGauss database and the Oracle database in the case of idle offline conditions.

[0082] In one optional example, embodiments of this application may be developed based on the persistence layer framework MyBatis and the database connection pool Druid. MyBatis is a Java-based persistence layer framework that uses simple XML or annotations for configuration and raw mapping, mapping interfaces and Java POJOs to records in the database. Druid is an open-source, high-performance database connection pool that supports all JDBC-compatible databases.

[0083] The heterogeneous database operation processing method provided in this application is applicable to dual-write operations between OpenGauss and Oracle databases. Firstly, since the syntax differences between OpenGauss and Oracle databases are not significant, only a small number of SQL statements need to be rewritten to achieve syntax compatibility, resulting in relatively low modification costs. Secondly, since this application uses the Mybatis framework, its SQL statements are all centrally configured in XML files, making the modification points concentrated, convenient to rewrite, and less prone to omissions. Thirdly, because the entity classes of the application system all inherit from a unified parent class (unified entity parent class), this application can create a new dual-write base class to implement dual-write operations on heterogeneous databases, and then replace the original unified entity parent class to achieve the dual-write goal, further reducing the amount of modification required.

[0084] The following adopts Figure 2The flowchart shown illustrates an optional heterogeneous database operation method, providing an example of an application system performing dual-write operations on a heterogeneous database. Figure 2 As shown, in an optional embodiment, when an application system such as a low-frequency trading system requests to perform a dual-write operation, the database entity class calls the unified entity parent class, and then the unified entity parent class calls the dual-write parent class, so that the dual-write parent class respectively calls the first access instance for performing database operations on the first database and the second access instance for performing database operations on the second database. The first database and the second database are heterogeneous databases.

[0085] In one example, the aforementioned entity class, the aforementioned unified parent class of entities, and the aforementioned dual-write parent class are all stored in the web container of the server, and there is a progressive calling relationship between the aforementioned entity class, the aforementioned unified parent class of entities, and the aforementioned dual-write parent class.

[0086] like Figure 2 As shown, when the second access instance of the parent class fails to perform database operations on the second database, the writing to the second database also fails. The failed data is then recorded in the failure log so that data synchronization can be performed during idle time.

[0087] After both the first and second access instances are successfully invoked, if the write operation on the first database is successful, the write operation on the second database will continue. If the write operation on the first database fails, the write operation on the second database will not be executed.

[0088] In one optional embodiment, taking an example where the first database can be an OpenGauss database and the second database can be an Oracle database, the application system implements CRUD operations on the database through database entity classes. Each entity class further calls a unified parent class to operate on the database. The unified parent class then calls a dual-write parent class to implement database operations on both the OpenGauss and Oracle databases. The dual-write parent class contains access entities for both OpenGauss and Oracle databases. After successfully writing to the OpenGauss database, it continues to write to the Oracle database. If writing to the OpenGauss database fails, the operation to write to the Oracle database is not performed.

[0089] The heterogeneous database operation processing method provided in this application includes a heterogeneous database switching scheme based on dual-write implementation suitable for low-frequency trading systems. This scheme can achieve real-time data synchronization between openGauss database and Oracle database. Combined with the idle time compensation scheme after data synchronization failure described in subsequent embodiments, the application system can switch back from openGauss database to Oracle database as needed, in a timely and rapid manner.

[0090] Therefore, the embodiments of this application can solve the problem in the prior art that the dual-write operation of heterogeneous databases cannot be effectively executed, which leads to the inability of application systems to quickly switch databases. This achieves the technical effect of effectively maintaining dual-write data synchronization between heterogeneous databases, enabling application systems to quickly switch databases.

[0091] In one example, before calling the unified parent class of entities using the database entity class in response to a database operation request from the application system, the above method also includes:

[0092] S201, Create the first data source configuration file for the first database and the second data source configuration file for the second database;

[0093] S202, create a first data source subclass corresponding to the first data source configuration file mentioned above, and a second data source subclass corresponding to the second data source configuration file mentioned above.

[0094] The first data source subclass and the second data source subclass are used to manage the data source of the first database and the data source of the second database, respectively.

[0095] S203, based on the database language mapping configuration file, according to the data source of the first database and the data source of the second database, respectively create the first access instance and the second access instance.

[0096] The dual-write data synchronization method proposed in this application embodiment is implemented using the Druid connection pool and the Mybatis framework. The specific configuration flowchart is as follows: Figure 3 As shown, for example, in this embodiment of the application, a Druid connection pool can be used to independently configure the first data source configuration file of the first database and the second data source configuration file of the second database. By reading data source parameters from the first data source configuration file and the second data source configuration file respectively, a first data source subclass corresponding to the first data source configuration file and a second data source subclass corresponding to the second data source configuration file can be created.

[0097] In one example, the first data source subclass manages the data source for the first database, and the second data source subclass manages the data source for the second database. In another example, the data source subclasses typically have some custom logic to distinguish between different data sources.

[0098] In one optional embodiment, based on the database language mapping configuration file, the first access instance and the second access instance are created respectively according to the data source of the first database and the data source of the second database, including:

[0099] S301, based on the above database language mapping configuration file, create a database language session factory class for the first database according to the data source of the first database, and create a database language session factory class for the second database according to the data source of the second database.

[0100] S302, based on the database language session factory class of the first database, create the first access instance; and based on the database language session factory class of the second database, create the second access instance.

[0101] Optionally, the aforementioned database language mapping configuration file is the SQL mapping configuration file in the persistence framework MyBatis. SQL statements are read from the SQL mapping configuration file to create a database language session factory class for the first database based on the data source of the first database, and a database language session factory class for the second database based on the data source of the second database.

[0102] In one optional embodiment, the first session factory class and the second session class are used to create corresponding sessions respectively. Each SQL statement is executed through a session. The access instance integrates the session factory, which simplifies code writing.

[0103] Optionally, the first session factory class and the second session class mentioned above are used to configure two persistent layer access instances, namely the first access instance and the second access instance, which are used to perform write operations on the first database and the second database, respectively.

[0104] In one optional implementation, creating a first data source configuration file for a first database and a second data source configuration file for a second database includes:

[0105] S401, a connection is established between the first database and the second database using a database connection pool, and the configuration files for the first data source and the second data source are created.

[0106] In the above example, the first data source configuration file and the second data source configuration file are respectively configured with the connection pool parameters of the database connection pool. The connection pool parameters include the following information: database connection information, database connection driver type, connection pool size information, and connection recycling information.

[0107] In one alternative example, a database connection pool is used to allocate, manage, and release database connections. This allows applications to reuse an existing database connection instead of establishing a new one. Database connections that have been idle for longer than the maximum idle time are released to avoid missing database connections due to failure to release them, thereby improving the performance of database operations.

[0108] In one example, taking the first database as an openGauss database and the second database as an Oracle database as an example, this embodiment configures connection pool parameters in the data source configuration files of the openGauss database and the Oracle database, configures the correspondence between SQL statements and namespaces and unique identifiers in the SQL mapping configuration file, specifies the input parameter data type and output parameter data type, and modifies the syntax format of the SQL statement to a syntax format compatible with both openGauss database and Oracle database. Then, it reads the data source configuration files of the openGauss database and the Oracle database respectively, creates two Druid data source subclasses, and manages the data source of the openGauss database and the data source of the Oracle database respectively.

[0109] Subsequently, the SQL mapping configuration file is read uniformly, and an SQL session factory class for the openGauss database is created based on the Druid data source that depends on the openGauss database, and an SQL session factory class for the Oracle database is created based on the Druid data source that depends on the Oracle database. Furthermore, in this embodiment, an access instance for the openGauss database is created based on the SQL session factory class that depends on the openGauss database, and an access instance for the Oracle database is created based on the SQL session factory class that depends on the Oracle database.

[0110] As an optional implementation method, Figure 4 This is a flowchart illustrating a heterogeneous database operation processing method provided in an embodiment of this application, as shown below. Figure 4 As shown, the above method also includes:

[0111] S501, In the database language mapping configuration file mentioned above, the correspondence between the database language SQL statements and the tag attributes of the database language mapping configuration file is configured in advance.

[0112] The tag attributes of the aforementioned database language mapping configuration file include namespaces and unique identifiers.

[0113] S502, modify the syntax of the above SQL statement to a syntax format compatible with both the first database and the second database.

[0114] S503, determine the input and output data types of the database language mapping configuration file mentioned above.

[0115] The above input parameter data type is used to specify the data type of the query conditions of the above SQL statement, and the above output parameter data type is used to specify the data type of the query result of the above SQL statement.

[0116] Optionally, in this embodiment, the two access instances corresponding to the two heterogeneous databases share a set of SQL mapping configuration files, and the amount of modification to the existing code is small. Only the syntax of the SQL statements needs to be modified to a syntax format that is compatible with both the first database and the second database.

[0117] In one optional embodiment, the correspondence between SQL statements and namespaces and unique identifiers is configured in the SQL mapping configuration file. It should be noted that namespace and unique identifier are terms related to the XML file domain. namespace is an attribute of the mapping mapper tag, and id is an attribute of tags such as select for query statements and insert for insert statements.

[0118] For example, if the namespace and unique identifier combination corresponding to a query statement `select xxx from table1` is `s13.blist-getByPk`, then `s13` is the namespace, and `blist-getByPk` is the unique identifier. The correspondence is one-to-one. It should be noted that the Java code in this embodiment references this SQL statement, achieving separation between the SQL statement and the Java code for centralized management.

[0119] In another alternative embodiment, the method further includes:

[0120] S601, based on the aforementioned database language mapping configuration file, determine whether to perform the aforementioned database operations on both the first database and the second database. The aforementioned database operations include at least one of the following: add, delete, modify, or query; and

[0121] S602, based on the above database language mapping configuration file, determine whether to enable rollback operation for the above first database when the database operation on the second database fails.

[0122] In one optional embodiment, this application allows for flexible configuration of the dual-write switch and rollback switch based on an SQL mapping configuration file. Dual-write in heterogeneous databases can be enabled or disabled via the SQL mapping configuration file. For example, the configuration can take effect 30 minutes after modification, but is not limited to this; it can also be controlled as needed.

[0123] In another embodiment, taking the example that the first database can be an openGauss database and the second database can be an Oracle database, the SQL mapping configuration file can be used to configure whether the failure of the Oracle database to perform a write operation will cause the openGauss database to perform a rollback operation. It can also be enabled or disabled through the SQL mapping configuration file. The configuration can take effect 30 minutes after modification, but is not limited to this. It can be controlled as needed.

[0124] According to one or more embodiments of the present application, the above method further includes:

[0125] S701, when a database operation fails to be performed on the second database mentioned above, the failed data is recorded in the failure log;

[0126] S702, when the above-mentioned application system is detected to be idle, a compensation synchronization operation is performed on the above-mentioned second database according to the above-mentioned failure log.

[0127] In this embodiment of the application, taking the first database as an openGauss database and the second database as an Oracle database as an example, when writing to the Oracle database fails, the failed data will be recorded in the failure log so that compensation synchronization can be performed when the application system is idle.

[0128] In one optional embodiment, the above-mentioned compensation synchronization operation on the second database based on the failure log includes:

[0129] S801, according to the record order of the failed data in the failure log, parse the failed data one by one, so as to resynchronize the failed data to the second database.

[0130] S802, at the end of synchronization, record the read position of the above failure log so that subsequent compensation synchronization operations can be performed based on the read position.

[0131] In this embodiment of the application, during idle time compensation synchronization, failure logs can be parsed sequentially, and the data that failed to be written can be resynchronized to the second database. An optional flowchart is shown below. Figure 5As shown in the embodiment of this application, a compensation synchronization module is used to read the database failure log when the application system is idle and record the reading position so that the next reading can continue from this reading position; the failure log is parsed sequentially, assembled into a database SQL statement and then written to the second database.

[0132] Another alternative embodiment further includes:

[0133] S901, in response to the database switchback request of the application system, the data source of the first database is configured to read the configuration file of the second data source, and the data source of the second database is configured to read the configuration file of the first data source.

[0134] S902, restart the aforementioned application system.

[0135] In one embodiment, if the application system needs to switch back from the Oracle database to the OpenGauss database, it is only necessary to configure the Druid1 data source (the data source of the OpenGauss database) to read the Oracle data source configuration file, configure the Druid2 data source (the data source of the Oracle database) to read the OpenGauss data source configuration file, and restart the application system to realize the database switchback, that is, to switch back from the Oracle database to the OpenGauss database.

[0136] In this embodiment, a Druid connection pool can be used to configure both an OpenGauss data source and an Oracle data source. Two persistent layer access instances are configured in the source code's XML configuration file, one for operating the OpenGauss database and the other for operating the Oracle database. The MyBatis mapping SQL statement is modified to a syntax format compatible with both OpenGauss and Oracle databases. A dual-write base class is created for the database, synchronously writing to the Oracle database after writing to the OpenGauss database. The application system's entity classes are modified to inherit from the dual-write base class to achieve dual-write database functionality. Data that fails to synchronize is recorded in a file and compensated for during low-frequency trading system downtime, ultimately ensuring data consistency between the two databases. If a database switch occurs, simply swapping the data source configurations and restarting the low-frequency trading system will achieve database rollback.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0138] According to one or more embodiments of this application, an operation processing apparatus for heterogeneous databases is provided. Figure 6 A structural block diagram of a heterogeneous database operation processing device provided in this application embodiment is shown below. Figure 6 As shown, the above-mentioned device includes:

[0139] The first calling module 601 is used to respond to the database operation request of the application system and call the unified parent class of entities using the database entity class. The database entity class corresponds one-to-one with the database table, and the unified parent class of entities is abstracted from the same fields contained in all database tables.

[0140] The second calling module 602 is used to call the dual-write parent class using the above-mentioned unified parent class of entities, so that the above-mentioned dual-write parent class respectively calls the first access instance for performing database operations on the first database and the second access instance for performing database operations on the second database, wherein the above-mentioned first database and the above-mentioned second database are heterogeneous databases.

[0141] The operation processing module 603 is used to continue executing the database operation on the second database if the database operation on the first database is successful, and to stop executing the database operation on the second database if the database operation on the first database fails.

[0142] According to one or more embodiments of this application, the above-described apparatus further includes:

[0143] The first creation module is used to create the first data source configuration file for the first database and the second data source configuration file for the second database.

[0144] The second creation module is used to create a first data source subclass corresponding to the first data source configuration file and a second data source subclass corresponding to the second data source configuration file, wherein the first data source subclass and the second data source subclass are used to manage the data source of the first database and the data source of the second database, respectively.

[0145] The third creation module is used to create the first access instance and the second access instance respectively based on the database language mapping configuration file and the data source of the first database and the data source of the second database.

[0146] According to one or more embodiments of this application, the third creation module described above includes:

[0147] The first creation unit is used to create a database language session factory class for the first database based on the database language mapping configuration file and the data source of the first database, and to create a database language session factory class for the second database based on the data source of the second database.

[0148] The second creation unit is used to create the first access instance based on the database language session factory class of the first database; and to create the second access instance based on the database language session factory class of the second database.

[0149] According to one or more embodiments of this application, the first creation module described above includes:

[0150] The third creation unit is used to establish a connection between the first database and the second database using a database connection pool, and to create the first data source configuration file and the second data source configuration file.

[0151] The first data source configuration file and the second data source configuration file are respectively configured with the connection pool parameters of the database connection pool. The connection pool parameters include the following information: database connection information, database connection driver type, connection pool size information, and connection recycling information.

[0152] According to one or more embodiments of this application, the above-described apparatus further includes:

[0153] The configuration module is used to pre-configure the correspondence between SQL statements in the database language and the tag attributes of the database language mapping configuration file in the database language mapping configuration file. The tag attributes of the database language mapping configuration file include namespaces and unique identifiers.

[0154] The format modification module is used to modify the syntax of the above SQL statement to a syntax format that is compatible with both the first database and the second database.

[0155] The data type determination module is used to determine the input parameter data type and output parameter data type of the above database language mapping configuration file. The input parameter data type is used to specify the data type of the query conditions of the above SQL statement, and the output parameter data type is used to specify the data type of the query result of the above SQL statement.

[0156] According to one or more embodiments of this application, the above-described apparatus further includes:

[0157] The first determining module is used to determine, based on the aforementioned database language mapping configuration file, whether to perform the aforementioned database operations on both the first database and the second database, wherein the aforementioned database operations include at least one of the following: add, delete, modify, or query; and

[0158] The second determining module is used to determine, based on the database language mapping configuration file, whether to enable rollback operation on the first database when the database operation on the second database fails.

[0159] According to one or more embodiments of this application, the above-described apparatus further includes:

[0160] The recording module is used to record the failed data to the failure log when a database operation fails to be performed on the second database mentioned above.

[0161] The compensation module is used to perform compensation synchronization operations on the second database based on the failure log when the above application system is detected to be idle.

[0162] According to one or more embodiments of this application, the above-mentioned compensation module includes:

[0163] The parsing unit is used to parse the failed data of the above operation one by one according to the recording order of the failed data of the above operation in the above failure log, so as to resynchronize the failed data of the above operation to the above second database;

[0164] The compensation unit is used to record the read position of the failure log at the end of the synchronization, so as to perform subsequent compensation synchronization operations based on the read position.

[0165] According to one or more embodiments of this application, the above-described apparatus further includes:

[0166] The switching module is used to respond to the database switchback request of the above application system by configuring the data source of the first database to read the configuration file of the second data source, and configuring the data source of the second database to read the configuration file of the first data source.

[0167] The restart module is used to restart the aforementioned application system.

[0168] According to one or more embodiments of this application, the aforementioned entity class, the aforementioned unified parent class of entities, and the aforementioned dual-write parent class are all stored in the web container of the server, and there is a progressive calling relationship between the aforementioned entity class, the aforementioned unified parent class of entities, and the aforementioned dual-write parent class.

[0169] In an exemplary embodiment, this application also provides an electronic device, including: a processor, and a memory connected to the processor;

[0170] The aforementioned memory stores instructions executed by the computer;

[0171] The processor executes computer execution instructions stored in the memory to implement any of the methods described above.

[0172] In an exemplary embodiment, this application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods described above.

[0173] In an exemplary embodiment, this application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the methods described above.

[0174] To implement the above embodiments, this application also provides an electronic device. (See reference...) Figure 7 The diagram illustrates a structural schematic of an electronic device 700 suitable for implementing embodiments of this application. The electronic device 700 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, messaging devices, game consoles, medical devices, fitness equipment, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0175] like Figure 7As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0176] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0177] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this application.

[0178] It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor device or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device or apparatus. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution device or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0179] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0180] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0181] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0183] The units described in the embodiments of this application can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0184] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0185] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution apparatus or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0186] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0187] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of processing operations of a heterogeneous database, characterized by, The method includes: In response to database operation requests from the application system, the database entity class calls the unified parent class of entities. The database entity class corresponds one-to-one with the database table, and the unified parent class of entities is abstracted from the same fields contained in all database tables. The entity unified parent class calls the dual-write parent class, so that the dual-write parent class respectively calls the first access instance for performing database operations on the first database and the second access instance for performing database operations on the second database, wherein the first database and the second database are heterogeneous databases; If the database operation on the first database is successful, the database operation on the second database continues; if the database operation on the first database fails, the database operation on the second database is no longer executed. Before calling the unified parent class of entities using the database entity class in response to a database operation request from the application system, the method further includes: Create a first data source configuration file for the first database and a second data source configuration file for the second database; Create a first data source subclass corresponding to the first data source configuration file and a second data source subclass corresponding to the second data source configuration file, wherein the first data source subclass and the second data source subclass are used to manage the data source of the first database and the data source of the second database, respectively; Based on the database language mapping configuration file, the first access instance and the second access instance are created respectively according to the data source of the first database and the data source of the second database.

2. The method of claim 1, wherein, Based on the database language mapping configuration file, the first access instance and the second access instance are created respectively according to the data source of the first database and the data source of the second database, including: Based on the database language mapping configuration file, a database language session factory class for the first database is created according to the data source of the first database, and a database language session factory class for the second database is created according to the data source of the second database. The first access instance is created based on the database language session factory class of the first database; and the second access instance is created based on the database language session factory class of the second database.

3. The method according to claim 1, characterized in that, Create a first data source configuration file for the first database and a second data source configuration file for the second database, including: A database connection pool is used to establish a connection between the first database and the second database, and configuration files for the first data source and the second data source are created. The first data source configuration file and the second data source configuration file are respectively configured with connection pool parameters of the database connection pool. The connection pool parameters include the following information: database connection information, database connection driver type, connection pool size information, and connection recycling information.

4. The method according to claim 1, characterized in that, The method further includes: In the database language mapping configuration file, the correspondence between database language SQL statements and the tag attributes of the database language mapping configuration file is configured in advance, wherein the tag attributes of the database language mapping configuration file include namespaces and unique identifiers; Modify the syntax of the SQL statement to a syntax format compatible with both the first database and the second database; The input parameter data type and output parameter data type of the database language mapping configuration file are determined, wherein the input parameter data type is used to specify the data type of the query conditions of the SQL statement, and the output parameter data type is used to specify the data type of the query results of the SQL statement.

5. The method according to claim 1, characterized in that, The method further includes: Based on the database language mapping configuration file, it is determined whether to perform the database operation on both the first database and the second database. The database operation includes at least one of the following: add, delete, modify, and query; and Based on the database language mapping configuration file, determine whether to enable rollback operation for the first database when database operation on the second database fails.

6. The method according to claim 1, characterized in that, The method further includes: When a database operation on the second database fails, the failed data is recorded in the failure log. When the application system is detected to be idle, a compensation synchronization operation is performed on the second database based on the failure log.

7. The method according to claim 6, characterized in that, The step of performing compensation synchronization operations on the second database based on the failure log includes: According to the recording order of the failed operation data in the failure log, the failed operation data is parsed one by one to resynchronize the failed operation data to the second database; At the end of the synchronization, the read position of the failure log is recorded so that subsequent compensation synchronization operations can be performed based on the read position.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: In response to the database rollback request of the application system, the data source of the first database is configured to read the configuration file of the second data source, and the data source of the second database is configured to read the configuration file of the first data source. Restart the application system.

9. The method according to any one of claims 1 to 7, characterized in that, The entity class, the unified parent class of the entity, and the dual-write parent class are all stored in the web container of the server, and there is a progressive calling relationship between the entity class, the unified parent class of the entity, and the dual-write parent class.

10. A heterogeneous database operation and processing device, characterized in that, The device includes: The first calling module is used to respond to database operation requests from the application system by calling the unified parent class of entities using the database entity class. The database entity class corresponds one-to-one with the database table, and the unified parent class of entities is abstracted from the same fields contained in all database tables. The second calling module is used to call the dual-write parent class using the unified parent class of the entity, so that the dual-write parent class respectively calls the first access instance for performing database operations on the first database and the second access instance for performing database operations on the second database, wherein the first database and the second database are heterogeneous databases; The operation processing module is used to continue executing the database operation on the second database if the database operation on the first database is successful, and to stop executing the database operation on the second database if the database operation on the first database fails. Before invoking the unified parent class of entities using the database entity class in response to a database operation request from the application system, the device further includes: The first creation module is used to create the first data source configuration file for the first database and the second data source configuration file for the second database. The second creation module is used to create a first data source subclass corresponding to the first data source configuration file and a second data source subclass corresponding to the second data source configuration file, wherein the first data source subclass and the second data source subclass are used to manage the data source of the first database and the data source of the second database, respectively. The third creation module is used to create the first access instance and the second access instance respectively based on the database language mapping configuration file and the data source of the first database and the data source of the second database.

11. An electronic device, characterized in that, include: A processor, and a memory connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.

Images