Method and apparatus for dual-write parallel operation of databases in domestic database switching environment

Abstract

The present invention discloses a method and device for parallel operation of database dual writing in a domestic database switching environment, which belongs to the field of database technology. The method includes the following steps: according to a database operation request, by intercepting a database component, asynchronously initiating a domestic database operation in accordance with a predetermined configuration and multi-threading, and adopting two methods to perform database dual writing; parallel execution of different SQL statements with the same SQL identifier in the process of switching a domestic database in an electronic banking system; adopting a method of persistently running a target database in a production environment, continuously collecting the operating indicators of the target database in the production environment, using persistent data to measure the indicators of the target database running in the production environment as a measure of whether the target database reaches the standard of switching a source database, and performing target database verification. The present invention ensures that the target database is fully verified during the switching process between the target database and the source database, and reduces the unpredictable impact of the domestic database on the application.

Description

Technical Field

[0001] The present invention relates to a method and device for dual-write parallel operation of a database in a domestic database switching environment, and specifically to a method and device for dual-write parallel operation of a database in a domestic trusted database switching environment for electronic banking, belonging to the field of database technology. Background Art

[0002] With the advent of the information age, data has become an important asset of enterprises. As a data storage and management platform, the security, stability and efficiency of the database are crucial to the business continuity and development of the enterprise.

[0003] Database migration can achieve the goal of data security and independent control. In the process of domestic database migration, business continuity is a crucial aspect, which is directly related to the normal operation of the enterprise business and customer experience. Business continuity refers to the ability of an enterprise to maintain business without interruption or quickly resume business in the face of various unexpected events. In the process of database migration, if it is not handled properly, it is likely to cause business interruption, which in turn affects the normal operation of the enterprise and customer trust.

[0004] At present, due to the characteristics of the electronic banking system itself, such as session state, security authentication, random factors, etc., the traditional data mirroring and merging scheme of the network routing layer cannot meet the merging requirements of the electronic banking system. At the same time, domestic databases often have their own grammatical dialect characteristics. When applied in the process of domestic database merging and adaptation, they must also have the ability to support multiple database dialects. In order to reduce the unpredictable impact of domestic databases on applications in the production environment, the present invention proposes a database dual-write merging operation method and device in the domestic database switching environment for electronic banking. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a database dual-write parallel operation method and device in a domestic database switching environment for electronic banking, which can verify the domestic database in the production environment and reduce the unpredictable impact of the domestic database on the application.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] In a first aspect, an embodiment of the present invention provides a method for running a database dual-write parallel track in a domestic database switching environment, comprising the following steps:

[0008] Step S1, according to the database operation request, by intercepting the database component, according to the established configuration, multi-threaded asynchronous initiation of the domestic database switching operation, and using link 1 and link 2 to perform database dual writing; the link 1 is the program running in the production environment, when the database operation is called, the main thread normally performs the database operation in the source library, and the database operation log is normally recorded; the link 2 is the program running in the production environment, when the database operation is called, the AOP interceptor intercepts the database operation, and performs asynchronous processing in the parallel thread;

[0009] Step S2, executing different SQL statements with the same SQL identifier in parallel during the process of switching the electronic banking system to the domestic database;

[0010] Step S3, adopt the method of persistently running the target database in the production environment, continuously collect the operating indicators of the target database in the production environment, use the persistent data to measure the indicators of the target database running in the production environment as the measurement standard for whether the target database meets the switching source database, and verify the target database. The operating indicators include coverage, abnormal error indicators, performance indicators, data consistency indicators and discard rate indicators.

[0011] As a possible implementation method of this embodiment, in step S1, the dual write operation is not performed when the domestic database operation does not hit the configuration item; during the dual write operation of the database parallel operation, all asynchronously initiated target database operations are completed using independent transactions, query transactions are executed using a shared thread pool, and non-query transactions are executed using a dedicated thread pool, so that the database operations executed in the parallel thread are consistent with the execution order of the main thread database operations, and the execution results are written into a separate log file.

[0012] As a possible implementation of this embodiment, the link 2 is specifically:

[0013] The interceptor determines whether the database operation needs to perform an asynchronous dual-write operation. If it is determined to be necessary, the interceptor initiates an asynchronous database operation instruction.

[0014] After initiating the asynchronous operation, multiple threads in the thread pool begin to execute database operations on the target database and separately record the log of this asynchronous operation; the separate log of this asynchronous operation is used to collect the operation status of the target database in the production environment, and long-term execution and recording are used as a basis for measuring whether the target database can achieve the switching of the source database.

[0015] As a possible implementation of this embodiment, step S1 includes the following steps:

[0016] Step S11, receiving a database operation request;

[0017] Step S12, adding database track dual-write configuration, the database track dual-write configuration includes data source information configuration and track test switch configuration, the data source information configuration is the connection configuration information of the track test target library, the track test switch configuration is whether to turn on the track test, supports two configuration modes of configuration file and configuration center, and supports real-time effectiveness of configuration item modification when using the configuration center;

[0018] Step S13, create a database parallel dual-write interceptor in the project, intercept the unified entrance of the database execution operation, intercept all database operations of the business library through the aspect, identify the database operation type, clone the parameters of the database operation, save the snapshot information of the database execution site, and realize that link 1 and link 2 perform the same database operation. The database operation type includes query operation and addition, deletion and modification operation;

[0019] Step S14, start an asynchronous thread for the data source that enables database merge and dual write, asynchronously perform the database merge operation, the asynchronous thread tasks include timeout monitoring of the current database operation and database merge execution operation, the asynchronous thread outputs the data source used by the current database operation, execution time, connection acquisition time, execution result status display, exception information, thread pool name and SQL type;

[0020] Step S15, after initiating the asynchronous operation, the multiple threads in the thread pool start to execute the database operation of the target database and separately record the log of this asynchronous operation.

[0021] As a possible implementation method of this embodiment, in step S14, a public thread pool is used to perform a merge operation on the query operation of the main business library, and an exclusive thread pool is used to perform an execution operation on the addition, deletion and modification operations of the main business library. There is only one working thread in the exclusive thread pool. When the merge task is executed, other tasks are first put into the queue and executed in the order of entering the queue.

[0022] As a possible implementation of this embodiment, step S2 includes the following steps:

[0023] Step S21, during the process of switching the electronic banking system to the domestic database, standard SQL identifiers are used for normal business development. The format of standard SQL identifiers is: main table name, operation name, SQL statement and function description;

[0024] Step S22, configuring the database name in the data source, and declaring the data source list supported by the current system;

[0025] Step S23, configuring a routing strategy for the merge test in the data source, supporting simultaneous merge verification of multiple libraries according to different conditions;

[0026] Step S24, set the SQL identifier of the database configured in step S22, the format of which is: main table name, operation name, SQL statement, function description and data source identifier. When performing business development, the original SQL statement is used for database operation. The state framework automatically identifies the database and selects the corresponding SQL statement for execution.

[0027] As a possible implementation of this embodiment, step S3 includes the following steps:

[0028] Step S31, dynamically compare the operation indicators in the dynamic library of the production environment, with the source database and the target database as the comparison objects, and the comparison content is the log content of the dynamic execution of the two, and draw a preliminary conclusion;

[0029] Step S32: Based on the preliminary conclusion, static comparison is performed in the static library, and SQL statements with inconsistent execution results or not meeting the index requirements in the dynamic comparison results are put into the static library for comparison again to verify whether it is caused by other operations in the dynamic library;

[0030] Step S33, compare the target database with the source database in terms of dynamic and static persistence, compare the database structure and the number of rows affected by SQL, form a phased comparison analysis result, and determine whether the target database and the source database can be switched based on the comparison analysis result.

[0031] As a possible implementation of this embodiment, in step S3, the coverage index is that the total coverage of the source database and the target database in the production environment running in parallel does not exceed 70%;

[0032] The abnormal error indicator is that the number of abnormal errors reported by the target database in the production environment is 0;

[0033] The performance indicators are that the overall SQL execution time of the target database in the production environment is within 50% higher than that of the source database, the average time consumption of the interface increases within 15%, and the application can support 8500 TPS;

[0034] The data consistency indicator is that the SQL execution results of the source database and the target database in the production environment are consistent;

[0035] The discard rate indicator is that the SQL execution discard rate of the target database in the production environment is 0, and there is no backlog in the update thread pool.

[0036] In a second aspect, an embodiment of the present invention provides a database dual-write parallel operation device in a domestic database switching environment, comprising:

[0037] The database dual-write module is used to initiate the domestic database switching operation asynchronously in accordance with the established configuration multi-threaded by intercepting the database component according to the database operation request, and adopts two methods, link one and link two, to perform database dual-write; the link one is the program running in the production environment. When the database operation is called, the main thread performs the database operation in the source library normally, and records the database operation log normally; the link two is the program running in the production environment. When the database operation is called, the AOP interceptor intercepts the database operation and performs asynchronous processing in the parallel thread;

[0038] The SQL parallel execution module is used to parallel execute different SQL statements with the same SQL identifier during the process of switching the electronic banking system to the domestic database;

[0039] The target database verification module is used to continuously collect the operating indicators of the target database in the production environment by running the target database persistently in the production environment, and use the persistent data to measure the indicators of the target database running in the production environment as the measurement standard for whether the target database meets the switching source database, and verify the target database. The operating indicators include coverage, abnormal error indicators, performance indicators, data consistency indicators and discard rate indicators.

[0040] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of the database dual-write and parallel operation method in any of the above-mentioned domestic database switching environments.

[0041] In a fourth aspect, an embodiment of the present invention provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the database dual-write and parallel operation method in any domestic database switching environment described above are executed.

[0042] The beneficial effects of the technical solution of the embodiment of the present invention are as follows:

[0043] A method for parallel operation of database dual writing in a domestic database switching environment according to the technical solution of an embodiment of the present invention comprises the following steps: step S1, according to a database operation request, by intercepting a database component, asynchronously initiating a domestic database switching operation in accordance with a predetermined configuration in a multi-threaded manner, and performing database dual writing in two ways, link one and link two; step S2, parallel execution of different SQL statements with the same SQL identifier in the process of switching the domestic database of an electronic banking system; step S3, by persistently running the target database in the production environment, continuously collecting the operating indicators of the target database in the production environment, using the persistent data to measure the indicators of the target database running in the production environment as the measurement standard for whether the target database meets the switching source database, and verifying the target database. In the process of switching the domestic database in the electronic banking system, the present invention adopts the method of dual-write parallel operation of the database to realize the verification of the domestic database in the production environment, and performs the single-track switching operation of the domestic database through configuration, thereby reducing the unpredictable impact of the domestic database on the application; the domestic database is used as the target database, and the foreign database to be replaced is the source database; for each database operation in the production environment, in addition to the normal source database operation, the target database being paralleled simultaneously initiates a target database operation asynchronously, and the parallel operation of the asynchronous database does not affect the final result of the business in the production environment, and the persistent data is only used for comparison and verification, and does not affect the source database data. The present invention adopts the method of dual-write parallel operation of the source database and the target database, which ensures that the target database is fully verified during the switching process between the target database and the source database, and at the same time ensures that the application development during the switching process can take into account the grammatical dialect characteristics of the target database and the source database, providing a reliable switching solution for the domestic database.

[0044] A database dual-write parallel operation device in a domestic database switching environment according to the technical solution of an embodiment of the present invention has the same beneficial effects as a database dual-write parallel operation method in a domestic database switching environment according to the technical solution of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart of a database dual-write and parallel operation method in a domestic database switching environment according to an exemplary embodiment;

[0046] Figure 2 It is a schematic diagram of the structure of a database dual-write parallel operation device in a domestic database switching environment according to an exemplary embodiment;

[0047] Figure 3 This is a specific implementation flow chart of a database double write according to an exemplary embodiment;

[0048] Figure 4is a flow chart showing a database merge operation according to an exemplary embodiment;

[0049] Figure 5 is a flow chart showing a SQL statement parallel execution according to an exemplary embodiment;

[0050] Figure 6 The present invention is a flowchart showing a comparison and verification of a target database and a source database according to an exemplary embodiment. DETAILED DESCRIPTION

[0051] In order to more clearly illustrate the technical features of the solution of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0052] like Figure 1 As shown, an embodiment of the present invention provides a method for running a database dual-write parallel track in a domestic database switching environment, comprising the following steps:

[0053] Step S1, according to the database operation request, by intercepting the database component, according to the established configuration, multi-threaded asynchronous initiation of the domestic database switching operation, and using link 1 and link 2 to perform database dual writing; the link 1 is the program running in the production environment, when the database operation is called, the main thread normally performs the database operation in the source library, and the database operation log is normally recorded; the link 2 is the program running in the production environment, when the database operation is called, the AOP interceptor intercepts the database operation, and performs asynchronous processing in the parallel thread;

[0054] Step S2, executing different SQL statements with the same SQL identifier in parallel during the process of switching the electronic banking system to the domestic database;

[0055] Step S3, adopt the method of persistently running the target database in the production environment, continuously collect the operating indicators of the target database in the production environment, use the persistent data to measure the indicators of the target database running in the production environment as the measurement standard for whether the target database meets the switching source database, and verify the target database. The operating indicators include coverage, abnormal error indicators, performance indicators, data consistency indicators and discard rate indicators.

[0056] As a possible implementation method of this embodiment, in step S1, the dual write operation is not performed when the domestic database operation does not hit the configuration item; during the dual write operation of the database parallel operation, all asynchronously initiated target database operations are completed using independent transactions, query transactions are executed using a shared thread pool, and non-query transactions are executed using a dedicated thread pool, so that the database operations executed in the parallel thread are consistent with the execution order of the main thread database operations, and the execution results are written into a separate log file.

[0057] As a possible implementation of this embodiment, the link 2 is specifically:

[0058] The interceptor determines whether the database operation needs to perform an asynchronous dual-write operation. If it is determined to be necessary, the interceptor initiates an asynchronous database operation instruction.

[0059] After initiating the asynchronous operation, multiple threads in the thread pool begin to execute database operations on the target database and separately record the log of this asynchronous operation; the separate log of this asynchronous operation is used to collect the operation status of the target database in the production environment, and long-term execution and recording are used as a basis for measuring whether the target database can achieve the switching of the source database.

[0060] As a possible implementation of this embodiment, step S1 includes the following steps:

[0061] Step S11, receiving a database operation request;

[0062] Step S12, adding database track dual-write configuration, the database track dual-write configuration includes data source information configuration and track test switch configuration, the data source information configuration is the connection configuration information of the track test target library, the track test switch configuration is whether to turn on the track test, supports two configuration modes of configuration file and configuration center, and supports real-time effectiveness of configuration item modification when using the configuration center;

[0063] Step S13, create a database parallel dual-write interceptor in the project, intercept the unified entrance of the database execution operation, intercept all database operations of the business library through the aspect, identify the database operation type, clone the parameters of the database operation, save the snapshot information of the database execution site, and realize that link 1 and link 2 perform the same database operation. The database operation type includes query operation and addition, deletion and modification operation;

[0064] Step S14, start an asynchronous thread for the data source that enables database merge and dual write, asynchronously perform the database merge operation, the asynchronous thread tasks include timeout monitoring of the current database operation and database merge execution operation, the asynchronous thread outputs the data source used by the current database operation, execution time, connection acquisition time, execution result status display, exception information, thread pool name and SQL type;

[0065] Step S15, after initiating the asynchronous operation, the multiple threads in the thread pool start to execute the database operation of the target database and separately record the log of this asynchronous operation.

[0066] As a possible implementation method of this embodiment, in step S14, a public thread pool is used to perform a merge operation on the query operation of the main business library, and an exclusive thread pool is used to perform an execution operation on the addition, deletion and modification operations of the main business library. There is only one working thread in the exclusive thread pool. When the merge task is executed, other tasks are first put into the queue and executed in the order of entering the queue.

[0067] As a possible implementation of this embodiment, step S2 includes the following steps:

[0068] Step S21, during the process of switching the electronic banking system to the domestic database, standard SQL identifiers are used for normal business development. The format of standard SQL identifiers is: main table name, operation name, SQL statement and function description;

[0069] Step S22, configuring the database name in the data source, and declaring the data source list supported by the current system;

[0070] Step S23, configuring a routing strategy for the merge test in the data source, supporting simultaneous merge verification of multiple libraries according to different conditions;

[0071] Step S24, set the SQL identifier of the database configured in step S22, the format of which is: main table name, operation name, SQL statement, function description and data source identifier. When performing business development, the original SQL statement is used for database operation. The state framework automatically identifies the database and selects the corresponding SQL statement for execution.

[0072] As a possible implementation of this embodiment, step S3 includes the following steps:

[0073] Step S31, dynamically compare the operation indicators in the dynamic library of the production environment, with the source database and the target database as the comparison objects, and the comparison content is the log content of the dynamic execution of the two, and draw a preliminary conclusion;

[0074] Step S32: Based on the preliminary conclusion, static comparison is performed in the static library, and SQL statements with inconsistent execution results or not meeting the index requirements in the dynamic comparison results are put into the static library for comparison again to verify whether it is caused by other operations in the dynamic library;

[0075] Step S33, compare the target database with the source database in terms of dynamic and static persistence, compare the database structure and the number of rows affected by SQL, form a phased comparison analysis result, and determine whether the target database and the source database can be switched based on the comparison analysis result.

[0076] As a possible implementation of this embodiment, in step S3, the coverage index is that the total coverage of the source database and the target database in the production environment running in parallel does not exceed 70%;

[0077] The abnormal error indicator is that the number of abnormal errors reported by the target database in the production environment is 0;

[0078] The performance indicators are that the overall SQL execution time of the target database in the production environment is within 50% higher than that of the source database, the average time consumption of the interface increases within 15%, and the application can support 8500TPS (Transactions Per Second).

[0079] The data consistency indicator is that the SQL execution results of the source database and the target database in the production environment are consistent;

[0080] The discard rate indicator is that the SQL execution discard rate of the target database in the production environment is 0, and there is no backlog in the update thread pool.

[0081] like Figure 2 As shown, an embodiment of the present invention provides a database dual-write parallel operation device in a domestic database switching environment, comprising:

[0082] The database dual-write module is used to initiate the domestic database switching operation asynchronously in accordance with the established configuration multi-threaded by intercepting the database component according to the database operation request, and adopts two methods, link one and link two, to perform database dual-write; the link one is the program running in the production environment. When the database operation is called, the main thread performs the database operation in the source library normally, and records the database operation log normally; the link two is the program running in the production environment. When the database operation is called, the AOP interceptor intercepts the database operation and performs asynchronous processing in the parallel thread;

[0083] The SQL parallel execution module is used to parallel execute different SQL statements with the same SQL identifier during the process of switching the electronic banking system to the domestic database;

[0084] The target database verification module is used to continuously collect the operating indicators of the target database in the production environment by running the target database persistently in the production environment, and use the persistent data to measure the indicators of the target database running in the production environment as the measurement standard for whether the target database meets the switching source database, and verify the target database. The operating indicators include coverage, abnormal error indicators, performance indicators, data consistency indicators and discard rate indicators.

[0085] like Figure 3 As shown, the specific process of using the database dual-write parallel operation device of the present invention to perform database dual-write parallel operation is as follows.

[0086] 1. Double write to the database.

[0087] Database dual writing can be divided into link 1 and link 2, where:

[0088] Link 1 is when the program is running in the production environment. When a database operation is called, the main thread performs the database operation normally in the source database and records the database operation log normally.

[0089] Link 2 is when the program is running in the production environment. When a database operation is called, the AOP interceptor intercepts the database operation and performs asynchronous processing in other threads.

[0090] like Figure 4 As shown, the process of database double writing of the present invention includes the following steps.

[0091] S11, add database parallel dual-write configuration;

[0092] The configuration is divided into data source information configuration and track test switch configuration. The data source information configuration is the connection configuration information of the track test target library, and the track test switch configuration is whether to turn on the track test. It supports two configuration modes: configuration file and configuration center. When using the configuration center, it supports real-time effectiveness of configuration item modifications.

[0093] S12, create a database dual-write interceptor in the project to intercept the unified entry for database execution operations;

[0094] This step is the prerequisite for database dual writing. It intercepts all database operations of the business library through the aspect, identifies the database operation type, clones the parameters of the database operation, saves the snapshot information of the database execution site, and implements the same database operation for link 1 and link 2.

[0095] S13, starting an asynchronous thread for a data source that enables database merge and dual write, and asynchronously performing a database merge operation;

[0096] For the operations of the main business database, after S12 identification, they are divided into two categories according to the operation type. The first category is query operations, which use a common thread pool for merge operations. The second category is add, delete, and modify operations. To ensure that the execution order of the merge database is consistent with that of the main database, an exclusive thread pool is used for execution. There is only one working thread in the exclusive thread pool. When the merge task is executed, other tasks are first put into the queue and executed in the order of entering the queue to ensure the consistency of the execution order of the change operation.

[0097] S14, after initiating the asynchronous operation, the multiple threads in the thread pool start to execute the database operation of the target database and separately record the log of this asynchronous operation;

[0098] The asynchronous thread task includes the timeout monitoring of the current database operation and the database parallel execution operation, outputting the data source used by the current database operation, execution time, connection acquisition time, execution result status display, exception information, thread pool name, and SQL type. After all the above operations are completed, the thread context information is cleared.

[0099] The log of this asynchronous operation is used to collect the operation status of the target database in the production environment. It is executed and recorded for a long time to serve as a basis for measuring whether the target database can switch to the source database.

[0100] 2. SQL statements are executed in parallel.

[0101] In the process of switching the domestic database of the electronic banking system, the parallel execution of SQL statements is used to solve the incompatibility problem of SQL statements in different databases. Since there is no SQL compatibility problem in ORM database operation, the present invention only targets named SQL. The parallel execution of SQL statements accurately describes the parallel execution of different SQL statements with the same SQL identifier. When a named SQL statement exists in two databases that are not compatible at the same time, it is necessary to develop SQL separately for a specific database to apply to this scenario. For the grammatical dialects of different databases, it is necessary to dynamically select compatible SQL statements for execution according to different databases.

[0102] The same SQL identifier means that there is only one SQL identifier logically, and physically the corresponding SQL identifier and SQL statement need to be defined in the Mapper.xml file according to different database types. From the developer's perspective, there is only one SQL identifier when it is passed in, and at runtime, the corresponding real physical SQL identifier will be dynamically routed according to different database types to achieve multi-database compatibility during the target database parallel operation without affecting the normal operation of the existing business logic.

[0103] like Figure 5 As shown, the process of executing SQL statements in parallel in the present invention includes the following steps.

[0104] S21. During the process of switching the electronic banking system to the domestic database, standard SQL identifiers are used in normal business development to comply with development specifications. The format is: main table name, operation name, SQL statement, and function description.

[0105] S22, configure a specific database name in the data source, declare the list of data sources supported by the current system, the data source can contain a normally running business library, and also supports the configuration of multiple libraries to be verified and tested. The configuration item is spring.datasource.multi.ds.dsNames=data source name list.

[0106] S23, configures the routing strategy for the parallel test in the data source, supports simultaneous parallel verification of multiple databases according to different conditions, for example, institution A uses A TEST DS (A TEST Decision Support Benchmark) for parallel verification, and institution B uses B TEST DS (B TEST Decision Support Benchmark). The number of parallel databases and strategies can be customized.

[0107] S24, SQL identifier for a specific database, in the format of: main table name, operation name, SQL statement, function description, data source identifier. Compared with the standard SQL defined in S21, a data source identifier field is added to support routing different SQL statements according to different data sources, solving the problem of inconsistent SQL statements due to different target databases for the same function under the parallel test. This type of routing specification is transparent to developers. When performing business development, the original SQL statement is used for database operations. The state framework automatically identifies the database and selects the corresponding SQL statement for execution.

[0108] After completing the above configuration, when a SQL instruction FUNCTION_A can be supported by database A but not by database B, if you want to execute FUNCTION_B in database B, the SQL identifier can be defined as the original standard SQL identifier _B. At this time, SQL only executes the SQL statement of FUNCTION_B on the data source of database B, and the SQL identifier passed in the program remains the original standard SQL identifier. In this way, during the target database integration, multi-database compatibility is supported without affecting the normal operation of the existing business logic.

[0109] 3. Target database verification.

[0110] The target database is persistently run in the production environment, and the operating indicators of the target database in the production environment are continuously collected. The indicators of the target database running in the production environment are measured with persistent data as the criterion for whether the target database meets the requirements for switching to the source database.

[0111] The target database and the source database's execution logs are recorded simultaneously to facilitate extraction and comparative analysis. The comparative analysis indicators mainly involve coverage, abnormal error indicators, performance indicators, data consistency indicators, and discard rate indicators.

[0112] like Figure 6 As shown, the process of verifying the target database of the present invention includes the following steps.

[0113] S31, first, dynamically compare the above indicators in the dynamic library of the production environment (here the dynamic library refers to the database that may change at any time), with the source database and the target database as the comparison objects, and the comparison content is the log content of the dynamic execution of the two, including coverage, abnormal error indicators, performance indicators, data consistency indicators, and discard rate indicators, to draw preliminary conclusions; the coverage indicator is that the total coverage of the source database and the target database in the production environment does not exceed 70%; the abnormal error indicator is that the number of abnormal errors of the target database in the production environment is 0; the performance indicator is that the overall SQL execution time of the target database in the production environment is higher than that of the source database by less than 50%, the average time consumption of the interface increases by less than 15%, and the application can support 8500TPS (Transactions Per Second); the data consistency indicator is that the SQL execution results of the source database and the target database in the production environment are consistent; the discard rate indicator is that the SQL execution discard rate of the target database in the production environment is 0, and there is no backlog in the update thread pool.

[0114] S32, then according to the preliminary conclusion of the dynamic comparison, a static comparison is performed in the static library (here the static library refers to the developer's private library, which will not cause database changes due to the operations of other people), and the SQL statements with inconsistent execution results or that do not meet the index requirements in the dynamic comparison results are put into the static library and compared again to verify whether it is caused by other operations of the dynamic library.

[0115] S33, finally, a dynamic and static persistence comparison is performed between the target database and the source database, and the database structure, number of rows affected by SQL, etc. are compared to form a phased comparative analysis result, and the result is used to determine whether the target database and the source database can be switched.

[0116] An electronic device provided by an embodiment of the present invention includes a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of the database dual-write and parallel operation method in any domestic database switching environment as described above.

[0117] Specifically, the above-mentioned memory and processor can be general-purpose memory and processor, which are not specifically limited here. When the processor runs the computer program stored in the memory, it can execute the database dual-write and parallel operation method in the above-mentioned domestic database switching environment.

[0118] Those skilled in the art will appreciate that the structure of the computer device does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently.

[0119] In some embodiments, the computer device may also include a touch screen that can be used to display a graphical user interface (e.g., a startup interface of an application) and receive user operations on the graphical user interface (e.g., startup operations on an application). The specific touch screen may include a display panel and a touch panel. The display panel may be configured in the form of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), etc. The touch panel may collect the user's contact or non-contact operations on or near it, and generate pre-set operation instructions, for example, the user uses any suitable object such as a finger, a stylus, or an accessory on or near the touch panel. In addition, the touch panel may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and posture, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into information that the processor can process, and then sends it to the processor, and can receive and execute commands from the processor. In addition, the touch panel can be implemented by various types such as resistive, capacitive, infrared and surface acoustic wave, and any technology developed in the future can also be used to implement the touch panel. Further, the touch panel can cover the display panel, and the user can operate on or near the touch panel covered on the display panel according to the graphical user interface displayed on the display panel. After the touch panel detects the operation on or near it, it is transmitted to the processor to determine the user input, and then the processor provides corresponding visual output on the display panel in response to the user input. In addition, the touch panel and the display panel can be implemented as two independent components or integrated.

[0120] Corresponding to the above-mentioned application startup method, an embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the database dual-write and parallel operation method in any of the above-mentioned domestic database switching environments are executed.

[0121] The application startup device provided in the embodiment of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided in the embodiment of the present application, its implementation principle and the technical effect produced are the same as those in the aforementioned method embodiment. For the sake of brief description, the parts not mentioned in the device embodiment can refer to the corresponding contents in the aforementioned method embodiment. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment, and will not be repeated here.

[0122] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0123] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0124] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, each functional module in the embodiments provided in the present application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0126] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for running a database dual-write parallel track in a domestic database switching environment, characterized in that: The steps include: Step S1, according to the database operation request, by intercepting the database component, according to the established configuration, multi-threaded asynchronous initiation of the domestic database switching operation, and using link 1 and link 2 to perform database dual writing; the link 1 is the program running in the production environment, when the database operation is called, the main thread normally performs the database operation in the source library, and the database operation log is normally recorded; the link 2 is the program running in the production environment, when the database operation is called, the AOP interceptor intercepts the database operation, and performs asynchronous processing in the parallel thread; Step S2, executing different SQL statements with the same SQL identifier in parallel during the process of switching the electronic banking system to the domestic database; Step S3, adopt the method of persistently running the target database in the production environment, continuously collect the operating indicators of the target database in the production environment, use the persistent data to measure the indicators of the target database running in the production environment as the measurement standard for whether the target database meets the switching source database, and verify the target database. The operating indicators include coverage, abnormal error indicators, performance indicators, data consistency indicators and discard rate indicators.

2. The method for running a database dual-write and parallel track under a domestic database switching environment according to claim 1 is characterized in that: In step S1, the dual write operation is not performed when the domestic database operation does not hit the configuration item; during the dual write operation of the database, all asynchronously initiated target database operations are completed using independent transactions, query transactions are executed using a shared thread pool, and non-query transactions are executed using a dedicated thread pool, so that the database operations executed in the merged thread are consistent with the execution order of the main thread database operations, and the execution results are written into a separate log file.

3. The method for running a database dual-write and parallel track under a domestic database switching environment according to claim 1, characterized in that: The link 2 is specifically: The interceptor determines whether the database operation needs to perform an asynchronous dual-write operation. If it is determined to be necessary, the interceptor initiates an asynchronous database operation instruction. After initiating the asynchronous operation, multiple threads in the thread pool begin to execute database operations on the target database and separately record the log of this asynchronous operation; the separate log of this asynchronous operation is used to collect the operation status of the target database in the production environment, and long-term execution and recording are used as a basis for measuring whether the target database can achieve the switching of the source database.

4. The method for running a database dual-write and parallel track under a domestic database switching environment according to claim 1, characterized in that: The step S1 comprises the following steps: Step S11, receiving a database operation request; Step S12, adding database track dual-write configuration, the database track dual-write configuration includes data source information configuration and track test switch configuration, the data source information configuration is the connection configuration information of the track test target library, the track test switch configuration is whether to turn on the track test, supports two configuration modes of configuration file and configuration center, and supports real-time effectiveness of configuration item modification when using the configuration center; Step S13, create a database parallel dual-write interceptor in the project, intercept the unified entrance of the database execution operation, intercept all database operations of the business library through the aspect, identify the database operation type, clone the parameters of the database operation, save the snapshot information of the database execution site, and realize that link 1 and link 2 perform the same database operation. The database operation type includes query operation and addition, deletion and modification operation; Step S14, start an asynchronous thread for the data source that enables database merge and dual write, asynchronously perform the database merge operation, the asynchronous thread tasks include timeout monitoring of the current database operation and database merge execution operation, the asynchronous thread outputs the data source used by the current database operation, execution time, connection acquisition time, execution result status display, exception information, thread pool name and SQL type; Step S15, after initiating the asynchronous operation, the multiple threads in the thread pool start to execute the database operation of the target database and separately record the log of this asynchronous operation.

5. The method for running a database dual-write and parallel track under a domestic database switching environment according to claim 1, characterized in that: In step S14, the query operation of the main business library is merged using a public thread pool, and the addition, deletion and modification operations of the main business library are executed using an exclusive thread pool. There is only one working thread in the exclusive thread pool. When the merge task is executed, other tasks are first put into the queue and executed in the order of entering the queue.

6. The method for running a database dual-write and parallel track under a domestic database switching environment according to claim 1, characterized in that: The step S2 comprises the following steps: Step S21, during the process of switching the electronic banking system to the domestic database, standard SQL identifiers are used for normal business development. The format of standard SQL identifiers is: main table name, operation name, SQL statement and function description; Step S22, configuring the database name in the data source, and declaring the data source list supported by the current system; Step S23, configuring a routing strategy for the merge test in the data source, supporting simultaneous merge verification of multiple libraries according to different conditions; Step S24, set the SQL identifier of the database configured in step S22, the format of which is: main table name, operation name, SQL statement, function description and data source identifier. When performing business development, the original SQL statement is used for database operation. The state framework automatically identifies the database and selects the corresponding SQL statement for execution.

7. The method for running a database dual-write and parallel track in a domestic database switching environment according to any one of claims 1 to 6, characterized in that: The step S3 comprises the following steps: Step S31, dynamically compare the operation indicators in the dynamic library of the production environment, with the source database and the target database as the comparison objects, and the comparison content is the log content of the dynamic execution of the two, and draw a preliminary conclusion; Step S32: Based on the preliminary conclusion, static comparison is performed in the static library, and SQL statements with inconsistent execution results or not meeting the index requirements in the dynamic comparison results are put into the static library for comparison again to verify whether it is caused by other operations in the dynamic library; Step S33, compare the target database with the source database in terms of dynamic and static persistence, compare the database structure and the number of rows affected by SQL, form a phased comparison analysis result, and determine whether the target database and the source database can be switched based on the comparison analysis result.

8. A database dual-write parallel operation device in a domestic database switching environment, characterized in that: include: The database dual-write module is used to initiate the domestic database switching operation asynchronously in accordance with the established configuration multi-threaded by intercepting the database component according to the database operation request, and adopts two methods, link one and link two, to perform database dual-write; the link one is the program running in the production environment. When the database operation is called, the main thread performs the database operation in the source library normally, and records the database operation log normally; the link two is the program running in the production environment. When the database operation is called, the AOP interceptor intercepts the database operation and performs asynchronous processing in the parallel thread; The SQL parallel execution module is used to parallel execute different SQL statements with the same SQL identifier during the process of switching the electronic banking system to the domestic database; The target database verification module is used to continuously collect the operating indicators of the target database in the production environment by running the target database persistently in the production environment, and use the persistent data to measure the indicators of the target database running in the production environment as the measurement standard for whether the target database meets the switching source database, and verify the target database. The operating indicators include coverage, abnormal error indicators, performance indicators, data consistency indicators and discard rate indicators.

9. An electronic device, characterized in that: It includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the database dual-write and parallel operation method in the domestic database switching environment as described in any one of claims 1-7.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for running a database dual-write and parallel track in a domestic database switching environment as described in any one of claims 1 to 7.

Images