Database switching method and device, medium and program product

Abstract

The embodiment of the invention discloses a database switching method and device, a medium and a program product. The method comprises the steps that synchronous processing is conducted in a first database, and asynchronous processing is conducted in a second database; if the target data processing operation in the first database and the second database is executed in the data persistence framework, maintaining a grammar corresponding to the first database and a grammar corresponding to the second database in a mapping file for loading; if the asynchronous processing of the target data in the second database in the first preset time period is not abnormal, hot switching is carried out to be synchronous processing in the second database, and asynchronous processing is carried out in the first database; and if the synchronization processing of the target data in the second database in the second preset time period is still not abnormal, the synchronization processing of the target data is only performed in the second database. According to the scheme, hot switching can be realized in a data persistence framework or a non-data persistence framework under the condition that normal business execution of the database is not influenced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database switching, and in particular to a database switching method, device, medium and program product. BACKGROUND

[0002] With the continuous expansion of China's digital economy, the dependence on information technology hardware and software in various fields is deepening. In the past, due to the lack of domestic related technology, these hardware and software procurement mainly relied on imported products, especially some key equipment and software. This dependence has brought many problems, such as information security risks, technical barriers, high procurement costs, etc. Especially for financial information systems, information security is related to the national economy and people's livelihood. The increasing maturity of domestic hardware and software products makes it possible for financial information system hardware and software resources to be self-controllable.

[0003] A certain system is a management system for asset investment and asset-side business of a bank company, which covers various aspects of business operation processes such as product management, portfolio management, investment management, transaction management, and risk management modules. The system is large in scale and highly professional. According to the compliance control requirements, database A needs to be withdrawn from system use, and the inventory application needs to be migrated to the self-developed database B. How to efficiently, smoothly, quickly and safely complete the software self-migration of heterogeneous databases has become a severe challenge faced by many current systems.

[0004] In the existing database software switching technology between systems, the current technical solution adopts a static switching mode, that is, the data of the first database master is synchronized to the first database backup, the connection between the first database master and the backup is disconnected and the log record sequence number of the first database is recorded; the full data in the first database backup is migrated to the second database, and the data migration of the first database backup to the second database is analyzed through the data table list and the data record number; when the data migration meets the preset condition, the incremental data generated after the first database master and backup are disconnected according to the log record sequence number is migrated to the second database. The above-mentioned static database migration and system switching mode is limited in operation and only supports system scenarios with weak business continuity, simple business logic and few associated parties. For large-scale, complex business logic and many associated parties, static and accurate migration cannot be achieved. SUMMARY

[0005] The embodiments of the present application provide a database switching method, device, medium and program product to realize hot switching of a database without affecting normal business processing of the database.

[0006] According to an aspect of the present application, a database switching method is provided, which comprises:

[0007] The target data is synchronously processed in the first database and asynchronously processed in the second database; if the target data processing operation in the first database and the second database is performed in a data persistence framework, the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in a mapping file for loading;

[0008] If no exception occurs in the asynchronous processing of the target data in the second database within a first preset time period, the synchronous processing of the target data in the second database is hot-switched, and the asynchronous processing of the target data in the first database is performed;

[0009] If no exception still occurs in the synchronous processing of the target data in the second database within a second preset time period, only the synchronous processing of the target data in the second database is performed.

[0010] According to an aspect of the present application, a database switching device is provided, and the device comprises:

[0011] The first processing module is configured to synchronously process target data in the first database and asynchronously process the target data in the second database; if the target data processing operation in the first database and the second database is performed in a data persistence framework, the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in a mapping file for loading;

[0012] The second processing module is configured to, if no exception occurs in the asynchronous processing of the target data in the second database within a first preset time period, hot-switch the synchronous processing of the target data in the second database and perform the asynchronous processing of the target data in the first database;

[0013] The switching module is configured to, if no exception still occurs in the synchronous processing of the target data in the second database within a second preset time period, only perform the synchronous processing of the target data in the second database.

[0014] According to another aspect of the present application, an electronic device is provided, and the electronic device comprises:

[0015] At least one processor; and

[0016] The memory is in communication connection with the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the database switching method of any embodiment of the present application.

[0018] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for causing a processor to implement the database switching method of any of the embodiments of the present application when executed.

[0019] According to another aspect of the present application, a computer program product is provided, which comprises a computer program for implementing the database switching method of any of the embodiments of the present application when executed by a processor.

[0020] The technical solution of the embodiments of the present application performs synchronous processing of target data in the first database and performs asynchronous processing of target data in the second database; if the target data processing operation in the first database and the second database is performed in a data persistence framework, the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in a mapping file for loading; if no exception occurs in the asynchronous processing of target data in the second database within a first preset time period, the synchronous processing of target data in the second database and the asynchronous processing of target data in the first database are switched; if no exception still occurs in the synchronous processing of target data in the second database within a second preset time period, only the synchronous processing of target data in the second database is performed. The above-mentioned solution can maintain the syntax corresponding to the first database and the syntax corresponding to the second database in a mapping file for loading, thereby achieving double writing of data in a data persistence framework and realizing hot switching of the database without affecting the normal execution of business in the process of double writing.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0023] Figure 1 A flowchart of a database switching method provided by an embodiment of the present application is shown in FIG. 1;

[0024] Figure 2 A flowchart of a database switching method provided by another embodiment of the present application is shown in FIG. 2;

[0025] Figure 3A flow chart of a database switching method provided for another embodiment of the present application is shown in FIG. 1;

[0026] Figure 4 A flow chart of a double-write operation in a non-data persistent framework provided for an embodiment of the present application is shown in FIG. 2;

[0027] Figure 5 A schematic diagram of an overall processing structure provided for an embodiment of the present application is shown in FIG. 3;

[0028] Figure 6 A schematic diagram of an AB phase provided for an embodiment of the present application is shown in FIG. 4;

[0029] Figure 7 A schematic diagram of a BA phase provided for an embodiment of the present application is shown in FIG. 5;

[0030] Figure 8 A schematic diagram of a database switching device provided for an embodiment of the present application is shown in FIG. 6;

[0031] Figure 9 A schematic diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0032] In order to make the personnel in the technical field better understand the present application scheme, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without creative labor should belong to the scope of protection of the present application.

[0033] It should be noted that the terms "first", "second", "third", "fourth", "actual", "preset" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0034] Figure 1A flowchart of a database switching method provided in an embodiment of the present application, the embodiment of the present application can be applicable to the case of hot switching of a database. The method can be executed by a database switching apparatus, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in Figure 1 the method includes the following steps.

[0035] S110, performing synchronization processing of target data in the first database and performing asynchronous processing of the target data in the second database; if the target data processing operation in the first database and the second database is executed in a data persistence framework, the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in a mapping file for loading.

[0036] In the embodiment of the present application, the first database and the second database can be the same type of database, or can be different types of databases. Generally, the first database is applied to the first system, and the second database is applied to the second system. In the case of switching the system from the first system to the second system, the database also needs to be switched from the first database to the second database. In the embodiment of the present application, the first database is the old database corresponding to the old system, and the second database is the new database corresponding to the new system, and switching from the first database to the second database is required. The processing includes operations such as adding, deleting, modifying and querying. Synchronization processing means performing in sequence, and the next step can be performed only after the current processing is completed. Asynchronous processing means that the task can be executed concurrently, and the current task returns the result immediately after being initiated without waiting, and the subsequent code can be executed. For example, after reading the data in the first database synchronously, the read data is returned to the business layer, and the next reading data can be performed. In the second database, the data is read asynchronously, and the read data can be returned to the business layer without waiting, and the next reading data can be directly performed.

[0037] Exemplarily, in the case that the business requirement of the business layer is processed, the synchronization processing of the target data is performed in the first database, and the processing result is returned to the business layer for application, display, etc., and the asynchronous processing of the target data is performed in the second database. Since the second database does not completely switch to process the business data, the processing result of the target data in the second database is not completely reliable, and thus the result of the asynchronous processing of the target data in the second database does not need to be returned to the business layer. The result of the asynchronous processing of the target data in the second database can be used to compare with the result of the asynchronous processing of the target data in the first database, so as to verify the effectiveness of the asynchronous processing of the target data in the second database.

[0038] In the embodiment of the present application, the synchronization processing of the target data in the first database and the asynchronous processing of the target data in the second database can be executed in a data persistence framework or in a non-data persistence framework. If executed in the data persistence framework, since there is a syntax difference between different databases, the same syntax cannot be used for execution in the first database and execution in the second database, and thus the syntax corresponding to the first database can be maintained in a mapping file in advance, and the syntax corresponding to the second database can be maintained in the mapping file in advance, so that different syntaxs are loaded for execution according to different databases when the operation of the target data is executed.

[0039] S120, if the asynchronous processing of the target data in the second database does not occur abnormally in the first preset time period, hot switching is performed to perform the synchronization processing of the target data in the second database and perform the asynchronous processing of the target data in the first database.

[0040] The occurrence of an abnormality can include various cases, for example, a statement cannot be executed normally, a database processing abnormality caused by an abnormality of the statement, a processing timeout, an incorrect processing result, etc.

[0041] Exemplarily, in the process of the asynchronous processing of the target data in the second database, if no abnormality occurs, it is reflected that the asynchronous processing of the target data in the second database can be processed normally, the second database can play the same role as the first database and process the same business, and thus hot switching can be performed to perform the synchronization processing of the target data in the second database and perform the asynchronous processing of the target data in the first database, that is, the database is tried to switch, and the two databases still undertake business processing at the same time, so that in the case of trying to use the second database as the main database, the first database is used as a guarantee, and in the case that the second database processing is wrong after switching, the first database can be switched back in time.

[0042] In the embodiment of the present application, the first preset time period can be a time period set according to actual conditions to verify whether the asynchronous processing in the second database is normal, so as to avoid accidental normality being identified as no exception, resulting in misjudgment. If the asynchronous processing of the target data in the second database has lasted for the first preset time period and no exception occurs, it is considered that the second database can normally and stably process the target data and is stable and reliable, and the hot switching is performed to the synchronous processing of the target data in the second database.

[0043] In S130, if the synchronous processing of the target data in the second database still does not occur within the second preset time period, only the synchronous processing of the target data in the second database is performed.

[0044] The second time period is a time period set according to actual conditions to verify whether the synchronous processing in the second database is normal. After the hot switching to the synchronous processing of the target data in the second database and the asynchronous processing of the target data in the first database, the duration of the synchronous processing in the second database is counted. If the duration reaches the second preset time period and no exception occurs, it is determined that the synchronous processing in the second database is reliable, and therefore full switching can be realized, that is, only the synchronous processing of the target data in the second database is performed, and no data processing is performed in the first database, the business is completely switched to the second database, and the switching of the new and old systems is also realized.

[0045] The technical scheme of the embodiment of the present application performs the synchronous processing of the target data in the first database and the asynchronous processing of the target data in the second database. If the target data processing operation in the first database and the second database is performed in a data persistence framework, the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in a mapping file for loading. If no exception occurs in the asynchronous processing of the target data in the second database within the first preset time period, the hot switching is performed to the synchronous processing of the target data in the second database, and the asynchronous processing of the target data in the first database. If the synchronous processing of the target data in the second database still does not occur within the second preset time period, only the synchronous processing of the target data in the second database is performed. The above scheme can maintain the syntax corresponding to the first database and the syntax corresponding to the second database in the mapping file for loading, so as to realize the double processing of data in the data persistence framework and realize the hot switching of the database without affecting the normal execution of the business of the database in the process of double processing.

[0046] As a non-limiting implementation, if the target data processing operation in the first database and the second database is performed in a data persistence framework, the synchronization processing of the target data is performed in the first database, and the asynchronous processing of the target data is performed in the second database, including:

[0047] intercepting the method executed by the data access layer in the data persistence framework;

[0048] modifying the method executed by the data access layer to first perform the synchronization processing of the target data in the first database, and then perform the asynchronous processing of the target data in the second database using an asynchronous thread.

[0049] For example, if the double writing of the first database and the second database is implemented in the data persistence framework, the method executed by the data access layer in the data persistence framework can be intercepted. The data access layer (Data Access Object) is a core component in the data persistence framework, which is an abstraction layer between business logic and data storage, mainly responsible for encapsulating all operations interacting with the database. It decouples CRUD (Create, Read, Update, Delete) data operations and specific implementations by defining a unified interface specification, so that business code does not directly depend on database technical details, thereby improving system maintainability, testability and flexibility. Modify the data access layer to first perform the synchronization processing of the target data in the first database, and then perform the asynchronous processing of the target data in the second database using an asynchronous thread.

[0050] As a non-limiting implementation, the method executed by the data access layer is modified to first perform the synchronization processing of the target data in the first database, and then perform the asynchronous processing of the target data in the second database using an asynchronous thread, including:

[0051] obtaining a first database identifier of the first database, obtaining and loading a processing statement corresponding to the first database identifier from a mapping file based on the first database identifier, and performing synchronization processing of the target data in the first database based on the processing statement corresponding to the first database identifier;

[0052] obtaining a second database identifier of the second database, obtaining and loading a processing statement corresponding to the second database identifier from a mapping file based on the second database identifier, and performing asynchronous processing of the target data in the second database based on the processing statement corresponding to the second database identifier.

[0053] Exemplarily, since there can be syntax differences between the first database and the second database, the syntax written for the first database can not be executed in the second database, and therefore the syntax corresponding to the first database and the syntax corresponding to the second database can be maintained in the mapping file. In the mapping file, the first database identifier of the first database is stored in association with the corresponding syntax, and the second database identifier of the second database is stored in association with the corresponding syntax. In the specific processing, the first database identifier of the first database is acquired, the processing statement formed by loading the syntax corresponding to the first database identifier from the mapping file is acquired based on the first database identifier, and the synchronization processing of the target data in the first database is performed based on the processing statement corresponding to the first database identifier. The second database identifier of the second database is acquired, the processing statement formed by loading the syntax corresponding to the second database identifier from the mapping file is acquired based on the second database identifier, and the asynchronous processing of the target data in the second database is performed based on the processing statement corresponding to the second database identifier. The above scheme maintains the syntax corresponding to each database, and adaptively loads the processing statement formed by the syntax corresponding to the database identifier for execution in the specific execution, solves the problem that it is difficult to realize the double writing of the database in the current data persistence framework for processing the data of one database, and realizes the double writing to the first database and the second database.

[0054] As a non-limiting implementation, if the target data processing operation in the first database and the second database is executed in a non-data persistence framework, the synchronization processing of the target data in the first database and the asynchronous processing of the target data in the second database are performed, comprising:

[0055] The database type and the corresponding processing statement of the first database are determined respectively, and the database type and the corresponding processing statement of the second database are determined respectively;

[0056] The processing statement is executed in the first database and the second database respectively for the target data.

[0057] Exemplarily, in the process of realizing the synchronization processing of the target data in the first database and the asynchronous processing of the target data in the second database in the non-data persistence framework, the database type and the corresponding processing statement of the first database can be determined respectively, and the database type and the corresponding processing statement of the second database can be determined respectively. The processing statement is executed in the first database and the second database respectively for the target data. The data type can include a master database or a slave database. The process of performing the synchronization processing of the target data in the first database and the asynchronous processing of the target data in the second database can be executed simultaneously, or the process of performing the synchronization processing of the target data in the first database can be performed first, and then the process of performing the asynchronous processing of the target data in the second database can be performed.

[0058] It should be noted that the above scheme takes the synchronization of target data in the first database and the asynchronous processing of target data in the second database as an example, respectively describes the processes executed in the data persistence framework and the processes executed in the non-data persistence framework, and the same applies to the synchronization of target data in the second database and the asynchronous processing of target data in the second database.

[0059] Figure 2 A flowchart of a database switching method provided for another embodiment of the present application is based on the optimization of the above embodiment, and the schemes not described in detail in the present embodiment are described in the above embodiment. As shown in the figure, the method of the present embodiment specifically includes the following steps: Figure 2

[0060] S210, the synchronization of target data in the first database and the asynchronous processing of target data in the second database are performed; if the target data processing operation in the first database and the second database is executed in the data persistence framework, the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in the mapping file for loading.

[0061] S220, if the asynchronous processing of target data in the second database does not occur within the first preset time period, the synchronization of target data in the second database is hot switched to the synchronization of target data in the first database and the asynchronous processing of target data in the second database.

[0062] S230, if the synchronization of target data in the second database still does not occur within the second preset time period, only the synchronization of target data in the second database is performed.

[0063] S240, if an exception occurs in the process of the synchronization of target data in the second database, the synchronization of target data in the first database is hot switched back to the synchronization of target data in the first database and the asynchronous processing of target data in the second database.

[0064] For example, after hot switching to the second database for the synchronization of target data and the first database for the asynchronous processing of target data, if an exception occurs in the synchronization of target data in the second database, the synchronization of target data in the first database is hot switched back to the synchronization of target data in the first database and the asynchronous processing of target data in the second database, thereby rescuing the exception in the synchronization of target data in the second database and enabling the synchronization of target data in the first database to be normally performed.

[0065] ​S250, if an exception occurs in the asynchronous processing of the target data in the second database, stopping the operation of performing the asynchronous processing of the target data in the second database, and only performing the operation of performing the synchronous processing of the target data in the first database.

[0066] For example, if the synchronous processing of the target data is performed in the first database, and an exception occurs in the asynchronous processing of the target data in the second database, it is reflected that the second database cannot perform the asynchronous processing of the target data from the beginning, the operation of performing the asynchronous processing of the target data in the second database is stopped, and only the operation of performing the synchronous processing of the target data in the first database is performed.

[0067] In the embodiment of the present application, if an exception occurs in the asynchronous processing of the target data in the second database, the operation of performing the asynchronous processing of the target data in the second database is stopped, and only the operation of performing the synchronous processing of the target data in the first database is performed; if an exception occurs in the synchronous processing of the target data in the second database, the synchronous processing of the target data in the first database is hot switched back, and the asynchronous processing of the target data in the second database is performed. That is, different strategies are executed in different stages. If an exception occurs in the asynchronous processing of the target data in the second database, it is reflected that the second database cannot perform the asynchronous processing of the target data, and the operation of performing the processing of the target data in the second database is directly stopped. If an exception occurs in the synchronous processing of the target data in the second database, it is reflected that the exception occurs in the second database after the hot switching of the master, and the synchronous processing of the target data in the first database is hot switched back first, so that the second database returns to the stage of performing the asynchronous processing of the target data, thereby ensuring that the business data is normally processed, and the exception in the second database is investigated.

[0068] Figure 3 A flowchart of a database switching method provided by another embodiment of the present application is shown in FIG. 10. The embodiment of the present application is optimized based on the above-described embodiment, and the solutions not described in detail in the embodiment of the present application are described in the above-described embodiment. As shown in FIG. 10, the method of the embodiment of the present application specifically includes the following steps: Figure 3

[0069] S310, performing the synchronous processing of the target data in the first database, and performing the asynchronous processing of the target data in the second database; if the target data processing operation in the first database and the second database is performed in a data persistence framework, maintaining the syntax corresponding to the first database and the syntax corresponding to the second database in a mapping file for loading. ​

[0070] S320, if no exception occurs in the asynchronous processing of the target data in the second database within the first preset time period, hot switching to synchronous processing of the target data in the second database and asynchronous processing of the target data in the first database.

[0071] S330, if no exception still occurs in the synchronous processing of the target data in the second database within the second preset time period, only synchronous processing of the target data in the second database is performed.

[0072] S340, if an exception occurs in the asynchronous processing or the synchronous processing of the target data in the second database, an exception message is sent to a message middleware; wherein the exception includes at least one of an execution processing statement prompt error, a processing timeout, and a processing result of the target data in the first database inconsistent with a processing result of the target data in the second database.

[0073] The message middleware is a communication infrastructure in a distributed system, which realizes asynchronous data transmission between different system components through a message queue mechanism, effectively solves the coupling problem between services and improves system reliability. Its core working principle is that the producer sends messages to the middleware server for temporary storage, and the consumer obtains and processes the messages from the queue as needed, and the whole process does not require real-time online or direct interaction of both parties. This technology is widely used in traffic peak clipping, system decoupling, asynchronous processing and other scenarios, and is a key component for building a flexible distributed architecture.

[0074] For example, if an exception occurs in the second database during the synchronous processing of the target data in the first database and the asynchronous processing of the target data in the second database, or during the asynchronous processing of the target data in the first database and the synchronous processing of the target data in the second database, an exception message is sent to the message middleware, so that the exception message is counted and saved. In addition, the exception message is sent to the message middleware without being directly sent to the first database, thereby solving the problem of accumulation of exception messages in the first database, occupation of resources and influence on normal business processing in the first database when there are too many exception messages.

[0075] S350, subscribing and consuming the exception message from the message middleware through a public microservice, and writing the exception message into the first database.

[0076] The public microservice refers to a basic service module in a microservice architecture that is commonly called by multiple business services.

[0077] In the embodiment of the present application, the public microservice can subscribe and consume the exception message from the message middleware, that is, in the message queue system, the consumer actively or passively processes the message marked as an exception state due to the failure of consumption, and writes the exception message into the first database.

[0078] In the process of performing the asynchronous processing or the synchronous processing of the target data in the second database, the scheme of the embodiment of the present application sends an exception message to the message middleware; wherein the exception includes at least one of the following: an error of executing a processing statement prompt, a processing timeout, and a processing result of the target data in the first database being inconsistent with a processing result of the target data in the second database; the public microservice subscribes and consumes the exception message from the message middleware, and writes the exception message into the first database. The above scheme can store the exception message in time when the second database processes the target data and an exception occurs, facilitating the analysis of the exception of the second database by the staff based on the exception message. Moreover, the message middleware stores and manages the exception message stack, which can avoid the problem of the first database being occupied by the message accumulation and affecting the normal processing of the business due to the exception message being sent to the first database.

[0079] As a non-limiting implementation, the process of hot switching includes:

[0080] Real-time monitoring of whether the hot switching switch is opened, if the hot switching switch is opened, refreshing the flag bit; wherein the flag bit is used to mark the database of the synchronous processing in the first database and the second database;

[0081] According to the flag bit, the hot switching of the first database and the second database is determined.

[0082] In the embodiment of the present application, in order to realize the switching of the first database and the second database without interrupting the ongoing data processing task, the switching of the first database and the second database can be performed, and the hot switching of the first database and the second database is realized by configuration change. Specifically, whether the hot switching switch is opened can be detected in real time, if the hot switching switch is opened, the flag bit is refreshed, and the hot switching of the first database and the second database is determined according to the flag bit. The flag bit is used to mark the database of the synchronous processing in the first database and the second database, that is, the master database. If the flag bit reflects that the first database is the master database, the synchronous processing of the target data is performed in the first database, and the asynchronous processing of the target data is performed in the second database. If the flag bit reflects that the second database is the master database, the asynchronous processing of the target data is performed in the first database, and the synchronous processing of the target data is performed in the second database.

[0083] The embodiment of the application provides a specific implementation, comprising:

[0084] In the double-writing scheme, based on the data persistence framework, the dynamic data source is bound with SqlSessionFactory, so that the multi-data source loading and smooth switching are realized. The custom PersistenceConfiguration, DatabaseIdProvider and SqlSessionFactory are bound, so that the heterogeneous database data persistence SQL dynamic loading is realized. The aspect is made in the DAO layer, the DAO layer execution method is intercepted, the main database executes the operation, the auxiliary database is switched, the same operation is executed asynchronously, and the double-writing of the heterogeneous database application is realized.

[0085] 6.1 Data persistence framework double-writing implementation steps

[0086] a) Dynamic data source definition and initialization

[0087] Inheritance AbstractRoutingDataSource, and implementation determineCurrentLookupKey method, when the data persistence framework obtains the data source, the method is called to obtain the corresponding identifier, and the corresponding data source is returned. The DataSourceContext class is used for managing the data source identifier used by the current thread, and the data source identifier can be set by calling setCurrentDataSource, so that the switching of the data source is realized.

[0088] b) PersistenceConfiguration, DatabaseIdProvider definition and initialization

[0089] The application double-writing involves distributed and database A, and the SQL syntax has differences. The DatabaseIdProvider can make a project support different databases, and for the SQL with syntax differences, the syntax of different databaseIds is maintained in XML, the addMappedStatement method of PersistenceConfiguration is rewritten, and two kinds of syntax MappedStatement are saved to different lists. The getMappedStatement method is rewritten to realize the loading of different data sources and different syntax SQL.

[0090] c) PersistenceConfiguration, DatabaseIdProvider, DynamicDataSource and SqlSessionFactory binding, realizing the dynamic loading of the data source and SQL

[0091] d) DAO aspect definition and initialization implementation application data double write

[0092] Define the aspect, intercept the method executed by the DAO layer, execute the primary library read-write first, and then use the asynchronous thread pool to implement the asynchronous read-write of the secondary library.

[0093] 6.2 Non-data persistence framework data double write

[0094] Non-data persistence framework data write operation, need to customize the implementation of data double write, logic like Figure 4 .

[0095] 1. Get the primary data source, the primary data source database type and the corresponding SQL, execute the primary data source read-write;

[0096] 2. Asynchronously start a thread, get the secondary data source, the secondary data source database type and the corresponding SQL, asynchronously execute the secondary data source read-write.

[0097] 6.3 Asynchronous slave write exception record and processing

[0098] In order to ensure the reliability of application double write, when asynchronous write exception occurs, the exception execution SQL needs to be recorded. For the SQL that fails to execute, manually analyze the failure reason to ensure that the problem is solved and the primary and secondary library data are consistent.

[0099] For the SQL that is executed for a long time, manually optimize the SQL to ensure that the database B SQL execution performance is consistent with that of database A.

[0100] Main process:

[0101] 1. The error message of the secondary library SQL execution exception or timeout is sent to the message middleware;

[0102] 2. The public microservice subscribes and consumes error messages from the message middleware and writes them to the primary database;

[0103] 3. Manually confirm whether the error SQL can be run again and mark, and execute the failed SQL in batches at regular intervals.

[0104] 6.4 Application migration AB stage

[0105] As shown in Figure 5 , the primary read-write of database A, asynchronous read-write of database B. This stage mainly executes SQL in database B asynchronously. Use the production traffic. Verify the consistency of the execution results of the DAO layer of database B and the DAO layer of database A. This stage lasts for two months, and if the following conditions are met, it is considered to enter the BA stage.

[0106] a). There is no syntax error in the asynchronous read-write of database B during the parallel period.

[0107] b). No timeout problem affecting important business processes during the parallel period.

[0108] c). No data inconsistency problem during the parallel period.

[0109] Effective SQL execution coverage reaches 90%, and core SQL execution rate coverage reaches 100%

[0110] 6.5 Application migration BA phase

[0111] This phase is based on database B, and writes to database A asynchronously. This phase mainly implements real-time synchronization of database B and database A data by asynchronously executing write SQL in relational databases, ensuring quick rollback to database B in the initial stage of switching. This phase lasts for one month. If the following conditions are met, the switching is complete.

[0112] a). No syntax error when reading and writing database B during the parallel period.

[0113] b). No timeout problem affecting important business processes during the parallel period.

[0114] c). No data inconsistency problem during the parallel period.

[0115] 6.6 Microservice hot rollback

[0116] During the peak period of business use, it is necessary to ensure the continuity of business. If abnormal conditions occur, it is necessary to implement rollback without interrupting the processing of transactions. Microservice master and asynchronous write hot rollback are achieved through configuration changes. The implementation steps are as follows.

[0117] a). Real-time monitoring of hot switch opening, if opened, refresh the flag bit.

[0118] b). DoubleWrite, DynamicDataSource, PersistenceConfiguration complete master and asynchronous write database switching according to the flag bit.

[0119] 6.7 Only database B phase

[0120] After long-term verification of production data double-write, ensure full business scenario coverage, and read and write database B is stable, then database B runs independently, and the database switching is officially completed.

[0121] 6.8 Exception handling

[0122] If an exception occurs during the data double-write phase, the microservice granularity function rollback can be achieved through configuration.

[0123] a). AB stage: abnormality is found after running for a period of time, and analysis is performed according to the abnormality: if asynchronous writing of the database B causes abnormality of the application server performance, and further causes abnormality of the business main process, the asynchronous writing of the database B switch is disconnected; if Figure 6 as shown.

[0124] b). BA stage: the performance of the main read-write database B stage appears abnormality or other needs to switch back to the database A, and the AB state is changed through configuration. As shown in Figure 7 , the dynamic data source includes a data source A (A database) and a data source B (B database), the running stage defaults to the data source B, the asynchronous execution data source is A, the data persistence framework dynamically configured includes the data source A configuration (A database syntax sql) and the data source B configuration (B database syntax sql), the running stage defaults to using the data source B configuration, the asynchronous execution uses the data source A configuration, the abnormality switching stage defaults to using the A data source and the A configuration, and the asynchronous execution uses the B data source B configuration. As shown in Table 1.

[0125] Table 1

[0126]

[0127] Figure 8 A structure diagram of a database switching device provided by an embodiment of the present application is shown in Figure 8 , the device includes:

[0128] The first processing module 410 is configured to perform synchronous processing of target data in a first database and perform asynchronous processing of the target data in a second database; if the target data processing operation in the first database and the second database is performed in a data persistence framework, the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in a mapping file for loading.

[0129] The second processing module 420 is configured to hot switch to perform synchronous processing of target data in the second database and perform asynchronous processing of the target data in the first database if no abnormality occurs in the asynchronous processing of the target data in the second database within a first preset time period.

[0130] The switching module 430 is configured to perform only synchronous processing of target data in the second database if no abnormality occurs in the synchronous processing of the target data in the second database within a second preset time period.

[0131] In the embodiment of the present application, if the target data processing operation in the first database and the second database is executed in a data persistence framework, the first processing module 410 performs synchronous processing of the target data in the first database and performs asynchronous processing of the target data in the second database, including:

[0132] intercepting a method executed by a data access layer in the data persistence framework;

[0133] modifying the method executed by the data access layer to first perform synchronous processing of the target data in the first database, and then perform asynchronous processing of the target data in the second database using an asynchronous thread.

[0134] In the embodiment of the present application, the first processing module 410 modifies the method executed by the data access layer to first perform synchronous processing of the target data in the first database, and then perform asynchronous processing of the target data in the second database using an asynchronous thread, including:

[0135] obtaining a first database identifier of the first database, obtaining and loading a processing statement corresponding to the first database identifier from a mapping file based on the first database identifier, and performing synchronous processing of the target data in the first database based on the processing statement corresponding to the first database identifier;

[0136] obtaining a second database identifier of the second database, obtaining and loading a processing statement corresponding to the second database identifier from a mapping file based on the second database identifier, and performing asynchronous processing of the target data in the second database based on the processing statement corresponding to the second database identifier.

[0137] In the embodiment of the present application, if the target data processing operation in the first database and the second database is executed in a non-data persistence framework, the first processing module 410 performs synchronous processing of the target data in the first database and performs asynchronous processing of the target data in the second database, including:

[0138] determining the database type and the corresponding processing statement of the first database, and the database type and the corresponding processing statement of the second database, respectively;

[0139] executing the processing statement in the first database and the second database for the target data, respectively.

[0140] In the embodiment of the present application, the apparatus further includes:

[0141] The abnormal message sending module is configured to send an abnormal message to the message middleware if an abnormality occurs in the process of performing the asynchronous processing or the synchronous processing of the target data in the second database; wherein the abnormality includes at least one of the following: an error in executing a processing statement, a processing timeout, and a processing result of the target data in the first database being inconsistent with a processing result of the target data in the second database.

[0142] The abnormal message writing module is configured to subscribe to and consume the abnormal message from the message middleware through the common microservice, and write the abnormal message into the first database.

[0143] In the embodiments of the present application, the apparatus further comprises:

[0144] The second database notification processing module is configured to, if an abnormality occurs in the process of performing the asynchronous processing of the target data in the second database, stop the operation of performing the asynchronous processing of the target data in the second database, and only perform the operation of performing the synchronous processing of the target data in the first database.

[0145] The hot switching module is configured to, if an abnormality occurs in the process of performing the synchronous processing of the target data in the second database, hot switch back to performing the synchronous processing of the target data in the first database and performing the asynchronous processing of the target data in the second database.

[0146] In the embodiments of the present application, the apparatus further comprises:

[0147] The flag bit refreshing module is configured to monitor whether the hot switching switch is opened in real time, and refresh the flag bit if the hot switching switch is opened; wherein the flag bit is used to mark the database of the synchronous processing in the first database and in the second database.

[0148] The hot switching determining module is configured to determine the hot switching of the first database and the second database according to the flag bit.

[0149] The database switching apparatus provided in the embodiments of the present application can perform the database switching method provided in any of the embodiments of the present application, and has the corresponding function modules and beneficial effects of performing the method.

[0150] Figure 9A schematic diagram of an electronic device 10, which can be used to implement embodiments of this application, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0151] like Figure 9 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0152] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless database switching transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0153] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the database switching method.

[0154] In some embodiments, the database switching method can be implemented as a computer program tangibly embodied in a computer readable storage medium, e.g., storage unit 18. In some embodiments, parts or all of the computer program can be loaded and / or installed onto electronic device 10 via, e.g., ROM 12 and / or communication unit 19. When the computer program is loaded onto RAM 13 and executed by processor 11, one or more steps of the above-described database switching method can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the database switching method by way of other means, e.g., by way of firmware.

[0155] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0156] Computer programs used to implement the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed by the processor of the computer or other programmable data processing apparatus, enables the systems and methods as claimed in the claims to be implemented. The computer program can be executed entirely on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0157] In the context of this application, a computer readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer readable storage medium can be a machine readable signal medium. More specific examples of the machine readable storage medium will include a one or more lines of a electrical connection, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0158] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0159] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), blockchain network, and the Internet.

[0160] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud synchronization machine, which is a synchronization machine product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical synchronization machines and VPS services.

[0161] The embodiment of the present application further provides a computer program product comprising a computer program which, when executed by a processor, implements the database switching method provided in any embodiment of the present application.

[0162] The computer program product can be written in one or more programming languages or combinations of languages including object-oriented languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0163] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be executed in parallel, in series, or in a different order, as long as the desired information of the technical solution of the present application can be achieved, and the present application is not limited herein.

[0164] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent replacement and improvement within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A database handover method, characterized by, The method comprises: Synchronous processing of target data is performed in a first database, and asynchronous processing of target data is performed in a second database; if target data processing operations in the first database and the second database are performed in a data persistence framework, then the syntax corresponding to the first database and the syntax corresponding to the second database are maintained in a mapping file for loading; If no exception occurs in the asynchronous processing of target data in the second database within a first preset time period, then hot switching is performed to synchronous processing of target data in the second database and asynchronous processing of target data in the first database; If no exception still occurs in the synchronous processing of target data in the second database within a second preset time period, then only synchronous processing of target data in the second database is performed.

2. The method of claim 1, wherein, If target data processing operations in the first database and the second database are performed in a data persistence framework, synchronous processing of target data is performed in a first database, and asynchronous processing of target data is performed in a second database, comprising: Intercepting a method executed by a data access layer in the data persistence framework; Modifying the method executed by the data access layer to first perform synchronous processing of target data in the first database, and then perform asynchronous processing of target data in the second database using an asynchronous thread.

3. The method of claim 2, wherein, Modifying the method executed by the data access layer to first perform synchronous processing of target data in the first database, and then perform asynchronous processing of target data in the second database using an asynchronous thread, comprising: Obtaining a first database identifier of the first database, obtaining and loading a processing statement corresponding to the first database identifier from a mapping file based on the first database identifier, and performing synchronous processing of target data in the first database based on the processing statement corresponding to the first database identifier; Obtaining a second database identifier of the second database, obtaining and loading a processing statement corresponding to the second database identifier from a mapping file based on the second database identifier, and performing asynchronous processing of target data in the second database based on the processing statement corresponding to the second database identifier.

4. The method of claim 1, wherein, If target data processing operations in the first database and the second database are performed in a non-data persistence framework, synchronous processing of target data is performed in a first database, and asynchronous processing of target data is performed in a second database, comprising: Respectively determining a database type and a corresponding processing statement of the first database, and a database type and a corresponding processing statement of the second database; Respectively executing the processing statements in the first database and the second database for the target data.

5. The method of claim 1, wherein, The method further comprises: If an exception occurs in the process of performing the asynchronous processing or the synchronous processing of the target data in the second database, an exception message is sent to the message middleware; wherein, the exception includes at least one of the following: an error of executing a processing statement prompt, a processing timeout, and a processing result of the target data in the first database being inconsistent with a processing result of the target data in the second database; The exception message is subscribed and consumed from the message middleware by a common microservice, and the exception message is written into the first database.

6. The method of claim 1, wherein, The method further includes: If an exception occurs in the process of performing the asynchronous processing of the target data in the second database, the operation of performing the asynchronous processing of the target data into the second database is stopped, and only the operation of performing the synchronous processing of the target data into the first database is executed; If an exception occurs in the process of performing the synchronous processing of the target data in the second database, the synchronous processing of the target data in the first database is hot switched back, and the asynchronous processing of the target data in the second database is performed.

7. The method according to any one of claims 1 to 6, characterized in that, The process of hot switching includes: Real-time monitoring whether a hot switching switch is opened, and if the hot switching switch is opened, a flag bit is refreshed; wherein, the flag bit is used to mark the synchronous processing of the database in the first database and the second database; According to the flag bit, the hot switching of the first database and the second database is determined.

8. An electronic device, comprising: The electronic device includes: at least one processor; and a memory connected with the at least one processor in communication; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the database switching method in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions for enabling the processor to execute the database switching method in any one of claims 1-7 when executed.

10. A computer program product, characterised in that, The computer program is executed by the processor to implement the database switching method in any one of claims 1-7.

Images