Database replacement method and device, storage medium and product

Abstract

The invention discloses a database replacement method and device, a storage medium and a product, and relates to the technical field of databases, the database replacement method comprises the following steps: in response to a database replacement instruction, executing a non-intrusive application data double-write operation to maintain data consistency between a source database and a target database; in the application data double-writing process, recording read-only operation traffic of a production environment to the source database, and playing back the read-only operation traffic to the target database for response comparison to generate a verification result; based on the verification result, after it is confirmed that the target database meets a preset condition, the service flow is switched from the source database to the target database, and database replacement is completed. The reliability and safety of the database replacement process are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database, and particularly relates to a database replacement method, device, storage medium and product. BACKGROUND

[0002] In modern enterprise-level application systems, as a core data storage and management system, the selection and performance of the database directly relate to the stability and efficiency of the entire system.

[0003] However, in the prior art, in order to ensure the data consistency of the source database and the target database during database replacement, a data double-write technology is usually used. The existing double-write scheme usually deeply invades the business application code, and needs to explicitly increase the write operation to the target database in the business logic layer. This not only increases the complexity and maintenance cost of the code, but also introduces the problem of distributed transactions, which seriously wastes system performance in a high-concurrency scenario, and may even affect the availability of the main business due to temporary unavailability of a database, and substantially affects the online business. In the verification stage before the flow is cut, the traditional method usually adopts a gray release strategy, that is, a small part of the production traffic is gradually switched to the application using the new database, so that the database replacement becomes a high-risk operation, resulting in low reliability of the entire replacement process. SUMMARY

[0004] The main purpose of the present application is to provide a database replacement method, device, storage medium and product, which aims to solve the technical problem of low reliability of database replacement.

[0005] To achieve the above purpose, the present application provides a database replacement method, which comprises: In response to a database replacement instruction, a non-invasive application data double-write operation is performed to maintain data consistency between a source database and a target database; During the application data double-write process, read-only operation traffic of a production environment to the source database is recorded, and the read-only operation traffic is played back to the target database for response comparison to generate a verification result; Based on the verification result, after confirming that the target database meets a preset condition, the business traffic is switched from the source database to the target database to complete the database replacement.

[0006] In an embodiment, the step of performing a non-invasive application data double-write operation comprises: A data change operation to the source database is intercepted through a self-defined annotation and a data layer interceptor; Based on the data change operation, an asynchronous real-time data sending event is triggered, and data change information corresponding to the data change operation is converted into a synchronous message; The synchronization message is sent to a preset message middleware cluster, and the message is consumed by a target application and synchronized to the target database.

[0007] In an embodiment, the step of sending the synchronization message to a preset message middleware cluster, and the message is consumed by a target application and synchronized to the target database includes: The synchronization message is sent to two independent message middleware clusters at the same time to improve the reliability of message transmission through redundant deployment; The target application consumes messages from the two message middleware clusters at the same time, and performs deduplication based on the data version number and data modification time in the synchronization message to ensure that the latest data version is used to perform data synchronization; A timing compensation synchronization task is started, wherein the task acquires changed data in the source database that has changed within a preset time period according to a preset time interval; After data deduplication by comparing the data version number and the modification time, the missing changed data is synchronized to the target database to achieve the final consistency guarantee of data.

[0008] In an embodiment, the step of recording the read-only operation traffic of the source database includes: An interception point is set at a read-only interface of a source application of the source database through aspect programming technology to capture request response data passing through the read-only interface; The captured request response data is filtered according to a preset configuration rule, and the request response data that meets the recording conditions is combined to form traffic messages; The traffic messages are sent to a designated message middleware topic through an asynchronous manner for storage, completing the traffic recording process.

[0009] In an embodiment, the step of playing back the read-only operation traffic to the target database for response comparison includes: The traffic messages are consumed from the message middleware topic by a preset stream processing engine, and the requests are played back to the target application according to a preset concurrency; The actual response result generated by the target application in response to the playback request is received, and the actual response result is compared and analyzed with the expected response result of the source application recorded in the traffic message; The comparison result containing the difference information is persistently stored in a preset columnar analysis database.

[0010] In an embodiment, the method further includes the step of dynamically controlling the traffic recording and playback process through a preset configuration center, including: Persisting configuration states of traffic recording switches and playback switches of each interface in a preset configuration center; Loading the configuration states into application memory to form a local cache, and updating the local cache in real time according to change notifications of the configuration center; Based on the configuration states in the local cache, dynamically controlling the opening and closing of the traffic recording and playback functions of each interface.

[0011] In an embodiment, the method further comprises the steps of rate control and result display of the playback process, including: Controlling the concurrent rate of traffic playback by adjusting the number of partitions of the message middleware topic and the parallelism of the stream processing engine; Reading the verification results from the columnar analysis database to generate a multi-dimensional analysis report containing interface accuracy statistics and response time comparison; Displaying the multi-dimensional analysis report on a visualization platform to provide decision support for database replacement verification.

[0012] In addition, to achieve the above-mentioned purposes, the present application also provides a database replacement device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the computer program is configured to implement the steps of the database replacement method as described above.

[0013] In addition, to achieve the above-mentioned purposes, the present application also provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the database replacement method as described above.

[0014] In addition, to achieve the above-mentioned purposes, the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the database replacement method as described above.

[0015] The one or more technical solutions proposed in the present application have at least the following technical effects: Compared with the prior art, the application adopts an application data double-writing operation without invasion in response to a database replacement instruction to maintain data consistency between a source database and a target database; during the application data double-writing process, read-only operation flow of a production environment to the source database is recorded and played back to the target database for response comparison to generate a verification result; after the target database is confirmed to meet a preset condition based on the verification result, business flow is switched from the source database to the target database to complete database replacement. It can be understood that the application adopts an application data double-writing mechanism without invasion, and when database replacement is needed, asynchronous data synchronization based on an interceptor and a message middleware is adopted to realize quasi-real-time double-writing of business data between a source database and a target database, so as to realize decoupling of the data synchronization process and the business logic; at the same time, real read-only flow of a production environment is recorded and played back to the target database for comparison and verification, so that the interface accuracy and performance of the target database can be fully verified based on the full flow characteristics of the production, and thus functional and performance problems that cannot be covered in the gray stage can be found and repaired in advance, and finally the smooth switching of the database is completed under the premise of ensuring data consistency and business continuity, thereby significantly improving the reliability and security of the database replacement process. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings from these drawings without creative labor.

[0018] Figure 1 A flowchart is provided for the database replacement method embodiment one of the present application; Figure 2 A scheme diagram of application high-availability double-writing is provided for the database replacement method embodiment one of the present application; Figure 3 A non-invasive write flow asynchronous recording diagram is provided for the database replacement method embodiment one of the present application; Figure 4 A double-cluster quasi-real-time data consumption and timing polling data synchronization compensation diagram is provided for the database replacement method embodiment one of the present application; Figure 5Application recording, playback, comparison schematic diagram provided for the database replacement method embodiment one of the present application; Figure 6 Traffic recording schematic diagram of the source application provided for the database replacement method embodiment one of the present application; Figure 7 Traffic playback, comparison, calculation flow schematic diagram provided for the database replacement method embodiment one of the present application; Figure 8 Configuration switch cache hot update schematic diagram provided for the database replacement method embodiment one of the present application; Figure 9 System architecture schematic diagram provided for the database replacement method embodiment one of the present application; Figure 10 Device structure schematic diagram of the hardware running environment involved in the database replacement method in the embodiment of the present application.

[0019] The object implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0021] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the drawings in the specification and specific embodiments.

[0022] The main solution of the embodiment of the present application is: In response to the database replacement instruction, the non-invasive application data double-write operation is performed to maintain data consistency between the source database and the target database; In the application data double-write process, the read-only operation traffic of the production environment to the source database is recorded, and the read-only operation traffic is played back to the target database for response comparison to generate a verification result; Based on the verification result, after confirming that the target database meets the preset condition, the business traffic is switched from the source database to the target database to complete the database replacement.

[0023] In the present embodiment, the database replacement device is taken as the execution subject, and for the convenience of description, the following will be specifically described in the form of abbreviation "device".

[0024] Because the existing technology adopts the invasive data double-write scheme, the system complexity and maintenance cost are high, and it is difficult to fully expose potential problems through small traffic gray verification, resulting in low database replacement reliability.

[0025] The application provides a solution, which adopts a non-invasive application data double-writing mechanism. When database replacement is needed, through asynchronous data synchronization based on an interceptor and a message middleware, quasi-real-time double writing of business data between a source database and a target database is realized, so as to realize decoupling of a data synchronization process and business logic. Meanwhile, real read-only traffic of a production environment is recorded and played back to the target database for comparison and verification, so that based on full-traffic characteristics of the production, the accuracy and performance of the target database interface can be fully verified, and then function and performance problems that are difficult to cover in the gray stage can be found and repaired in advance, and finally, smooth switching of the database is completed under the premise of ensuring data consistency and business continuity, and the reliability and security of the database replacement process are significantly improved.

[0026] Based on this, the application embodiment provides a database replacement method, which refers to Figure 1 , Figure 1 FIG. 1 is a flowchart of a first embodiment of the database replacement method of the application.

[0027] In this embodiment, the database replacement method comprises steps S10-S30: Step S10, in response to a database replacement instruction, a non-invasive application data double-writing operation is performed to maintain data consistency between a source database and a target database. It should be noted that the database replacement instruction refers to a system instruction or configuration operation for starting the database replacement process. The non-invasive application data double-writing operation refers to a technical process of writing data into the source database and the target database at the same time through aspect programming and a message middleware without modifying the business logic code. The source database refers to an original database system to be replaced, and the target database refers to a new database system to be replaced. The data consistency refers to a state in which the business data in the source database and the target database is kept consistent in logic and content through technical means.

[0028] It can be understood that by decoupling the business logic and the data synchronization process, the double-writing logic is avoided to be hard-coded in the business code, so that a data synchronization channel between the source database and the target database is established without affecting the performance of the main business, and a consistent data basis is provided for subsequent verification and switching.

[0029] Step S20, in the application data double-writing process, read-only operation traffic of a production environment to the source database is recorded, and the read-only operation traffic is played back to the target database for response comparison to generate a verification result. It should be noted that the read-only operation traffic refers to network request data generated by the query operation of the application to the source database in the production environment. The response comparison refers to the process of comparing the actual response returned by the target database to the playback request with the expected response of the source database recorded in the traffic message. The verification result is the conclusion data generated after comparison and analysis, which is used to evaluate the accuracy and performance of the target database interface.

[0030] It can be understood that this step uses the real traffic characteristics of the production environment to fully test the target database in the independent verification environment, which can expose interface compatibility, data accuracy and potential performance problems in advance before actual switching, provide real and comprehensive data support for switching decision, and effectively avoid the defects of insufficient gray verification samples.

[0031] Exemplarily, as shown in Figure 2 The application shows a specific implementation scheme of high-availability double writing. The scheme builds a complete data synchronization channel from the source database to the target database. The source application converts the data change operation into a synchronization message by a non-invasive method and sends it to the Kafka message cluster for asynchronous decoupling. The target application consumes the synchronization message in Kafka and applies it to the target database to realize quasi-real-time synchronization of data. At the same time, the scheme also designs a timing compensation synchronization mechanism, which periodically checks the change of the source database by polling to ensure the final consistency of data in extreme cases. This double writing scheme not only guarantees the real-time of data synchronization, but also provides high-availability protection through the compensation mechanism to ensure the reliability of data synchronization during database replacement.

[0032] Step S30, based on the verification result, confirming that the target database meets the preset condition, switching the business traffic from the source database to the target database to complete the database replacement.

[0033] It should be noted that the preset condition refers to the threshold of each technical index for judging whether the target database has the ability to take over all business traffic, such as interface accuracy, response time, system stability, etc. The business traffic switching refers to the system operation of routing the data access request of the application from the source database to the target database.

[0034] It can be understood that this step makes a scientific switching decision based on the sufficient verification data in the early stage, ensuring the reliability and safety of the switching action, and finally realizes the smooth and reliable migration of the business from the old database system to the new database system, completing the entire replacement process.

[0035] The embodiment provides a database replacement method, which adopts a non-invasive application data double-writing mechanism. When database replacement is needed, through asynchronous data synchronization based on an interceptor and a message middleware, quasi-real-time double-writing of business data between a source database and a target database is realized, so that the data synchronization process and the business logic are decoupled. Meanwhile, real read-only traffic in a production environment is recorded and played back to the target database for comparison and verification. Therefore, based on the full-amount traffic characteristics in the production, the accuracy and performance of the target database interface can be fully verified, so that functional and performance problems that are difficult to cover in the gray stage can be found and repaired in advance. Finally, the smooth switching of the database is completed under the premise of ensuring data consistency and business continuity, and the reliability and security of the database replacement process are significantly improved.

[0036] In a feasible implementation, the step of performing the non-invasive application data double-writing operation comprises: intercepting a data change operation on the source database through a self-defined annotation and a data layer interceptor; based on the data change operation, triggering a quasi-real-time data sending event asynchronously, and converting data change information corresponding to the data change operation into a synchronization message; sending the synchronization message to a preset message middleware cluster, and consuming the message by a target application and synchronizing to the target database.

[0037] It should be noted that the self-defined annotation refers to a metadata tag defined in a Java program, which is used to mark a Mapper layer method to enable the data synchronization function. The data layer interceptor refers to a component based on the Mybatis framework, which is used to intercept database insert, delete and update operations. The data change operation refers to a write operation of inserting, updating or deleting data of a database table. The quasi-real-time data sending event refers to an event that is executed asynchronously after a database transaction is committed, and is used to prepare and send synchronization data. The data change information refers to a set of information including a change table name, an operation type and data primary keys involved. The synchronization message refers to a message structure that encapsulates complete data records and operation semantics according to a predetermined format. The message middleware cluster refers to a message service cluster composed of multiple Kafka nodes, which is used to realize asynchronous communication and decoupling. The target application refers to an application program instance that connects and operates the target database.

[0038] Understandably, this implementation method transparently captures data changes at the data persistence layer through annotations and interceptors, and uses a transaction synchronization mechanism to ensure that data is sent asynchronously only after the business transaction is committed. On the one hand, this achieves complete decoupling from the business code and ensures the non-intrusiveness of the dual-write process; on the other hand, it decouples the direct dependency between the source and target ends through message middleware, avoiding the impact on the source end business due to the temporary unavailability of the target database, while providing a buffer and reliability guarantee for data synchronization.

[0039] For example, such as Figure 3 As shown in the diagram, this illustrates the non-intrusive asynchronous recording process of write traffic. When the business application executes database write operations at the Mapper layer, methods marked with custom annotations are intercepted by the data layer interceptor. The interceptor does not immediately handle the synchronization logic; instead, it registers a near real-time data sending event through the Spring transaction synchronization manager, which is triggered only after the transaction is successfully committed. Once this event is triggered, it asynchronously queries the source database based on the primary key to retrieve the complete modified data row and encapsulates it along with the operation type into a synchronization message. Finally, this message is sent to two independent Kafka clusters to achieve highly reliable message transmission.

[0040] In one feasible implementation, the step of sending the synchronization message to a preset message middleware cluster, whereby the target application consumes the message and synchronizes it to the target database includes: The synchronization message is sent to two independent message middleware clusters simultaneously to improve the reliability of message transmission through redundant deployment; The target application consumes messages simultaneously from the two message middleware clusters and performs deduplication based on the data version number and data modification time in the synchronization message to ensure that data synchronization is performed based on the latest data version. A timed compensation and synchronization task is initiated, wherein the task acquires the changed data that has changed within the preset time period from the source database in segments according to preset time intervals. After deduplicating the data by comparing the data version number and modification time, the missing changed data is synchronized to the target database, thereby ensuring the eventual consistency of the data.

[0041] It should be noted that the two independent message middleware clusters refer to two sets of physically or logically isolated Kafka clusters, forming a redundant architecture. The data version number refers to a sequence number used to identify the update number of a data record. The data modification time refers to the timestamp of the last modification of the data record. The timing compensation synchronization task refers to a data synchronization job for checking and filling gaps launched according to a fixed time period. The preset time interval refers to the execution period set for the timing task. The time period refers to the time window for polling the data changes of the source database. The changed data refers to the data records that have undergone insertion, update or deletion operations within the specified time window.

[0042] It can be understood that the embodiment prevents the risk of message loss caused by single point failure through dual-cluster redundancy sending and consumption; ensures the accuracy of data synchronization in the repeated consumption scenario through the deduplication mechanism based on data version and time; and finally introduces the timing compensation task as a bottom-up mechanism to capture and synchronize a small amount of data changes missed due to extreme conditions, thereby building a multi-level data guarantee system and providing high reliability guarantee for realizing the eventual consistency of data.

[0043] Exemplarily, as shown in Figure 4 The figure shows the complete process of dual-cluster quasi-real-time data consumption and timing polling data synchronization compensation. The synchronization message is sent to two Kafka clusters in parallel, and the target application consumes messages from the two clusters at the same time. At the consumption end, the repeated messages or out-of-order messages from different clusters are deduplicated by comparing the data version number and the data modification time in the message, ensuring that only the latest version of the data is written to the target database. At the same time, the system starts an independent timing task, which records the last synchronization time point and pulls the changed data in the source database after the time point in a small batch in a paging manner, and then compensates the changed data that is not captured by the quasi-real-time synchronization by writing it to the target database after deduplication comparison with the version number and timestamp, thereby realizing high-availability guarantee of data eventual consistency.

[0044] In a feasible embodiment, the step of recording the read-only operation traffic of the source database includes: Setting an interception point at the read-only interface of the source application of the source database through aspect programming technology to capture request response data passing through the read-only interface; According to a preset configuration rule, the captured request response data is filtered, and the request response data meeting the recording conditions is combined to form a traffic message; The traffic message is sent to a specified message middleware topic in an asynchronous manner for storage, completing the traffic recording process.

[0045] It should be noted that the aspect programming technology refers to a technology for realizing unified maintenance of program functions through pre-compilation and dynamic proxy at runtime. The read-only interface refers to a service interface for performing a database query operation in an application. The interception point refers to a specific position at which aspect logic is inserted in a program execution flow. The request response data refers to a data set of client request parameters and server return results. The preset configuration rule refers to a condition set defined in advance for filtering traffic to be recorded. The recording condition refers to a traffic collection trigger condition set based on an interface identifier, a request feature, and the like. The traffic message refers to a message unit encapsulated in a specific format and containing complete request and response information. The message middleware topic refers to a logical channel for classified storage of messages in Kafka.

[0046] It can be understood that the embodiment realizes non-aware collection of read-only traffic through the aspect technology, ensures zero-invasion of business logic, realizes fine filtering of traffic through a configurable rule engine, avoids collection of invalid data, and adopts an asynchronous sending mechanism to ensure that the traffic recording process does not block the main business flow, thereby guaranteeing the performance and stability of the source application while obtaining real production traffic samples.

[0047] Exemplarily, as shown in Figure 5 , the present application provides a complete flow of traffic recording, playback, and comparison. The flow starts from the read traffic recording phase of the source application, intercepts the request response data of the read-only interface through the aspect technology, generates a traffic message, and sends it to Kafka. Subsequently, the flow enters the traffic playback phase, the stream processing engine consumes the recorded traffic message from Kafka, and plays it back to the target application at a controllable concurrency. In the comparison and analysis phase, the system compares the actual response returned by the target application with the expected response recorded in real time, identifies differences in function correctness, performance, and the like. Finally, the comparison result is stored persistently and an analysis report is generated. This flow forms a complete verification closed loop, making it possible to fully verify the accuracy and performance of the target database before actual switching.

[0048] Exemplarily, as shown in Figure 6 , the figure shows the traffic recording process of the source application. When a request arrives at the read-only interface of the source application, the interception point set through the aspect programming technology captures the request and the subsequent response data. The system queries the configuration cache in the memory, and decides whether to record according to the recording switch state and specific rules of the interface. If the recording condition is met, the request parameters and response results are combined into a traffic message, which is sent to the specified Kafka topic for persistent storage in an asynchronous and non-blocking manner, thereby completing a traffic recording. The entire process has an exception capturing mechanism, so that even if an exception occurs in the recording link, it will not affect the normal execution of the main business flow.

[0049] In an implementable embodiment, the step of playing back the read-only operation traffic to the target database for response comparison comprises: The traffic message is consumed from the message middleware subject by a preset stream processing engine, and the request is played back to the target application according to a preset concurrency degree; The actual response result generated by the target application in response to the playback request is received, and the actual response result is compared and analyzed with the expected response result recorded in the traffic message; The comparison result containing the difference information is persistently stored in a preset columnar analysis database.

[0050] It should be noted that the stream processing engine refers to a computing framework for real-time processing of data streams, and in the present scheme, Flink is used for implementation. The concurrency degree refers to the number of instances of parallel execution of stream processing tasks, which is used to control the throughput of the playback request. The target application refers to an application program instance connected to and operating the target database. The actual response result refers to the response data returned by the target application after processing the playback request. The expected response result refers to the response data returned by the source application when processing the original request, which is recorded in the traffic message. The comparison and analysis refers to the process of consistency comparison between the actual response result and the expected response result. The difference information refers to the detailed information of inconsistent response content, response time or status code found in the comparison process. The columnar analysis database refers to a database system using columnar storage structure, and in the present scheme, ClickHouse is used for implementation.

[0051] It can be understood that the present embodiment realizes real-time playback and calculation of traffic through the stream processing engine, which can quickly find the behavior differences between the target database and the source database; the playback pressure is controlled by adjustable concurrency degree, which can accelerate the verification process and avoid overloading the target system; the comparison result is persisted to the columnar analysis database, which provides a data basis for subsequent generation of multi-dimensional analysis report, thereby realizing full automation of the verification process and traceability of the results.

[0052] Exemplarily, as shown in Figure 7 , the figure shows the traffic playback, comparison and calculation process. The Flink stream processing engine consumes the recorded traffic message from the Kafka subject, and plays back the request to the target application in parallel according to the configured parallelism. The target application returns the actual response result after processing these requests, and the system compares and analyzes these results with the expected response result recorded in the traffic message in real time. The difference information generated by the comparison, including response content difference and response time anomaly, is written into the ClickHouse columnar analysis database in real time for persistence, providing data support for subsequent visual report generation.

[0053] In an embodiment, the method further comprises the step of dynamically controlling the traffic recording and playback process by a preset configuration center, including: persistently storing the configuration state of the traffic recording switch and playback switch of each interface in the preset configuration center; loading the configuration state into the application memory to form a local cache, and updating the local cache in real time according to the change notification of the configuration center; based on the configuration state in the local cache, dynamically controlling the opening and closing of the traffic recording and playback function of each interface.

[0054] It should be noted that the configuration center refers to a system service for centralized management of application configuration information, used to store and distribute various switch configurations. The traffic recording switch refers to a Boolean configuration item that controls whether to record traffic on a specific interface. The playback switch refers to a Boolean configuration item that controls whether to play back the recorded traffic. The configuration state refers to the current value and version information of each switch configuration. The local cache refers to a copy of the configuration information maintained in the application memory for fast reading. The change notification refers to an update signal sent to the subscribed application when the configuration in the configuration center is modified.

[0055] It can be understood that the embodiment realizes centralized management and dynamic distribution of traffic recording and playback rules through the configuration center, and combines the local cache mechanism to not only guarantee the performance of configuration reading, but also realize the hot update capability of the configuration, so that the user can adjust the traffic collection and playback strategy on demand without restarting the application, and realize fine control of the verification process and efficient use of system resources.

[0056] Exemplarily, as shown in Figure 8 , the figure shows the working mechanism of the configuration switch cache hot update. The configuration center persistently stores the recording and playback switch state of all interfaces. The application loads these configurations into the memory cache when it starts, and listens to the change notification of the configuration center during running. When the user modifies the switch state of an interface in the configuration center, the change notification will trigger the application to update its local cache in real time, so that the new control strategy takes effect immediately, realizing the dynamic start and stop of the traffic recording and playback function without restarting the application service.

[0057] In an embodiment, the method further comprises the steps of rate control and result display of the playback process, including: controlling the concurrency rate of traffic playback by adjusting the number of partitions of the message middleware topic and the parallelism of the stream processing engine; reading the verification result from the columnar analysis database to generate a multi-dimensional analysis report containing interface accuracy statistics and response time comparison; The multi-dimensional analysis report is displayed in a visualization platform to provide decision support for database replacement verification.

[0058] It should be noted that the number of partitions refers to the number of logical units in which the Kafka topic is divided, and directly affects the parallel processing capability of the message. The parallelism refers to the number of concurrent instances of task execution in the stream processing engine. The concurrency rate refers to the frequency of sending playback requests to the target application per unit time. The interface correctness rate statistics refers to a data index calculated based on the response comparison result. The response time comparison refers to the difference analysis of the time required by the source application and the target application to process the same request. The multi-dimensional analysis report refers to a statistical report that displays the verification results from multiple angles such as interface, time, and error type. The visualization platform refers to a report system for data visualization.

[0059] It can be understood that the embodiment realizes precise control of playback pressure by adjusting the two key parameters of message partition and processing parallelism, which can not only fully utilize system resources to accelerate the verification process, but also avoid excessive load on the target database; by generating multi-dimensional analysis reports and visualizing the display, the verification results are converted into intuitive decision-making basis, so that technical personnel and managers can fully evaluate the readiness of the target database, and provide scientific support for the final switching decision.

[0060] Exemplarily, as shown in Figure 7 , the flow shows the complete process from traffic consumption to result display. By increasing the number of partitions of the Kafka topic and improving the parallelism of the Flink job, the playback throughput can be linearly improved. The comparison results generated during the verification process are continuously written into the ClickHouse database, and the report platform queries these data to automatically generate visualization charts showing key indicators such as interface correctness rate trend and response time distribution, providing intuitive data support for evaluating the feasibility of database replacement.

[0061] Exemplarily, referring to Figure 9The application provides a whole architecture of a non-invasive application data double-writing and flow verification system. The architecture is divided into three levels from top to bottom: a display layer, a core layer and an infrastructure layer. The display layer provides a visual analysis interface of data playback comparison results for a user and supports configuration operation of a flow switching switch. The core layer includes four core function modules: a data double-writing module responsible for realizing non-invasive application data high-availability double-writing; a read flow recording module responsible for realizing flow recording of a source application; a flow playback module responsible for consuming recorded flow and playing back to a target application for comparison monitoring; and a flow routing switch module responsible for recording flow and flow switching rules according to an interface. The infrastructure layer provides bottom support for the system, including configuration persistence, playback flow storage, monitoring information persistence, and integration of a Kafka message middleware and a report platform. The hierarchical architecture clarifies the responsibilities and cooperation relationships of components of the system and constitutes a complete database replacement solution.

[0062] It should be noted that the above examples are only used for understanding the application and do not constitute a limitation on the database replacement method of the application. More forms of simple changes based on the technical concept are within the protection scope of the application.

[0063] The application provides a database replacement device, which includes: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the database replacement method in Embodiment I.

[0064] Reference will be made to the following description of the embodiments of the application. Figure 10 which shows a structural schematic diagram of a database replacement device suitable for being used to implement the embodiments of the application. The database replacement device in the embodiments of the application can include, but is not limited to, mobile terminals such as mobile phones, tablet computers, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistant: personal digital assistant), PMPs (Portable Media Player: portable multimedia player), vehicle-mounted terminals (for example, vehicle-mounted navigation terminals) and the like, and fixed terminals such as digital televisions, desktop computers and the like. Figure 10 The illustrated database replacement device is only an example and should not bring any limitation on the functions and use range of the embodiments of the application.

[0065] As Figure 10As shown, the database replacement device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the operation of the database replacement device are also stored in the RAM 1004. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the database replacement device to communicate with other devices wirelessly or by wire to exchange data. Although the database replacement device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0066] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0067] The database replacement device provided by the present disclosure adopts the database replacement method in the above embodiments, and can solve the technical problem of low reliability of database replacement. Compared with the prior art, the database replacement device provided by the present disclosure has the same beneficial effects as the database replacement method provided by the above embodiments, and other technical features in the database replacement device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0068] It should be understood that various aspects of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any appropriate manner in any one or more embodiments or examples.

[0069] The above description is only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0070] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer programs) for performing the database replacement method in the above embodiments.

[0071] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, an RF (Radio Frequency), etc., or any appropriate combination thereof.

[0072] The above computer readable storage medium can be contained in the database replacement device; or can exist separately without being assembled into the database replacement device.

[0073] The computer readable storage medium carries one or more programs, when the one or more programs are executed by the database replacement device, the database replacement device is caused to: in response to a database replacement instruction, perform a non-intrusive application data double-write operation to maintain data consistency between a source database and a target database; In the application data double-write process, read-only operation traffic of the source database in the recording production environment is recorded and played back to the target database for response comparison to generate a verification result; Based on the verification result, after confirming that the target database meets a preset condition, switching business traffic from the source database to the target database to complete the database replacement.

[0074] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++, or conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0075] The flow diagrams and the block diagrams in the drawings are illustrations of possible architectures, functions, and operations for systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0076] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. In some cases, the name of the module does not constitute a limitation on the module itself.

[0077] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the above-mentioned database replacement method, and can solve the technical problem of low reliability of database replacement. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the database replacement method provided by the above-mentioned embodiments, which will not be repeated here.

[0078] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned database replacement method.

[0079] The computer program product provided by the present application can solve the technical problem of low reliability of database replacement. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the database replacement method provided by the above-mentioned embodiments, which will not be repeated here.

[0080] All the acquisition of signals, information or actions in the present application are carried out under the premise of complying with the corresponding data protection regulations and policies of the country where the device is located, and obtaining the authorization given by the corresponding device owner.

[0081] The above only describes some embodiments of the present application, and does not limit the protection scope of the present application, and any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the protection scope of the present application.

Claims

1. A database replacement method characterized by comprising: The method comprises: In response to the database replacement instruction, a non-invasive application data double-write operation is performed to maintain data consistency between the source database and the target database; During the application data double-write process, read-only operation traffic of the production environment to the source database is recorded, and the read-only operation traffic is played back to the target database for response comparison to generate a verification result; After confirming that the target database meets the preset condition based on the verification result, the business traffic is switched from the source database to the target database to complete the database replacement.

2. The method of claim 1, wherein, The step of performing the non-invasive application data double-write operation comprises: Through a self-defined annotation and a data layer interceptor, a data change operation on the source database is intercepted; Based on the data change operation, an asynchronous real-time data sending event is triggered, and the data change information corresponding to the data change operation is converted into a synchronous message; The synchronous message is sent to a preset message middleware cluster, and the target application consumes the message and synchronizes it to the target database.

3. The method of claim 2, wherein, The step of sending the synchronous message to the preset message middleware cluster, and the target application consuming the message and synchronizing it to the target database comprises: The synchronous message is sent to two independent message middleware clusters at the same time to improve the reliability of message transmission through redundant deployment; The target application consumes messages from the two message middleware clusters at the same time, and performs deduplication processing based on the data version number and the data modification time in the synchronous message to ensure that the latest data version is used to perform data synchronization; A timing compensation synchronization task is started, wherein the task acquires changed data in the source database within a preset time period according to a preset time interval, After deduplication by comparing the data version number and the modification time, the missing changed data is synchronized to the target database to achieve final data consistency guarantee.

4. The method of claim 1, wherein, The step of recording read-only operation traffic of the production environment to the source database comprises: An interception point is set at a read-only interface of a source application of the source database through aspect programming technology to capture request response data passing through the read-only interface; The captured request response data is filtered according to a preset configuration rule, and the request response data meeting the recording conditions is combined to form a traffic message; The traffic message is sent to a designated message middleware topic through an asynchronous manner for storage, completing the traffic recording process.

5. The method of claim 4, wherein, The step of playing back the read-only operation traffic to the target database for response comparison comprises: The traffic message is consumed from the message middleware topic by a preset stream processing engine, and the request is played back to the target application according to a preset concurrency degree; The actual response result of the target application to the playback request is received, and the actual response result is compared and analyzed with the expected response result of the source application recorded in the traffic message; The comparison result containing the difference information is persistently stored in a preset columnar analysis database.

6. The method of claim 5, wherein, The method further comprises the step of dynamically controlling the traffic recording and playback process through a preset configuration center, comprising: Persisting configuration states of traffic recording switches and playback switches of each interface in a preset configuration center; Loading the configuration states into application memory to form a local cache, and updating the local cache in real time according to a change notification of the configuration center; Controlling the opening and closing of the traffic recording and playback functions of each interface based on the configuration states in the local cache.

7. The method of claim 5, wherein, The method further comprises steps of rate control and result display of the playback process, including: Controlling the concurrent rate of traffic playback by adjusting the number of partitions of the message middleware topic and the parallelism of the stream processing engine; Reading the verification results from the columnar analysis database to generate a multi-dimensional analysis report containing interface accuracy statistics and response time comparison; Displaying the multi-dimensional analysis report on a visualization platform to provide decision support for database replacement verification.

8. A database replacement apparatus characterized by comprising: The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the database replacement method according to any one of claims 1 to 7.

9. A storage medium, characterized by The storage medium is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the database replacement method according to any one of claims 1 to 7.

10. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by the processor to implement the steps of the database replacement method according to any one of claims 1 to 7.

Images