Data backup recovery method and device, computer equipment and readable storage medium

Abstract

The invention relates to a data backup recovery method and device, computer equipment and a readable storage medium. The method comprises the steps that in response to a backup request for a database of a target service system, whether uncompleted service operation exists in multiple pieces of middleware associated with the target service system or not is detected, the backup request carries a test scene name and a service timestamp, and under the condition that the uncompleted service operation does not exist in the multiple pieces of middleware, the multiple pieces of middleware correspond to the test scene name and the service timestamp. The method comprises the following steps: acquiring and storing key cache data of a target service system, carrying out physical backup on a database, carrying out associated storage on the key cache data and physical backup data obtained by physical backup based on a test scene name and a service timestamp, and responding to a data recovery request for the database. And writing the key cache data into the corresponding middleware under the condition that the uncompleted business operation does not exist in the plurality of middleware, and carrying out physical recovery on the database based on the physical backup data. By adopting the method, the performance test efficiency can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of data testing technology, and in particular to a data backup and recovery method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Currently, for business systems with large data volumes and numerous testing scenarios, the usual practice is to back up the performance test data using physical or logical backups before executing the tests. If retesting is required during the testing period, the data is restored using backup tools. However, the data in business systems includes not only data from MySQL databases but also data from middleware such as Kafka and Redis. When backing up and restoring the test environment, this data also needs to be processed and must correspond to the database backup time points to ensure that all types of data in the business system can be restored to the state of the backup node.

[0003] However, currently, middleware data and MySQL data often require testers to manually judge and process them, which can easily lead to omissions and errors, thus affecting the efficiency of performance testing. Summary of the Invention

[0004] Therefore, it is necessary to provide a data backup and recovery method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the efficiency of performance testing in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a data backup and recovery method, the method comprising:

[0006] In response to a backup request for the database of a target business system, the system detects whether there are any incomplete business operations in multiple middleware associated with the target business system. The backup request carries the test scenario name and the business timestamp.

[0007] In the absence of any incomplete business operations in the middleware, the key cache data of the target business system is acquired and saved, the database is physically backed up, and the key cache data is associated with the physical backup data obtained from the physical backup based on the test scenario name and the business timestamp.

[0008] In response to a data recovery request for the database, detect whether there are any incomplete business operations in the multiple middlewares;

[0009] If there are no incomplete business operations in the middleware, the key cached data is written to the corresponding middleware, and the database is physically restored based on the physical backup data.

[0010] In one embodiment, the middleware includes a caching system; the steps of acquiring and saving key cached data of the target business system, performing a physical backup of the database, and associating and storing the key cached data with the physical backup data obtained from the physical backup include:

[0011] Read the key cache data of the target business system from the cache system and store the key cache data in the form of a structured file;

[0012] Perform a physical backup of the database, determine the directory path based on the test scenario name and the business timestamp, and store the physical backup data obtained from the physical backup and the structured file in the same directory path.

[0013] In one embodiment, the middleware includes a message queue and a caching system; the step of detecting whether there are any incomplete business operations in the multiple middlewares associated with the target business system includes:

[0014] The system detects whether there are any unconsumed messages in the message queue that are key topics interacting with the target business system, and detects whether there are any distributed locks in the cache system that indicate that business processing is in progress.

[0015] In one embodiment, detecting whether there are unconsumed messages in the message queue related to key topics interacting with the target business system includes:

[0016] For each partition of the key topic, compare the partition message offset of each partition with the corresponding consumer group's committed offset;

[0017] If the partition message offset of each partition is equal to the corresponding committed offset, it is determined that the messages of the key topic have been consumed.

[0018] In one embodiment, detecting whether a distributed lock representing ongoing business processing exists in the caching system includes:

[0019] Detect whether there is a lock key in the cache system that conforms to the preset naming rules;

[0020] If any lock key that conforms to the preset naming rules exists, it is determined that there is a distributed lock in the cache system that indicates that business processing is in progress.

[0021] In one embodiment, the middleware further includes a search engine; prior to physically backing up the database, the method further includes:

[0022] Remove the index data related to the target business system from the search engine.

[0023] Secondly, this application also provides a data backup and recovery device, comprising:

[0024] The backup request response module is used to respond to backup requests for the database of the target business system, and to detect whether there are any incomplete business operations in the multiple middleware associated with the target business system. The backup request carries the test scenario name and business timestamp.

[0025] The backup module is used to acquire and save key cache data of the target business system when there are no incomplete business operations in multiple middlewares, perform physical backup of the database, and associate and store the key cache data with the physical backup data obtained from the physical backup based on the test scenario name and the business timestamp.

[0026] The recovery request response module is used to respond to a data recovery request for the database and detect whether there are any unfinished business operations in the multiple middlewares;

[0027] The recovery module is used to write the key cached data to the corresponding middleware when there are no incomplete business operations in the multiple middlewares, and to physically restore the database based on the physical backup data.

[0028] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above-described data backup and recovery method embodiments.

[0029] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps in any of the above-described data backup and recovery method embodiments.

[0030] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in any of the above-described data backup and recovery method embodiments.

[0031] The aforementioned data backup and recovery methods, devices, computer equipment, computer-readable storage media, and computer program products proactively detect whether there are any incomplete business operations in multiple middlewares associated with the target business system before and after performing data backup and recovery operations. This ensures that data backup and restoration are performed only during the silent business window, thereby reducing logical breaks caused by inconsistencies between the middleware state and the database. Simultaneously, by utilizing the test scenario name and business timestamp carried in the backup request, critical cached data is associated and stored with the physical backup data of the database. During recovery, the cache is accurately written back and the database is restored, enabling the performance testing environment to be restored to an executable state completely aligned with the original backup time with a single click. As a result, not only is consistency of data across multiple components guaranteed at the business semantic level, but also multi-scenario, multi-time point isolation management and on-demand backtracking are supported, significantly improving the automation level of backup and recovery, environment readiness efficiency, and reliability of performance test results. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is an application environment diagram of a data backup and recovery method in one embodiment;

[0034] Figure 2 This is a flowchart illustrating a data backup and recovery method in one embodiment;

[0035] Figure 3 This is a flowchart illustrating a data backup and recovery method in another embodiment;

[0036] Figure 4 This is a flowchart illustrating the process of detecting whether there are incomplete business operation steps in the middleware in one embodiment;

[0037] Figure 5 This is a flowchart illustrating the process of detecting whether there are incomplete business operation steps in the middleware in another embodiment;

[0038] Figure 6 This is a structural block diagram of a data backup and recovery device in one embodiment;

[0039] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0041] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0042] The data recovery and backup method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with backup server 104 (hereinafter referred to as the server) via a network. The data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104, or it can be located in the cloud or on other network servers.

[0043] Specifically, testers can send a backup request for the database of the target business system to server 104 via terminal 102. The backup request carries the test scenario name and business timestamp. Server 104 responds to the backup request and checks whether there are any incomplete business operations in the multiple middlewares associated with the target business system. If no incomplete business operations are detected in the multiple middlewares, the server obtains and saves the key cache data of the target business system, performs a physical backup of the database based on the test scenario name and business timestamp, and stores the key cache data in association with the physical backup data obtained from the physical backup. Then, server 104 receives a data recovery request for the database, responds to the data recovery request, and checks again whether there are any incomplete business operations in the multiple middlewares. If no incomplete business operations are found in the multiple middlewares, the server writes the key cache data to the corresponding middleware and performs a physical recovery of the database based on the physical backup data.

[0044] Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0045] In one exemplary embodiment, such as Figure 2 As shown, a data backup and recovery method is provided, which can be applied to... Figure 1 Taking server 104 as an example, the explanation includes steps 200 to 800. Among them:

[0046] Step 200: In response to the backup request for the database of the target business system, check whether there are any incomplete business operations in the multiple middleware associated with the target business system. The backup request carries the test scenario name and business timestamp.

[0047] The target business system refers to the core business system to be tested, which relies on a database and multiple middleware components for operation. In this embodiment, the target business system refers to a business system with large test data, numerous test scenarios, and long build processes, such as an accounting system, a loan system, and other types of business systems. For ease of explanation, this embodiment uses an accounting system as an example. Middleware includes, but is not limited to, message queues, caching systems, search engines, and other components that work in conjunction with the business system. The accounting system's data is stored in the middleware.

[0048] Backup requests are instructions initiated by external automated processes. These requests carry a test scenario name (to identify the test type, such as "interest rate adjustment") and a "business timestamp" (such as the accounting date, to identify the business time context). Incomplete business operations refer to business states in the middleware that have not yet been processed, such as messages on critical topics in the message queue that have not been consumed. If such states exist during backup / restore, they will disrupt data consistency among multiple components.

[0049] In practical applications, developers can encapsulate the backup function and recovery module as two services and provide an HTTP call interface through Spring Boot's controller. For example, these services can be automatically called during the execution of batch date-switching scripts or other automated control tools, or manually called at any time. Specifically, the server receives backup requests from external processes, carrying the test scenario name and business timestamp. It parses the test scenario name and business timestamp from the backup request and, based on a pre-configured list of middleware addresses, sequentially queries the status of each middleware to determine if there are any incomplete business operations, such as reading the consumption offset of the message queue through the management interface. If all middleware has no active business, the subsequent backup process begins. If any middleware has incomplete business operations, the server waits for that middleware to complete the business operation until a timeout occurs and an exception is returned. In other embodiments, the server can subscribe to the event bus of each middleware, and the backup process is triggered only after all relevant middleware broadcasts a "business silence" signal.

[0050] Step 400: If there are no incomplete business operations in multiple middlewares, obtain and save the key cache data of the target business system, perform a physical backup of the database, and associate and store the key cache data with the physical backup data obtained from the physical backup based on the test scenario name and business timestamp.

[0051] In this embodiment, the key cached data consists of high-frequency configuration or status information (such as business dates, transaction type mappings, and mapping relationships) that the business system relies on for operation. Physical backup refers to the complete copying of the underlying database data files using tools (such as XtraBackup), which is different from logical export.

[0052] Following the previous step, if the server determines that there are no incomplete business operations in multiple middlewares, it reads predefined key cache data from the caching system. Then, it can control backup tools such as XtraBackup to completely copy or restore the underlying data files of the database (e.g., a MySQL database). Next, based on the test scenario name and business timestamp, a path is constructed to associate the physically backed-up data with the key cache data for storage, ensuring that all types of data can be restored simultaneously during recovery.

[0053] Step 600: In response to a data recovery request for the database, check whether there are any incomplete business operations in multiple middlewares.

[0054] In this embodiment, the data recovery request can also be an indicator initiated by an external automated tool or test script to trigger the data recovery process.

[0055] In practice, after receiving a recovery request, the server also parses the scenario and timestamp, and performs status checks on each middleware again to check whether there are any incomplete business operations in each middleware, ensuring that the current environment is clean and free from interfering business activities, and preventing state conflicts from occurring during the recovery process.

[0056] Step 800: If there are no incomplete business operations in multiple middlewares, write the key cached data to the corresponding middleware and perform physical recovery of the database based on the physical backup data.

[0057] Physical recovery refers to the process of completely restoring the underlying data files of a database.

[0058] Following the previous step, if the server determines that there are no incomplete business operations in the middleware, it can locate the corresponding backup unit based on the test scenario name and business timestamp in the recovery request. First, the saved key cache data is deserialized and written back to the corresponding middleware, such as the Redis caching system. Then, the database recovery tool is called to load the physical backup data, completing the overall environment reconstruction. At this point, the performance test environment is directly restored to an executable state without any further processing, and a new round of testing can be carried out directly.

[0059] In the aforementioned data backup and recovery method, before and after performing data backup and recovery operations, it actively detects whether there are any incomplete business operations in the multiple middlewares associated with the target business system. This ensures that data backup and restoration are performed only during the silent business window, thereby reducing logical breaks caused by inconsistencies between the middleware state and the database. At the same time, it uses the test scenario name and business timestamp carried in the backup request to associate and store key cached data with the physical backup data of the database. During recovery, it accurately writes back to the cache and performs database restoration, enabling the performance test environment to be restored to an executable state that is completely aligned with the original backup time with one click. As a result, it not only achieves consistency assurance of multi-component data at the business semantic level, but also supports isolation management and on-demand backtracking for multiple scenarios and time points, significantly improving the automation level of backup and recovery, environment readiness efficiency, and reliability of performance test results.

[0060] In one exemplary embodiment, such as Figure 3 As shown, the middleware includes a caching system. Step 400 includes:

[0061] Step 420: If there are no incomplete business operations in multiple middlewares, read the key cache data of the target business system from the cache system, store the key cache data in the form of a structured file, perform a physical backup of the database, determine the directory path based on the test scenario name and business timestamp, and store the physical backup data obtained from the physical backup and the structured file in the same directory path of the associated storage.

[0062] A caching system refers to middleware (such as Redis) used for temporary storage of frequently accessed data from a target business system. A structured file is a data file serialized in a standardized format (such as JSON or XML) for easy storage and restoration. A directory path is a storage location identifier composed of the test scenario name and the business timestamp, used to organize physical backup data and structured files within the same logical unit, achieving associative storage.

[0063] In practical implementation, if the server determines that there are no incomplete business operations in any of the middleware, the accounting date can be retrieved from the Redis cache system based on a predefined list of key cache keys. Key cached data from the accounting system can then be read from Redis and integrated into a structured object. This object is then serialized into a JSON file and temporarily stored in local memory or a temporary storage area. Next, a unique directory path is generated by concatenating the test scenario name and business timestamp carried in the backup request. A physical backup tool is then called to perform a full backup of the database, and the output is written to this directory path. Finally, the generated structured file is persisted to the same directory path. This ensures that key cached data and physical backup data are bound at the physical storage level, facilitating subsequent overall restoration according to the business context.

[0064] In this embodiment, by capturing and associating key cached data of the target business system during physical database backup, and using the test scenario name and business timestamp as anchors to achieve multi-dimensional isolation and precise positioning, the performance test environment can synchronously restore the database and middleware states during recovery, reducing the problem of environment unavailability caused by cache missing or state misalignment, thereby achieving one-click recovery to a complete business state that can be directly executed for testing, significantly improving test efficiency, data consistency and automation level.

[0065] In one exemplary embodiment, such as Figure 4 As shown, the middleware includes a message queue and a caching system; step 200 includes:

[0066] Step 220: Check if there are any unconsumed messages in the message queue that are key topics that interact with the target business system.

[0067] Step 240: Detect whether there is a distributed lock in the cache system that indicates that business processing is in progress.

[0068] Message queues refer to middleware systems (such as Kafka) used for asynchronous transmission of business messages. A key topic is a specific message channel upon which the target business system (accounting system) and downstream systems (such as accounting and payment systems) rely for interaction. A key topic refers to messages that affect test results; for example, transaction entries sent from the accounting system to the accounting system require confirmation that the accounting system has consumed them before data backup can be performed, otherwise, data loss will occur in the accounting system. Unconsumed messages are those that have been published to a key topic but have not yet been successfully processed and acknowledged by the corresponding consumer group.

[0069] A distributed lock is an identifier that exists in a caching system with a specific key name. It is used to mutually exclude the concurrent execution of a certain business operation (such as repayment or interest calculation) in a distributed environment. Its existence indicates that the business is still being processed.

[0070] In practice, the server can query the message offset of each partition of each key topic and the corresponding consumer group's committed offset through the message queue management client. By comparing the message offset of each partition with the corresponding consumer group's committed offset, it can determine whether there are any unconsumed messages in the key topics that the accounting system interacts with.

[0071] To detect the existence of a distributed lock in Redis, the server can initiate a key existence check with the caching system based on preset lock key naming rules. If the return result is empty or there is no matching key, it is determined that no distributed lock representing the ongoing business process exists. Only if it is determined that there are no unconsumed messages in the message queue and no distributed lock in Redis is present in the accounting system's transaction processing, will the backup process be executed.

[0072] In this embodiment, by collaboratively detecting the critical message consumption status of the message queue and the distributed lock in the caching system before backup and recovery, it is ensured that data operations are only performed when the target business system is in a business silent window. This effectively reduces the inconsistency of multiple component states caused by residual active transactions in the middleware, thereby ensuring that the performance test environment is logically aligned and behaves predictably after restoration, significantly improving the reliability of backup and recovery, the speed of environment readiness, and the accuracy of test results.

[0073] like Figure 5 As shown, in an exemplary embodiment, step 220 includes:

[0074] Step 222: For each partition of the key topic, compare the partition message offset of each partition with the corresponding committed offset of the consumer group. If the partition message offset of each partition is equal to the corresponding committed offset, it is determined that the messages of the key topic have been consumed.

[0075] A partition is a logical shard in a message queue (such as Kafka) used for parallel processing, and messages within each partition are ordered. The partition message offset is the position (End Offset) after the latest message in that partition, representing the identifier of the latest message that has been written. The committed offset of the consumer group indicates the position of the last message that the consumer group has successfully processed and acknowledged in that partition; when the two are equal, it means that all messages in that partition have been consumed and acknowledged, and there is no backlog.

[0076] In practice, the server first obtains a list of all partitions for the specified key topic through the message queue management interface. Then, for each partition, it queries its current partition message offset (End Offset) and the committed offset of the target consumer group in that partition, and compares the two values. If all partitions satisfy the condition that "the partition message offset equals the committed offset", it is determined that the messages of the key topic have been completely consumed and there is no unfinished business. Otherwise, it is considered that there are unconsumed messages, and the process terminates or enters a waiting retry phase.

[0077] In this embodiment, by accurately comparing the message production progress and consumption confirmation progress of each partition of the key topic, it is ensured that backup or recovery operations are only allowed after all relevant messages have been reliably processed by the downstream system. This fundamentally reduces the business data breakage caused by message backlog or duplicate consumption, thereby ensuring the state integrity and logical consistency of the performance testing environment in multi-system collaborative scenarios and improving the reliability and repeatability of automated testing.

[0078] In one exemplary embodiment, such as Figure 5 As shown, step 240 includes: step 242, detecting whether there is a lock key in the cache system that conforms to the preset naming rules, and if there is any lock key that conforms to the preset naming rules, determining that there is a distributed lock in the cache system that represents the business processing in progress.

[0079] The default naming rules are pre-agreed key name formats used to identify the ongoing specific business operations. The lock key is the specific key representing the existence of a distributed lock in the caching system. A "distributed lock representing the progress of business processing" refers to a mechanism that achieves cross-node mutual exclusion control by setting a specific key in the cache. Its existence indicates that a transaction or process has not yet been completed; performing backup or recovery in this state will lead to data inconsistency.

[0080] In practice, the server can initiate a key matching query to the Redis cache system based on the pre-configured lock key naming rules (e.g., using KEYS or the more secure SCAN command) to obtain all existing keys that match the rules. If the returned result contains at least one key, it is determined that there is a distributed lock representing that the business processing is in progress, indicating that there are still unfinished transactions in the system. Otherwise, it is considered that there are no active locks in the cache, the business is in a silent state, and subsequent operations can continue.

[0081] In this embodiment, by detecting whether there are lock keys in the cache system that conform to the preset naming rules, it is possible to accurately identify whether the target business system is in the process of critical transaction processing. This reduces the need to perform backup or recovery when business operations are not completed, effectively preventing process interruption, data corruption or duplicate processing caused by lock state residue or loss. It ensures that the performance test environment has a complete, clean and executable business context after restoration, and significantly improves the consistency, stability and automation efficiency of the test environment.

[0082] In one exemplary embodiment, the middleware further includes a search engine; prior to physically backing up the database, the method further includes: clearing index data in the search engine that is related to the target business system.

[0083] A search engine refers to middleware (such as Elasticsearch) used to store and retrieve logs, events, or operation records of a target business system. Index data related to the target business system refers to various structured data collections (such as transaction log indexes, event tracking indexes, etc.) written to the search engine during the system's operation, typically organized by index names containing business identifiers or timestamps.

[0084] In practice, the server can send a deletion request to the search engine (such as Elasticsearch) based on the pre-configured index naming rules or the list of indexes associated with the business system, and remove all existing indexes related to the target business system in batches (e.g., by wildcard matching or filtering by business time range). This operation is performed after confirming that there are no incomplete business operations in the middleware and before the physical backup of the database is started, ensuring that the environment is "clean" at the time of backup and that search data generated by historical tests will not be carried into the next round of testing, so as to reduce the interference of residual data from old tests on subsequent performance test results.

[0085] In this embodiment, by proactively clearing the index data related to the target business system in the search engine before physical database backup, the interference of historical test or operation residues on performance indicators (such as query latency, throughput, and resource consumption) is effectively eliminated, ensuring that the performance test environment is in a clean and consistent initial state after each recovery, thereby improving the accuracy, comparability, and repeatability of test results, and further supporting a highly reliable automated test closed loop.

[0086] To provide a clearer explanation of the data recovery and backup method provided in this application, a specific embodiment is described below, which includes the following:

[0087] This embodiment uses a microservice application built on Spring Boot as the control center, deployed on a separate backup server, to manage the automated backup and recovery of the accounting system performance testing environment.

[0088] For example, after the testing team completes a round of verification for a "bulk deduction during holidays" scenario, the current environment state needs to be solidified for future reuse. At this time, the automated scheduling script sends an HTTP backup request to the microservice, carrying test scenario parameters and business timestamps. Upon receiving the request, the microservice first initializes the AdminClient through KafkaAdmin, queries the latest offsets of each partition of the key topics on which the accounting system depends and the committed offsets of the consumer groups, confirming that all messages have been consumed; simultaneously, it checks for distributed locks to ensure that no transactions are in transit. After confirming that all middleware is in a business-quiet state, the microservice reads key cached data such as accounting dates, transaction type mapping tables, and circuit breaker rates from Redis and serializes them into a JSON file. Subsequently, by connecting to a MySQL server deployed with XtraBackup, it executes a physical backup command, constructs a directory path using the test scenario name and business timestamp, and writes the backed-up database files and key cached data to the same directory path. The backup is then complete.

[0089] If the testing team needs to rerun the scenario, they call the recovery interface, passing in the same parameters. The microservice performs a silent middleware check again. After confirming the environment is clean, it loads key cached data from the aforementioned path based on the test scenario name and business timestamp, and writes the data back to Redis. Simultaneously, RestHighLevelClient is used to clear old indexes related to the accounting system in Elasticsearch, reducing the impact of residual data in the search engine on performance test results. Next, XtraBackup is called again via Jsch to execute the physical recovery process, overwriting the MySQL data directory and restarting the service. After recovery, all components of the accounting system (MySQL, Redis, Kafka consumption status, ES indexes) are aligned to the state at the end of the previous "holiday batch deduction" scenario. The test environment can be immediately deployed for a new round of performance testing without manual intervention, significantly improving testing efficiency and data consistency.

[0090] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0091] Based on the same inventive concept, this application also provides a data backup and recovery apparatus for implementing the data backup and recovery method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more data backup and recovery apparatus embodiments provided below can be found in the limitations of the data backup and recovery method described above, and will not be repeated here.

[0092] In one exemplary embodiment, such as Figure 6 As shown, a data backup and recovery device 600 is provided, including: a backup request response module 610, a backup module 620, a recovery request response module 630, and a recovery module 640, wherein:

[0093] The backup request response module 610 is used to respond to a backup request for the database of the target business system and detect whether there are any incomplete business operations in the multiple middleware associated with the target business system. The backup request carries the test scenario name and business timestamp.

[0094] The backup module 620 is used to acquire and save key cache data of the target business system when there are no incomplete business operations in the multiple middlewares, perform physical backup of the database, and associate and store the key cache data with the physical backup data obtained from the physical backup based on the test scenario name and the business timestamp.

[0095] The recovery request response module 630 is used to detect whether there are any incomplete business operations in the plurality of middleware in response to a data recovery request for the database.

[0096] The recovery module 640 is used to write the key cache data to the corresponding middleware when there are no incomplete business operations in the multiple middlewares, and to perform physical recovery of the database based on the physical backup data.

[0097] In one embodiment, the middleware includes a caching system; the backup module 620 is also used to read key cache data of the target business system from the caching system, store the key cache data in the form of a structured file, perform physical backup of the database, determine the directory path based on the test scenario name and business timestamp, and store the physical backup data obtained from the physical backup and the structured file in the same directory path of the associated storage.

[0098] In one embodiment, the middleware includes a message queue and a caching system; the backup request response module 610 and the recovery request response module 630 are also used to detect whether there are unconsumed messages in the message queue for key topics that interact with the target business system, and to detect whether there are distributed locks in the caching system that indicate that business processing is in progress.

[0099] In one embodiment, the backup request response module 610 and the recovery request response module 630 are further configured to compare the partition message offset of each partition of the critical topic with the corresponding committed offset of the consumer group, and determine that the messages of the critical topic have been consumed if the partition message offset of each partition is equal to the corresponding committed offset.

[0100] In one embodiment, the backup request response module 610 and the recovery request response module 630 are further configured to detect whether there is a lock key in the cache system that conforms to a preset naming rule. If any lock key that conforms to the preset naming rule exists, it is determined that there is a distributed lock in the cache system that indicates that business processing is in progress.

[0101] In one embodiment, the middleware also includes a search engine; the backup module 620 is also used to clear index data in the search engine that is related to the target business system.

[0102] Each module in the aforementioned data backup and recovery device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0103] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores backup data and critical cached data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data backup and recovery method.

[0104] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0105] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above-described data backup and recovery method embodiments.

[0106] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in any of the above-described data backup and recovery method embodiments.

[0107] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the above-described data backup and recovery method embodiments.

[0108] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0109] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0110] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0111] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data backup recovery method characterized by, The method includes: In response to a backup request for the database of a target business system, the system detects whether there are any incomplete business operations in multiple middleware associated with the target business system. The backup request carries the test scenario name and the business timestamp. In the absence of any incomplete business operations in the middleware, the key cache data of the target business system is acquired and saved, the database is physically backed up, and the key cache data is associated with the physical backup data obtained from the physical backup based on the test scenario name and the business timestamp. In response to a data recovery request for the database, detect whether there are any incomplete business operations in the multiple middlewares; If there are no incomplete business operations in the middleware, the key cached data is written to the corresponding middleware, and the database is physically restored based on the physical backup data.

2. The method of claim 1, wherein, The middleware includes a caching system; the steps of acquiring and saving key cached data of the target business system, performing physical backups of the database, and associating and storing the key cached data with the physical backup data obtained from the physical backup include: Read the key cache data of the target business system from the cache system and store the key cache data in the form of a structured file; Perform a physical backup of the database, determine the directory path based on the test scenario name and the business timestamp, and store the physical backup data obtained from the physical backup and the structured file in the same directory path.

3. The method of claim 1, wherein, The middleware includes message queues and a caching system; the detection of whether there are any incomplete business operations in the multiple middlewares associated with the target business system includes: The system detects whether there are any unconsumed messages in the message queue that are key topics interacting with the target business system, and detects whether there are any distributed locks in the cache system that indicate that business processing is in progress.

4. The method of claim 3, wherein, The detection of whether there are unconsumed messages in the message queue for key topics that interact with the target business system includes: For each partition of the key topic, compare the partition message offset of each partition with the corresponding consumer group's committed offset; If the partition message offset of each partition is equal to the corresponding committed offset, it is determined that the messages of the key topic have been consumed.

5. The method of claim 3, wherein, Detecting whether a distributed lock representing ongoing business processing exists in the caching system includes: Detect whether there is a lock key in the cache system that conforms to the preset naming rules; If any lock key that conforms to the preset naming rules exists, it is determined that there is a distributed lock in the cache system that indicates that business processing is in progress.

6. The method according to any one of claims 1 to 4, characterized in that, The middleware also includes a search engine; prior to performing a physical backup of the database, the method further includes: Remove the index data related to the target business system from the search engine.

7. A data backup and recovery device, characterized in that, The device includes: The backup request response module is used to respond to backup requests for the database of the target business system, and to detect whether there are any incomplete business operations in the multiple middleware associated with the target business system. The backup request carries the test scenario name and business timestamp. The backup module is used to acquire and save key cache data of the target business system when there are no incomplete business operations in multiple middlewares, perform physical backup of the database, and associate and store the key cache data with the physical backup data obtained from the physical backup based on the test scenario name and the business timestamp. The recovery request response module is used to respond to a data recovery request for the database and detect whether there are any unfinished business operations in the multiple middlewares; The recovery module is used to write the key cached data to the corresponding middleware when there are no incomplete business operations in the multiple middlewares, and to physically restore the database based on the physical backup data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images