Data backup method, device and storage medium

Abstract

The application provides a data backup method and device and a storage medium, relates to the technical field of computers, and is used for solving the technical problem that general technology is prone to causing business stoppage and low efficiency when data is backed up. The method comprises the following steps: acquiring a first starting point and transaction information of a plurality of shard nodes at a first time in the process of providing data services by the plurality of shard nodes; the first starting point is used for representing the starting position of the next data change information stored in the data change log of the shard node at the first time; when the transaction information of the plurality of shard nodes at the first time passes consistency verification, acquiring a second starting point of the plurality of shard nodes at a second time; the second time is located after the first time; and when the first starting point and the second starting point of each shard node are the same, performing a backup operation on the data change log of each shard node.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a data backup method, apparatus and storage medium. Background Technology

[0002] Currently, distributed systems have become widely adopted due to their high efficiency, reliability, and ease of use. However, when backing up data in a distributed system, general techniques typically require temporarily halting data storage or retrieval services provided by the system to ensure safe backup and maintain data consistency. When dealing with large datasets and numerous sharded nodes, this approach is generally time-consuming, inefficient, and can easily lead to business disruptions on the distributed system. Summary of the Invention

[0003] This application provides a data backup method, apparatus, and storage medium to solve the technical problem that general technologies are prone to causing business downtime and low efficiency during data backup.

[0004] To achieve the above objectives, this application adopts the following technical solution:

[0005] Firstly, a data backup method is provided, comprising: during the process of multiple shard nodes providing data services, obtaining the first starting point and transaction information of multiple shard nodes at a first moment; the first starting point is used to characterize the starting position of the shard node in the data change log at the first moment for storing the next data change information; the transaction information includes the number and status information of transactions run by multiple shard nodes at the first moment; when the transaction information of multiple shard nodes at the first moment passes the consistency verification, obtaining the second starting point of multiple shard nodes at a second moment; the second moment is after the first moment; when the first starting point and the second starting point of each shard node are the same, performing a backup operation on the data change log of each shard node.

[0006] Optionally, before acquiring the first starting point and transaction information of multiple shard nodes at the first moment, the data backup method further includes: receiving a backup instruction message for instructing the execution of a backup operation; the backup instruction message includes a target node identifier; and verifying the target node identifier based on a preset node identifier.

[0007] Optionally, the method for obtaining the first starting point and transaction information of multiple shard nodes at the first moment specifically includes: creating multiple backup subtasks; the backup subtasks are used to obtain the first starting point and transaction information; and executing the multiple backup subtasks in parallel to obtain the first starting point and transaction information of each shard node.

[0008] Optionally, the method for performing backup operations on the data change logs of each shard node specifically includes: obtaining historical backup points of multiple shard nodes; the historical backup points are used to characterize the termination position of the data change information backed up in the data change logs of the shard node in the previous backup operation; and backing up the data change information between the historical backup points and the second starting point in the data change logs of each shard node.

[0009] Secondly, a data backup device is provided, comprising: an acquisition unit and a processing unit; the acquisition unit is configured to acquire, during the process of multiple shard nodes providing data services, a first starting point and transaction information of multiple shard nodes at a first moment; the first starting point is used to characterize the starting position of the shard node in the data change log at the first moment for storing the next data change information; the transaction information includes the number and status information of transactions run by multiple shard nodes at the first moment; the acquisition unit is further configured to acquire a second starting point of multiple shard nodes at a second moment when the transaction information of multiple shard nodes at the first moment passes consistency verification; the second moment is located after the first moment; the processing unit is configured to perform a backup operation on the data change log of each shard node when the first starting point and the second starting point of each shard node are the same.

[0010] Optionally, the data backup device further includes: a receiving unit;

[0011] The receiving unit is used to receive a backup instruction message that instructs the execution of a backup operation; the backup instruction message includes a target node identifier;

[0012] The processing unit is also used to verify the target node identifier based on the preset node identifier.

[0013] Optionally, the unit is acquired, specifically for:

[0014] Create multiple backup subtasks; the backup subtasks are used to obtain the first starting point and transaction information.

[0015] Multiple backup subtasks are executed in parallel to obtain the first starting point and transaction information of each shard node.

[0016] Optionally, the processing unit is specifically used for:

[0017] Obtain the historical backup positions of multiple shard nodes; the historical backup positions are used to characterize the termination position of the data change information backed up in the data change log of the previous backup operation in the shard node;

[0018] Back up the data change information between the historical backup site and the second starting site in the data change log of each shard node.

[0019] Thirdly, a data backup device is provided, including a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory via a bus; when the data backup device is running, the processor executes the computer execution instructions stored in the memory to cause the data backup device to perform the data backup method as described in the first aspect.

[0020] The data backup device can be a network device or a component of a network device, such as a chip system within the network device. This chip system supports the network device in implementing the functions involved in the first aspect and any of its possible implementations, such as receiving, determining, and distributing the data and / or information involved in the aforementioned data backup method. The chip system includes a chip and may also include other discrete devices or circuit structures.

[0021] Fourthly, a computer-readable storage medium is provided, including computer-executable instructions that, when executed on a computer, cause the computer to perform the data backup method as described in the first aspect.

[0022] It should be noted that the aforementioned computer instructions may be stored, in whole or in part, on the first computer-readable storage medium. The first computer-readable storage medium may be packaged together with the processor of the data backup device, or it may be packaged separately from the processor of the data backup device; this application does not impose any limitations on this.

[0023] In this application, the name of the aforementioned data backup device does not limit the device or functional module itself. In actual implementation, these devices or functional modules may appear under other names. As long as the function of each device or functional module is similar to that of this application, it falls within the scope of the claims of this application and its equivalents.

[0024] These or other aspects of this application will become more readily apparent in the following description.

[0025] The technical solution provided in this application brings at least the following beneficial effects:

[0026] Based on any of the above aspects, in this application, the proxy node can obtain the first starting point and transaction information of multiple shard nodes at a first moment during the process of multiple shard nodes providing data services. When the transaction information of multiple shard nodes at the first moment passes consistency verification, it can be shown that the execution of transactions by multiple shard nodes at the first moment will not further compromise the integrity of the database. In this case, the proxy node can further obtain the second starting point of multiple shard nodes at a second moment, and thus, when the first starting point and the second starting point of each shard node are the same, perform a backup operation on the data information of each shard node to complete the data backup.

[0027] Since the transaction information run by multiple shard nodes is consistent, and the starting points of multiple shard nodes are the same in the first and second time points, this application can determine the start time of the backup operation during the process of multiple shard nodes providing data services. This ensures data consistency while performing data backup, thus avoiding the problem of business downtime and low efficiency that often occurs when general technologies require temporary suspension of data services during data backup. Therefore, this application can solve the technical problem of business downtime and low efficiency that often arises when performing data backup on distributed systems using general technologies. Attached Figure Description

[0028] Figure 1 A schematic diagram of the structure of a data backup system provided in this application embodiment. Figure 1 ;

[0029] Figure 2 A schematic diagram of the structure of a data backup system provided in this application embodiment. Figure 2 ;

[0030] Figure 3 A schematic diagram of the hardware structure of a proxy node provided in an embodiment of this application;

[0031] Figure 4 A flowchart illustrating a data backup method provided in this application embodiment. Figure 1 ;

[0032] Figure 5 A flowchart illustrating a data backup method provided in this application embodiment. Figure 2 ;

[0033] Figure 6 A flowchart illustrating a data backup method provided in this application embodiment. Figure 3 ;

[0034] Figure 7 A schematic diagram of a data backup process provided in this application embodiment. Figure 1 ;

[0035] Figure 8 A flowchart illustrating a data backup method provided in this application embodiment. Figure 4 ;

[0036] Figure 9 A schematic diagram of a data backup process provided in this application embodiment. Figure 2 ;

[0037] Figure 10 This is a schematic diagram of the structure of a data backup device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0040] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0041] Furthermore, the terms "comprising" and "having" in the embodiments, claims, and drawings of this application are not exclusive. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules, but may also include steps or modules not listed.

[0042] To facilitate understanding of this application, the relevant elements involved in this application are described below.

[0043] Currently, distributed systems have become widely adopted due to their high efficiency, reliability, and ease of use. However, when backing up data in a distributed system, general techniques typically require temporarily halting data storage or retrieval services provided by the system to ensure safe backup and maintain data consistency. When dealing with large datasets and numerous sharded nodes, this approach is generally time-consuming, inefficient, and can easily lead to business disruptions on the distributed system.

[0044] To address the aforementioned issues, this application provides a data backup method. In this method, a proxy node can acquire the first starting point and transaction information of multiple shard nodes at a first moment during the data service provision process. When the transaction information of multiple shard nodes at the first moment passes consistency verification, it indicates that the execution of transactions by multiple shard nodes at the first moment will not further compromise the integrity of the database. In this case, the proxy node can further acquire the second starting point of multiple shard nodes at a second moment. Therefore, when the first and second starting points of each shard node are the same, a backup operation is performed on the data information of each shard node to complete the data backup.

[0045] Since the transaction information run by multiple shard nodes is consistent, and the starting points of multiple shard nodes are the same in the first and second time points, this application can determine the start time of the backup operation during the process of multiple shard nodes providing data services. This ensures data consistency while performing data backup, thus avoiding the problem of business downtime and low efficiency that often occurs when general technologies require temporary suspension of data services during data backup. Therefore, this application can solve the technical problem of business downtime and low efficiency that often arises when performing data backup on distributed systems using general technologies.

[0046] This data backup method is applicable to data backup systems. Figure 1 One structure of the data backup system 100 is shown. For example... Figure 1 As shown, the data backup system 100 includes: a proxy node 101 and multiple shard nodes 102. The proxy node 101 can communicate with each of the multiple shard nodes 102.

[0047] Figure 1 The proxy node 101 can have functions such as data sharding and read / write separation. The proxy node 101 can use data sharding to distribute various data from the application layer to different shard nodes 102 for processing. Furthermore, the proxy node 101 can also use read / write separation to read the data change logs from each shard node 102 and transfer these logs to a backup database, completing the backup operation.

[0048] Figure 1 The sharding node 102 in the system can have functions such as data acquisition, data transmission, data storage, data processing, and data exchange, and is used to provide various data and services.

[0049] Combination Figure 1 ,like Figure 2As shown in the embodiments of this application, the data backup system 100 may further include a relay node 103 and a terminal 104. The relay node 103 may communicate with the terminal 104 or with the agent node 101.

[0050] In practical applications, relay node 103 can also communicate with multiple terminals 104. Furthermore, relay node 103 can also communicate with multiple proxy nodes 101.

[0051] For ease of understanding, this application uses the communication connection between relay node 103 and a terminal 104, and the communication connection between a proxy node 101 as examples for illustration.

[0052] Figure 2 The relay node 103 can be configured with a Hypertext Transfer Protocol (HTTP) server to provide various data and services. For example, Figure 2 The relay node 103 in the middle can be the server of ZooKeeper (a distributed application coordination service) to provide data and services deployed in various shard nodes 102 to the terminal 104.

[0053] Optionally, the proxy node 101, each shard node 102, and the relay node 103 can be servers. A server can be a single server, or it can be a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. This application does not impose any limitations in this regard.

[0054] Figure 2 Terminal 104 can be configured with an HTTP client. The HTTP client can be a service implemented based on a World Wide Web (WEB) browser.

[0055] In one possible approach, terminal 104 may also be configured with a ZooKeeper client, which can be used to call data and services deployed on each shard node 102 through relay node 103.

[0056] In one possible configuration, terminal 104 can be configured with functional modules such as a display module, an input module, and a communication module. The input module can be an external computer input device such as a mouse and keyboard to support various user operations, such as data editing and data backup. The display module can be a device such as an LCD screen to display the relevant interface of the HTTP client, such as a data editing interface. The communication module can be a transceiver or similar device for communication with relay node 103.

[0057] Optionally, Figure 2 Terminal 104 can be a device providing voice and / or data connectivity to a user, a handheld device with wireless connectivity, or other processing devices connected to a wireless modem. The wireless terminal can communicate with one or more core networks via a radio access network (RAN). The wireless terminal can be a mobile terminal, such as a computer with a mobile terminal, or a portable, pocket-sized, handheld, or computer-embedded mobile device that exchanges voice and / or data with the radio access network, such as a mobile phone, tablet computer, laptop computer, netbook, or personal digital assistant (PDA). This application does not impose any limitations on these embodiments.

[0058] like Figure 3 The diagram shown is a hardware structure schematic of a proxy node 101 provided in an embodiment of this application. The proxy node 101 includes a processor 21, a memory 22, a communication interface 23, and a bus 24. The processor 21, memory 22, and communication interface 23 are connected via the bus 24.

[0059] Processor 21 is the control center of agent node 101. It can be a single processor or a collective term for multiple processing elements. For example, processor 21 can be a CPU or other general-purpose processors. Among them, general-purpose processors can be microprocessors or any conventional processors.

[0060] As one embodiment, processor 21 may include one or more CPUs, for example Figure 3 CPU0 and CPU1 are shown in the diagram.

[0061] The memory 22 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0062] In one possible implementation, the memory 22 can exist independently of the processor 21. The memory 22 can be connected to the processor 21 via a bus 24 and is used to store instructions or program code. When the processor 21 calls and executes the instructions or program code stored in the memory 22, it can implement the data backup method provided in the following embodiments of this application.

[0063] In another possible implementation, the memory 22 can also be integrated with the processor 21.

[0064] Communication interface 23 is used for the proxy node 101 to connect with other devices via a communication network, which may be Ethernet, wireless access network, wireless local area network (WLAN), etc. Communication interface 23 may include a receiving unit for receiving data and a sending unit for sending data.

[0065] Bus 24 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0066] It should be pointed out that, Figure 3 The structure shown does not constitute a limitation on proxy node 101, except... Figure 3 In addition to the components shown, the proxy node 101 may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0067] like Figure 4 The diagram shown is a flowchart illustrating a data backup method provided in an embodiment of this application. This data backup method is applied to... Figure 1 The agent node 101 in the data backup system 100 shown. The data backup method includes: S401-S403.

[0068] S401. During the process of providing data services to multiple shard nodes, the proxy node obtains the first starting point and transaction information of multiple shard nodes at the first moment.

[0069] The first starting point can be used to characterize the starting position of the shard node in the data change log at the first moment when storing the next data change information.

[0070] Optionally, data services can include data acquisition services, data transmission services, data storage services, data processing services, and data exchange services, to meet the business needs of specific operations. For example, specific operations could include logistics delivery order generation, currency transaction services in banking institutions, and comment services within community content services.

[0071] In one possible approach, each shard node can be configured with a MySQL database (a relational database management system). The data change log can be the BinLog in the MySQL database. The BinLog is a binary log file used to store all data change information in the MySQL database in binary form.

[0072] Furthermore, when storing data change information in the data change log, each shard node can arrange the data changes sequentially according to time. Each data change message has a start and end position, and the data change messages can be linked together.

[0073] In one possible approach, transaction information may include the number and status information of transactions run by the shard node at the first moment. Transaction information may also include transaction identifiers used to uniquely identify each transaction.

[0074] In one possible way, combining Figure 2 Users can pre-set the start and end times of backup operations in the relay node via their terminals. Correspondingly, the relay node can create scheduled tasks corresponding to the start and end times of the backup operation. When the scheduled task corresponding to the start time of the backup operation is triggered, the relay node can send a backup operation instruction to the agent node. The agent node can then receive the backup operation instruction from the relay node.

[0075] Next, the proxy node can obtain the first starting point and transaction information of multiple shard nodes at a given moment while they are providing data services, without changing the operational state of each shard node. Specifically, the proxy node can send log status requests and transaction information requests to each shard node at the first moment. Correspondingly, each shard node, upon receiving the log status and transaction information requests from the proxy node, reads the log status and transaction information and sends them back to the proxy node. The log status information may include the first starting point. Accordingly, the proxy node can receive the log status and transaction information from each shard node at the first moment and further parse the log status information to determine the first starting point.

[0076] Optionally, staff can pre-set backup cycles in the proxy nodes. After receiving a backup operation instruction from the relay node, the proxy node can periodically obtain relevant information from multiple shard nodes according to the preset backup cycle. Therefore, the "first moment" can be the moment the proxy node receives the backup operation instruction, or it can be the start time of any backup cycle after the proxy node receives the instruction.

[0077] In one possible approach, the log status request could be the "show master status" command.

[0078] In one possible approach, the transaction information request could be the "XA recover" command.

[0079] S402. When the transaction information of multiple shard nodes passes the consistency verification at the first moment, the proxy node obtains the second starting point of multiple shard nodes at the second moment.

[0080] The second time point is located after the first time point. The second starting point can be used to characterize the starting position of the shard node in the data change log at the second time point for storing the next data change information.

[0081] It should be noted that when processing a service request initiated by a terminal, the proxy node can divide the data service associated with the service request into multiple transactions and dispatch these transactions to various shard nodes for execution. In other words, multiple shard nodes can process different data in a single data service to collectively provide service for a single service request from the terminal.

[0082] Based on this, after obtaining the transaction information of multiple shard nodes at the first moment, the proxy node can perform consistency verification on this information. If the transaction information of multiple shard nodes fails the consistency verification, it indicates that after the first moment, when multiple shard nodes run deployed transactions to provide data services to the terminal, they will further update the data change log. In this case, the proxy node can stop other operations in this backup cycle and wait for the next backup cycle.

[0083] If the transaction information of multiple shard nodes passes consistency verification at the first moment, it indicates that in the short period following the first moment, when multiple shard nodes run deployed transactions to provide data services to the terminal, they will not further update the data change log. In other words, each shard node will not store new data change information in the data change log in the short period following the first moment.

[0084] In this scenario, the proxy node can further obtain the second starting points of multiple shard nodes at the second time step. Subsequently, the proxy node can compare the first and second starting points to determine whether the data change logs of each shard node have been updated.

[0085] It should be understood that the method by which the proxy node obtains the second starting position of multiple shard nodes at the second time point can be referred to the specific description in S401 regarding the proxy node obtaining the first starting position of multiple shard nodes at the first time point, which will not be repeated here.

[0086] In one possible approach, when the proxy node performs consistency verification on the transaction information of multiple shard nodes at the first moment, it can do so by determining whether transactions running on multiple shard nodes completely belong to one or more data services, and whether transactions within the same data service are in the same running state. Alternatively, other common implementation methods can be used, which will not be elaborated here.

[0087] In one possible approach, the second timeframe could be the moment when the proxy node determines multiple shard nodes and the transaction information from the first timeframe passes consistency verification. Alternatively, a preset interval can exist between the second and first timeframes. This preset interval can be pre-set by staff within the proxy node; for example, 20 milliseconds.

[0088] In one possible approach, if the transaction information of multiple shard nodes fails consistency verification, the proxy node can reacquire the transaction information of multiple shard nodes at a third time step. Then, if the transaction information of multiple shard nodes passes consistency verification at the third time step, the proxy node can acquire the second starting point of the multiple shard nodes at a second time step. The third time step is located after the first time step and before the second time step. In this way, the proxy node can more effectively guarantee the backup tasks within this backup cycle, improving the efficiency of data backup.

[0089] S403. When the first and second starting points of each shard node are the same, the proxy node performs a backup operation on the data change log of each shard node.

[0090] In one feasible approach, after obtaining the second starting points of multiple shard nodes at the second time step, the proxy node can compare the first and second starting points of each shard node. If at least one shard node has a different first and second starting point, it indicates that at least one shard node updated its data change log between the first and second time steps. That is, the data change logs of the various shard nodes may not be consistent. To ensure the security and reliability of the backup data, the proxy node can abandon the current backup cycle and wait for the next backup cycle.

[0091] If the first and second starting points of each shard node are the same, then the data change logs of each shard node are consistent. The proxy node can perform backup operations on the data change logs of each shard node. Specifically, the proxy node can read the data change logs of each shard node and store them in a pre-defined backup database.

[0092] In one example, there are three shard nodes: shard node 1, shard node 2, and shard node 3. If the first and second starting points of shard node 1, shard node 2, and shard node 3 are the same, it indicates that the first and second starting points of all shard nodes are identical. The proxy node can perform backup operations on the data change logs of each shard node.

[0093] If the first starting point and the second starting point of shard node 1 are different, or the first starting point and the second starting point of shard node 2 are different, or the first starting point and the second starting point of shard node 3 are different, then the proxy node will not perform backup operations on the data change logs of each shard node.

[0094] In one embodiment, before the proxy node obtains the first starting point and transaction information of multiple shard nodes at the first moment, such as Figure 5 As shown, the data backup method provided in this application embodiment further includes: S501-S502.

[0095] S501, The agent node receives a backup instruction message that instructs the execution of a backup operation.

[0096] The backup instruction message includes the target node identifier.

[0097] It should be noted that, in combination Figure 2 In a distributed system consisting of relay nodes, proxy nodes, and multiple shard nodes, the number of proxy nodes is typically set to multiple to ensure high availability. However, for a single business request from a terminal, only one proxy node can be assigned to respond in the distributed system to guarantee the atomicity required by the system.

[0098] Therefore, to ensure the atomicity of data services provided by multiple agent nodes in a distributed system, the relay node can select a target agent node from among multiple agent nodes to perform the backup operation according to preset rules (such as load balancing rules) when the scheduled task corresponding to the start time of the backup operation is triggered, and add the target node identifier to the backup instruction message. Then, the relay node can send a backup instruction message carrying the target node identifier. Correspondingly, multiple agent nodes can listen for the backup instruction message carrying the target node identifier sent by the relay node.

[0099] In one possible approach, the listening mechanism between the relay node and multiple proxy nodes could be implemented based on ZooKeeper.

[0100] S502. The proxy node verifies the target node identifier based on the preset node identifier.

[0101] In one possible approach, a preset node identifier can be pre-set by staff in the proxy node to uniquely identify the proxy node.

[0102] In one feasible approach, after receiving a backup instruction message, the proxy node can parse the message to determine the target node identifier. Next, the proxy node can read a preset node identifier and compare it with the target node identifier. If the target node identifier matches the preset node identifier, then the proxy node is the selected target proxy node for performing the backup operation. Subsequently, the proxy node can retrieve the first starting point and transaction information of multiple shard nodes at the first moment, as well as the corresponding backup process.

[0103] In one embodiment, the proxy node can obtain the first starting point and transaction information of multiple shard nodes in parallel at the first moment. For example... Figure 6 As shown, the data backup method provided in this application embodiment further includes: S601-S602.

[0104] S601, the agent node creates multiple backup subtasks.

[0105] Multiple backup subtasks can correspond one-to-one with multiple shard nodes. A backup subtask can be used to obtain the first starting point and transaction information of the shard node corresponding to that backup subtask.

[0106] In one possible approach, to improve data backup efficiency, the proxy node can create multiple backup subtasks, each corresponding one-to-one with a shard node. Subsequently, the proxy node can execute these multiple backup subtasks in parallel, thereby acquiring the initial starting point and transaction information of multiple shard nodes at the first moment.

[0107] S602, the agent node executes multiple backup subtasks in parallel to obtain the first starting point and transaction information of each shard node.

[0108] It should be noted that each proxy node in a distributed system can be configured with master-slave thread functionality to support master-slave thread mode.

[0109] Based on this, such as Figure 7 As shown, the proxy node can listen for backup indication messages sent by the relay node through a listening thread. Then, the listening thread can synchronize the backup indication messages to the main thread based on inter-thread synchronization. Next, the main thread can create a backup control thread and multiple backup detection threads when verifying that the target node identifier matches a preset node identifier. The backup control thread can be used to control each backup detection thread to execute its respective backup subtask to obtain the first starting point and transaction information of each shard node.

[0110] In one possible way, combining Figure 7 The backup control thread can also be used to verify the consistency of transaction information of multiple shard nodes at the first moment. Furthermore, when the transaction information of multiple shard nodes passes the consistency verification at the first moment, the backup control thread can also control each backup detection thread to obtain the second starting point of the multiple shard nodes at the second moment. Further, if the first and second starting points of each shard node are the same, the backup control thread can store the data change logs for each shard node in the backup database deployed on the backup node.

[0111] In one embodiment, when the proxy node performs a backup operation on the data change logs of each shard node, such as Figure 8 As shown, this application embodiment provides an optional implementation method, including: S701-S702.

[0112] S701, the agent node obtains the historical backup locations of multiple shard nodes.

[0113] Among them, the historical backup point can be used to characterize the termination position of the data change information backed up in the data change log of the previous backup operation in the shard node.

[0114] It should be noted that in a distributed system, backup instruction messages sent by relay nodes at different times can be executed by different proxy nodes. Furthermore, data backups of the data change logs for each shard node are generally incremental backups; that is, backups are made on top of previous backups, focusing on newly generated data change information.

[0115] Based on this, each agent node can send the backed-up positions of the data change logs of each shard node in the backup database to the relay node after each backup operation is completed. Correspondingly, the relay node receives and stores the backed-up positions of the data change logs of each shard node. Subsequently, any agent node can obtain the backed-up positions of the data change logs of each shard node from the relay node when performing a backup operation on the data change logs of each shard node, thus obtaining the historical backup positions of each shard node.

[0116] S702. The agent node backs up the data change information between the historical backup point and the second starting point in the data change log of each shard node.

[0117] In one possible approach, after obtaining the historical backup locations of multiple shard nodes, the proxy node can determine the data between the historical backup locations and the second starting location in the data change logs of each shard node as the incremental backup range for that shard node's data change log. Then, the proxy node reads the data change information within each incremental backup range and stores it in the backup database.

[0118] In one embodiment, such as Figure 9The diagram illustrates a data flow according to an embodiment of this application. Preset shard nodes are shard node 1, shard node 2, and shard node 3. When the scheduled task corresponding to the start time of the backup operation is triggered, the relay node can send a backup instruction message to the proxy node. Correspondingly, the proxy node can receive the backup instruction message sent by the relay node. Then, if the proxy node verifies that the target node identifier is a preset node identifier, it can send log information requests to each shard node at the first moment. Correspondingly, each shard node can send a log information response to the proxy node. In this way, the proxy node can parse the log information of each shard node and determine the first starting point of each shard node. Simultaneously, the proxy node can also send transaction information requests to each shard node. Correspondingly, each shard node can send a transaction information response to the proxy node. In this way, the proxy node can obtain the transaction information of each shard node.

[0119] Next, the proxy node can perform consistency verification on the transaction information of multiple shard nodes at the first moment. Furthermore, when the transaction information of multiple shard nodes passes the consistency verification at the first moment, the proxy node can also send log information requests to each shard node at the second moment. Correspondingly, each shard node can send a log information response to the proxy node. In this way, the proxy node can parse the log information of each shard node and determine the second starting point of each shard node.

[0120] Next, the proxy node can obtain historical backup locations from the relay node, provided that the first and second starting locations of each shard node are the same. After the relay node returns the historical backup locations, the proxy node can determine the data between the historical backup locations and the second starting locations in the data change logs of each shard node as the incremental backup range for each shard node's data change logs. Then, the proxy node can send data change information requests to each shard node. Correspondingly, each shard node can send a data change information response to the proxy node. In this way, the proxy node can parse the data change information responses from each shard node, determine the data change information of each shard node within the incremental backup range, and update the backup database.

[0121] In this embodiment, the proxy node can obtain the first starting point and transaction information of multiple shard nodes at a first moment during the process of multiple shard nodes providing data services. When the transaction information of multiple shard nodes at the first moment passes consistency verification, it indicates that the execution of transactions by multiple shard nodes at the first moment will not further compromise the integrity of the database. In this case, the proxy node can further obtain the second starting point of multiple shard nodes at a second moment, and thus, when the first starting point and the second starting point of each shard node are the same, perform a backup operation on the data information of each shard node to complete the data backup.

[0122] Since the transaction information run by multiple shard nodes is consistent, and the starting points of multiple shard nodes are the same in the first and second time points, this application can determine the start time of the backup operation during the process of multiple shard nodes providing data services. This ensures data consistency while performing data backup, thus avoiding the problem of business downtime and low efficiency that often occurs when general technologies require temporary suspension of data services during data backup. Therefore, this application can solve the technical problem of business downtime and low efficiency that often arises when performing data backup on distributed systems using general technologies.

[0123] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0124] This application embodiment can divide the proxy node into functional modules according to the above method example. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. Optionally, the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0125] like Figure 10 The diagram shown is a structural schematic of a data backup device provided in an embodiment of this application. This data backup device can be applied to a proxy node to perform tasks such as... Figures 4-6 and Figure 8The data backup method shown is illustrated. The data backup device includes: an acquisition unit 901 and a processing unit 902.

[0126] The acquisition unit 901 is used to acquire the first starting point and transaction information of multiple shard nodes at a first moment during the process of multiple shard nodes providing data services; the first starting point is used to characterize the starting position of the shard node in the data change log at the first moment for storing the next data change information; the transaction information includes the number and status information of the transactions run by the multiple shard nodes at the first moment; for example, combined with Figure 4 The acquisition unit 901 can be used to execute S401.

[0127] The acquisition unit 901 is further configured to acquire the second starting point of the multiple shard nodes at a second time when the transaction information of the multiple shard nodes at the first time has passed the consistency verification; the second time is after the first time; for example, combined with Figure 4 The acquisition unit 901 can be used to execute S402.

[0128] Processing unit 902 is used to perform a backup operation on the data change logs of each shard node when the first start point and the second start point of each shard node are the same. For example, combined with Figure 4 The processing unit 902 can be used to execute S403.

[0129] Optionally, the data backup device further includes: a receiving unit 903;

[0130] Receiving unit 903 is configured to receive a backup instruction message indicating the execution of a backup operation; the backup instruction message includes a target node identifier; for example, in conjunction with... Figure 5 The receiving unit 903 can be used to execute S501.

[0131] The processing unit 902 is also used to verify the target node identifier based on a preset node identifier. For example, combined with Figure 5 The processing unit 902 can be used to execute S502.

[0132] Optionally, the acquisition unit 901 is specifically used for:

[0133] Create multiple backup subtasks; these subtasks are used to obtain the first starting point and transaction information; for example, combining... Figure 6 The acquisition unit 901 can be used to execute S601.

[0134] Multiple backup subtasks are executed in parallel to obtain the first starting point and transaction information of each shard node. For example, combining... Figure 6 The acquisition unit 901 can be used to execute S602.

[0135] Optionally, the processing unit 902 is specifically used for:

[0136] Obtain historical backup positions for multiple shard nodes; historical backup positions are used to characterize the termination position of the backed-up data change information in the data change log of the previous backup operation on the shard node; for example, combining... Figure 8 The processing unit 902 can be used to execute S701.

[0137] Back up the data change information between the historical backup point and the second starting point in the data change log of each shard node. For example, combine... Figure 8 The processing unit 902 can be used to execute S702.

[0138] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this application can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and other division methods may exist in actual implementation. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate; components shown as units may be one physical unit or multiple physical units, i.e., they may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data backup method, characterized in that, include: During the process of multiple shard nodes providing data services, the acquisition of the first starting point and transaction information of the multiple shard nodes at a first moment includes: creating multiple backup subtasks; the backup subtasks are used to acquire the first starting point and the transaction information; executing the multiple backup subtasks in parallel to obtain the first starting point and the transaction information of each shard node; the first starting point is used to characterize the starting position of the shard node in the data change log at the first moment for storing the next data change information; the transaction information includes the number of transactions run by the multiple shard nodes at the first moment, a transaction identifier for uniquely identifying each transaction, and status information; the data services include data acquisition services, data transmission services, data storage services, data processing services, and data exchange services; When the transaction information of multiple shard nodes passes the consistency verification at the first time, the second starting point of the multiple shard nodes is obtained at the second time; the second time is after the first time. When the transaction information of multiple shard nodes fails the consistency verification at the first time, the transaction information of multiple shard nodes is reacquired at the third time; the third time is after the first time and before the second time; when the transaction information of multiple shard nodes passes the consistency verification at the third time, the second starting point of multiple shard nodes at the second time is acquired. When the first starting point and the second starting point of each shard node are the same, a backup operation is performed on the data change log of each shard node.

2. The data backup method according to claim 1, characterized in that, Before obtaining the first starting point and transaction information of the multiple shard nodes at the first moment, the method further includes: Receive a backup instruction message that instructs the execution of the backup operation; the backup instruction message includes a target node identifier; The target node identifier is verified based on the preset node identifier.

3. The data backup method according to claim 1, characterized in that, The backup operation on the data change logs of each of the aforementioned shard nodes includes: Obtain the historical backup positions of multiple shard nodes; the historical backup positions are used to characterize the termination position of the data change information backed up in the data change log of the shard node during the previous backup operation. The data change information between the historical backup site and the second starting site in the data change log of each of the aforementioned shard nodes is backed up.

4. A data backup device, characterized in that, include: Acquisition unit and processing unit; The acquisition unit is used to acquire the first starting point and transaction information of the multiple shard nodes at the first moment during the process of multiple shard nodes providing data services. The first starting point is used to characterize the starting position of the shard node in the data change log at the first time point for storing the next data change information; The transaction information includes the number of transactions run by the multiple shard nodes at the first moment, a transaction identifier for uniquely identifying each transaction, and status information; the data services include data acquisition services, data transmission services, data storage services, data processing services, and data exchange services. The acquisition unit is further configured to: acquire the second starting position of the multiple shard nodes at a second time when the transaction information of the multiple shard nodes at the first time passes the consistency verification; the second time is after the first time; reacquire the transaction information of the multiple shard nodes at a third time when the transaction information of the multiple shard nodes at the first time fails the consistency verification; the third time is after the first time and before the second time; acquire the second starting position of the multiple shard nodes at the second time when the transaction information of the multiple shard nodes at the third time passes the consistency verification; The processing unit is configured to perform a backup operation on the data change log of each shard node when the first starting point and the second starting point of each shard node are the same. The acquisition unit is specifically used for: Multiple backup subtasks are created; the backup subtasks are used to obtain the first starting point and the transaction information. Multiple backup subtasks are executed in parallel to obtain the first starting point and transaction information of each shard node.

5. The data backup device according to claim 4, characterized in that, Also includes: Receiving unit; The receiving unit is configured to receive a backup instruction message that instructs the execution of the backup operation; The backup instruction message includes the target node identifier; The processing unit is also used to verify the target node identifier based on a preset node identifier.

6. The data backup device according to claim 4, characterized in that, The processing unit is specifically used for: Obtain the historical backup positions of multiple shard nodes; the historical backup positions are used to characterize the termination position of the data change information backed up in the data change log of the shard node during the previous backup operation. The data change information between the historical backup site and the second starting site in the data change log of each of the aforementioned shard nodes is backed up.

7. A data backup device, characterized in that, It includes a memory and a processor; the memory is used to store computer execution instructions, and the processor is connected to the memory via a bus; when the data backup device is running, the processor executes the computer execution instructions stored in the memory, so that the data backup device performs the data backup method as described in any one of claims 1-3.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the data backup method as described in any one of claims 1-3.

Images