Data processing method and device

Abstract

The invention provides a data processing method and device, and relates to the field of computers. And the storage node receives the backup request sent by the computing node, reads the incremental data in the storage device according to the data identifier of the data to be backed up in the backup request, and then sends the incremental data to the computing node, so that the computing node backs up the incremental data. Wherein the storage device belongs to the storage node. Therefore, the incremental data reading operation of the computing node is unloaded to the storage node, the storage node reads the incremental data from the storage device of the storage node, the computing node does not need to execute the incremental data reading operation, and the input / output load of the computing node in the incremental backup process is reduced.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular, to a data processing method and apparatus. Background Art

[0002] Incremental backup is a type of backup, which means that after a full backup or the previous incremental backup, each subsequent backup only needs to back up the files that have increased or been modified compared to the previous one.

[0003] During the incremental backup process, the computing node scans all the data and then compares it with the data backed up last time to determine the incremental data to be backed up this time. To reduce the amount of data scanned, the computing node updates the block change tracking (BCT) file every time it modifies the data. Thus, during the incremental backup, only the part of the data where changes have occurred is scanned and read according to the block change tracking file as the incremental data for backup. However, the computing node needs to read the incremental data according to the block change tracking file, resulting in a large number of input / output (I / O) operations of the computing node. Summary of the Invention

[0004] This application provides a data processing method and apparatus, which solves the problem of a large number of input / output operations of the computing node during the incremental backup process.

[0005] In a first aspect, a data processing method is provided. The data processing method is applied to a storage system, and the storage system includes a computing node and a storage node. If the data processing method is executed by the storage node in the storage system. The data processing method includes: First, the storage node receives a backup request sent by the computing node, and the backup request includes a data identifier of the data to be backed up. Then, the storage node reads incremental data in the storage device according to the data identifier of the data to be backed up, and the incremental data is the part of the data to be backed up that has not been backed up. Finally, the storage node sends the incremental data to the computing node to instruct the computing node to back up the incremental data.

[0006] In this embodiment, compared with the computing node reading the incremental data to be backed up during the incremental backup, the operation of the computing node reading the incremental data is offloaded to the storage node. The storage node reads the incremental data and then returns the incremental data to the computing node, and the computing node then backs up and stores the incremental data. In this way, the computing node does not need to execute the operation of reading the incremental data, reducing the input / output load of the computing node during the incremental backup process.

[0007] As a possible implementation, the storage node uses a block change tracking file to read the unbacked-up incremental data in the data to be backed up. The storage node reads the block change tracking file from the storage device according to the data identifier of the data to be backed up, and the block change tracking file is used to indicate the changes of each data block in the data to be backed up. Then, the storage node determines the unbacked-up incremental data in the data to be backed up according to the block change tracking file, and reads the unbacked-up incremental data.

[0008] Optionally, the backup request further includes a first log sequence number and a second log sequence number. The first log sequence number is used to indicate the starting position of the incremental data, and the second log sequence number is used to indicate the ending position of the incremental data. The storage node determines that the unbacked-up incremental data in the data to be backed up is the data indicated by the data identifiers between the first log sequence number and the second log sequence number in the block change tracking file.

[0009] Based on the above implementation, compared with the computing node using the block change tracking file to identify and read the unbacked-up incremental data in the data to be backed up, the operation of the storage node to identify and read the unbacked-up incremental data in the data to be backed up reduces the occupation of the computing resources of the computing node. At the same time, since the computing node usually only has computing power, data such as the block change tracking file is stored in the storage node. When the computing node uses the block change tracking file to identify and read the unbacked-up incremental data in the data to be backed up, it needs to perform I / O operations to read the block change tracking file from the storage node. However, in the above implementation provided by the present application, the storage node reads the block change tracking file from its own storage device, reducing the I / O operations of the computing node, thereby reducing the I / O load of the computing node during the incremental backup process.

[0010] As a possible implementation, if the block change tracking file and the incremental data are stored by the storage node, the storage node also needs to update the block change tracking file and the incremental data. The storage node receives a data update request from the computing node, and the data update request includes updated data, as well as the data identifier and the log sequence number of the updated data. Then, the storage node writes the data identifier and the log sequence number of the updated data into the block change tracking file, and writes the updated data into the storage device. In this way, the storage node stores and updates the block change tracking file, and stores the updated data, enabling the storage node to read the incremental data from the storage node using the block change tracking file when receiving the backup request from the computing node, without the computing node updating the block change tracking file and the incremental data, thereby reducing the I / O load of the computing node during the incremental backup process.

[0011] Optionally, in addition to updating, the storage node's maintenance of the block change tracking file includes deleting the block change tracking file after the backup ends. The storage node receives a block change tracking file deletion request from the computing node and deletes the block change tracking file corresponding to the incremental data according to the block change tracking file deletion request.

[0012] In a second aspect, a data processing method is provided. This data processing method is applied to a storage system, which includes a computing node and a storage node. If this data processing method is executed by the computing node in the storage system. The computing node sends a backup request, and the backup request includes the data identifier of the data to be backed up. The backup request is used to instruct the storage node to return the unbacked incremental data in the data to be backed up in the storage device to the computing node according to the data identifier of the data to be backed up. Then, the computing node receives the incremental data returned by the storage node and backs up the incremental data to the storage device.

[0013] In this embodiment, compared with the computing node reading the incremental data to be backed up during incremental backup, the computing node instructs the storage node to return the unbacked incremental data, and then the computing node backs up and stores the incremental data. In this way, the computing node does not need to perform the operation of reading incremental data, reducing the input / output load of the computing node during the incremental backup process.

[0014] As a possible implementation, the computing node can also instruct the storage node to store and update the block change tracking file, etc. The computing node sends a data update request, which includes the updated data, as well as the data identifier and log sequence number of the updated data. The data update request is used to instruct the storage node to write the data identifier and log sequence number of the updated data into the block change tracking file in the storage device, and write the updated data into the storage device. In this way, the computing node triggers the storage node to update the block change tracking file and write the updated data, reducing the I / O operations of the computing node, thereby reducing the I / O load of the computing node during the incremental backup process.

[0015] Optionally, the storage device includes a first storage device and a second storage device. The first storage device is used to store the block change tracking file and the updated data, and the second storage device is used to back up the incremental data.

[0016] As a possible implementation, in addition to updating, the maintenance of the block change tracking file includes deleting the block change tracking file after the backup ends. The computing node sends a block change tracking file deletion request, which is used to instruct the storage node to delete the block change tracking file corresponding to the incremental data.

[0017] In a third aspect, the present application provides a data processing device, including a transceiver module and a processing module. The transceiver module is configured to receive a backup request sent by the computing node, where the backup request includes a data identifier of the data to be backed up. The processing module is configured to read the unbacked-up incremental data in the data to be backed up from the storage device according to the data identifier of the data to be backed up. The transceiver module is further configured to send the incremental data, so that the computing node backs up the incremental data.

[0018] As a possible implementation, the data processing device may further include other modules that perform the operation steps of the data processing method described in the first aspect.

[0019] In a fourth aspect, the present application provides a data processing device, including a transceiver module and a processing module. The transceiver module is configured to send a backup request, where the backup request includes a data identifier of the data to be backed up, and the backup request is used to instruct the storage node to return the unbacked-up incremental data in the data to be backed up in the storage device to the computing node according to the data identifier of the data to be backed up. The transceiver module is further configured to receive the incremental data. The processing module is configured to back up the incremental data to the storage device.

[0020] As a possible implementation, the data processing device may further include other modules that perform the operation steps of the data processing method described in the second aspect.

[0021] Regarding the technical principles and beneficial effects of the third aspect and the fourth aspect, reference may be made to the relevant descriptions of the first aspect above, and details will not be elaborated here.

[0022] In a fifth aspect, the present application provides an electronic device, including a memory and a processor, where the memory is configured to store a set of computer instructions; when the processor executes the set of computer instructions, the data processing method described in any possible implementation manner of the first aspect above is executed.

[0023] In a sixth aspect, the present application provides an electronic device, including a memory and a processor, where the memory is configured to store a set of computer instructions; when the processor executes the set of computer instructions, the data processing method described in any possible implementation manner of the second aspect above is executed.

[0024] In a seventh aspect, the present application provides a computer system, including a computing node and a storage node. The computing node is configured to send a backup request in response to an incremental backup operation sent by a client, where the backup request includes a data identifier of the data to be backed up. The storage node is configured to receive the backup request, read the unbacked-up incremental data in the data to be backed up from the storage device according to the data identifier of the data to be backed up, and send the incremental data. The computing node is further configured to back up the incremental data.

[0025] In an eighth aspect, a computer program product is provided. The computer program product includes a computer program or instructions, which, when running on a computer, cause the computer to execute the data processing method described in any possible implementation manner of the first aspect above.

[0026] In a ninth aspect, a computer program product is provided. The computer program product includes a computer program or instructions, which, when running on a computer, cause the computer to execute the data processing method described in any possible implementation manner of the second aspect above.

[0027] In a tenth aspect, a computer-readable storage medium is provided. The readable storage medium includes: a computer program or instructions; which, when running on a computer, cause the computer to execute the data processing method described in any possible implementation manner of the first aspect above.

[0028] In an eleventh aspect, a computer-readable storage medium is provided. The readable storage medium includes: a computer program or instructions; which, when running on a computer, cause the computer to execute the data processing method described in any possible implementation manner of the second aspect above. Description of the Drawings

[0029] Figure 1a It is a schematic diagram of the architecture of a computer system provided by an embodiment of the present application;

[0030] Figure 1b It is a schematic diagram of the modules of a storage system provided by an embodiment of the present application;

[0031] Figure 2 It is a schematic flowchart of a data processing method provided by an embodiment of the present application;

[0032] Figure 3 It is a schematic flowchart of the update steps of a block change tracking file and data to be backed up provided by an embodiment of the present application;

[0033] Figure 4 It is a schematic flowchart of a cleaning step provided by an embodiment of the present application;

[0034] Figure 5 It is a schematic diagram of the structure of a data processing device provided by an embodiment of the present application;

[0035] Figure 6 It is a schematic diagram of the structure of another data processing device provided by an embodiment of the present application;

[0036] Figure 7A schematic structural diagram of a computing device composition system provided by an embodiment of the present application. Detailed implementation manners

[0037] The data processing method and device provided by the embodiments of the present application can be applied to the scenario of database incremental backup. The database can be based on architectures such as distributed storage systems and centralized storage systems. The following briefly introduces the technologies that may be involved in the above database incremental backup scenario of the present application.

[0038] (1) Storage system

[0039] The storage system is the general term for the devices and technologies used by a computer to store data and programs, including various storage media and storage devices used by the computer. Distributed storage and centralized storage are two different architectures of the storage system.

[0040] A centralized storage system refers to a system in which one or more main devices form a central node, data is centrally stored in this central node, and all data processing services of the entire system are centrally deployed on this central node. In other words, in a centralized storage system, the terminal or client is only responsible for data input and output, while the storage and control processing of data are completely handed over to the central node to complete. The biggest feature of the centralized system is that the deployment structure is simple, there is no need to consider how to deploy services on multiple nodes, and thus there is no need to consider the distributed cooperation problem between multiple nodes.

[0041] A distributed storage system is a system in which data is dispersed and stored on multiple independent storage nodes. Traditional network storage systems use centralized storage arrays to store all data. The performance of the storage array is both the bottleneck of the system performance and the focus of reliability and security, and it cannot meet the needs of large-scale storage applications. The distributed network storage system adopts an extensible system structure and uses multiple storage nodes to share the storage load. It not only improves the reliability, availability and access efficiency of the system, but also is easy to expand.

[0042] (2) Checkpoint

[0043] In a database, a checkpoint (CKPT) is a background process used to coordinate the operations of buffer flushing and log file archiving in the database. For example, a checkpoint is used to write the modified dirty blocks in the buffer to disk and archive the dirty redo log files to a specified location. In addition, the checkpoint is also responsible for maintaining the control file and recording the structure and configuration information of the database, etc., which will not be elaborated here.

[0044] (3) Incremental backup

[0045] Incremental backup means that after a full backup or the previous incremental backup, each subsequent backup only needs to back up the data that has increased or been modified compared to the previous one. This means that the object of the first incremental backup is the data that has increased and been modified after the full backup; the object of the second incremental backup is the data that has increased and been modified after the first incremental backup, and so on. The most significant advantage of the incremental backup method is that there is no duplicate backup data, so the amount of backup data is small and the time required for backup is very short.

[0046] When performing an incremental backup, the computing node in the storage system scans all data and then compares it with the data that has been backed up to identify the newly added data as the data to be backed up. To reduce the number of data scans, the computing node can use a block change tracking file to indicate the newly added parts of the data. Among them, the principle of the block change tracking file is to record the changes of each data block in the data file and save this change information in the block change tracking file. For example, the block change tracking file uses a bitmap to save the changes of each database.

[0047] Specifically, when the computing node refreshes the data, it updates the identifier and log sequence number (LSN) of the data to be refreshed (such as dirty pages, that is, data pages in which the data has been modified but not written to the disk) to the block change tracking file and persists the data to be refreshed. During incremental backup, the computing node first reads the block change tracking file, then reads the increased and modified data that has changed from the persistently stored data to be refreshed according to the block change tracking file, and then organizes the read data into a file and writes it to the backup storage medium. In this way, during the incremental backup process, the computing node can only read the changed data according to the block change tracking file.

[0048] However, in the traditional incremental backup method, usually the computing node is responsible for updating and reading the block change tracking file and the operations of reading and writing data for backup. In addition to reading and writing the data to be backed up, the computing node also needs additional input / output resources to update the block change tracking file, resulting in a relatively large input / output load on the computing node.

[0049] This application provides a data processing method, specifically a data processing method involving "using a storage node to replace part of the incremental backup operations of a computing node". First, when the storage node receives a backup request from the computing node to perform an incremental backup, the storage node reads the incremental data from the storage device according to the data identifier of the data to be backed up. Among them, the backup request includes the data identifier of the data to be backed up, and the incremental data is the part of the data to be backed up that has not been backed up. The storage device belongs to the storage node. Then, the storage node sends the incremental data to the computing node to instruct the computing node to back up the incremental data.

[0050] Based on the above data processing method, compared with the computing node reading the incremental data to be backed up during incremental backup, the operation of the computing node reading the incremental data is offloaded to the storage node. The storage node reads the incremental data and then returns the incremental data to the computing node, and the computing node then backs up and stores the incremental data. In this way, the computing node does not need to perform the operation of reading the incremental data, reducing the input / output load of the computing node during incremental backup.

[0051] The implementation manners of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0052] Figure 1a It is a schematic diagram of the architecture of a computer system provided by the present application. In Figure 1a the application scenario shown, users access data through application programs. The computers running these application programs are called "application servers". The application server 101 can be a physical machine or a virtual machine. Physical application servers include but are not limited to desktop computers, servers, laptops, and mobile devices. The application server accesses the storage system 120 through the fiber optic switch 110 to access data. However, the switch 110 is only an optional device, and the application server 101 can also directly communicate with the storage system 120 through the network. Alternatively, the fiber optic switch 110 can also be replaced with an Ethernet switch, an InfiniBand switch, a RoCE (RDMA over converged ethernet) switch, etc.

[0053] Figure 1a The storage system 120 shown is a centralized storage system. The characteristic of a centralized storage system is that there is a unified entry, and all data coming from external devices has to pass through this entry, and this entry is the computing node 121 of the centralized storage system. The computing node 121 is the most core component in the centralized storage system, and many functions of the storage system are implemented therein.

[0054] As Figure 1a shown, there is one or more controllers in the computing node 121. Figure 1a Taking the computing node including one controller as an example for illustration. There is a mirror channel between controller 0 and controller 1. Then when controller 0 writes a piece of data into its memory 124, it can send a copy of the data to controller 1 through the mirror channel, and controller 1 stores the copy in its own local memory 124. Thus, controller 0 and controller 1 back up each other. When controller 0 fails, controller 1 can take over the business of controller 0. When controller 1 fails, controller 0 can take over the business of controller 1, thereby avoiding the unavailability of the entire storage system 120 caused by hardware failures.

[0055] In some possible embodiments, the computing node 121 may also be referred to as an engine.

[0056] The computing node 121 further includes a front-end interface 125 and a back-end interface 126. The front-end interface 125 is used to communicate with the application server 101 to provide storage services for the application server 101. The back-end interface 126 is used to communicate with the storage device 133 to expand the capacity of the storage system. Through the back-end interface 126, the computing node 121 can connect more storage devices 133 to form a very large storage resource pool.

[0057] In terms of hardware, as Figure 1a shown, the controller 0 at least includes a processor 123 and a memory 124. The processor 123 is a central processing unit (CPU) for processing data access requests from outside the storage system (servers or other storage systems) and also for processing requests generated within the storage system. For example, when the processor 123 receives a write data request sent by the application server 101 through the front-end port 125, it temporarily stores the data in these write data requests in the memory 124. When the total amount of data in the memory 124 reaches a certain threshold, the processor 123 sends the data stored in the memory 124 to the storage device 133 through the back-end port for persistent storage. Another example is that after the processor 123 receives a write data request sent by the application server 101 through the front-end port 125, it sends a data update request including the updated data, as well as the data identifier and log sequence number of the updated data, to the storage node 130 through the back-end port. The data update request is used to instruct the storage node 130 to write the data identifier and log sequence number of the updated data into the block change tracking file in the storage device and write the updated data into the storage device. Still another example is that the processor 123 receives incremental data sent by the storage node 130 through the back-end port and sends the incremental data to the storage device 133 through the back-end port for persistent storage.

[0058] Optionally, the processor 123 initiates a data update request through a checkpoint thread and initiates the persistent storage of incremental data through a backup thread.

[0059] The memory 124 refers to the internal memory that directly exchanges data with the processor. It can read and write data at any time and is very fast. It serves as the temporary data memory for the operating system or other running programs. The memory includes at least two types of memories. For example, the memory can be either a random access memory or a read-only memory (ROM). For instance, the random access memory can be a dynamic random access memory (DRAM) or a storage class memory (SCM). DRAM is a semiconductor memory, and like most random access memories (RAM), it belongs to a volatile memory device. SCM is a composite storage technology that combines the characteristics of traditional storage devices and memories. The storage class memory can provide faster read and write speeds than hard disks, but its access speed is slower than that of DRAM and it is also cheaper than DRAM. However, DRAM and SCM are only exemplary in this embodiment. The memory can also include other random access memories, such as static random access memory (SRAM), etc. For the read-only memory, for example, it can be a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), etc. In addition, the memory 124 can also be a dual in-line memory module or a dual-line memory module (Dual In-line Memory Module, DIMM for short), that is, a module composed of dynamic random access memory (DRAM), or it can be a solid state disk (SSD). In practical applications, multiple memories 124 and different types of memories 124 can be configured in the controller 0. The number and type of the memory 113 are not limited in this embodiment. In addition, the memory 124 can be configured to have a power retention function. The power retention function means that when the system experiences a power failure and then powers on again, the data stored in the memory 124 will not be lost. The memory with the power retention function is called a non-volatile memory.

[0060] The software program is stored in the memory 124. The processor 123 runs the software program in the memory 124 to manage the hard disk. For example, the hard disk is abstracted as a storage resource pool and then divided into logical unit numbers (LUNs) for the server to use. Here, the LUN is actually the hard disk seen on the server. Of course, some centralized storage systems are also file servers themselves and can provide shared file services for the server.

[0061] The hardware components and software structure of controller 1 (and other Figure 1a controllers not shown in the figure) are similar to those of controller 0 and will not be elaborated here.

[0062] Figure 1a Shown is a centralized storage system with disk control separation. In this system, the computing node 121 may not have a storage device slot. The storage device 134 needs to be placed in the storage node 130, and the backend interface 126 communicates with the storage node 130. The backend interface 126 exists in the computing node 121 in the form of an adapter card. Two or more backend interfaces 126 can be used simultaneously on one computing node 121 to connect multiple storage nodes. Alternatively, the adapter card can also be integrated on the motherboard. In this case, the adapter card can communicate with the processor 123 through the peripheral component interconnect express (PCIE) bus.

[0063] It should be noted that Figure 1a only one computing node 121 is shown in the figure. However, in actual applications, the storage system may include two or more computing nodes 121, and redundancy or load balancing is performed among the multiple computing nodes 121.

[0064] The storage node 130 includes a control unit 131 and several storage devices 134.

[0065] The control unit 131 can have various forms. In one case, the storage node 130 belongs to an intelligent disk enclosure in the hard disk enclosure. As Figure 1a shown in the figure, the control unit 131 includes a CPU and a memory. The CPU is used to perform operations such as address conversion and reading and writing data. The memory is used to temporarily store the data to be written to the storage device 134 or the data read from the storage device 134 to be sent to the controller.

[0066] In another case, the control unit 131 is a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of a CPU, but is more specialized and can operate efficiently on network data packets, storage requests, or analysis requests. The DPU is distinguished from the CPU by a high degree of parallelism (required to process a large number of requests). Optionally, the DPU here can also be replaced by a processing chip such as a graphics processing unit (GPU), a neural-network processing unit (NPU), etc.

[0067] Generally, the number of control units 131 can be one, or two or more. When the storage node 130 includes at least two control units 131, there may be an ownership relationship between the storage device 134 and the control unit 131. If there is an ownership relationship between the storage device 134 and the control unit 131, then each control unit can only access the hard disks belonging to it, which often involves forwarding read / write data requests between the control units 131, resulting in a long data access path. In addition, if the storage space is insufficient, when adding a new storage device 134 to the storage node 130, it is necessary to re-bind the ownership relationship between the storage device 134 and the control unit 131, and the operation is complex, resulting in poor scalability of the storage space. Therefore, in another embodiment, the function of the control unit 131 can be offloaded to the network card 132. In other words, in this embodiment, the storage node 130 does not have a control unit 131 internally, but the network card 132 is used to complete data reading and writing, address conversion, and other computing functions. At this time, the network card 132 is a smart network card. It can include a CPU and memory. The network card 132 can also be a programmable electronic component, such as a data processing unit (DPU). The DPU has the versatility and programmability of a CPU, but is more specialized and can operate efficiently on network data packets, storage requests, or analysis requests. The DPU is distinguished from the CPU by a high degree of parallelism (required to process a large number of requests). Optionally, the DPU here can also be replaced by a processing chip such as a graphics processing unit (GPU), a neural-network processing unit (NPU), etc. There is no ownership relationship between the network card 132 and the storage device 134 in the storage node 130, and the network card 132 can access any storage device 134 in the storage node 130. Therefore, it is more convenient to expand the hard disk when the storage space is insufficient.

[0068] In this embodiment, the control unit 131 or the network card 132 may also be configured with a storage device 133, which is a persistent memory medium, such as persistent memory (PM), or non-volatile random access memory (NVRAM), or phase change memory (PCM), etc. The CPU is used to perform operations such as address translation and data reading and writing. The memory is used to temporarily store the data to be written to the storage device 133, or the data read from the storage device 133 and to be sent to the controller.

[0069] In a possible embodiment of the present application, after receiving the backup request sent by the computing node 121, the control unit 131 or the network card 132 reads the unbacked-up incremental data of the data to be backed up from the storage device 133 according to the data identifier of the data to be backed up, and returns the incremental data to the computing node 121. In a possible embodiment of the present application, the storage device 134 of the control unit 131 or the network card 132 is used to store block change tracking files, incremental data, etc., and is called the first storage device. The storage device 133 is used to store the backup data after the incremental backup is completed, and is called the second storage device. If the above functions are implemented by software modules, such as Figure 1b As shown, the computing node 121 may include a checkpoint thread and a backup thread, and the storage node 130 may include a backup storage system. The checkpoint thread sends a data update request to the backup storage system when flushing a page. The backup storage system writes a block change tracking file and flushes the page according to the data update request. Then the backup thread sends a backup request to the backup storage system. The backup storage system reads the unbacked-up incremental data of the data to be backed up from the storage device 133 according to the backup request, and returns the incremental data to the backup thread.

[0070] According to the type of the communication protocol between the computing node 121 and the storage node 130, the storage node 130 may be a SAS (Serial Attached SCSI) storage node, or an NVMe storage node, an IP storage node, or other types of storage nodes. The SAS storage node adopts the SAS 3.0 protocol, and each enclosure supports 25 SAS hard disks. The computing node 121 is connected to the storage node 130 through an on-board SAS interface or a SAS interface module. The NVMe storage node is more like a complete computer system, and the NVMe hard disk is inserted into the NVMe storage node. The NVMe storage node is then connected to the computing node 121 through an RDMA port.

[0071] Figure 1a It is only a schematic diagram of a computer system architecture provided by the embodiments of the present application. Figure 1aThe positional relationships among the devices, components, modules, etc. shown do not constitute any limitation. For example, Figure 1a The computer system is described with a disk control separation architecture. In another embodiment, the computer system can also be a disk control integrated architecture.

[0072] Next, the data processing method provided by this application will be specifically described with reference to the accompanying drawings.

[0073] In the scenario of incremental backup, the computing node 121 and the storage node 130 in the computer system above Figure 1a can be the execution entities of the data processing method. Next, taking the computer system as an example, the data processing method will be elaborated in detail. As Figure 2 shown, Figure 2 the storage service in it is used to maintain the storage device 134, and the file system is used to maintain the storage device 133. Among them, the storage service can support storage devices 134 of types such as Storage Area Network (SAN) and Network Area Storage (NAS). The data processing method can include the following steps 210 - step 250.

[0074] Step 210, the application server 101 sends an incremental backup request to the computing node 121.

[0075] As a possible example, the application server 101 responds to the operation of the user triggering incremental backup and sends an incremental backup request to the computing node 121. The incremental backup request is used to instruct the computing node 121 to perform the incremental backup operation, such as sending a backup request to the storage node 130.

[0076] Step 220, the computing node 121 sends a backup request to the storage node 130.

[0077] As a possible example, when the computing node 121 receives the incremental backup request sent by the application server 101, it sends a backup request to the storage node 130.

[0078] As a possible implementation, the computing node 121 uses the data identifier of the data that needs to be written from the memory to the storage device for persistent storage as the content of the backup request to generate a backup request.

[0079] Optionally, the data to be backed up that needs to be persistently stored is at least one flush page, and the data identifier is the page identifier of the flush page, such as page id.

[0080] Step 230, the storage node 130 reads the unbacked-up incremental data in the data to be backed up from the storage device according to the data identifier of the data to be backed up.

[0081] As a possible example, when the storage node 130 receives a backup request, it reads the block change tracking file according to the data identifier of the data to be backed up, and determines the incremental data that has not been backed up in the data to be backed up according to the block change tracking file.

[0082] As a possible implementation, the backup request further includes a log sequence number for indicating the start position and the end position of the data to be backed up. The storage node 130 determines the incremental data that has not been backed up in the data to be backed up in the block change tracking file according to the log sequence number of the start position and the end position of the data to be backed up.

[0083] Optionally, the log sequence number of the start position of the data to be backed up is the first log sequence number, the log sequence number of the end position of the data to be backed up is the second log sequence number, and the incremental data that has not been backed up is the data indicated by the data identifier between the first log sequence number and the second log sequence number in the block change tracking file.

[0084] Wherein, the log sequence number can be the logical sequence number of the log, i.e., LSN, and LSN increases as the log is written. The first log sequence number can be the LSN of the end position of the data that has been backed up, i.e., the LSN of the last checkpoint, also known as lastcheckpoint at. The second log sequence number can be the LSN of the end position of the data to be backed up this time, i.e., the LSN in the redo log, also known as log sequence number.

[0085] For example, the data to be backed up includes 6 refreshed pages with page id from 5 to 10. For the refreshed page with page id 5, the corresponding first log sequence number is 6. For the refreshed page with page id 10, the corresponding second log sequence number is 11. The storage node 130 reads the block change tracking file from the storage device 134. The block change tracking file stores the page id and the log sequence number corresponding to the refreshed page. According to the first log sequence number 6 to the second log sequence number 11, it is determined that the 6 refreshed pages with page id equal to 5 to page id equal to 10 are the data to be backed up, and the incremental data that has not been backed up in the data to be backed up is the data blocks that have changed in the 6 refreshed pages with page id equal to 5 to page id equal to 10 indicated by the block change tracking file.

[0086] Optionally, after the storage node 130 determines the page id of the data to be backed up, it can also deduplicate the data to be backed up to remove the duplicate refreshed pages in the data to be backed up.

[0087] As a possible implementation, after determining the incremental data in the data to be backed up that has not been backed up, the storage node 130 reads the incremental data in the data to be backed up that has not been backed up from the storage device 134.

[0088] Optionally, the storage node 130 reads the incremental data in the data to be backed up that has not been backed up from the storage device 134 according to the block change tracking file.

[0089] For example, the block change tracking file indicates the data blocks that have changed in 6 refreshed pages with page id from 5 to 10, and the storage device 134 stores 6 refreshed pages with page id from 5 to 10. The storage node 130 reads the corresponding data blocks that have changed in the above 6 refreshed pages in the storage device 134 according to the data blocks that have changed in the above 6 refreshed pages indicated by the block change tracking file, and obtains the incremental data that has not been backed up.

[0090] In a possible embodiment of the present application, the storage node 130 updates the block change tracking file and stores the data to be backed up in the storage device 134. For the specific steps, please refer to Figure 3 Steps 310 - 330 shown, which will not be elaborated here.

[0091] Step 240, the storage node 130 sends the incremental data.

[0092] As a possible example, the storage node 130 sends the incremental data to the computing node 121.

[0093] As a possible implementation, the storage node 130 can return the incremental data to the computing node 121 in batches.

[0094] Step 250, the computing node 121 backs up the incremental data.

[0095] As a possible example, after receiving the incremental data sent by the computing node 121, the computing node 121 backs up the incremental data to the storage device 133.

[0096] As a possible implementation, after completing the backup of the incremental data, the computing node 121 can also instruct the storage node 130 to clean up the backed-up data and the block change tracking file corresponding to the backed-up data. For the specific steps, please refer to Figure 4 Steps 410 - 430 shown, which will not be elaborated here.

[0097] In a possible embodiment of the present application, the operations of the computing node 121 in the above steps 210 - 250 may be executed by a backup thread running on the computing node 121. The operations of the storage node 130 in the above steps 210 - 250 may be executed by the server side of the backup storage system of the storage node 130.

[0098] Based on the above steps 210 - 250, as opposed to the computing node reading the incremental data to be backed up during incremental backup, the operation of the computing node reading the incremental data is offloaded to the storage node. The storage node reads the incremental data and then returns the incremental data to the computing node, and the computing node then performs backup storage on the incremental data. In this way, the computing node does not need to execute the operation of reading the incremental data, reducing the input / output load of the computing node during incremental backup.

[0099] As described above in conjunction with Figure 2 the data processing method has been described generally. Next, in conjunction with Figure 3 , the update steps of the block change tracking file and the storage steps of the data to be backed up will be described in detail. As Figure 3 shown, the update steps of the block change tracking file and the storage steps of the data to be backed up include the following steps 310 - 330.

[0100] Step 310, the computing node 121 sends a data update request.

[0101] As a possible example, the computing node 121 sends a data update request to the storage node 130. Among them, the data update request includes the updated data, as well as the data identifier and log sequence number of the updated data.

[0102] As a possible implementation, the above data update request is that the computing node 121 calls the interface of the server side of the backup storage system of the storage node 130 to flush the dirty pages of the storage node 130. The data contained in the dirty pages to be flushed is the updated data.

[0103] Step 320, the storage node 130 writes the data identifier and log sequence number of the updated data into the block change tracking file.

[0104] As a possible example, the storage node 130 writes the data identifier and log sequence number of the updated data into the block change tracking file in directories according to the thread identifier of the checkpoint thread. Among them, the directories can be divided according to the namespace.

[0105] As a possible implementation, the storage node 130 can write to the block change tracking file in batches.

[0106] Optionally, the above log sequence number may refer to the redo log corresponding to the log sequence number.

[0107] Step 330: The storage node 130 writes the updated data into the storage device 134.

[0108] As a possible example, the storage node 130 writes the updated data into the internal storage device of the storage node 130, i.e., the storage device 134, which is a storage device that the server of the backup storage system of the storage node 130 can directly read and write.

[0109] Based on the above steps 310 - 330, the storage node 130 performs the writing of the updated data and the update of the block change tracking file in the local storage device 134. The computing node 121 only sends a request to the storage node 130 to trigger the corresponding operation, thereby offloading the writing operation of the updated data and the update operation of the block change tracking file to the storage node 130, reducing the I / O operations caused by the data operations of the computing node 121 and alleviating the I / O pressure of the computing node 121.

[0110] In a possible embodiment of the present application, the computing node 121 and the storage node 130 maintain the updated data and the block change tracking file through the above steps 310 - 330. After the computing node 121 sends a backup request and completes the above steps 210 - 250, the computing node 121 can also instruct the storage node 130 to execute a cleaning step to clean the backed-up data and the corresponding block change tracking file of the backed-up data, so as to reduce the occupation of the storage resources of the storage node 130 and improve the overall performance of the computer system.

[0111] As Figure 4 shown, the cleaning step may include the following steps 410 - 430.

[0112] Step 410: The computing node 121 sends a block change tracking file deletion request.

[0113] As a possible example, the computing node 121 sends a block change tracking file deletion request to the storage node 130. Among them, the block change tracking file deletion request includes the data identifier of the incremental data that has completed the incremental backup.

[0114] As a possible implementation manner, the data identifier of the incremental data that has completed the incremental backup may be the page id of the refreshed page, for example, the value of the page id ranges from 5 to 10.

[0115] Step 420: The storage node 130 deletes the block change tracking file according to the block change tracking file deletion request.

[0116] As a possible example, the storage node 130 determines the block change tracking file to be deleted according to the data identifier of the incremental data for which the incremental backup has been completed, and deletes the block change tracking file to be deleted in the storage device 134. For example, the storage node 130 deletes the block change tracking files corresponding to the page id values from 5 to 10.

[0117] Step 430: The storage node 130 deletes the updated data according to the block change tracking file deletion request.

[0118] As a possible example, the storage node 130 determines the updated data to be deleted according to the data identifier of the incremental data for which the incremental backup has been completed, and deletes the updated data to be deleted in the storage device 134. For example, the storage node 130 deletes the refreshed pages corresponding to the pageid values from 5 to 10.

[0119] Figure 5 It is a schematic structural diagram of a possible data processing device provided in this embodiment. This data processing device can be used to implement the functions of the storage node 130 in the above method embodiment, and thus also has the beneficial effects of the above method embodiment. In this embodiment, this data processing device can be a node composed of the storage node 130 and / or other servers as shown in Figure 1a and can also be a module (such as a chip) applied to the server.

[0120] Such as Figure 5 shown, the data processing device 500 includes a transceiver module 510 and a processing module 520.

[0121] The data processing device 500 can be used to implement the functions of the storage node 130 in the above Figure 2 shown method embodiment, and each module included in the data processing device 500 is specifically used to implement the following functions.

[0122] The transceiver module 510 is used to receive the backup request sent by the computing node, and the backup request includes the data identifier of the data to be backed up. For example, the transceiver module 510 is used to receive the backup request sent by the computing node 121 in Figure 2 shown in step 220.

[0123] The processing module 520 is used to read the unbacked-up incremental data in the to-be-backed-up data from the storage device according to the data identifier of the to-be-backed-up data. For example, the processing module 520 is used to execute Figure 2 shown in step 230.

[0124] The transceiver module 510 is also used to send the incremental data so that the computing node backs up the incremental data. For example, the transceiver module 510 is used to execute Figure 2 shown in step 240.

[0125] As a possible implementation manner, the processing module 520 is specifically configured to: read a block change tracking file from the storage device according to the data identifier of the data to be backed up, where the block change tracking file is used to indicate changes of each data block in the data to be backed up; determine the incremental data that has not been backed up in the data to be backed up according to the block change tracking file; and read the incremental data that has not been backed up.

[0126] As a possible implementation manner, the backup request further includes a first log sequence number and a second log sequence number, where the first log sequence number is used to indicate the starting position of the incremental data, and the second log sequence number is used to indicate the ending position of the incremental data. The processing module 520 is specifically configured to: determine that the incremental data that has not been backed up in the data to be backed up is the data indicated by the data identifiers between the first log sequence number and the second log sequence number in the block change tracking file.

[0127] As a possible implementation manner, the transceiver module 510 is further configured to: receive a data update request of the computing node, where the data update request includes updated data, and a data identifier and a log sequence number of the updated data. The processing module 520 is further configured to: write the data identifier and the log sequence number of the updated data into the block change tracking file; and write the updated data into the storage device.

[0128] As a possible implementation manner, the storage device includes a first storage device and a second storage device, where the first storage device is used to store the block change tracking file and the updated data, and the second storage device is used to back up the incremental data.

[0129] As a possible implementation manner, the transceiver module 510 is further configured to: receive a block change tracking file deletion request of the computing node. The processing module 520 is further configured to: delete the block change tracking file corresponding to the incremental data according to the block change tracking file deletion request.

[0130] It should be understood that the data processing device 500 in the embodiments of the present invention and the present application can be implemented by a CPU, or by an ASIC, or by a programmable logic device (PLD). The above PLD can be a complex programmable logic device (CPLD), an FPGA, a generic array logic (GAL), or any combination thereof. When the data processing device 500 is implemented by software Figure 2 for the data processing method shown, the data processing device 500 and its respective modules can also be software modules.

[0131] It should be understood that nodes such as storage nodes in the embodiments of the present application may correspond to the data processing device 500 in the embodiments of the application, and may correspond to the corresponding entities in the method according to the embodiments of the present application, and the above and other operations and / or functions of each module in the data processing device 500 are respectively for implementing Figure 2 the corresponding processes of the methods in , for the sake of brevity, they will not be elaborated here.

[0132] Figure 6 FIG. is a schematic structural diagram of another possible data processing device provided in this embodiment. This data processing device can be used to implement the functions of the computing node 121 in the above method embodiments, and thus also has the beneficial effects of the above method embodiments. In this embodiment, this data processing device can be a node composed of the computing node 121 and / or other servers as shown in Figure 1a , and can also be a module (such as a chip) applied to the server.

[0133] Such as Figure 6 shown, the data processing device 600 includes a transceiver module 610 and a processing module 620.

[0134] The data processing device 600 can be used to implement the functions of the computing node 121 in the above Figure 2 shown method embodiments, and each module included in the data processing device 600 is specifically used to implement the following functions.

[0135] The transceiver module 610 is used to send a backup request, and the backup request includes a data identifier of the data to be backed up. The backup request is used to instruct the storage node to return the unbacked-up incremental data in the storage device corresponding to the data identifier of the data to be backed up to the computing node. For example, the transceiver module 610 is used to execute Figure 2 the step 220 shown in .

[0136] The transceiver module 610 is also used to receive the incremental data. For example, the transceiver module 610 is used to receive Figure 2 the incremental data sent by the storage node 130 in the step 240 shown in .

[0137] The processing module 620 is used to back up the incremental data to the storage device. For example, the processing module 620 is used to execute Figure 2 the step 250 shown in .

[0138] As a possible implementation, the transceiver module 610 is further configured to: send a data update request, where the data update request includes updated data, as well as a data identifier and a log sequence number of the updated data, and the data update request is used to instruct the storage node to write the data identifier and the log sequence number of the updated data into a block change tracking file in the storage device, and write the updated data into the storage device.

[0139] As a possible implementation, the storage device includes a first storage device and a second storage device. The first storage device is used to store the block change tracking file and the updated data, and the second storage device is used to back up the incremental data.

[0140] As a possible implementation, the transceiver module 610 is further configured to: send a block change tracking file deletion request, and the block change tracking file deletion request is used to instruct the storage node to delete the block change tracking file corresponding to the incremental data.

[0141] It should be understood that the data processing device 600 in the embodiments of the present invention and the present application can be implemented by a CPU, or can be implemented by an ASIC, or a PLD. The above PLD can be a complex programmable logic device CPLD, FPGA, GAL, or any combination thereof. When the data processing device 600 is implemented by software Figure 2 for the data processing method shown, the data processing device 600 and its various modules can also be software modules.

[0142] It should be understood that nodes such as computing nodes in the embodiments of the present application can correspond to the data processing device 600 in the embodiments of the application, and can correspond to the corresponding entities executing the methods according to the embodiments of the present application, and the above and other operations and / or functions of the various modules in the data processing device 600 are respectively for implementing Figure 2 the corresponding processes of the methods in, and for the sake of brevity, they will not be described in detail here.

[0143] The present application also provides a system composed of computing devices as shown in Figure 7 where the system includes a plurality of computing devices 700, and each computing device 700 includes a memory 701, a processor 702, a communication interface 703, and a bus 704. Among them, the memory 701, the processor 702, and the communication interface 703 are communicatively connected to each other through the bus 704. The computing device 700 can be Figure 1a the computing node or the storage node in.

[0144] The memory 701 can be a read-only memory, a static storage device, a dynamic storage device, or a random access memory. The memory 701 can store computer instructions and a set of data required for executing the computer instructions. When the computer instructions stored in the memory 701 are executed by the processor 702, the processor 702 and the communication interface 703 are used to execute the data processing method.

[0145] The processor 702 can be a general-purpose central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or any combination thereof. The processor 702 can include one or more chips. The processor 702 can include an AI accelerator, for example, a neural processing unit (NPU). In addition, Figure 7 in this example, each computing device 700 includes one processor 702. Specifically, in implementation, the number and type of the processors 702 in each computing device 700 can be set according to service requirements. For the same computing device 700, it can include one or more processors. When the same computing device 700 includes multiple processors, the type of the processors is not limited in this application.

[0146] The communication interface 703 uses a transceiver module such as, but not limited to, a transceiver to implement the communication between the computing device 700 and other devices or a communication network. For example, requests and the like can be received or sent through the communication interface 703.

[0147] The bus 704 can include a path for transmitting information between various components of the computing device 700 (for example, the memory 701, the processor 702, the communication interface 703).

[0148] A communication path is established between each of the above-mentioned computing devices 700 through a communication network. Any one of the computing devices 700 can be a computer (for example: a server) in a distributed storage system, or a computer in an edge data center, or a terminal computing device.

[0149] The functions of nodes such as computing nodes and / or storage nodes can be deployed on each computing device 700. For example, execute Figure 2 any one of the steps 210 - step 250 shown, or execute the functions of each module in the data processing device 500 and the data processing device 600.

[0150] The method steps in this embodiment can be implemented in a hardware manner or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), register, hard disk, removable hard disk, CD-ROM, or any other form of storage medium well-known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. Additionally, the ASIC can be located in a terminal device. Of course, the processor and the storage medium can also exist as discrete components in a network device or a terminal device.

[0151] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, or other programmable devices. The computer program or instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired or wireless manner. The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid state drive (SSD). As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A data processing method, characterized in that, it is applied to a storage node, the storage node is connected to a computing node, and the method includes: receiving a backup request sent by the computing node, the backup request including a data identifier of data to be backed up; reading, according to the data identifier of the data to be backed up, the unbacked-up incremental data in the data to be backed up from a storage device; sending the incremental data so that the computing node backs up the incremental data.

2. The method according to claim 1, characterized in that, the reading, according to the data identifier of the data to be backed up, the unbacked-up incremental data in the data to be backed up from a storage device includes: reading a block change tracking file from the storage device according to the data identifier of the data to be backed up, the block change tracking file being used to indicate changes of each data block in the data to be backed up; determining, according to the block change tracking file, the unbacked-up incremental data in the data to be backed up; reading the unbacked-up incremental data.

3. The method according to claim 2, characterized in that, the backup request further includes a first log sequence number and a second log sequence number, the first log sequence number being used to indicate a start position of the incremental data, and the second log sequence number being used to indicate an end position of the incremental data, and the determining, according to the block change tracking file, the unbacked-up incremental data in the data to be backed up includes: determining that the unbacked-up incremental data in the data to be backed up is data indicated by data identifiers between the first log sequence number and the second log sequence number in the block change tracking file.

4. The method according to claim 2 or 3, characterized in that, the method further includes: receiving a data update request from the computing node, the data update request including updated data, as well as a data identifier and a log sequence number of the updated data; writing the data identifier and the log sequence number of the updated data into the block change tracking file; writing the updated data into the storage device.

5. The method according to any one of claims 1-4, characterized in that, the storage device includes a first storage device and a second storage device, the first storage device is used to store the block change tracking file and the updated data, and the second storage device is used to back up the incremental data.

6. The method according to any one of claims 2-5, characterized in that, the method further includes: receiving a block change tracking file deletion request from the computing node; deleting, according to the block change tracking file deletion request, the block change tracking file corresponding to the incremental data.

7. A data processing method, characterized in that, it is applied to a computing node, the computing node is connected to the storage node, and the method includes: sending a backup request, the backup request including a data identifier of data to be backed up, the backup request being used to instruct the storage node to return the unbacked-up incremental data in the data to be backed up in the storage device to the computing node according to the data identifier of the data to be backed up; receiving the incremental data; backing up the incremental data to the storage device.

8. The method according to claim 7, wherein, the method further comprises: sending a data update request, the data update request including updated data, as well as a data identifier and a log sequence number of the updated data, the data update request being used to instruct the storage node to write the data identifier and the log sequence number of the updated data into a block change tracking file in the storage device, and to write the updated data into the storage device.

9. The method according to claim 8, wherein, the storage device includes a first storage device and a second storage device, the first storage device being used to store the block change tracking file and the updated data, and the second storage device being used to back up the incremental data.

10. The method according to any one of claims 7-9, wherein, the method further comprises: sending a block change tracking file deletion request, the block change tracking file deletion request being used to instruct the storage node to delete the block change tracking file corresponding to the incremental data.

11. A data processing apparatus, wherein, the apparatus includes a module for performing the operation steps of the method according to any one of claims 1-6 above.

12. A data processing apparatus, wherein, the apparatus includes a module for performing the operation steps of the method according to any one of claims 7-10 above.

13. An electronic device, wherein, the electronic device includes a memory and a processor, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, it performs the operation steps of the method according to any one of claims 1-6 above.

14. An electronic device, wherein, the electronic device includes a memory and a processor, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, it performs the operation steps of the method according to any one of claims 7-10 above.

15. A computer system, wherein, comprises: a computing node for sending a backup request in response to an incremental backup operation sent by a client, the backup request including a data identifier of data to be backed up; a storage node for receiving the backup request and reading, from the storage device, the incremental data that has not been backed up in the data to be backed up according to the data identifier of the data to be backed up, and sending the incremental data; the computing node is further used for backing up the incremental data.

16. A readable storage medium, wherein, the readable storage medium includes a computer program or instructions, and when the computer program or instructions run on a computer, the computer is caused to perform the operation steps of the method according to any one of claims 1-6 above.

17. A readable storage medium, wherein, the readable storage medium includes a computer program or instructions, and when the computer program or instructions run on a computer, the computer is caused to perform the operation steps of the method according to any one of claims 7-10 above.

Images