Data processing method and device, computer device, storage medium and program product

Abstract

The application relates to a data processing method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: receiving a clear volume operation instruction sent by a management node, creating a takeover volume structure of a to-be-cleared logical volume according to the clear volume operation instruction, taking over index information of the to-be-cleared logical volume through the takeover volume structure, and then outputting response information. The above method can take over the index information of the to-be-cleared logical volume by creating the takeover volume structure of the to-be-cleared logical volume. As long as the index information is taken over by the takeover volume structure, it indicates that the target service node has completed the data clearing operation on the to-be-cleared logical volume. At this time, the response information can be directly outputted. The process does not require the target service node to directly perform the data clearing operation on the to-be-cleared logical volume, so that the user can receive the response information, the user has no perception of the data clearing operation, the speed of receiving the response information by the user is improved, and the time of waiting for the response information by the user is saved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, computer equipment, storage medium, and program product. Background Technology

[0002] With the development of data storage technology, distributed storage systems are becoming increasingly popular.

[0003] In distributed storage systems, there are often two application scenarios: remote replication of logical volumes and copying of logical volumes. However, in these two scenarios, when users select a target logical volume, they may choose an existing non-empty logical volume, and there may be insufficient remaining storage space on the target logical volume. In this case, it is necessary to perform a truncating (i.e., truncate) operation on the target logical volume to clear the original data in the target logical volume and use the target logical volume as a new logical volume.

[0004] However, using related technologies to perform the volume clearing operation results in a slower response time for users. Summary of the Invention

[0005] Therefore, it is necessary to provide a data processing method, apparatus, computer equipment, storage medium, and program product to address the aforementioned technical problems, which can accelerate the user's response speed to receiving the volume clearing operation.

[0006] Firstly, this application provides a data processing method, which includes:

[0007] The target service node receives the volume clearing operation command sent by the management node; the volume clearing operation command is used to instruct the data clearing operation to be performed on the logical volume to be cleared, and the target service node can be any service node in the distributed storage system;

[0008] The target service node creates a takeover volume structure for the logical volume to be cleared based on the clear volume operation command;

[0009] The target service node takes over the index information of the logical volume to be cleared by taking over the volume structure and outputs response information indicating that the data clearing of the logical volume to be cleared has been completed.

[0010] In the technical solution of this application embodiment, a volume clearing operation instruction sent by a management node can be received. A takeover volume structure for the logical volume to be cleared is created according to the volume clearing operation instruction, and the index information of the logical volume to be cleared is taken over through the takeover volume structure. Then, a response information is output. This method can take over the index information of the logical volume to be cleared by creating a takeover volume structure. Once the takeover volume structure takes over the index information, it indicates that the target service node has completed the data clearing operation on the logical volume to be cleared. At this time, a response information can be directly output. This process does not require the target service node to directly perform the data clearing operation on the logical volume to be cleared, yet the user can receive the response information. This method ensures that users are unaware of the data clearing operation, improving the speed at which users receive response information and saving them time waiting for it. Furthermore, this method is not limited by the amount of data in the logical volume to be cleared; the target service node can quickly respond to the volume clearing operation and output response information. Additionally, after taking over the index information of the logical volume to be cleared through the takeover volume structure, the backend server can asynchronously perform the volume clearing operation on behalf of the target service node and synchronously send response information to the user. This allows the user to quickly receive the response information, promptly notifying them that an empty volume is available on the target service node, further accelerating the user's business data storage speed.

[0011] In one embodiment, the volume clearing operation instruction includes identification information of the logical volume to be cleared; the target service node, based on the volume clearing operation instruction, creates a takeover volume structure for the logical volume to be cleared in the distributed storage system, including:

[0012] The target service node obtains the index tree of the logical volume to be cleared through the identification information of the logical volume to be cleared;

[0013] The target service node obtains the root index of the logical volume to be cleared from the index tree of the logical volume to be cleared;

[0014] The target service node creates a takeover volume structure for the logical volume to be cleared based on the root index of the logical volume to be cleared.

[0015] In the technical solution of this application embodiment, the index tree of the logical volume to be cleared can be obtained through the identification information of the logical volume to be cleared, the root index of the logical volume to be cleared can be obtained from the index tree of the logical volume to be cleared, and a takeover volume structure of the logical volume to be cleared can be created based on the root index of the logical volume to be cleared. This method can create a takeover volume structure of the logical volume to be cleared based on the root index of the logical volume to be cleared, in order to prepare for the backend server to replace the target service node to perform the clearing operation on the logical volume to be cleared. At the same time, this method can create a takeover volume structure of the logical volume to be cleared by only obtaining the root index in the logical volume to be cleared, and the creation process is relatively simple.

[0016] In one embodiment, the target service node obtains the index tree of the logical volume to be cleared through the identification information of the logical volume to be cleared, including:

[0017] The target service node searches for the identification information corresponding to the identification information of the logical volume to be cleared in the mapping relationship of the storage pool; the storage pool is the storage space on any node in the distributed storage block system, and the mapping relationship includes the correspondence between the index tree of each logical volume created on the target service node and the identification information of each logical volume;

[0018] The target service node will identify the index tree of the logical volume corresponding to the found identification information as the index tree of the logical volume to be cleared.

[0019] In the technical solution of this application embodiment, the identification information corresponding to the identification information of the logical volume to be cleared can be found in the mapping relationship of the storage pool, and the index tree of the logical volume corresponding to the found identification information is determined as the index tree of the logical volume to be cleared. This method does not require operation on the logical volume to be cleared, and the index tree of the logical volume to be cleared can be obtained by directly performing a search operation on the storage pool. This not only reduces the task processing load of the target service node in the distributed storage system, but also improves the speed of obtaining the index tree of the logical volume to be cleared.

[0020] In one embodiment, the target service node takes over the index information of the logical volume to be cleared through the takeover volume structure, including:

[0021] The target service node combines all index information in the index tree of the logical volume to be cleared, except for the root index, with the takeover volume structure, and deletes all index information in the index tree except for the root index, in order to complete the data clearing operation performed by the target service node on the logical volume to be cleared.

[0022] In the technical solution of this application embodiment, all index information except the root index in the index tree of the logical volume to be cleared can be combined with the takeover volume structure, and all index information except the root index in the index tree can be deleted to complete the data clearing operation of the logical volume to be cleared by the target service node. This method does not require the target service node in the distributed storage system to directly process the logical volume to be cleared to complete the clearing operation. After all index information except the root index in the index tree of the logical volume to be cleared is combined with the takeover volume structure, it is only necessary to delete all index information except the root index in the index tree of the logical volume to be cleared to determine that the clearing operation of the logical volume to be cleared is completed. This process greatly saves the time of the target service node to perform the data clearing operation and further improves the speed of the clearing operation.

[0023] In one embodiment, the target service node takes over the index information of the logical volume to be cleared through the takeover volume structure, including:

[0024] The target service node truncates the index tree according to the truncation position, obtains all index information except the root index in the index tree, and combines all index information except the root index in the index tree of the logical volume to be cleared with the takeover volume structure to complete the data clearing operation performed by the target service node on the logical volume to be cleared; the truncation position is the position after the root index in the index tree.

[0025] In the technical solution of this application embodiment, all index information except the root index in the index tree can be directly removed from the logical volume to be cleared by truncating the index tree, and all the removed index information is combined with the takeover volume structure, so that the backend server can directly perform the data clearing operation of the logical volume to be cleared by replacing the target service node through the combination result. This process greatly saves the time of the target service node to perform the data clearing operation and further improves the speed of the clearing operation. At the same time, this method can clear the index information on the logical volume to be cleared by truncating the index tree, so that the target service node can realize the clearing operation. This process saves the target service node from the process of deleting all index information except the root index of the index tree in the logical volume to be cleared, thereby speeding up the target service node to complete the clearing operation of the logical volume to be cleared and improving the speed of the target service node to realize the clearing operation.

[0026] In one embodiment, the method further includes:

[0027] After the target service node combines all index information except the root index in the index tree with the takeover volume structure, it obtains the index tree to be reclaimed for the logical volume to be cleared.

[0028] The target service node sends a clear message to the backend server; the clear message is used to instruct the backend server to perform data clearing operations on the logical volume to be cleared through the index tree to be reclaimed.

[0029] In the technical solution of this application embodiment, after combining all index information except the root index in the index tree with the takeover volume structure, a reclaimable index tree of the logical volume to be cleared can be obtained, that is, the index tree of the logical volume to be cleared. Then, a clear message is sent to the backend server so that the backend server can replace the target service node to perform data clearing operation on the logical volume to be cleared through the reclaimable index tree, saving the target service node the time to perform data clearing operation. At the same time, this method can replace the target service node to realize the data clearing operation of the logical volume to be cleared, further shortening the time for the user to perceive the clearing volume operation on the target service node side.

[0030] Secondly, this application provides a data processing apparatus, which includes:

[0031] The instruction receiving module is used to receive the volume clearing operation instruction sent by the management node; the volume clearing operation instruction is used to instruct the data clearing operation to be performed on the logical volume to be cleared.

[0032] The creation module is used to create the takeover volume structure of the logical volume to be cleared based on the clear volume operation command;

[0033] The takeover module is used to perform data clearing operations on the logical volume to be cleared through the takeover volume structure and output response information; the response information indicates that the data clearing of the logical volume to be cleared has been completed.

[0034] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the embodiments of the first aspect described above.

[0035] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0036] Fifthly, this application also provides a computer program product comprising a computer program that, when executed by a processor, implements the steps of the method in any of the embodiments of the first aspect described above.

[0037] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0038] Figure 1 This is a diagram illustrating the application environment of a data processing method in one embodiment.

[0039] Figure 2 This is a flowchart illustrating a data processing method in one embodiment;

[0040] Figure 3 This is a flowchart illustrating the data processing method in another embodiment;

[0041] Figure 4 This is a structural diagram of the index tree in one embodiment;

[0042] Figure 5 This is a flowchart illustrating the data processing method in another embodiment;

[0043] Figure 6 for Figure 4A schematic diagram illustrating the data between root nodes in the index tree and the corresponding takeover tree structure in the embodiment;

[0044] Figure 7 This is a flowchart illustrating the data processing method in another embodiment;

[0045] Figure 8 This is a structural block diagram of a data processing device in one embodiment;

[0046] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] In data storage technology, distributed storage systems are becoming increasingly widely used. For distributed storage systems, remote logical volume replication and logical volume copying are two common application scenarios, both of which may involve clearing the logical volume, i.e., clearing the volume operation.

[0049] In related technologies, the main approach is to directly perform a volume clearing operation on the logical volume to erase the existing data. However, this method results in a slow response time for the user after the volume clearing operation is completed. Therefore, this application provides a data processing method that can accelerate the user's reception of the volume clearing operation response.

[0050] The above data processing method can be applied to Figure 1 The distributed storage system shown can be a distributed cluster system capable of providing block storage services. This distributed storage system includes multiple servers (i.e., multiple nodes), and the servers can communicate with each other via methods such as Wi-Fi, mobile networks, or Bluetooth. Each server can be, but is not limited to, an independent server or a server cluster composed of multiple servers; this embodiment does not limit the specific form of the servers. The distributed storage system includes multiple nodes, which can be one management node and multiple service nodes. Figure 1 This example illustrates a distributed storage system with three service nodes. The following embodiments will describe the specific process of the data processing method, using the target service node in the distributed storage system as the executing entity. The target service node can be any one of the service nodes in the distributed storage system.

[0051] like Figure 2 The diagram shown is a flowchart illustrating a data processing method provided in an embodiment of this application. This method may include the following steps:

[0052] S100: The target service node receives the volume clearing operation command sent by the management node. The volume clearing operation command instructs the data clearing operation to be performed on the logical volume to be cleared. The target service node can be any service node in the distributed storage system.

[0053] In this embodiment, the distributed storage system can be a distributed block storage system, a distributed file storage system, or a distributed object storage system. The distributed storage system includes multiple nodes. The target service node can be any one of the multiple nodes in the distributed storage system. It should be noted that the management node can also be understood as the master node in the distributed storage system, and the target service node can also be understood as any one of the slave nodes in the distributed storage system.

[0054] In practical applications, the following situations necessitate the target service node to perform a volume clearing operation: First, if the management node requires the target service node to perform remote logical volume replication or copying, but all logical volumes on the target service node are full, then the target service node must perform a volume clearing operation before the remote replication or copying can be completed. Second, if the management node requires the target service node to clear historical data from its logical volumes, this also requires the target service node to perform a volume clearing operation. In both cases, the management node can send a volume clearing operation command to the target service node, which then receives the command. The volume clearing operation essentially involves demapping all data in the logical volume to be cleared against the root index of the index tree in the logical volume, thereby releasing or reclaiming the storage space of the logical volume to be cleared in the storage pool; this storage pool can be the storage space on any one or more nodes in the distributed storage system.

[0055] Optionally, the management node can send the volume clearing operation command to the target service node automatically based on actual needs, or the user can manually trigger the volume clearing operation command based on the management node's actual needs and then send it to the target service node. Optionally, the user can manually trigger the volume clearing operation command through voice, gestures, buttons, etc.

[0056] In this embodiment, multiple logical volumes can be configured on the target service node, and any one of these logical volumes can be referred to as the logical volume to be cleared. The logical volumes on the target service node can be allocated using storage space on the target service node. When the logical volumes to be cleared are different logical volumes on the target service node, the corresponding clear volume operation instructions are different.

[0057] S200, The target service node creates a takeover volume structure for the logical volume to be cleared based on the clear volume operation command.

[0058] Upon receiving a volume clearing operation command, the target service node retrieves all information from the logical volume to be cleared corresponding to the command, and then creates a takeover volume structure for the logical volume to be cleared based on this information. Optionally, all information from the logical volume to be cleared can be stored in the form of an index tree, which includes a root index and index information for the logical volume to be cleared; the index information includes multi-level indexes and all data within the logical volume to be cleared.

[0059] Optionally, the takeover volume structure of the logical volume to be cleared may include some information from the logical volume to be cleared. It should be noted that the aforementioned takeover volume structure of the logical volume to be cleared can be obtained by repositioning some information from the logical volume to be cleared, or it can be obtained directly from some information from the logical volume to be cleared.

[0060] S300: The target service node takes over the index information of the logical volume to be cleared through the takeover volume structure and outputs a response message. The response message indicates that the data clearing operation of the logical volume to be cleared has been completed.

[0061] Specifically, the target service node can determine the remaining information in the logical volume to be cleared, excluding the information in the taken-over volume structure, based on partial information in the taken-over volume structure. Then, it removes the remaining information from the logical volume to be cleared and combines it with the taken-over volume structure. At this point, the target service node determines that the data clearing operation of the logical volume to be cleared is complete. Optionally, the above combination can be a simple combination without any substantial processing.

[0062] In practical applications, after combining the remaining information with the takeover volume structure, the backend server can replace the target service node to perform the data clearing operation of the logical volume to be cleared. In this embodiment, as long as the takeover volume structure created by the target service node has taken over all the index information of the logical volume to be cleared, it indicates that the target service node has completed the data clearing operation of the logical volume to be cleared.

[0063] Furthermore, after confirming the completion of the data clearing operation on the logical volume to be cleared, a response message can be sent to the user terminal to remind the user that the data clearing of the logical volume to be cleared has been completed and subsequent data storage processing can continue. Optionally, the response message can be a string of characters composed of numbers, symbols, and / or English letters, or it can be a visual icon, etc. In this embodiment, as long as the user terminal receives the response message, it indicates that the target service node has completed the volume clearing operation. Optionally, the response message can be output on the user terminal in the form of text or a view.

[0064] In practical applications, after the clearing operation of the logical volume to be cleared is completed, the logical volume to be cleared can be used as an empty volume, or it can be understood as being used as a new volume.

[0065] The data processing method provided in this application embodiment can receive a volume clearing operation command sent by a management node, create a takeover volume structure for the logical volume to be cleared according to the volume clearing operation command, take over the index information of the logical volume to be cleared through the takeover volume structure, and then output response information. This method can take over the index information of the logical volume to be cleared by creating a takeover volume structure. Once the takeover volume structure takes over the index information, it indicates that the target service node has completed the data clearing operation on the logical volume to be cleared, and at this time, response information can be directly output. This process does not require the target service node to directly perform the data clearing operation on the logical volume to be cleared, yet the user can still receive the response information. This method allows users to be unaware of the data clearing operation, improving the speed at which users receive response information and saving them time waiting for it. Furthermore, this method is not limited by the amount of data in the logical volume to be cleared; the target service node can quickly respond to the volume clearing operation and output response information. In addition, after taking over the index information of the logical volume to be cleared through the takeover volume structure, the backend server can asynchronously perform the volume clearing operation on behalf of the target service node and synchronously send response information to the user. This ensures that the user receives the response information quickly, promptly reminding them that an empty volume is available on the target service node, further accelerating the user's business data storage speed.

[0066] In some scenarios, to improve the retrieval efficiency of data in a logical volume to be cleared, all information in the logical volume is stored in an index tree structure. The following describes how, when all information in a logical volume to be cleared is stored in an index tree structure, a takeover volume structure is created in the distributed storage system based on the clear volume operation instructions. In one embodiment, as... Figure 3 As shown, the above-mentioned volume clearing operation instruction includes the identification information of the logical volume to be cleared; the steps in S200 above may include:

[0067] S210. The target service node obtains the index tree of the logical volume to be cleared through the identification information of the logical volume to be cleared.

[0068] In practical applications, a target service node may have multiple logical volumes. It may be necessary to clear only one or more of these logical volumes on the target service node. Therefore, the clear volume operation command can carry identification information of the logical volume to be cleared, allowing the target service node to identify the volume based on this information. Optionally, this identification information can be the name, attribute information, number, etc., of the logical volume to be cleared, and can be represented by characters composed of numbers, symbols, and / or English letters.

[0069] Specifically, the target service node can obtain the index tree stored in the logical volume to be cleared through the identification information of the logical volume to be cleared. This can also be understood as obtaining all the information stored in the logical volume to be cleared through the identification information of the logical volume to be cleared. Optionally, the index tree in the logical volume to be cleared is pre-stored, and its specific size is not limited in this embodiment. Optionally, the logical volume to be cleared can store only one index tree.

[0070] S220. The target service node obtains the root index of the logical volume to be cleared from the index tree of the logical volume to be cleared.

[0071] The index tree in the logical volume to be cleared may include multi-level indexes of the index tree and all data in the logical volume to be cleared. The multi-level indexes of the index tree may include at least two levels of indexes, and these at least two levels include the root index of the index tree. Optionally, each level of index in the index tree may include at least one node, and each node may include a unique node identifier for the current node, a parent node identifier, and / or a sorting identifier, etc. Optionally, the unique node identifier may represent the unique identifier of the current node in the index tree; the parent node identifier may represent the unique identifier of the parent node of the current node.

[0072] In practical applications, the first-level index of the index tree is the root index, which is also the root node of the index tree, and can be understood as the parent node of the index tree. There is only one root node. In the second-level index, the parent node of each node can be the root node, and the child nodes of the root node can be any node in the second-level index. The last level of the index tree can include all data in the logical volume to be cleared. The last level includes multiple nodes, and each node can store one or more data items. The parent node, which is related to each node in this level, can store the attribute information of the data stored in each node of this level. Optionally, when multiple data items are stored in each node of the last level of the index tree, they can be stored according to a pre-set sorting identifier corresponding to each data item.

[0073] In one embodiment, the target service node can obtain the root index of the logical volume to be cleared from the index tree of the logical volume to be cleared based on the attribute information of the root node in the index tree. Optionally, the attribute information of the root node in the index tree can be information such as the total number of root nodes in the index tree and / or the position (i.e., the level) of the root node in the index tree.

[0074] In another embodiment, the target service node can obtain a pre-trained algorithm model and then input the index tree into the algorithm model, which outputs the root index of the logical volume to be cleared in the index tree.

[0075] For example, Figure 4 This example illustrates the structure of an index tree using a 5-level index. The first level of the index tree is the root index. Figure 4 The root index is the leftmost first level (i.e., 0, 1, 2, ..., 61, ...). The second level index of the index tree consists of two corresponding columns at the third level. The third level index consists of two corresponding columns at the second level. The fourth level index consists of three corresponding columns at the first level. The bottom level index of the index tree is the data stored in the logical volume to be cleared (i.e., ... Figure 4 (Data layer in the middle). Among them, Figure 4 Level 1, Level 2, and Level 3 can correspond to different attribute information for the data in the logical volume to be cleared.

[0076] S230. The target service node creates a takeover volume structure for the logical volume to be cleared based on the root index of the logical volume to be cleared.

[0077] Specifically, the target service node can create a takeover volume structure for the logical volume to be cleared using a pre-configured takeover volume structure creation tool, based on the root index of the logical volume to be cleared.

[0078] Additionally, after obtaining the root index of the logical volume to be cleared, the target service node can search for a root index that matches the root index in a pre-created structure library, and then directly obtain the takeover volume structure of the logical volume to be cleared based on the found index. Optionally, the pre-created structure library may include the root index of the logical volume, the takeover volume structure, and the correspondence between the two.

[0079] It should be noted that the information in the takeover volume structure of the logical volume to be cleared can be the root index of the logical volume to be cleared. However, the characteristic of the created takeover volume structure is that the root index in the takeover volume structure can be connected to multiple levels of indexes. That is, the root index in the takeover volume structure can take over multiple levels of indexes. In this embodiment, by taking over this characteristic of the takeover volume structure, multiple levels of indexes after the root index of the index tree in the logical volume to be cleared can be taken over after the root index in the takeover volume structure, so that the root index in the takeover volume structure and the taken over multiple levels of indexes can generate a complete index tree. Furthermore, the node where the takeover volume structure is located replaces the target service node to clear all the data in the generated index tree.

[0080] The technical solution of this application embodiment can obtain the index tree of the logical volume to be cleared through the identification information of the logical volume to be cleared, obtain the root index of the logical volume to be cleared from the index tree of the logical volume to be cleared, and create a takeover volume structure of the logical volume to be cleared based on the root index of the logical volume to be cleared. This method can create a takeover volume structure of the logical volume to be cleared based on the root index of the logical volume to be cleared, in order to prepare for the backend server to replace the target service node to perform the clearing operation on the logical volume to be cleared. At the same time, this method can create the takeover volume structure of the logical volume to be cleared by only obtaining the root index in the logical volume to be cleared, and the creation process is relatively simple.

[0081] The following describes the process by which the target service node obtains the index tree of the logical volume to be cleared using the identification information of the logical volume to be cleared. In one embodiment, as... Figure 5 As shown, the steps in S210 above can be implemented in the following ways:

[0082] S211. The target service node searches for the identification information corresponding to the identification information of the logical volume to be cleared in the mapping relationship of the storage pool. Here, the storage pool is the storage space on any service node in the distributed storage block system; the mapping relationship includes the correspondence between the index tree of each logical volume created on the target service node and the identification information of each logical volume.

[0083] Specifically, the target service node can search for the identification information that matches the identification information of the logical volume to be cleared in the mapping relationship of the storage pool. In this embodiment, the storage pool is the storage space on any service node in the distributed storage block system. It should be noted that the storage pool can store mapping relationships, which may include the correspondence between the index trees of each logical volume created on the target service node and the identification information of each logical volume. The identification information and index trees of different logical volumes are different.

[0084] S212. The target service node determines the index tree of the logical volume corresponding to the found identification information as the index tree of the logical volume to be cleared.

[0085] The target service node can determine the index tree of the logical volume corresponding to the identification information found in the mapping relationship as the index tree of the logical volume to be cleared.

[0086] The technical solution of this application embodiment can find the identification information corresponding to the identification information of the logical volume to be cleared in the mapping relationship of the storage pool, and determine the index tree of the logical volume corresponding to the found identification information as the index tree of the logical volume to be cleared. This method does not require operation on the logical volume to be cleared, and can directly perform a search operation on the storage pool to obtain the index tree of the logical volume to be cleared. This not only reduces the task processing load of the target service node in the distributed storage system, but also improves the speed of obtaining the index tree of the logical volume to be cleared.

[0087] The following describes how the target service node takes over the index information of the logical volume to be cleared through the takeover volume structure. In one embodiment, the step in S300 above, in which the target service node takes over the index information of the logical volume to be cleared through the takeover volume structure, includes: the target service node combining all index information in the index tree of the logical volume to be cleared except for the root index with the takeover volume structure, and deleting all index information in the index tree except for the root index, so as to complete the data clearing operation performed by the target service node on the logical volume to be cleared.

[0088] Specifically, the target service node can obtain all index information in the index tree of the logical volume to be cleared except for the root index, then combine all index information in the index tree of the logical volume to be cleared except for the root index with the information in the takeover volume structure, and delete all index information in the index tree of the logical volume to be cleared except for the root index. After the deletion is completed, it indicates that the target service node has completed the data clearing operation of the logical volume to be cleared.

[0089] For example, one way to obtain all index information other than the root index in the index tree of the logical volume to be cleared is to search for the index tree of the logical volume to be cleared in the mapping relationship of the storage pool based on the identification information of the logical volume to be cleared, and then obtain all index information other than the root index in the index tree of the logical volume to be cleared.

[0090] For example, another way to obtain all index information other than the root index in the index tree of the logical volume to be cleared is to obtain the index tree of the logical volume to be cleared from the storage device based on the identification information of the logical volume to be cleared, and then obtain all index information other than the root index from that index tree. Optionally, the storage device can be a hard drive, the cloud, a local disk, or other similar location; the storage device stores the index trees of each logical volume on the target service node.

[0091] Optionally, combining all index information in the index tree of the logical volume to be cleared, except for the root index, with the information in the takeover volume structure can be understood as connecting all index information in the index tree of the logical volume to be cleared, except for the root index, with the information in the takeover volume structure. It can also be understood as taking over all index information in the index tree of the logical volume to be cleared, except for the root index, after the information in the takeover volume structure. The index tree of the logical volume to be cleared is obtained after combining all index information in the index tree of the logical volume to be cleared, except for the root index, with the information in the takeover volume structure.

[0092] The technical solution of this application embodiment can combine all index information except the root index in the index tree of the logical volume to be cleared with the takeover volume structure, and delete all index information except the root index in the index tree to complete the data clearing operation of the logical volume to be cleared by the target service node. This method does not require the target service node in the distributed storage system to directly process the logical volume to be cleared to complete the clearing operation. After combining all index information except the root index in the index tree of the logical volume to be cleared with the takeover volume structure, it is only necessary to delete all index information except the root index in the index tree of the logical volume to be cleared to determine that the clearing operation of the logical volume to be cleared is completed. This process greatly saves the time of the target service node to perform the data clearing operation and further improves the speed of the clearing operation.

[0093] In another embodiment, the step in S300 above, where the target service node performs a data clearing operation on the logical volume to be cleared by taking over the volume structure, includes: the target service node truncating the index tree according to the truncation position to obtain all index information except the root index in the index tree, and combining all index information except the root index in the index tree of the logical volume to be cleared with the volume takeover structure to complete the data clearing operation performed by the target service node on the logical volume to be cleared. The truncation position is the position after the root index in the index tree.

[0094] Specifically, the target service node can truncate the index tree according to the truncation position, obtaining all index information except the root index in the index tree, and combine all index information except the root index in the index tree of the logical volume to be cleared with the takeover volume structure. Furthermore, the target service node can also clear the root index in the index tree of the logical volume to be cleared and mark the logical volume to be cleared to record that the logical volume to be cleared has been cleared.

[0095] In this embodiment of the application, there is a correspondence between adjacent indexes in the index tree. The root index in the index tree and the second-level index in the index tree are truncated, indicating that the information in the index tree has been cleared, that is, all the data in the logical volume to be cleared has been cleared.

[0096] In one embodiment, the method of truncating the index tree according to the truncation position to obtain all index information except the root index can be to construct a truncation network model, and then input the index tree of the logical volume to be cleared and the truncation position into the truncation network model. The truncation network model splits the index tree to obtain the root index and all index information except the root index. Optionally, the truncation network model can be composed of a fully convolutional neural network model, a support vector machine, and / or a dilated convolutional network model.

[0097] In this embodiment, to improve the speed and efficiency of creating the takeover volume structure for the logical volume to be cleared, it is created only using the root index of the index tree in the logical volume to be cleared. Based on the root index required for creating the takeover volume structure, the information in the index tree of the logical volume to be cleared is essentially divided into two parts: one part includes the root index of the index tree, and the other part includes the other layers of indexes and data layers after the root index. Based on this, a truncation position can be set for the index tree, and then the index tree is truncated based on this truncation position to take over all index information except the root index in the index tree to the root index in the takeover volume structure, thus obtaining the complete index tree of the logical volume to be cleared. Correspondingly, the aforementioned truncation position can be the position after the root index in the index tree.

[0098] For example, in Figure 4 Based on the index tree, such as Figure 6 The diagram illustrates how, after taking over the root index in the volume structure, all index information following the root index of the index tree in the logical volume to be cleared is taken over. Figure 6 The "×" sign between the root index and the second-level index in the index tree indicates a truncation.

[0099] The technical solution of this application embodiment can directly remove all index information except the root index from the logical volume to be cleared by truncating the index tree, and combine all the removed index information with the takeover volume structure, so that the backend server can directly perform the data clearing operation of the logical volume to be cleared by replacing the target service node through the combination result. This process greatly saves the time of the target service node to perform the data clearing operation and further improves the speed of the clearing operation. At the same time, this method can clear the index information on the logical volume to be cleared by truncating the index tree, so that the target service node can realize the clearing operation. This process saves the target service node from the process of deleting all index information except the root index of the index tree in the logical volume to be cleared, thereby speeding up the target service node to complete the clearing operation of the logical volume to be cleared and improving the speed of the target service node to realize the clearing operation.

[0100] The following describes how to have a backend server perform the data cleanup operation on behalf of the target service node for the logical volume to be cleared. In one embodiment, as follows: Figure 7As shown, the above data processing method may further include:

[0101] S400: The target service node combines all index information except the root index in the index tree with the takeover volume structure to obtain the index tree to be reclaimed for the logical volume to be cleared.

[0102] After the target service node combines all index information except the root index in the index tree with the takeover volume structure, it obtains the index tree to be reclaimed for the logical volume to be cleared.

[0103] In practical applications, the target service node combines all index information except the root index in the index tree with the takeover volume structure to obtain the index tree to be reclaimed for the logical volume to be cleared, which indicates that the target service node has completed the data clearing operation of the logical volume to be cleared.

[0104] S500, the target service node sends a clear message to the backend server; the clear message is used to instruct the backend server to perform data clearing operations on the logical volume to be cleared through the index tree to be reclaimed.

[0105] After obtaining the index tree of the logical volume to be emptied, a clear message can be obtained and sent to the backend server. Upon receiving the clear message, the backend server can perform the data clearing operation on behalf of the target service node, ultimately enabling the storage pool to reclaim or release the storage space of the logical volume to be emptied. Optionally, the clear message can be a combination of numbers, letters, and / or English characters, and this clear message is different from the response information mentioned above.

[0106] In practical applications, the backend server can control the truncation of the mapping relationship between the information of each adjacent layer in the index tree to be reclaimed in a preset order, so as to complete the data clearing operation of the logical volume to be cleared.

[0107] In this embodiment, the above-mentioned volume clearing operation actually includes the process of deleting all data in the logical volume to be cleared; the process of deleting all data in the logical volume to be cleared can be understood as the process of demapping the data in the logical volume to be cleared. Optionally, data demapping can be understood as the process of truncating the mapping relationship between adjacent layers of information in the index tree to be reclaimed.

[0108] The aforementioned clear message may carry a preset order to instruct the backend server on the truncation order in the index tree to be reclaimed. This preset order can be the order in which the mapping relationships between adjacent layers of information in the index tree to be reclaimed are truncated. Optionally, the preset order can be from the first layer to the last layer in the index tree to be reclaimed, or from the last layer to the first layer, or a custom order set according to specific rules. Optionally, the information of adjacent layers in the index tree to be reclaimed may include the index information of two adjacent layers in the index tree to be reclaimed, as well as the index information of adjacent layers between data layers.

[0109] It should be noted that after receiving the clear message, the backend server can truncate the mapping relationship between the information of each adjacent layer in the index tree to be reclaimed in a preset order to complete the data clearing operation of the logical volume to be cleared.

[0110] It should be noted that the data clearing operation performed by the backend server is imperceptible to the user in terms of time, so that users on the management node side can quickly receive the corresponding response information for the completion of the data clearing operation.

[0111] The technical solution of this application embodiment can combine all index information except the root index in the index tree with the takeover volume structure to obtain the index tree to be reclaimed for the logical volume to be cleared, that is, the index tree of the logical volume to be cleared. Then, a clear message is sent to the backend server so that the backend server can replace the target service node to perform data clearing operation on the logical volume to be cleared through the index tree to be reclaimed, saving the target service node the time to perform data clearing operation. At the same time, this method can replace the target service node to realize the data clearing operation of the logical volume to be cleared, further shortening the time for the user to perceive the volume clearing operation on the target service node side.

[0112] In one embodiment, this application also provides a data processing method, which includes the following steps:

[0113] (1) The target service node receives the volume clearing operation instruction sent by the management node; the volume clearing operation instruction is used to instruct the data clearing operation to be performed on the logical volume to be cleared. The volume clearing operation instruction includes the identification information of the logical volume to be cleared. The target service node is any service node in the distributed storage system.

[0114] (2) The target service node searches for the identification information corresponding to the identification information of the logical volume to be cleared in the mapping relationship of the storage pool; the storage pool is the storage space on any service node in the distributed storage block system, and the mapping relationship includes the correspondence between the index tree of each logical volume created on the target service node and the identification information of each logical volume.

[0115] (3) The target service node determines the index tree of the logical volume corresponding to the found identification information as the index tree of the logical volume to be cleared.

[0116] (4) The target service node obtains the root index of the logical volume to be cleared from the index tree of the logical volume to be cleared.

[0117] (5) The target service node creates the takeover volume structure of the logical volume to be cleared based on the root index of the logical volume to be cleared.

[0118] (6) The target service node takes over the index information of the logical volume to be cleared through the takeover volume structure and outputs response information, which indicates that the logical volume to be cleared has completed data clearing.

[0119] The step of the target service node taking over the index information of the logical volume to be cleared through the takeover volume structure can be implemented in two ways;

[0120] The first method includes:

[0121] (61) The target service node combines all index information except the root index in the index tree of the logical volume to be cleared with the takeover volume structure and deletes all index information except the root index in the index tree to complete the data clearing operation of the logical volume to be cleared by the target service node.

[0122] The second method includes:

[0123] (62) The target service node truncates the index tree according to the truncation position, obtains all index information except the root index in the index tree, and combines all index information except the root index in the index tree of the logical volume to be cleared with the takeover volume structure to complete the data clearing operation of the logical volume to be cleared by the target service node; the truncation position is the position after the root index in the index tree.

[0124] (7) The target service node combines the index information other than the root index in the index tree with the takeover volume structure to obtain the index tree to be reclaimed for the logical volume to be cleared.

[0125] (8) The target service node sends a clear message to the backend server; the clear message is used to instruct the backend server to perform data clearing operation on the logical volume to be cleared through the index tree to be reclaimed.

[0126] The specific execution process in (1) to (8) above can be found in the description of the above embodiments. The implementation principle and technical effect are similar, and will not be repeated here.

[0127] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0128] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0129] In one embodiment, such as Figure 8 As shown, a data processing apparatus is provided, comprising: an instruction receiving module 11, a creation module 12, and a takeover module 13, wherein:

[0130] The instruction receiving module 11 is used to receive the clear volume operation instruction sent by the management node; the clear volume operation instruction is used to instruct the data clearing operation to be performed on the logical volume to be cleared.

[0131] Create module 12, which is used to create the takeover volume structure of the logical volume to be cleared according to the clear volume operation command;

[0132] The takeover module 13 is used to take over the index information of the logical volume to be cleared through the takeover volume structure and output response information; the response information indicates that the data clearing operation of the logical volume to be cleared has been completed.

[0133] The data processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0134] In one embodiment, the volume clearing operation instruction includes identification information of the logical volume to be cleared; the creation module includes: an index tree acquisition unit, a root index acquisition unit, and a creation unit, wherein:

[0135] The index tree acquisition unit is used to acquire the index tree of the logical volume to be cleared by using the identification information of the logical volume to be cleared;

[0136] The root index retrieval unit is used to retrieve the root index of the logical volume to be emptied from the index tree of the logical volume to be emptied.

[0137] Create a unit to create a takeover volume structure for the logical volume to be cleared based on the root index of the logical volume to be cleared.

[0138] The data processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0139] In one embodiment, the index tree acquisition unit includes: a search sub-unit and an index tree determination unit, wherein:

[0140] The lookup sub-unit is used to find the identification information corresponding to the identification information of the logical volume to be cleared in the mapping relationship of the storage pool; the storage pool is the storage space on any service node in the distributed storage block system, and the mapping relationship includes the correspondence between the index tree of each logical volume created on the target service node and the identification information of each logical volume.

[0141] The index tree determination unit is used to determine the index tree of the logical volume corresponding to the found identification information as the index tree of the logical volume to be cleared.

[0142] The data processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0143] In one embodiment, the takeover module 13 includes: a first processing unit, wherein:

[0144] The first processing unit is used to combine all index information in the index tree of the logical volume to be cleared except for the root index with the takeover volume structure, and delete all index information in the index tree except for the root index, so as to complete the data clearing operation of the logical volume to be cleared by the target service node.

[0145] The data processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0146] In one embodiment, the takeover module 13 includes: a second processing unit, wherein:

[0147] The second processing unit is used to truncate the index tree according to the truncation position, obtain all index information in the index tree except for the root index, and combine all index information in the index tree of the logical volume to be cleared except for the root index with the takeover volume structure to complete the data clearing operation of the logical volume to be cleared by the target service node; the truncation position is the position after the root index in the index tree.

[0148] The data processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0149] In one embodiment, the takeover module 13 further includes: an index tree acquisition unit and a message sending unit, wherein:

[0150] The index tree acquisition unit is used to combine all index information except the root index in the index tree with the takeover volume structure to obtain the index tree to be reclaimed for the logical volume to be cleared.

[0151] The message sending unit is used to send a clear message to the backend server; the clear message is used to instruct the backend server to perform data clearing operations on the logical volume to be cleared through the index tree to be reclaimed.

[0152] The data processing device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0153] Specific limitations regarding the data processing device can be found in the limitations regarding the data processing method described above, and will not be repeated here. Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0154] In one embodiment, a computer device is provided, the internal structure of which can be shown in the following diagram. Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores an index tree of logical volumes to be cleared. The network interface communicates with external endpoints via a network connection. When executed by the processor, the computer program implements a data processing method.

[0155] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0157] Receive the volume clearing operation command sent by the management node; the volume clearing operation command is used to instruct the data clearing operation to be performed on the logical volume to be cleared;

[0158] Create a takeover volume structure for the logical volume to be cleared based on the clear volume operation command;

[0159] The system takes over the index information of the logical volume to be cleared by taking over the volume structure and outputs a response message; the response message indicates that the data clearing operation of the logical volume to be cleared has been completed.

[0160] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0161] Receive the volume clearing operation command sent by the management node; the volume clearing operation command is used to instruct the data clearing operation to be performed on the logical volume to be cleared;

[0162] Create a takeover volume structure for the logical volume to be cleared based on the clear volume operation command;

[0163] The system takes over the index information of the logical volume to be cleared by taking over the volume structure and outputs a response message; the response message indicates that the data clearing operation of the logical volume to be cleared has been completed.

[0164] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0165] Receive the volume clearing operation command sent by the management node; the volume clearing operation command is used to instruct the data clearing operation to be performed on the logical volume to be cleared;

[0166] Create a takeover volume structure for the logical volume to be cleared based on the clear volume operation command;

[0167] The system takes over the index information of the logical volume to be cleared by taking over the volume structure and outputs a response message; the response message indicates that the data clearing operation of the logical volume to be cleared has been completed.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0169] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: The target service node receives a volume clearing operation instruction sent by the management node; the volume clearing operation instruction is used to instruct the data clearing operation to be performed on the logical volume to be cleared, and the target service node is any service node in the distributed storage system; The target service node, according to the clear volume operation instruction, searches for the index tree corresponding to the identification information of the logical volume to be cleared through the mapping relationship of the storage pool, extracts the root index of the logical volume to be cleared from the index tree, and creates a takeover volume structure of the logical volume to be cleared based on the root index; wherein, the takeover volume structure is a logical structure independent of the logical volume to be cleared, and the takeover volume structure is uniquely bound to the root index; The target service node takes over the index information of the logical volume to be cleared through the takeover volume structure and outputs response information; the response information indicates that the logical volume to be cleared has completed the data clearing operation; after outputting the response information, the target service node uses the background server to asynchronously perform the data clearing operation on the logical volume to be cleared based on the takeover volume structure.

2. The method according to claim 1, characterized in that, The target service node, based on the clear volume operation instruction, searches the index tree corresponding to the identifier information of the logical volume to be cleared through the mapping relationship of the storage pool, including: The target service node searches for the identification information corresponding to the identification information of the logical volume to be cleared in the mapping relationship of the storage pool; the storage pool is the storage space on any service node in the distributed storage block system, and the mapping relationship includes the correspondence between the index tree of each logical volume created on the target service node and the identification information of each logical volume; The target service node determines the index tree of the logical volume corresponding to the found identification information as the index tree of the logical volume to be cleared.

3. The method according to claim 1 or 2, characterized in that, The target service node takes over the index information of the logical volume to be cleared through the takeover volume structure, including: The target service node combines all index information in the index tree of the logical volume to be cleared, except for the root index, with the takeover volume structure, and deletes all index information in the index tree except for the root index, so as to complete the data clearing operation of the logical volume to be cleared by the target service node.

4. The method according to claim 1 or 2, characterized in that, The target service node takes over the index information of the logical volume to be cleared through the takeover volume structure, including: The target service node truncates the index tree according to the truncation position, obtains all index information in the index tree except for the root index, and combines all index information in the index tree of the logical volume to be cleared except for the root index with the takeover volume structure to complete the data clearing operation performed by the target service node on the logical volume to be cleared; the truncation position is the position after the root index in the index tree.

5. The method according to claim 4, characterized in that, The method further includes: The target service node combines all index information except the root index in the index tree with the takeover volume structure to obtain the index tree to be reclaimed for the logical volume to be cleared. The target service node sends a clear message to the backend server; the clear message is used to instruct the backend server to perform a data clearing operation on the logical volume to be cleared through the index tree to be reclaimed.

6. The method according to claim 1, characterized in that, The asynchronous execution of the physical data reclamation operation on the logical volume to be cleared includes: According to a preset order, the mapping relationship between the index information of each layer in the takeover volume structure is released.

7. A data processing apparatus, characterized in that, The device includes: The instruction receiving module is used to receive the clear volume operation instruction sent by the management node; the clear volume operation instruction is used to instruct the data clearing operation to be performed on the logical volume to be cleared. A creation module is used to find the index tree corresponding to the identification information of the logical volume to be cleared through the mapping relationship of the storage pool according to the clear volume operation instruction, extract the root index of the logical volume to be cleared from the index tree, and create a takeover volume structure of the logical volume to be cleared according to the root index; wherein, the takeover volume structure is a logical structure independent of the logical volume to be cleared, and the takeover volume structure is uniquely bound to the root index; The takeover module is used to take over the index information of the logical volume to be cleared through the takeover volume structure and output response information; the response information indicates that the logical volume to be cleared has completed data clearing; after outputting the response information, the target service node uses the background server to asynchronously perform the data clearing operation on the logical volume to be cleared based on the takeover volume structure.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-6.

Images