Data processing method and device of distributed storage system, equipment and medium

Abstract

The invention relates to the technical field of distributed storage, and discloses a data processing method and device of a distributed storage system, equipment and a medium, the method comprises the following steps: obtaining a data write request initiated to the distributed storage system, and analyzing the data write request to obtain first target data; determining a first copy block device corresponding to the data write request, wherein the first copy block device is bound with a plurality of target storage nodes; and synchronously storing the first target data to a plurality of target storage nodes based on a replication strategy of the first replication block device. By implementing the method provided by the invention, the consistency of the copy data among the plurality of storage nodes can be ensured, the first target data is synchronously stored to the first storage node and the one or more second storage nodes based on the binding relationship, a complex synchronization algorithm is not needed, and the overall performance of the distributed storage system can be improved.

Description

Technical Field

[0001] This invention relates to the field of distributed storage technology, and specifically to a data processing method, apparatus, device, and medium for a distributed storage system. Background Technology

[0002] To ensure high data availability, distributed storage systems are typically used to store data across multiple storage nodes. This way, if one storage node fails or goes down, the remaining storage nodes can be accessed to successfully retrieve the data.

[0003] During the process of storing data across multiple storage nodes, some abnormal situations may occur, leading to inconsistencies in the data stored on the multiple storage nodes. For example, network abnormalities of storage nodes, storage node failures, or downtime may occur. Summary of the Invention

[0004] This invention provides a data processing method, apparatus, device, and medium for a distributed storage system to solve the problem of inconsistent data stored in multiple storage nodes.

[0005] In a first aspect, the present invention provides a data processing method for a distributed storage system, the method comprising: acquiring a data write request initiated to the distributed storage system and parsing the data write request to obtain first target data; determining a first replication block device corresponding to the data write request, the first replication block device being bound to multiple target storage nodes; and synchronously storing the first target data to multiple target storage nodes based on the replication strategy of the first replication block device.

[0006] The first target data carried in the write request is determined by parsing the write request, and the first replication block device corresponding to the write request is also identified. Since the first replication block device is bound to multiple target storage nodes, the replication strategy of the first replication block device can synchronously store the first target data on multiple target storage nodes. This ensures that the first target data is successfully stored on each target storage node, avoiding inconsistencies between multiple replicas. Furthermore, this solution guarantees the consistency and integrity of multiple replicas without requiring complex algorithms, thus improving the overall performance of the distributed storage system.

[0007] In one optional implementation, the plurality of target storage nodes bound to the first replication block device include a first storage node and a second storage node. The method further includes: determining a plurality of target storage nodes with free disks based on the disk working status of each storage node in the distributed storage system; selecting one target storage node from the plurality of target storage nodes as the first storage node; and selecting at least one other storage node from the plurality of target storage nodes, excluding the first storage node, as the second storage node.

[0008] Selecting a storage node with free disk space from the distributed storage system as the target storage node ensures that the selected target storage node has sufficient physical space, preventing subsequent write failures to the first target data. Furthermore, identifying the first and second storage nodes from multiple target storage nodes allows for the identification of the primary and secondary replicas when creating the first replication block device corresponding to a data write request, further improving the success rate of writing to the first target data and ensuring its consistency.

[0009] In an optional implementation, the method further includes: determining a first free disk on a first storage node and a second free disk on a second storage node based on resource attribute information in a configuration file, wherein the configuration file is constructed based on the first resource attribute information of the first storage node and the second resource attribute information of at least one second storage node; and creating a first replication block device based on the first free disk and the second free disk.

[0010] The first and second free disks are bound together as a logical first replication block device. This first replication block device enables real-time redundant backup of data, further resolving data consistency issues between multiple replicas and providing highly available block storage services for upper-layer applications.

[0011] In one optional implementation, the process of constructing a configuration file based on the first resource attribute information and the second resource attribute information includes: obtaining a configuration file template; filling the configuration file template with the first resource attribute information and the second resource attribute information to generate a configuration file.

[0012] By using a template generation method, the first and second resource attribute information can be filled into the configuration file template, allowing for rapid configuration file generation. Furthermore, the configuration file template is reusable, reducing redundant coding, and the standardized configuration format facilitates maintenance and troubleshooting.

[0013] In one optional implementation, a first replication block device is created based on a first free disk and a second free disk, including: creating a first initial replication block device between the first free disk and the second free disk; creating a physical volume on the first initial replication block device; if no volume group corresponding to the physical volume exists, creating a volume group for the physical volume, creating a storage pool for the volume group, and starting a data block server corresponding to the volume group to obtain the first replication block device, wherein the data block server is used to store and manage data blocks in the first replication block device; if a volume group corresponding to the physical volume exists, adding the physical volume to the volume group.

[0014] Creating physical volumes for the first initial replication block device, creating volume groups for the physical volumes, and creating storage pools for the volume groups are equivalent to setting up a logical volume management mechanism for the first initial replication block device. The first storage node corresponding to the first initial replication block device and at least one second storage node can use this logical volume management mechanism. Through the logical volume management mechanism, the storage resources on the first storage node and at least one second storage node can be flexibly managed.

[0015] In one optional implementation, based on the replication strategy of the first replication block device, the first target data is synchronously stored to the first storage node and the second storage node, including: storing the first target data to the first storage node through a data block server; and synchronously storing the first target data stored on the first storage node to the second storage node based on the replication strategy of the first replication block device.

[0016] The first target data is synchronously stored to the first storage node, and then the first target data stored on the first storage node is synchronously stored to the second storage node. This can further ensure that the first storage node and one or more second storage nodes store the same first target data, and further solve the problem of inconsistency between multiple replica data.

[0017] In an optional implementation, the method further includes: acquiring a data read request initiated to the distributed storage system; determining a second replica block device corresponding to the data read request, a third storage node bound to the second replica block device, and at least one fourth storage node; if the third storage node is not in a fault state, reading the second target data corresponding to the data read request from the third storage node; if the third storage node is in a fault state, reading the second target data from at least one fourth storage node.

[0018] If the third storage node is not faulty, the second target data can be read from it. If the third storage node is faulty, the second target data can be read from any fourth storage node until it is successfully read, thus improving the success rate of reading the second target data.

[0019] In a second aspect, the present invention provides a data processing apparatus for a distributed storage system, the apparatus comprising: a first acquisition module, configured to acquire a data write request initiated to the distributed storage system and parse the data write request to obtain first target data; a first determination module, configured to determine a first replication block device corresponding to the data write request, the first replication block device being bound to multiple target storage nodes; and a synchronization module, configured to synchronously store the first target data to multiple target storage nodes based on the replication strategy of the first replication block device.

[0020] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the data processing method of the distributed storage system of the first aspect or any corresponding embodiment described above.

[0021] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the data processing method of the distributed storage system of the first aspect or any corresponding embodiment described above.

[0022] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the data processing method of the distributed storage system described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the division of a Chunk into multiple Blocks according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the metadata controller issuing commands to the SSA according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the Resources mode according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the first type of data processing method for a distributed storage system according to an embodiment of the present invention; Figure 6 This is a schematic diagram of a second process for a data processing method of a distributed storage system according to an embodiment of the present invention; Figure 7 This is a schematic diagram of the third process of a data processing method for a distributed storage system according to an embodiment of the present invention; Figure 8 This is a schematic diagram illustrating the writing of data to a distributed storage system based on the DRBD mechanism according to an embodiment of the present invention; Figure 9 This is a structural block diagram of a data processing device for a distributed storage system according to an embodiment of the present invention; Figure 10 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0027] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0028] To ensure high data availability, distributed storage systems are typically used to store data across multiple storage nodes. This way, if one storage node fails or goes down, the remaining storage nodes can be accessed to successfully retrieve the data.

[0029] During the process of storing data across multiple storage nodes, some abnormal situations may occur, leading to inconsistencies in the data stored on the multiple storage nodes. For example, network abnormalities of storage nodes, storage node failures, or downtime may occur.

[0030] In view of this, this application provides a data processing method, apparatus, device, and medium for a distributed storage system. The method relies on a Distributed Replicated Block Device (DRBD) mechanism to determine the first replica block device corresponding to a data write request; and utilizes the replication strategy of the first replica block device to synchronously store the first target data carried by the data write request to multiple target storage nodes corresponding to the first replica block device. This ensures that the first target data can be successfully stored on each target storage node, avoiding inconsistencies between multiple replica data. Moreover, this solution can guarantee the consistency and integrity of multiple replica data without complex algorithmic logic, thus improving the overall performance of the distributed storage system.

[0031] As one optional application scenario in the embodiments of this application, such as Figure 1 As shown, the architecture of the distributed storage system in this application can include a client and a server. The client is used to interface with different application scenarios, such as virtual machine scenarios, container scenarios, and bare metal scenarios. The server forms the foundation of the distributed storage system, and data can be transmitted between the client and the server via a network.

[0032] For clients, in scenarios where virtual machines are adapted for the client, virtual machines can be created on a Cloud Virtualization Kernel (CVK) host using open-source general-purpose virtualization and emulation tools (Quick Emulator, Qemu). Figure 1 The diagram shows VM1 and VM2, and corresponding logical volumes are created for each virtual machine, such as logical volume 1 for VM1 and logical volume 2 for VM2. The created virtual machines can access the server via the qemu-librbd or qemu-vhost-librbd interface. However, in client-side adaptations for bare metal and container scenarios, a kernel block device is required for interface connection, and the server must be accessed via the nbd-librbd interface.

[0033] Among them, libebs connects to the data block server (ChunkServer) to read and write data, and at the same time obtains metadata from the metadata controller (Metactrl) to complete the storage mapping and addressing between the client-side logical volumes (such as Volume1 and Volume2) and the server-side storage, and correctly distributes the data actually written to the logical volumes to each storage node in the distributed storage system.

[0034] On the server side, the system includes a metadata controller (MetaCtrl), a chunk server (ChunkServer), and a storage service agent (SSA). The metadata controller maintains metadata information such as the topology, storage pools, and storage volumes of the entire distributed storage system. The chunk server stores the actual data, is responsible for data retrieval, and ensures high availability. The SSA manages the underlying physical volumes, handling disk reads and writes and providing underlying data services. The following is a detailed description of the metadata controller, chunk server, and storage service agent.

[0035] 1) Metadata Controller In a distributed storage system, metadata information is primarily stored and accessed centrally by a metadata controller. This metadata includes the distributed storage system's identifier, topology, volume metadata, volume chunk information, and chunk pool information. The distributed storage system identifier uniquely identifies the system; for example, it could be a Universally Unique Identifier (UUID). The topology stores information about all storage nodes and their respective disks, with each disk corresponding to a chunk server. Volume metadata includes the volume's name, total available storage space, current operating status, and characteristics.

[0036] When creating a volume, you need to specify the storage pool (ChunkPool) to which the volume belongs, and request chunks from the storage pool through the block server and allocate them to the volume. Furthermore, based on the total storage space available for the volume, you can request chunk replicas from the target storage pool bound to the storage pool. Chunk replica information is stored as part of the volume's metadata.

[0037] 2) Block server The block server is responsible for creating, deleting, and reading / writing all logical volumes (LVs) under its corresponding volume group (VG). The block server stores the actual data, is responsible for data retrieval, and ensures its high availability.

[0038] In a distributed storage hierarchy design, the logical volume allocated by the client is divided into multiple chunks and mapped to chunk replicas provided by a block server. Each block server further subdivides its disk into multiple chunk replicas. Chunk replicas provided by different block servers are all mapped to the chunks corresponding to the logical volume, ensuring high data availability through data redundancy.

[0039] At each storage node, each disk corresponds to a block server, which is responsible for the read and write operations on that disk. The metadata information of the block server includes the disk type, such as hard disk drive (HDD), flash memory chip (SATA SSD), flash memory chip with NVMe protocol (NVMe SSD), the disk's unique identifier (DISKUUID), the Internet Protocol (IP) address of the storage node to which the block server belongs, its current status, and capacity usage statistics. Thus, through the metadata information of the block server, it is possible to know the total number of disks on that storage node, the number of disks used, and the number of disks remaining.

[0040] Each chunk replica can read and write data in units of blocks. That is, each chunk replica is divided into multiple blocks, typically 64KB in size. The size of a chunk replica can usually be set to 10GB. Chunk replicas use a write-on-demand approach for block mapping, which is a thin-film configuration. Figure 2 As shown, the address of each block is set to 4 bytes, so a 10GB chunk such as Chunk1 or Chunk2 can hold 163,840 blocks. The logical block address (LBA) of the chunk is 640KB in size, which can be easily put into memory.

[0041] 3) Storage service agent like Figure 3As shown, SSA is installed and configured as a background service when deploying storage nodes. Each storage node has an SSA responsible for invoking relevant commands for that node. For example, the metadata controller issues a command to query storage node information in real time. Upon receiving the command, the SSA queries the information of each storage node sequentially, records this information in a data structure, and then reports it to the metadata controller. The metadata controller maintains the information reported by the SSA. When receiving a data write request, the metadata controller can use a disk allocation algorithm to find multiple target storage nodes with free disks, providing the identifiers of each target storage node and the identifiers of the free disks on each target storage node. It then issues a command to the SSA, which executes the command and creates the resource. After creating the resource, the SSA associates it with the block server and sets the resource mode for the created resource. The resource mode can be configured for data synchronization and access permissions between master and slave nodes.

[0042] like Figure 4 As shown, the resource mode of a DRBD block device (i.e., the replicated block device mentioned later) can be a 3-replica mode with primary (master node) + 2 secondary (slave nodes). Alternatively, the resource mode can be a 2-replica mode with primary + secondary + diskless. Based on this, Figure 3 The Resources1 mode can represent a 2-replica mode of primary + secondary + diskless, and the Resources2 mode can represent a 3-replica mode of primary + 2 secondarys. Figure 3 The Free disks in the text represent the free disks on the corresponding storage node.

[0043] In addition, storage nodes are responsible for periodically uploading relevant information (the running status of the storage node, the status of the disks, and resource information) to the metadata controller. The metadata controller ensures the resource usage balance of the SSA (such as whether disks are evenly distributed across the nodes).

[0044] In addition, SSA itself will also set a timer to collect information about the current storage nodes every 2 hours, including but not limited to disk idle information, disk usage information, etc., and actively report it to the metadata controller. Once there is any change in node information, including changes in node status, changes in node Internet Protocol (IP) address, disk failure, or changes in the number of disks after disk expansion, it will actively report to the metadata controller, and the reporting mechanism can adopt gRPC (a high-performance, cross-language remote procedure call framework).

[0045] like Figure 1 As shown, based on the aforementioned distributed storage system architecture, this application selects storage nodes with free disks as target storage nodes from the distributed storage system, then determines a first storage node (primary node) from multiple target storage nodes, and determines at least one other storage node from the remaining target storage nodes as a second storage node (secondary node); through the Distributed Replicated Block Device (DRBD) mechanism, the selected first free disk on the first storage node and the selected second free disk on the second storage node are bound together as a logical first replica block device (e.g., ...). Figure 1 The DRBD disk (as shown) utilizes the replication strategy of the first replicated block device to offload the actual data to the DRBD disk for high availability. The DRBD disk presents a multi-replica, highly available block device to the ChunkServer for reading and writing data, thereby ensuring data storage consistency across multiple storage nodes through the DRBD mechanism.

[0046] It should be understood that from a user's perspective, the first replication block device can be referred to as a DRBD disk, and users can read and write to it like a regular disk through the file system or directly. However, from a developer's perspective, the first replication block device can be considered a binding mechanism, through which data on one storage node can be instantly synchronized to the other storage nodes corresponding to the first replication block device.

[0047] According to an embodiment of the present invention, a data processing method embodiment for a distributed storage system is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0048] This embodiment provides a data processing method for a distributed storage system, which can be used on the server side, such as in a distributed storage system. Figure 5 This is a flowchart of a data processing method for a distributed storage system according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps: Step S501: Obtain the data write request initiated to the distributed storage system and parse the data write request to obtain the first target data.

[0049] A distributed storage system can be a storage architecture that distributes data across multiple independent storage nodes (servers, hard drives, or clusters) and works collaboratively through a network. As a specific example, this application may employ a storage architecture such as... Figure 1 The distributed storage system shown corresponds to multiple storage nodes.

[0050] A data write request can be an instruction initiated by a client to a distributed storage system, requesting that data be saved to the distributed storage system. As a concrete example, an application runs on the client, and the user initiates a data write request to the distributed storage system through the application.

[0051] Parsing a data write request involves extracting key information from the request, such as the primary target data carried in the request and the corresponding processing type (e.g., write or read). Furthermore, the method for parsing a data write request depends on the protocol format, data type, and deployment rules of the distributed storage system; no specific limitations are imposed here.

[0052] Step S502: Determine the first replication block device corresponding to the data write request. The first replication block device is bound to multiple target storage nodes.

[0053] Here, a pre-configured node selection algorithm can be used to determine multiple target storage nodes from the distributed storage system that can store the first target data. Then, through the Distributed Replicated Block Device (DRBD) mechanism, the selected disks on each target storage node are bound to a logical first replica block device (e.g., ...). Figure 1 The DRBD disk shown is used to determine the first replication block device corresponding to the data write request. The pre-set node selection algorithm can select a storage node with free disk space from the distributed storage system as the target storage node.

[0054] Step S503: Based on the replication strategy of the first replication block device, the first target data is synchronously stored to multiple target storage nodes.

[0055] Here, the first target data can be synchronously stored to multiple target storage nodes using fully synchronous, semi-synchronous, or asynchronous methods. For example, a first storage node can be selected from multiple target storage nodes, and the first target data can be written to the first storage node. Then, by combining fully synchronous, semi-synchronous, or asynchronous methods, the first target data can be written to at least one other storage node besides the first storage node among the multiple target storage nodes.

[0056] In the fully synchronous method, the first target data can be written to the primary node, such as the first storage node shown above, and simultaneously through... Figure 1 The backend storage network shown synchronizes the first target data to at least one of the multiple target storage nodes (excluding the first storage node) as described above. After the first target data is successfully written to the secondary node, a write success message is returned to the client.

[0057] In the semi-synchronous method, the first target data can be written to the primary node first, and then the first target data can be synchronized to the secondary node through the network. At this time, as soon as the secondary node receives the first target data, it will immediately return a write success to the client.

[0058] In asynchronous mode, after the first target data is written to the primary node, a write success message is immediately returned to the client.

[0059] The data processing method for the distributed storage system provided in this embodiment, after obtaining a data write request, determines the first target data carried in the data write request by parsing the data write request, and identifies the first replication block device corresponding to the data write request. Since the first replication block device is bound to multiple target storage nodes, the replication strategy of the first replication block device can synchronously store the first target data to multiple target storage nodes. This ensures that the first target data can be successfully stored on each target storage node, avoiding inconsistencies between multiple replicas. Moreover, this solution can guarantee the consistency and integrity of multiple replicas without complex algorithmic logic, thus improving the overall performance of the distributed storage system.

[0060] This embodiment provides a data processing method for a distributed storage system, which can be used on the server side, such as in a distributed storage system. Figure 6 This is a flowchart of a data processing method for a distributed storage system according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps: Step S601: Obtain the data write request initiated to the distributed storage system and parse the data write request to obtain the first target data. For details, please refer to [link to relevant documentation]. Figure 5 Step S501 of the illustrated embodiment will not be described again here.

[0061] Step S602: Determine the first replication block device corresponding to the data write request. The first replication block device is bound to multiple target storage nodes. For details, please refer to [link to relevant documentation]. Figure 5 Step S502 of the illustrated embodiment will not be described again here.

[0062] Step S603: Based on the replication strategy of the first replication block device, the first target data is synchronously stored to multiple target storage nodes. For details, please refer to [link to details]. Figure 5 Step S503 of the illustrated embodiment will not be described again here.

[0063] Step S604: Based on the disk working status of each storage node in the distributed storage system, determine multiple target storage nodes with free disks.

[0064] The disk's working status can be used to characterize its current usage state, which can be either idle or used. When a disk is bound to a replica block device with disks belonging to other storage nodes via the DRBD mechanism, its working status can be used; conversely, when a disk is not bound to any disks belonging to other storage nodes via the DRBD mechanism, its working status can be free.

[0065] For example, if a disk on a storage node has been configured in DRBD Resource mode and is running successfully, its working state can be "USED". Conversely, its working state can be "FREE". Furthermore, to improve the stability of the distributed storage system, DRBD resource configuration can adopt a one-disk-one-Resources model.

[0066] In addition, after a storage node joins the distributed storage system, the SSA corresponding to that storage node will collect statistics on the DRBD node resources, including the number of idle disks, the number of disks in use, the number of disks as resource masters (used for load balancing), and the number of disks in a faulty state (used for monitoring and maintenance). Each time the SSA collects this information, it includes an epoch to mark any configuration changes to the disks on that storage node. This information is then reported to the metadata controller. The metadata controller maintains the information reported by each storage node and saves it to the Distributed Key-Value Store (Etcd).

[0067] When the metadata controller receives a data write request to the distributed storage system, it can select a storage node with free disks as the target storage node based on the working status of each disk it maintains. The selected free disks on each target storage node are then grouped into a DRBD resource to achieve a balanced resource allocation. To ensure the stability of each DRBD resource, the allocated disks are not readjusted.

[0068] Furthermore, the number of target storage nodes can be pre-configured, and this application does not impose a specific limit on the exact number of target storage nodes. This application can adopt a create-on-demand model to create a first replica block device for a data write request, rather than pre-creating replica block devices for the entire distributed storage system.

[0069] Step S605: Select one target storage node from multiple target storage nodes as the first storage node.

[0070] like Figure 4 As shown, in Resources mode, a typical setup is a 3-replica mode with a primary node and 2 secondary nodes, or a 2-replica mode with a primary node, 1 secondary node, and 1 diskless node. Therefore, the first storage node corresponding to the first target data can be the primary node in Resources mode.

[0071] As a specific example, a target storage node can be randomly selected from multiple target storage nodes as the first storage node.

[0072] Step S606: Select at least one other storage node besides the first storage node from multiple target storage nodes as the second storage node.

[0073] The second storage node can be a slave node (secondary node) in Resources mode. As shown earlier, in 3-replica mode, the number of second storage nodes can be 2; in 2-replica mode, the number of second storage nodes can be 1. Of course, to ensure high availability of data, the number of slave nodes (i.e., second storage nodes) can be more, and there is no specific limit on the number of second storage nodes here.

[0074] Selecting a storage node with free disk space from the distributed storage system as the target storage node ensures that the selected target storage node has sufficient physical space, preventing subsequent write failures to the first target data. Furthermore, identifying the first and second storage nodes from multiple target storage nodes allows for the identification of the primary and secondary replicas when creating the first replication block device corresponding to a data write request, further improving the success rate of writing to the first target data and ensuring its consistency.

[0075] Step S607: Based on the resource attribute information in the configuration file, determine the first free disk on the first storage node and the second free disk on the second storage node, wherein the configuration file is constructed based on the first resource attribute information of the first storage node and the second resource attribute information of at least one second storage node.

[0076] The first resource attribute information may include the node attribute information and disk attribute information of the first storage node, and the second resource attribute information may include the node attribute information and disk attribute information of the second storage node. The node attribute information may represent the node name, unique identifier, and IP address of the corresponding storage node; the disk attribute information may represent which disk of the corresponding storage node is selected, its drive letter, and minor device number; of course, the resource attribute information (either the first or second resource attribute information) may also include other fields such as max-buffers, max-epoch-size, etc. max-buffers controls how much memory buffer is needed to temporarily store the packaged data; max-epoch-size controls how many write operations are packaged at a time.

[0077] The first storage node can obtain the first free disk of the first storage node defined in the configuration file by reading the resource attribute information in the configuration file; the second storage node can obtain the second free disk of the second storage node defined in the configuration file by reading the resource attribute information in the configuration file.

[0078] Specifically, step S607 includes: Step S6071: Obtain the configuration file template.

[0079] The configuration file template may include a first filled area for first resource attribute information and a second filled area for second resource attribute information. As a specific example, the configuration file template can be obtained by accessing memory or a predetermined storage location.

[0080] Step S6072: Fill the first resource attribute information and the second resource attribute into the configuration file template to generate the configuration file.

[0081] As shown above, the first resource attribute information can be filled into the first filling area, and the second resource attribute information can be filled into the second filling area to generate a configuration file.

[0082] By using a template generation method, the first and second resource attribute information can be filled into the configuration file template, allowing for rapid configuration file generation. Furthermore, the configuration file template is reusable, reducing redundant coding, and the standardized configuration format facilitates maintenance and troubleshooting.

[0083] Step S608: Create a first copy block device based on the first free disk and the second free disk.

[0084] The DRBD mechanism allows binding the first free disk on the first storage node and the second free disk on the second storage node into a logical first replication block device. To upper-layer applications (such as file systems and databases), the first replication block device appears as a regular local disk, allowing for direct formatting, mounting, and read / write operations.

[0085] To facilitate understanding of the creation process of a replica block device, the following description uses the creation of a DRBD disk (the first replica block device) based on the DRBD mechanism as an example. This includes steps S1 and S2.

[0086] Step S1, as described above, involves the metadata controller distributing the resource attribute information of the first and second storage nodes as a whole to both nodes. Taking the first storage node as an example, the first storage node reads the configuration file and starts up the DRBD resource. The DRBD resource includes the disks on the first storage node, the node information of the second storage node, the synchronization strategy between the first and second storage nodes, and the DRBD resource name, etc.

[0087] Step S2: After DRBD resources are started, the DRBD disk is generated. Correspondingly, the process of starting DRBD resources can be the same as the process of generating the DRBD disk. For example, DRBD resources can be started using the following commands: `wipefs -a / dev / sdc` (this command is optional and can be executed by non-diskless nodes), `drbdadm create-md test` (this command is required and can be executed by each storage node; `test` is the name of the first replicated block device created), `drbdadm up test` (this command is required and can be executed by non-diskless nodes).

[0088] The first and second free disks are bound together as a logical first replication block device. This first replication block device enables real-time redundant backup of data, further resolving data consistency issues between multiple replicas and providing highly available block storage services for upper-layer applications.

[0089] Specifically, step S608 includes: Step S6081: Create a first initial copy block device between the first free disk and the second free disk. The process for creating the first initial copy block device is described above.

[0090] Step S6082: Create a physical volume on the first initial copy block device.

[0091] Since the first initial replicated block device is a block device, it does not have flexible volume management capabilities. Therefore, this application creates a physical volume on the first initial replicated block device. This allows the first initial replicated block to be incorporated into the Logical Volume Manager (LVM) management system through the physical volume. It also lays the foundation for the subsequent creation of volume groups and storage pools, enabling the first initial replicated block to have flexible volume management capabilities.

[0092] The process of creating a physical volume (PV) for the first initial replica block device can be as follows: Enter the command: pvcreate / dev / drbd1009 The execution result, "Physical volume “ / dev / drbd1009” successfully created," indicates that the physical volume (PV) was created successfully. If creation is successful, volume groups (VGs) can then be created. It should be understood that this application uses a one-to-one mapping method, meaning that physical volumes can be created for each free disk corresponding to the first initial copy block device.

[0093] In step S6083, if there is no volume group corresponding to the physical volume, a volume group is created for the physical volume, a storage pool is created for the volume group, and a data block server corresponding to the volume group is started to obtain the first replica block device. The data block server is used to store and manage the data blocks in the first replica block device.

[0094] The process of creating a volume group for a physical volume here may include: Enter the command: vgcreate vg01 / dev / drbd1009 The execution result "Volume group “vg01” successfully created" indicates that the VG was created successfully. If the VG is created successfully, a ChunkServer service is started on the first storage node to manage the Chunks after the storage pool is created based on the VG. Therefore, in this application, one replicated block device corresponds to one data block server (ChunkServer), instead of one disk corresponding to one data block server.

[0095] The process of creating a storage pool for a volume group here may include: Enter the command: lvconvert --thinpool vg / pool0 --poolmetadata vg / pool0meta.

[0096] Once a storage pool (ChunkPool) is created, Chunks can be created based on that storage pool.

[0097] Step S6084: If a volume group corresponding to the physical volume exists, add the physical volume to the volume group.

[0098] Creating physical volumes for the first initial replication block device, creating volume groups for the physical volumes, and creating storage pools for the volume groups are equivalent to setting up a logical volume management mechanism for the first initial replication block device. The first storage node corresponding to the first initial replication block device and at least one second storage node can use this logical volume management mechanism. Through the logical volume management mechanism, the storage resources on the first storage node and at least one second storage node can be flexibly managed.

[0099] The data processing method of the distributed storage system provided in this embodiment binds a first free disk and at least one second free disk into a logical first initial replication block device. By configuring a physical volume, volume group, and storage pool for the first initial replication block device in sequence, a logical volume management mechanism is configured for the first initial replication block device. This allows the first free disk and one or more second free disks corresponding to the final first replication block device to reuse the logical volume management mechanism, thereby enabling flexible management of the storage resources of the first free disk and one or more second free disks.

[0100] This embodiment provides a data processing method for a distributed storage system, which can be used on the server side, such as in a distributed storage system. Figure 7 This is a flowchart of a data processing method for a distributed storage system according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps: Step S701: Obtain the data write request initiated to the distributed storage system and parse the data write request to obtain the first target data. For details, please refer to [link to relevant documentation]. Figure 5 Step S501 of the illustrated embodiment will not be described again here.

[0101] Step S702: Determine the first replication block device corresponding to the data write request. The first replication block device is bound to multiple target storage nodes. For details, please refer to [link to relevant documentation]. Figure 5 Step S502 of the illustrated embodiment will not be described again here.

[0102] Step S703: Based on the replication strategy of the first replication block device, the first target data is synchronously stored to multiple target storage nodes.

[0103] Specifically, step S703 includes: Step S7031: The first target data is stored in the first storage node through the data block server.

[0104] As shown above, the first free disk on the first storage node can reuse the logical management mechanism on the first replication block device. In this way, the storage resources of the first free disk can be flexibly divided based on the size of the first target data to obtain one or more data blocks (i.e., the Chunk mentioned above, this process is equivalent to applying for a Chunk from the storage pool); and then the first target data is stored in the applied one or more data blocks.

[0105] Step S7032: Based on the replication strategy of the first replication block device, the first target data stored on the first storage node is synchronously stored to the second storage node.

[0106] Since the first storage node is bound to at least one second storage node on the same first replication block device, the first target data can be synchronized to the second storage node after it has been stored on the first storage node. Furthermore, the method by which the first target data on the first storage node is synchronized to one or more second storage nodes after it has been stored on the first storage node depends on the synchronization method described above, such as full synchronization, semi-synchronous, or asynchronous.

[0107] In some alternative implementations, the method further includes: Step a1: Obtain the data read request initiated to the distributed storage system, and determine the second replication block device corresponding to the data read request, as well as the third storage node and at least one fourth storage node bound to the second replication block device.

[0108] Step a2: If the third storage node is not in a fault state, then read the second target data corresponding to the data read request from the third storage node.

[0109] Step a3: If the third storage node is in a fault state, the second target data is read from at least one fourth storage node.

[0110] Here, the data read request can be parsed to obtain the IP address of the second replica block device corresponding to the data read request. The IP address can be used to determine the primary node (i.e., the third storage node) corresponding to the data read request, as well as at least one fourth storage node bound to the second replica block device.

[0111] If the third storage node is not faulty, the second target data can be read from it. If the third storage node is faulty, the second target data can be read from any fourth storage node until it is successfully read, thus improving the success rate of reading the second target data.

[0112] The data processing method of the distributed storage system provided in this embodiment first synchronously stores the first target data to the first storage node, and then synchronously stores the first target data stored on the first storage node to the second storage node. This can further ensure that the first storage node and one or more second storage nodes store the same first target data, and further solve the problem of inconsistency between multiple replica data.

[0113] As a specific application embodiment of the present invention, such as Figure 8 As shown, assume the distributed storage system includes storage node 1, storage node 2, and storage node 3, and each storage node includes two disks, namely disk A and disk B. Assume each DRBD disk is created from two disks (two disks are needed for 2 replicas, and three disks are needed for 3 replicas). For example, DRBD1 is created from disk A on storage node 2 and disk A on storage node 3; DRBD2 is created from disk A on storage node 1 and disk B on storage node 3; and DRBD3 is created from disk B on storage node 1 and disk B on storage node 2. After DRBD1, DRBD2, and DRBD3 are created, any node on storage node 1, storage node 2, or storage node 3 can query DRBD1, DRBD2, and DRBD3. Assuming the first target data is written to disk B on storage node 1, the replication strategy of DRBD3 can be used to synchronize the first target data stored on disk B on storage node 1 to disk B on storage node 2.

[0114] This application uses a DRBD-based scheme. On the server side, a client's volume consists of several chunk replicas, each corresponding to a thin logical volume (LV). When creating a volume, the metadata controller can request chunks from the chunk pool. Once a chunk is assigned to a specific chunkserver, the chunkserver creates a mapping between the thin LV and the requested chunk replica in the storage pool. Chunk replica management of blocks is delegated to the thin LV, i.e., managed by LVM. The chunkserver is not responsible for the LBA mapping management of blocks within the chunk replica.

[0115] This embodiment also provides a data processing apparatus for a distributed storage system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0116] This embodiment provides a data processing device for a distributed storage system, such as... Figure 9 As shown, it includes: The first acquisition module 901 is used to acquire data write requests initiated to the distributed storage system and parse the data write requests to obtain the first target data.

[0117] The first determining module 902 is used to determine the first replica block device corresponding to the data write request. The first replica block device is bound to multiple target storage nodes.

[0118] The synchronization module 903 is used to synchronously store the first target data to multiple target storage nodes based on the replication strategy of the first replication block device.

[0119] In some alternative implementations, the plurality of target storage nodes bound to the first replication block device include a first storage node and a second storage node, and the device further includes: The second determining module is used to determine multiple target storage nodes with free disks based on the disk working status of each storage node in the distributed storage system.

[0120] The first selection module is used to select one target storage node as the first storage node from multiple target storage nodes.

[0121] The second selection module is used to select at least one other storage node besides the first storage node from a plurality of target storage nodes as the second storage node.

[0122] In some alternative embodiments, the device further includes: The third determining module is used to determine the first free disk on the first storage node and the second free disk on the second storage node based on the resource attribute information in the configuration file, wherein the configuration file is constructed based on the first resource attribute information of the first storage node and the second resource attribute information of at least one second storage node.

[0123] A creation module is used to create a first copy block device based on a first free disk and a second free disk.

[0124] In some optional implementations, the third determining module is further configured to obtain a configuration file template; fill the first resource attribute information and the second resource attribute into the configuration file template to generate a configuration file.

[0125] In some optional implementations, the creation module is further configured to create a first initial replication block device between the first free disk and the second free disk; create a physical volume on the first initial replication block device; if no volume group corresponding to the physical volume exists, create a volume group for the physical volume, create a storage pool for the volume group, and start a data block server corresponding to the volume group to obtain the first replication block device, wherein the data block server is used to store and manage the data blocks in the first replication block device; if a volume group corresponding to the physical volume exists, add the physical volume to the volume group.

[0126] In some optional implementations, the synchronization module 903 is further configured to store the first target data to the first storage node via the data block server; and to synchronously store the first target data stored on the first storage node to the second storage node based on the replication strategy of the first replication block device.

[0127] In some alternative embodiments, the device includes: The second acquisition module is used to acquire data read requests initiated to the distributed storage system, and to determine the second replica block device corresponding to the data read request, as well as the third storage node and at least one fourth storage node bound to the second replica block device.

[0128] The first read module is used to read the second target data corresponding to the data read request from the third storage node if the third storage node is not in a fault state.

[0129] The second read module is used to read the second target data from at least one fourth storage node if the third storage node is in a faulty state.

[0130] The data processing apparatus for the distributed storage system provided in this embodiment of the invention can execute the data processing method for the distributed storage system provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0131] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0132] The following is a detailed reference. Figure 10This diagram illustrates a suitable structural schematic for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from memory 1008 into random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device. The processor 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0133] Typically, the following devices can be connected to the I / O interface 1005: input devices 1006 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1007 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; memory devices 1008 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows electronic devices to exchange data via wireless or wired communication with other devices. Although Figure 10 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0134] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 1009, or installed from a memory 1008, or installed from a ROM 1002. When the computer program is executed by the processor 1001, it performs the functions defined in the data processing method of the distributed storage system of the embodiments of the present invention.

[0135] Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0136] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the data processing method of the distributed storage system shown in the above embodiments is implemented.

[0137] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0138] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A data processing method for a distributed storage system, characterized in that, The method includes: Obtain the data write request initiated to the distributed storage system, and parse the data write request to obtain the first target data; The first replication block device corresponding to the data write request is determined, and the first replication block device is bound to multiple target storage nodes; Based on the replication strategy of the first replication block device, the first target data is synchronously stored to the multiple target storage nodes.

2. The method according to claim 1, characterized in that, The plurality of target storage nodes bound to the first replication block device include a first storage node and a second storage node, and the method further includes: Based on the disk working status of each storage node in the distributed storage system, the plurality of target storage nodes with free disks are determined. From the plurality of target storage nodes, one target storage node is selected as the first storage node; From the plurality of target storage nodes, at least one other storage node besides the first storage node is selected as the second storage node.

3. The method according to claim 2, characterized in that, The method further includes: Based on the resource attribute information in the configuration file, the first free disk on the first storage node and the second free disk on the second storage node are determined, wherein the configuration file is constructed based on the first resource attribute information of the first storage node and the second resource attribute information of the at least one second storage node; The first copy block device is created based on the first free disk and the second free disk.

4. The method according to claim 3, characterized in that, The process of constructing the configuration file based on the first resource attribute information and the second resource attribute information includes: Get the configuration file template; The configuration file is generated by filling the first resource attribute information and the second resource attribute information into the configuration file template.

5. The method according to claim 3, characterized in that, The step of creating the first replication block device based on the first free disk and the second free disk includes: Create a first initial copy block device between the first free disk and the second free disk; Create a physical volume on the first initial replication block device; If no volume group corresponding to the physical volume exists, the volume group is created for the physical volume, a storage pool is created for the volume group, and a data block server corresponding to the volume group is started to obtain the first replicated block device. The data block server is used to store and manage the data blocks in the first replicated block device. If a volume group corresponding to the physical volume exists, then the physical volume is added to the volume group.

6. The method according to claim 5, characterized in that, The replication strategy based on the first replication block device, which synchronously stores the first target data to the first storage node and the second storage node, includes: The first target data is stored in the first storage node through the data block server; Based on the replication strategy of the first replication block device, the first target data stored on the first storage node is synchronously stored to the second storage node.

7. The method according to claim 1, characterized in that, The method further includes: Obtain a data read request initiated to the distributed storage system, and determine the second replication block device corresponding to the data read request, as well as the third storage node and at least one fourth storage node bound to the second replication block device; If the third storage node is not in a fault state, then the second target data corresponding to the data read request is read from the third storage node; If the third storage node is in the fault state, the second target data is read from the at least one fourth storage node.

8. A data processing device for a distributed storage system, characterized in that, The device includes: The first acquisition module is used to acquire data write requests initiated to the distributed storage system and parse the data write requests to obtain the first target data; The first determining module is used to determine the first replication block device corresponding to the data write request, wherein the first replication block device is bound to multiple target storage nodes; The synchronization module is used to synchronously store the first target data to the plurality of target storage nodes based on the replication strategy of the first replication block device.

9. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the data processing method of the distributed storage system according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the data processing method of the distributed storage system according to any one of claims 1 to 7.

Images