Data writing method, medium, apparatus, and computing device

Abstract

Embodiments of the present disclosure provide a data writing method, medium, device and computing device. The method comprises: in response to a data writing request received by a leader node, the leader node writes the data writing request into a Raft file of the leader node; in response to majority reply information received by the leader node, updating a state in a state machine of the leader node based on the majority reply information, and determining a second starting position of a target data writing address in a virtual disk file; and exchanging a first disk block mapping relationship with a second disk block mapping relationship. The above scheme reduces the number of data writing operations. The original operation process of writing data from a disk block corresponding to a Raft file on a local disk to a disk block corresponding to a virtual disk file on a local disk is replaced by exchanging the first disk block mapping relationship with the second disk block mapping relationship, thereby improving the efficiency of data writing and reducing the resource consumption required for data writing.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of distributed systems, and more specifically, the embodiments of this disclosure relate to a data writing method, medium, apparatus, and computing device. Background Technology

[0002] This section is intended to provide background or context for embodiments of this disclosure. The description herein is not intended to imply that it is prior art simply because it is included in this section.

[0003] In distributed block storage systems, multi-node redundancy is used to improve the reliability of data writes. For each data write request, the corresponding data is first written to the corresponding location in the Raft file on the local disk. Multiple nodes then verify the data in the Raft file. If verification is successful, the data in the Raft file is written to the corresponding location in the state machine on the local disk, completing the data write process.

[0004] In this approach, multi-node confirmation avoids writing invalid or redundant data to the local disk, thus preventing disk space consumption and improving data write reliability. However, this approach involves two write operations: writing data to the corresponding location of the Raft file on the local disk and writing data to the corresponding location of the state machine on the local disk. This can lead to high resource consumption. Summary of the Invention

[0005] This disclosure provides a data writing method, medium, apparatus, and computing device to reduce resource consumption caused by data writing.

[0006] In a first aspect of this disclosure, a data writing method is provided, applied to a distributed block storage system that conforms to the Raft protocol, comprising: responding to a data write request received by a leader node, the leader node writing the data write request to its Raft file, and sending a request copy of the data write request to a plurality of corresponding follower nodes, wherein the Raft file has a first block mapping relationship with the leader node's local disk, and the data write request includes target data to be written, a first data size of the target data, and a persistent address of the target data in the Raft file. First starting position; In response to the majority response information received by the leader node, update the state in the leader node's state machine based on the majority response information, and determine the second starting position of the target write address of the target data in the virtual disk file. The majority response information consists of multiple response messages sent to the leader node by the majority of follower nodes, and the content of the multiple response messages is consistent. The virtual disk file corresponds to the leader node's state machine, and the virtual disk file has a second disk block mapping relationship with the leader node's local disk. Exchange the first disk block mapping relationship with the second disk block mapping relationship to complete the writing of the target data.

[0007] In one embodiment of this disclosure, the distributed block storage system further includes a user-space file system and a disk block mapping exchange interface; the step of exchanging the first disk block mapping relationship with the second disk block mapping relationship further includes: calling the disk block mapping exchange interface through the user-space file system to record the replay position of the Raft file corresponding to this disk block mapping relationship exchange, wherein the replay position includes a replica set identifier and a log replay index, wherein the replica set identifier and the log replay index are function parameters of the disk block mapping exchange interface.

[0008] In another embodiment of this disclosure, the number of the plurality of follower nodes is N, and the majority response information is sent by M follower nodes, wherein M is greater than or equal to (N+1) / 2+1, and M and N are positive integers.

[0009] In another embodiment of this disclosure, the Raft file includes a log file and an auxiliary file; the leader node writes the data write request to the leader node's Raft file by: if the first data size is greater than or equal to a preset size, determining the first data and the second data in the target data according to the first starting position and the preset disk block size; writing the first data to the first disk block corresponding to the log file, and writing the second data to the second disk block corresponding to the auxiliary file, so as to write the data write request to the leader node's Raft file.

[0010] In another embodiment of this disclosure, the first data includes header data and / or tail data of the target data, the size of the header data and the size of the tail data are respectively smaller than the preset disk block size, the size of the header data is the difference between the preset disk block size and a first offset, the first offset is the remainder between the first starting position and the preset disk block size; the second data includes middle data of the target data, the size of the second data is a positive integer multiple of the preset disk block size.

[0011] In another embodiment of this disclosure, determining the first data and the second data in the target data based on the first starting position and the preset disk block size includes: determining the head data size based on the first starting position and the preset disk block size; determining the difference between the first data size and the head data size as the remaining data size; determining the integer part of the quotient of the remaining data size and the preset disk block size as the number of disk blocks; determining the middle data size by multiplying the number of disk blocks by the preset disk block size; obtaining a first difference between the first data size and the remaining data size, and determining the difference between the first difference and the middle data size as the tail data size; and determining the first data and the second data in the target data based on the head data size, the middle data size, and the tail data size.

[0012] In another embodiment of this disclosure, determining the first data and the second data in the target data based on the head data size, the middle data size, and the tail data size includes: dividing the target data into head data, middle data, and tail data based on the head data size, the middle data size, and the tail data size; determining the head data and / or the tail data as the first data; and determining the middle data as the second data.

[0013] In another embodiment of this disclosure, the first disk block mapping relationship is the disk block mapping relationship between the auxiliary file and the local disk of the leader node; exchanging the first disk block mapping relationship and the second disk block mapping relationship includes: updating the disk block mapping relationship between the virtual disk file and the local disk of the leader node to the first disk block mapping relationship; updating the disk block mapping relationship between the auxiliary file and the local disk of the leader node to the second disk block mapping relationship; and writing the data in the disk block corresponding to the log file to the disk block corresponding to the virtual disk file.

[0014] In another embodiment of this disclosure, exchanging the first disk block mapping relationship with the second disk block mapping relationship includes: obtaining the target request identifier corresponding to the previous disk block mapping relationship exchange; and, based on the target request identifier, if it is determined that the next request to be exchanged for the disk block mapping relationship is the data write request, then exchanging the first disk block mapping relationship with the second disk block mapping relationship.

[0015] In another embodiment of this disclosure, exchanging the first disk block mapping relationship with the second disk block mapping relationship includes: determining first file information of the virtual disk file, second file information of the Raft file, and the first data size, wherein the first file information includes the file handle of the virtual disk file and the second starting position of the target write address of the target data in the virtual disk file, and the second file information includes the file handle of the Raft file and the first starting position of the persistent address of the target data in the Raft file; and exchanging the first disk block mapping relationship with the second disk block mapping relationship through the disk block mapping exchange interface according to the first file information, the second file information, and the first data size.

[0016] In another embodiment of this disclosure, exchanging the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface based on the first file information, the second file information, and the first data size includes: determining the first file information, the second file information, and the first data size as function input parameters of the disk block mapping exchange interface, and calling the disk block mapping exchange interface through the user-mode file system to exchange the first disk block mapping relationship and the second disk block mapping relationship.

[0017] In another embodiment of this disclosure, the function input parameters of the disk block mapping exchange interface further include the identifier of the data write request; after exchanging the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface, the method further includes: updating the target request identifier to the identifier of the data write request.

[0018] In another embodiment of this disclosure, the first data size is smaller than the preset size, and the Raft file includes a log file; the leader node writes the data write request to the leader node's Raft file, including: writing the target data to the log file according to the first data size and the first starting position; the method further includes: determining the disk block corresponding to the virtual disk file, and writing the target data to the disk block corresponding to the virtual disk file.

[0019] In a second aspect of this disclosure, a computer-readable storage medium is provided, comprising: computer-executable instructions stored therein, which, when executed by a processor, are used to implement the data writing method as described in any one of the first aspects.

[0020] In a third aspect of this disclosure, a data writing apparatus is provided for a distributed block storage system that conforms to the Raft protocol. The apparatus includes a processing module configured to respond to a data write request received by a leader node, wherein the leader node writes the data write request to its Raft file and sends a request copy of the data write request to a plurality of corresponding follower nodes. The Raft file has a first block mapping relationship with the leader node's local disk. The data write request includes target data to be written, a first data size of the target data, and a first persistent address of the target data in the Raft file. The system comprises: a starting position determination module, configured to respond to majority response information received by the leader node, update the state in the leader node's state machine based on the majority response information, and determine a second starting position of the target data in the virtual disk file at the target write address; the majority response information being multiple responses sent to the leader node by the majority of follower nodes, and the content of the multiple responses being consistent; the virtual disk file corresponding to the leader node's state machine; and a second disk block mapping relationship between the virtual disk file and the leader node's local disk; and a write module, configured to exchange the first disk block mapping relationship with the second disk block mapping relationship to complete the writing of the target data.

[0021] In one embodiment of this disclosure, the distributed block storage system further includes a user-space file system and a disk block mapping exchange interface; the device further includes: an exchange module, configured to call the disk block mapping exchange interface through the user-space file system to record the replay position of the Raft file corresponding to this disk block mapping relationship exchange, wherein the replay position includes a replica set identifier and a log replay index, wherein the replica set identifier and the log replay index are function parameters of the disk block mapping exchange interface.

[0022] In another embodiment of this disclosure, the number of the plurality of follower nodes is N, and the majority response information is sent by M follower nodes, wherein M is greater than or equal to (N+1) / 2+1, and M and N are positive integers.

[0023] In another embodiment of this disclosure, the Raft file includes a log file and an auxiliary file; the apparatus further includes: an execution module, configured to determine first data and second data in the target data according to the first starting position and a preset disk block size if the first data size is greater than or equal to a preset size; the execution module is further configured to write the first data to a first disk block corresponding to the log file and write the second data to a second disk block corresponding to the auxiliary file, so as to write the data write request to the Raft file of the leader node.

[0024] In another embodiment of this disclosure, the first data includes header data and / or tail data of the target data, the size of the header data and the size of the tail data are respectively smaller than the preset disk block size, the size of the header data is the difference between the preset disk block size and a first offset, the first offset is the remainder between the first starting position and the preset disk block size; the second data includes middle data of the target data, the size of the second data is a positive integer multiple of the preset disk block size.

[0025] In another embodiment of this disclosure, the execution module is specifically configured to determine the header data size based on the first starting position and the preset disk block size; the execution module is further configured to determine the remaining data size by the difference between the first data size and the header data size; the execution module is further configured to determine the number of disk blocks by the integer part of the quotient of the remaining data size and the preset disk block size; the execution module is further configured to determine the middle data size by the product of the number of disk blocks and the preset disk block size; the execution module is further configured to obtain a first difference between the first data size and the remaining data size, and determine the tail data size by the difference between the first difference and the middle data size; the execution module is further configured to determine the first data and the second data in the target data based on the header data size, the middle data size, and the tail data size.

[0026] In another embodiment of this disclosure, the execution module is specifically configured to divide the target data into header data, middle data, and tail data according to the header data size, the middle data size, and the tail data size; the execution module is further configured to determine the header data and / or the tail data as the first data; the execution module is further configured to determine the middle data as the second data.

[0027] In another embodiment of this disclosure, the first disk block mapping relationship is the disk block mapping relationship between the auxiliary file and the local disk of the leader node; the execution module is specifically used to update the disk block mapping relationship between the virtual disk file and the local disk of the leader node to the first disk block mapping relationship; the execution module is further used to update the disk block mapping relationship between the auxiliary file and the local disk of the leader node to the second disk block mapping relationship; the execution module is further used to write the data in the disk block corresponding to the log file to the disk block corresponding to the virtual disk file.

[0028] In another embodiment of this disclosure, the apparatus includes: a verification module, configured to obtain a target request identifier corresponding to the previous disk block mapping relationship exchange; the verification module is further configured to, based on the target request identifier, if it is determined that the next request to be exchanged for disk block mapping relationship is the data write request, exchange the first disk block mapping relationship with the second disk block mapping relationship.

[0029] In another embodiment of this disclosure, the apparatus further includes: a parameter module, configured to determine first file information of the virtual disk file, second file information of the Raft file, and the first data size, wherein the first file information includes a file handle of the virtual disk file and a second starting position of the target write address of the target data in the virtual disk file, and the second file information includes a file handle of the Raft file and a first starting position of the persistent address of the target data in the Raft file; the parameter module is further configured to exchange the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface according to the first file information, the second file information, and the first data size.

[0030] In another embodiment of this disclosure, the parameter module is specifically used to determine the first file information, the second file information, and the first data size as function input parameters of the disk block mapping exchange interface, and to call the disk block mapping exchange interface through the user-space file system to exchange the first disk block mapping relationship and the second disk block mapping relationship.

[0031] In another embodiment of this disclosure, the function input parameters of the disk block mapping exchange interface further include the identifier of the data write request; the apparatus further includes: an update module, used to update the target request identifier to the identifier of the data write request.

[0032] In another embodiment of this disclosure, the first data size is smaller than the preset size, and the Raft file includes a log file; the device further includes: a judgment module, configured to write the target data to the log file according to the first data size and the first starting position; the judgment module is further configured to determine the disk block corresponding to the virtual disk file and write the target data to the disk block corresponding to the virtual disk file.

[0033] In a fourth aspect of the present disclosure, a computing device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the computing device to perform a data writing method as described in any one of the first aspects of the present disclosure.

[0034] According to the embodiments of this disclosure, by exchanging disk block mappings, the number of data write operations is reduced. The original process of writing data from the disk block corresponding to the Raft file on the local disk to the disk block corresponding to the virtual disk file on the local disk in the state machine is replaced by exchanging the first disk block mapping relationship with the second disk block mapping relationship, thereby improving the efficiency of data writing and reducing the resource consumption required for data writing. Attached Figure Description

[0035] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0036] Figure 1 A schematic diagram illustrating an application scenario according to an embodiment of the present disclosure is provided.

[0037] Figure 2 A schematic flowchart of a data writing method provided according to an embodiment of the present disclosure is shown.

[0038] Figure 3 A schematic flowchart of a data writing method provided according to an embodiment of the present disclosure is shown.

[0039] Figure 4 An example diagram illustrating the determination of data size according to an embodiment of the present disclosure is shown schematically;

[0040] Figure 5 An example diagram illustrating the writing of data into a Raft file according to an embodiment of this disclosure is shown schematically;

[0041] Figure 6 An example diagram illustrating a swap block mapping according to an embodiment of the present disclosure is shown schematically;

[0042] Figure 7 An example diagram illustrating a verification data write request provided according to an embodiment of the present disclosure is shown schematically.

[0043] Figure 8 A schematic diagram of the structure of a storage medium provided according to an embodiment of the present disclosure is shown.

[0044] Figure 9 A schematic diagram of the structure of a data writing apparatus provided according to an embodiment of the present disclosure is shown.

[0045] Figure 10 A schematic diagram of the structure of a data writing apparatus provided according to an embodiment of the present disclosure is shown.

[0046] Figure 11 A schematic diagram of the structure of a computing device provided according to an embodiment of the present disclosure is shown.

[0047] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0048] The principles and spirit of this disclosure will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0049] Those skilled in the art will understand that the embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. The data involved in this disclosure can be data authorized by the user or fully authorized by all parties. The collection, dissemination, and use of the data comply with relevant national laws and regulations. The embodiments / examples of this disclosure can be combined with each other.

[0050] According to embodiments of this disclosure, a method, medium, apparatus, and computing device for writing data are proposed.

[0051] Furthermore, the number of any elements in the accompanying drawings is for illustrative purposes only and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0052] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments. Invention Overview

[0054] The inventors have discovered a double-write problem in related technologies when executing data write requests. Specifically, the data corresponding to the write request is first written to the leader node's Raft file in the corresponding local disk block, and a copy of the write request is sent to multiple follower nodes. After receiving the majority response from the follower nodes, the leader node then copies the data from the Raft file in the local disk block to the state machine's virtual disk file in the corresponding local disk block. This double-write problem increases both the storage space occupied on the local disk and resource consumption, and the overall time required for data writing, reducing data writing efficiency.

[0055] To address the above issues, the inventors designed a method that reduces the number of data write operations by exchanging disk block mappings. The original process of writing data from the Raft file's corresponding disk block on the local disk to the virtual disk file's corresponding disk block on the state machine is replaced by exchanging the first and second disk block mapping relationships. This improves data write efficiency and reduces the resource consumption required for data writing.

[0056] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0057] Application Scenarios Overview

[0058] First refer to Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario provided for an embodiment of this disclosure.

[0059] like Figure 1 As shown, the distributed block storage system uses the Raft consensus protocol to ensure data consistency across multiple nodes. The distributed block storage system includes a leader node and follower nodes. When the leader node receives a data write request, it writes the corresponding data to the Raft file. The leader node then sends the data to its multiple follower nodes. Once the leader node receives a majority response, it writes the data from the Raft file to the state machine, thus ensuring data consistency across the nodes.

[0060] Exemplary methods

[0061] The following is combined Figure 1 Application scenarios, refer to Figure 2-7 This document describes a data writing method provided according to exemplary embodiments of the present disclosure. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way. Rather, the embodiments of the present disclosure can be applied to any applicable scenario.

[0062] The implementation entity of the embodiments disclosed herein can be a data writing device, which is used to execute data writing requests. The data writing device can be implemented in various ways. For example, the data writing device can be program software; or, the device can also be a physical device, such as a chip, smart terminal, computer, server, or USB flash drive.

[0063] refer to Figure 2 , Figure 2 This is a schematic flowchart illustrating a data writing method provided in one embodiment of this disclosure. Figure 2 As shown, it includes:

[0064] S201. In response to a data write request received by the leader node, the leader node writes the data write request to its Raft file and sends a copy of the data write request to the corresponding multiple follower nodes. The Raft file has a first block mapping relationship with the leader node's local disk. The data write request includes the target data to be written, the first data size of the target data, and the first starting position of the persistent address of the target data in the Raft file.

[0065] The requested copy is a copy file of the data write request.

[0066] For the example of the first disk block mapping relationship, the first disk block mapping relationship is used to access data in the local disk from the Raft file. The first disk block mapping relationship corresponds to the first position in the local disk, and only the data in the first position of the local disk can be accessed through the first disk block mapping relationship.

[0067] S202. In response to the majority response information received by the leader node, update the state in the leader node's state machine based on the majority response information, and determine the second starting position of the target write address of the target data in the virtual disk file. The majority response information consists of multiple response messages sent to the leader node by the majority of follower nodes, and the content of the multiple response messages is consistent. The virtual disk file corresponds to the leader node's state machine, and the virtual disk file has a second disk block mapping relationship with the leader node's local disk.

[0068] Raft files are used to ensure data consistency among multiple nodes, and state machines are used to persistently store data. A state machine consists of multiple virtual disk files.

[0069] For the example of the second disk block mapping relationship, the virtual disk file corresponds to the second location on the local disk, and the virtual disk file can access the data at the second location on the local disk through the second disk block mapping relationship.

[0070] Optionally, if no response is received from any follower node, or if the number of follower nodes corresponding to the matching response is not in the majority, then data writing will not be performed.

[0071] One feasible implementation is that the number of the plurality of follower nodes is N, and the majority response information is sent by M follower nodes, wherein M is greater than or equal to (N+1) / 2+1, and M and N are positive integers.

[0072] For the example of determining the majority response information, if the number of follower nodes is 9, then when M is greater than or equal to 6, that is, when the leader node receives response information with the same content sent by at least 6 follower nodes, it is determined that the leader node has received the majority response information.

[0073] In this feasible implementation, data verification is performed by determining the majority of responses to ensure the accuracy of data writing.

[0074] S203. Exchange the first disk block mapping relationship with the second disk block mapping relationship to complete the writing of the target data.

[0075] For the example of swapped disk block mapping, after swapping the disk block mapping, Raft file access can only access data in the second position on the local disk, and virtual disk files can only access data in the first position on the local disk.

[0076] In one feasible implementation, the distributed block storage system further includes a user-space file system and a disk block mapping exchange interface; the step of exchanging the first disk block mapping relationship with the second disk block mapping relationship further includes: calling the disk block mapping exchange interface through the user-space file system to record the replay position of the Raft file corresponding to this disk block mapping relationship exchange, wherein the replay position includes a replica set identifier and a log replay index, wherein the replica set identifier and the log replay index are function parameters of the disk block mapping exchange interface.

[0077] For the disk block mapping exchange interface example, the disk block mapping exchange interface API corresponding to the user-space file system can execute user-defined function parameters. The operating system kernel restricts the input parameters of custom functions. By using the user-space file system, the operating system kernel can be bypassed, and disk operations can be performed directly, thereby removing the restrictions of the operating system kernel and improving the flexibility of executing function parameters.

[0078] Optionally, the operations on the API execution function parameters are atomic operations with no intermediate states, and the atomicity of the operations on the execution function parameters is guaranteed by the operation log file.

[0079] For examples that guarantee atomicity, multiple steps of the API execution function's input parameters and the execution status of each step are written to an operation log file. The execution status includes normal or abnormal. If a system crash or other failure causes the API execution to be interrupted, the operation log file is used to determine the most recent normal step, and the API continues execution from that step, thereby ensuring the accuracy of API execution.

[0080] In this system, a single disk block mapping exchange request only needs to be executed once. Repeatedly executing the request does not affect the result but consumes additional resources, leading to resource waste—a problem known as idempotency. For multiple disk block mapping exchange requests, a replay position is used to record currently completed requests. When executing new requests, already completed requests are skipped, avoiding duplicate execution and thus saving resources.

[0081] Specifically, the replica set identifier is a unique identifier corresponding to each disk block mapping exchange request in the disk block mapping exchange interface, used to distinguish different disk block mapping exchange requests. The replica set identifier can be in the form of a number, etc. Disk block mapping exchange requests are executed according to the order of the replica set identifiers. All disk block mapping exchange requests are arranged to form a request queue, and requests in the request queue are executed sequentially. The log replay index is the position of each disk block mapping exchange request in the request queue, used to locate the most recently executed disk block mapping exchange request.

[0082] In this feasible implementation, by recording the most recently executed disk block mapping relationship exchange request through the replay position, the next disk block mapping relationship exchange request that needs to be executed can be accurately located, avoiding the repeated execution of disk block mapping relationship exchange requests that have already been executed, thereby avoiding resource waste.

[0083] Based on any of the above embodiments, the following, in conjunction with Figure 3 The detailed process of writing data is explained.

[0084] Figure 3 This is a flowchart illustrating a data writing method provided in an embodiment of this disclosure. Figure 3 As shown, the method includes:

[0085] S301. If the size of the first data is greater than or equal to the preset size, then the first data and the second data are determined in the target data according to the first starting position and the preset disk size.

[0086] The local disk is divided into multiple disk blocks through logical partitions. These disk blocks are used to store data, and all disk blocks are the same size.

[0087] One feasible implementation is that the first data includes the header data and / or the tail data of the target data, the data size of the header data and the tail data is smaller than the preset disk block size, the header data size is the difference between the preset disk block size and a first offset, and the first offset is the remainder between the first starting position and the preset disk block size; the second data includes the middle data of the target data, and the data size of the second data is a positive integer multiple of the preset disk block size.

[0088] For the disk block example, the local disk consists of multiple disk blocks, and the target data stored on the local disk will occupy space in these disk blocks. The disk blocks occupied by the target data include those whose entire disk block space is occupied by the target data, and those whose partial disk block space is occupied by the target data. The header and tail data represent data that occupies a portion of the disk block space. The middle data represents data that occupies the entire disk block space.

[0089] The other space in the disk block occupied by the header and tail data stores other data. Therefore, changes in the mapping relationship of this disk block will cause changes in the mapping relationship of other data.

[0090] It should be noted that this disclosure is used to illustrate scenarios where both header and tail data exist. In actual use, there are scenarios where neither header nor tail data exists, and the solution disclosed herein can also be applied.

[0091] In this feasible implementation, by distinguishing between the first data and the second data, it is possible to determine which data mapping relationships in the target data can be changed based on the disk blocks where the data is stored, thereby exchanging the mapping relationships without affecting the validity of the data.

[0092] A feasible implementation method for determining first data and second data includes: determining a header data size based on a first starting position and a preset disk block size; determining the difference between the first data size and the header data size as the remaining data size; determining the integer part of the quotient of the remaining data size and the preset disk block size as the number of disk blocks; determining the product of the number of disk blocks and the preset disk block size as the middle data size; obtaining a first difference between the first data size and the remaining data size, and determining the difference between the first difference and the middle data size as the tail data size; and determining the first data and the second data in the target data based on the header data size, the middle data size, and the tail data size.

[0093] Below, in conjunction with Figure 4 Explain the determination of the data size.

[0094] Figure 4 This is a schematic diagram illustrating the data size provided for embodiments of this disclosure. For example... Figure 4As shown, the middle data can completely occupy the space of at least one disk block. The first starting position is the position of the starting data on the local disk. If the first starting position is located in the middle of any disk block, then header data exists, which can only occupy a portion of the disk block's space. The first starting position is offset, and the preset disk block size is block_size. Therefore, the header data size is head_size, which satisfies head_size = block_size - (offset % block_size), where % represents the remainder. Taking a first starting position of 1300KB, a preset disk block size of 500KB, and a first data size of size 1950KB as an example, head_size = 500 - (1300 % 500) = 200KB. This indicates that the starting data of the target data is located in the 3rd disk block on the local disk and occupies 200KB of that 3rd disk block. The remaining data is 1950 - 200 = 1750KB, meaning the sum of the middle and tail data sizes is 1750KB. The number of disk blocks corresponding to the middle data is block_num = (size - head_size) / block_size. Calculating block_num = (1950 - 200) / 500 and rounding down to the nearest integer gives 3. The size of the middle data is block_size * block_num = 500 * 3 = 1500KB. The size of the tail data is the size of the first data minus the size of the header data and the size of the middle data, i.e., 1950 - 200 - 1500 = 250KB.

[0095] In this feasible implementation, the size of the head data, the size of the middle data, and the size of the tail data can be accurately determined by the first starting position and the preset disk block size.

[0096] Furthermore, one feasible implementation involves determining the first data and the second data in the target data, including: dividing the target data into head data, middle data, and tail data based on the size of the head data, the size of the middle data, and the size of the tail data; determining the head data and / or the tail data as the first data; and determining the middle data as the second data.

[0097] In the data classification example, the disk blocks corresponding to the header and / or tail data can be used to store other data in addition to the target data; therefore, the header and / or tail data are designated as the first data. The disk blocks corresponding to the middle data store only the target data; therefore, the middle data are designated as the second data.

[0098] In this feasible implementation, the division position of the target data can be accurately determined by the size of the header data, the size of the middle data, and the size of the tail data, thereby accurately dividing the header data, the middle data, and the tail data.

[0099] S302. Write the first data to the first disk block corresponding to the log file, and write the second data to the second disk block corresponding to the auxiliary file, so as to write the data write request to the Raft file of the leader node.

[0100] Below, in conjunction with Figure 5 This section explains how data is written to a Raft file.

[0101] Figure 5 This diagram illustrates the writing of data to a Raft file, as provided in an embodiment of this disclosure. Figure 5 As shown, the target data is divided into first data and second data. The first data is written to the first disk block and the second data is written to the second disk block. The Raft file includes a log file and an auxiliary file. The log file corresponds to the first disk block and the auxiliary file corresponds to the second disk block.

[0102] Optionally, the command to write the data to the request is written to the first disk block corresponding to the log file.

[0103] S303, the first disk block mapping relationship is the disk block mapping relationship between the auxiliary file and the local disk of the leader node; the disk block mapping relationship between the virtual disk file and the local disk of the leader node is updated to the first disk block mapping relationship.

[0104] In the example of updating the first disk block mapping, the initial first disk block mapping has a mapping relationship between the auxiliary file and block 1 of the local disk, meaning the auxiliary file can access the data in block 1. In the updated first disk block mapping, the virtual disk file is mapped to block 1 of the local disk, meaning the virtual disk file can access the data in block 1.

[0105] In related technologies, data writing includes writing data to blocks on the local disk corresponding to the Raft file, and copying data to blocks on the local disk corresponding to the virtual disk file, involving two data write operations. In this disclosure, disk block mapping and swapping are used instead of writing data to blocks on the local disk corresponding to the virtual disk file.

[0106] It is understandable that swapping disk block mappings can reduce resource consumption compared to writing data to blocks on the local disk corresponding to the virtual disk file.

[0107] S304. Update the disk block mapping relationship between the auxiliary file and the local disk of the leader node to the second disk block mapping relationship.

[0108] Below, in conjunction with Figure 6 The mapping of swap disk blocks is explained.

[0109] Figure 6 This is a schematic diagram of the swap block mapping provided in an embodiment of this disclosure. Figure 6 As shown, in the initial second disk block mapping, the virtual disk file is mapped to block 2 of the local disk, meaning the virtual disk file can access the data in block 2. In the updated second disk block mapping, the auxiliary file is mapped to block 2 of the local disk, meaning the auxiliary file can access the data in block 2.

[0110] One feasible implementation involves exchanging the first disk block mapping relationship with the second disk block mapping relationship, including: obtaining the target request identifier corresponding to the previous disk block mapping relationship exchange; and, based on the target request identifier, if it is determined that the next request to be exchanged for the disk block mapping relationship is the data write request, then exchanging the first disk block mapping relationship with the second disk block mapping relationship.

[0111] For example, for the target request identifier, the request identifiers corresponding to multiple disk block mapping relationship exchange requests are sorted, and the disk block mapping relationship exchange requests are executed sequentially according to the sorting. By comparing the target request identifier and the sorting of the previous one, the next request identifier is determined. The next request identifier is used to verify whether it is the turn of the data write request. If the next request identifier is the identifier of the data write request, the disk block mapping relationship exchange of the data write request is executed.

[0112] Below, in conjunction with Figure 7 This section explains the request to write verification data.

[0113] Figure 7 This is a schematic diagram illustrating a verification data write request provided in an embodiment of this disclosure. For example... Figure 7 As shown, multiple request identifiers are sorted, and disk block mapping relationship exchange is performed according to the sorting. Executed request identifiers are marked, and the last request identifier is determined from the executed request identifiers and designated as the target request identifier. It is then determined whether the next request identifier of the target request identifier is the request identifier corresponding to the data write request. If so, the disk block mapping exchange corresponding to the data write request is performed.

[0114] In this feasible implementation, sorting verification is used to avoid missed or repeated execution of disk block mapping and swapping, thereby improving the accuracy of disk block mapping and swapping.

[0115] One feasible implementation includes the following: the function input parameters of the disk block mapping exchange interface further include the identifier of the data write request; after exchanging the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface, the implementation further includes: updating the target request identifier to the identifier of the data write request.

[0116] For example, before executing the disk block mapping exchange corresponding to the data write request, the target request identifier is the request identifier corresponding to the previous disk block mapping relationship exchange. After executing the disk block mapping exchange corresponding to the data write request, the data write request has been completed and does not need to be executed again. The target request identifier is updated to the identifier of the data write request to avoid the repeated execution of the data write request.

[0117] In this feasible implementation, by updating the target request identifier, the execution status of the disk block mapping relationship exchange request can be accurately determined. The execution status can be distinguished as executed or not executed, so as to avoid repeated execution of disk block mapping relationship exchange requests with an execution status of executed, thereby avoiding resource waste caused by idempotency problems.

[0118] One feasible implementation involves exchanging the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface based on the first file information, the second file information, and the first data size. This includes: determining the first file information, the second file information, and the first data size as function parameters of the disk block mapping exchange interface, and calling the disk block mapping exchange interface through the user-space file system to exchange the first disk block mapping relationship and the second disk block mapping relationship.

[0119] For example, by determining the first file information, the second file information, and the first data size as function input parameters, the disk block mapping exchange interface can accurately identify the files and disk blocks to be processed by the disk block mapping exchange.

[0120] Specifically, one feasible implementation involves exchanging the first disk block mapping relationship with the second disk block mapping relationship, including: determining the first file information of the virtual disk file, the second file information of the Raft file, and the first data size; the first file information includes the file handle of the virtual disk file and the second starting position of the target write address of the target data in the virtual disk file; the second file information includes the file handle of the Raft file and the first starting position of the persistent address of the target data in the Raft file; and exchanging the first disk block mapping relationship with the second disk block mapping relationship through the disk block mapping exchange interface based on the first file information, the second file information, and the first data size.

[0121] In this context, the file handle of the virtual disk file serves as the unique identifier of the virtual disk file, and the file handle of the Raft file serves as the unique identifier of the Raft file.

[0122] For the handle example, the virtual disk file and Raft file to be swapped are accurately determined by the file handle, the blocks to be swapped are accurately determined by the first and second starting positions, the starting block is determined by the starting position, the number of blocks is determined by the data size, and the blocks to be swapped are determined by the starting block and the number of blocks.

[0123] In this feasible implementation, the virtual disk file and Raft file are identified by the file handle, and the block is identified by the starting position, which can improve the accuracy of disk block mapping and swapping.

[0124] S305. Write the data in the disk block corresponding to the log file to the disk block corresponding to the virtual disk file.

[0125] In the log file example, the data in the disk block corresponding to the log file is fragmented, meaning it doesn't occupy a complete disk block. This fragmented data is stored in a block used for other data. However, disk block mapping and swapping are performed on a block-by-block basis. Performing disk block mapping and swapping on the log file would disrupt the mapping relationships of other data. Therefore, the data in the disk block corresponding to the log file is directly written to the disk block corresponding to the virtual disk file. The data in the log file is copied to the disk block corresponding to the virtual disk file. This copying operation can be repeated, allowing the data in the log file to be replayed repeatedly. In contrast, disk block mapping and swapping cannot be repeated, leading to the problem of data in the auxiliary file not being replayable. Therefore, during disk block mapping and swapping, the replay position is recorded and stored in the log file. By querying the replay position in the log file, the data in the auxiliary file can be replayed, thus avoiding the problem of data in the auxiliary file not being replayable.

[0126] One feasible implementation is that the first data size is smaller than the preset size, and the Raft file includes a log file; the leader node writes the data write request to the leader node's Raft file, including: writing the target data to the log file according to the first data size and the first starting position; the method further includes: determining the disk block corresponding to the virtual disk file, and writing the target data to the disk block corresponding to the virtual disk file.

[0127] For the log file example, if the size of the first data is smaller than the preset size (e.g., the preset size could be 128KB), it means the target data cannot occupy a complete disk block. In this case, the target data is written to the log file, and after receiving the majority response information, the target data is written to the disk block corresponding to the virtual disk file. It is not necessary to determine the first and second data from the target data.

[0128] In this feasible implementation, for scenarios where the size of the first data is smaller than the preset size, the operation of determining the first data and the second data is not performed, which can reduce unnecessary operations.

[0129] The data writing method provided in this embodiment reduces the number of data writing operations by exchanging disk block mappings. The original process of writing data from the disk block corresponding to the Raft file on the local disk to the disk block corresponding to the virtual disk file on the local disk in the state machine is replaced by exchanging the first disk block mapping relationship with the second disk block mapping relationship, thereby improving the efficiency of data writing and reducing the resource consumption required for data writing.

[0130] Exemplary media

[0131] After introducing the methods of exemplary embodiments of this disclosure, the following references are made. Figure 8 The storage medium of the exemplary embodiments of this disclosure will be described.

[0132] refer to Figure 8 As shown, the storage medium 80 stores a program product for implementing the above-described method according to embodiments of the present disclosure. This program product may be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a device such as a personal computer. However, the program product of the present disclosure is not limited thereto.

[0133] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0134] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium.

[0135] Program code for performing the operations disclosed herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0136] Exemplary device

[0137] Having introduced the medium of exemplary embodiments of this disclosure, the following references are made to... Figure 9 The data writing apparatus of the exemplary embodiments of this disclosure will be described to implement the method in any of the above method embodiments. The implementation principle and technical effect are similar, and will not be repeated here.

[0138] refer to Figure 9 , Figure 9 This is a schematic diagram of the structure of a data writing device provided in an embodiment of this disclosure. Figure 9 As shown, it includes:

[0139] Processing module 91 is used to respond to a data write request received by the leader node. The leader node writes the data write request to its Raft file and sends a copy of the data write request to multiple corresponding follower nodes. The Raft file has a first disk block mapping relationship with the leader node's local disk. The data write request includes the target data to be written, the first data size of the target data, and the first starting position of the persistent address of the target data in the Raft file.

[0140] The determination module 92 is used to respond to the majority response information received by the leader node, update the state in the state machine of the leader node based on the majority response information, and determine the second starting position of the target write address of the target data in the virtual disk file. The majority response information consists of multiple response messages sent to the leader node by the majority of follower nodes, and the content of the multiple response messages is consistent. The virtual disk file corresponds to the state machine of the leader node, and the virtual disk file has a second disk block mapping relationship with the local disk of the leader node.

[0141] The writing module 93 is used to exchange the first disk block mapping relationship with the second disk block mapping relationship to complete the writing of the target data.

[0142] Optionally, processing module 91 can execute Figure 2 S201 in the embodiment.

[0143] Optionally, module 92 can be executed. Figure 2 S202 in the embodiment.

[0144] Optionally, the write module 93 can execute... Figure 2 S203 in the embodiment.

[0145] Figure 10 This is a schematic diagram of a data writing device provided in an embodiment of this disclosure. Figure 9 Based on the illustrated embodiments, as Figure 10 As shown, the data writing device 100 further includes: an exchange module 104, an execution module 105, a verification module 106, a parameter module 107, an update module 108, and a judgment module 109, wherein:

[0146] The distributed block storage system also includes a user-space file system and a disk block mapping exchange interface; the exchange module 104 is used to: call the disk block mapping exchange interface through the user-space file system to record the replay position of the Raft file corresponding to this disk block mapping relationship exchange, wherein the replay position includes a replica set identifier and a log replay index, wherein the replica set identifier and the log replay index are function parameters of the disk block mapping exchange interface.

[0147] In one possible implementation, the number of the plurality of follower nodes is N, and the majority response information is sent by M follower nodes, where M is greater than or equal to (N+1) / 2+1, and M and N are positive integers.

[0148] The Raft file includes a log file and an auxiliary file; the execution module 105 is configured to: if the size of the first data is greater than or equal to a preset size, determine the first data and the second data in the target data according to the first starting position and the preset disk block size; the execution module 105 is further configured to write the first data to the first disk block corresponding to the log file and write the second data to the second disk block corresponding to the auxiliary file, so as to write the data write request to the Raft file of the leader node.

[0149] In one possible implementation, the first data includes the header data and / or the tail data of the target data, the size of the header data and the tail data being smaller than the preset disk block size, the size of the header data being the difference between the preset disk block size and a first offset, the first offset being the remainder between the first starting position and the preset disk block size; the second data includes the middle data of the target data, the size of the second data being a positive integer multiple of the preset disk block size.

[0150] In one possible implementation, the execution module 105 is specifically used for:

[0151] The header data size is determined based on the first starting position and the preset disk block size;

[0152] The difference between the first data size and the header data size is determined as the remaining data size;

[0153] The integer part of the quotient of the remaining data size and the preset disk block size is determined as the number of disk blocks;

[0154] The product of the number of disk blocks and the preset disk block size is used to determine the size of the central data.

[0155] Obtain the first difference between the first data size and the remaining data size, and determine the difference between the first difference and the middle data size as the tail data size;

[0156] The first data and the second data are determined in the target data based on the size of the head data, the size of the middle data, and the size of the tail data.

[0157] In one possible implementation, the execution module 105 is specifically used for:

[0158] Based on the size of the head data, the size of the middle data, and the size of the tail data, the target data is divided into head data, middle data, and tail data.

[0159] The header data and / or the tail data are determined as the first data;

[0160] The middle data is identified as the second data.

[0161] In one possible implementation, the first disk block mapping relationship is the disk block mapping relationship between the auxiliary file and the local disk of the leader node; the execution module 105 is specifically used for:

[0162] Update the disk block mapping relationship between the virtual disk file and the local disk of the leader node to the first disk block mapping relationship;

[0163] Update the disk block mapping relationship between the auxiliary file and the local disk of the leader node to the second disk block mapping relationship;

[0164] The data in the disk block corresponding to the log file is written to the disk block corresponding to the virtual disk file.

[0165] The verification module 106 is used to obtain the target request identifier corresponding to the previous disk block mapping relationship exchange; the verification module 106 is also used to, based on the target request identifier, if it is determined that the next request to be exchanged for disk block mapping relationship is the data write request, then exchange the first disk block mapping relationship with the second disk block mapping relationship.

[0166] The parameter module 107 is used to determine the first file information of the virtual disk file, the second file information of the Raft file, and the first data size. The first file information includes the file handle of the virtual disk file and the second starting position of the target write address of the target data in the virtual disk file. The second file information includes the file handle of the Raft file and the first starting position of the persistent address of the target data in the Raft file. The parameter module 107 is also used to exchange the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface according to the first file information, the second file information, and the first data size.

[0167] In one possible implementation, the parameter module 107 is specifically used for:

[0168] The first file information, the second file information, and the first data size are determined as the function input parameters of the disk block mapping exchange interface, and the disk block mapping exchange interface is called through the user-space file system to exchange the first disk block mapping relationship and the second disk block mapping relationship.

[0169] The function input parameters of the disk block mapping exchange interface also include the identifier of the data write request; the update module 108 is used to update the target request identifier to the identifier of the data write request.

[0170] The first data size is smaller than the preset size, and the Raft file includes a log file; the judgment module 109 is used to write the target data to the log file according to the first data size and the first starting position; the judgment module 109 is also used to determine the disk block corresponding to the virtual disk file and write the target data to the disk block corresponding to the virtual disk file.

[0171] Exemplary computing device

[0172] Having described the methods, media, and apparatus of exemplary embodiments of this disclosure, the following references... Figure 11 A computing device according to an exemplary embodiment of the present disclosure will be described.

[0173] Figure 11 The computing device 110 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0174] like Figure 11 As shown, the computing device 110 is presented in the form of a general-purpose computing device. The components of the computing device 110 may include, but are not limited to: at least one processing unit 1101, at least one storage unit 1102, and a bus 1103 connecting different system components (including the processing unit 1101 and the storage unit 1102).

[0175] Bus 1103 includes a data bus, a control bus, and an address bus.

[0176] Storage unit 1102 may include readable media in the form of volatile memory, such as random access memory (RAM) 11021 and / or cache memory 11022, and may further include readable media in the form of non-volatile memory, such as read-only memory (ROM) 11023.

[0177] Storage unit 1102 may also include a program / utility 11025 having a set (at least one) of program modules 11024, such program modules 11024 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0178] The computing device 110 can also communicate with one or more external devices 1104 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 1105. Furthermore, the computing device 110 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1106. Figure 11 As shown, network adapter 1106 communicates with other modules of computing device 110 via bus 1103. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with computing device 110, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0179] It should be noted that although several units / modules or sub-units / modules of the data writing device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0180] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0181] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A data writing method applied to a distributed block storage system, the distributed block storage system conforming to the Raft protocol, characterized in that, include: In response to a data write request received by the leader node, the leader node writes the data write request to the leader node's Raft file and sends a copy of the data write request to the corresponding multiple follower nodes. The Raft file has a first disk block mapping relationship with the leader node's local disk. The data write request includes the target data to be written, the first data size of the target data, and the first starting position of the persistent address of the target data in the Raft file. In response to the majority response information received by the leader node, the state in the leader node's state machine is updated based on the majority response information, and the second starting position of the target write address of the target data in the virtual disk file is determined. The majority response information consists of multiple response messages sent to the leader node by the majority of follower nodes, and the content of the multiple response messages is consistent. The virtual disk file corresponds to the leader node's state machine, and the virtual disk file has a second disk block mapping relationship with the leader node's local disk. The first disk block mapping relationship is exchanged with the second disk block mapping relationship to complete the writing of the target data.

2. The method according to claim 1, characterized in that, The distributed block storage system further includes a user-space file system and a disk block mapping exchange interface; the step of exchanging the first disk block mapping relationship with the second disk block mapping relationship further includes: The user-space file system calls the disk block mapping exchange interface to record the replay position of the Raft file corresponding to this disk block mapping relationship exchange. The replay position includes a replica set identifier and a log replay index, wherein the replica set identifier and the log replay index are function parameters of the disk block mapping exchange interface.

3. The method according to claim 2, characterized in that, The number of the plurality of follower nodes is N, and the majority response information is sent by M follower nodes, where M is greater than or equal to (N+1) / 2+1, and M and N are positive integers.

4. The method according to claim 3, characterized in that, The Raft file includes log files and auxiliary files; The leader node writes the data write request to the leader node's Raft file, including: If the size of the first data is greater than or equal to the preset size, then the first data and the second data are determined in the target data according to the first starting position and the preset disk size; The first data is written to the first disk block corresponding to the log file, and the second data is written to the second disk block corresponding to the auxiliary file, so as to write the data write request to the Raft file of the leader node.

5. The method according to claim 4, characterized in that, The first data includes the header data and / or the tail data of the target data. The data size of the header data and the tail data is smaller than the preset disk block size. The header data size is the difference between the preset disk block size and the first offset. The first offset is the remainder between the first starting position and the preset disk block size. The second data includes the middle part of the target data, and the size of the second data is a positive integer multiple of the preset disk block size.

6. The method according to claim 5, characterized in that, Based on the first starting position and the preset disk size, determine the first data and the second data from the target data, including: The header data size is determined based on the first starting position and the preset disk block size; The difference between the first data size and the header data size is determined as the remaining data size; The integer part of the quotient of the remaining data size and the preset disk block size is determined as the number of disk blocks; The product of the number of disk blocks and the preset disk block size is used to determine the size of the central data. Obtain the first difference between the first data size and the remaining data size, and determine the difference between the first difference and the middle data size as the tail data size; The first data and the second data are determined in the target data based on the size of the head data, the size of the middle data, and the size of the tail data.

7. The method according to claim 6, characterized in that, Determining the first data and the second data from the target data based on the head data size, the middle data size, and the tail data size includes: Based on the size of the head data, the size of the middle data, and the size of the tail data, the target data is divided into head data, middle data, and tail data. The header data and / or the tail data are determined as the first data; The middle data is identified as the second data.

8. The method according to claim 7, characterized in that, The first disk block mapping relationship is the disk block mapping relationship between the auxiliary file and the local disk of the leader node; Exchanging the first disk block mapping relationship with the second disk block mapping relationship includes: Update the disk block mapping relationship between the virtual disk file and the local disk of the leader node to the first disk block mapping relationship; Update the disk block mapping relationship between the auxiliary file and the local disk of the leader node to the second disk block mapping relationship; The data in the disk block corresponding to the log file is written to the disk block corresponding to the virtual disk file.

9. The method according to claim 8, characterized in that, Exchanging the first disk block mapping relationship with the second disk block mapping relationship includes: Obtain the target request identifier corresponding to the last disk block mapping relationship exchange; Based on the target request identifier, if the next request to be exchanged for disk block mapping is determined to be the data write request, then the first disk block mapping relationship and the second disk block mapping relationship are exchanged.

10. The method according to claim 9, characterized in that, Exchanging the first disk block mapping relationship with the second disk block mapping relationship includes: The first file information of the virtual disk file, the second file information of the Raft file, and the first data size are determined. The first file information includes the file handle of the virtual disk file and the second starting position of the target write address of the target data in the virtual disk file. The second file information includes the file handle of the Raft file and the first starting position of the persistent address of the target data in the Raft file. Based on the first file information, the second file information, and the first data size, the first disk block mapping relationship and the second disk block mapping relationship are exchanged through the disk block mapping exchange interface.

11. The method according to claim 10, characterized in that, Based on the first file information, the second file information, and the first data size, the first disk block mapping relationship and the second disk block mapping relationship are exchanged through the disk block mapping exchange interface, including: The first file information, the second file information, and the first data size are determined as the function input parameters of the disk block mapping exchange interface, and the disk block mapping exchange interface is called through the user-space file system to exchange the first disk block mapping relationship and the second disk block mapping relationship.

12. The method according to claim 11, characterized in that, The function parameters of the disk block mapping exchange interface also include the identifier of the data write request; after exchanging the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface, it also includes: Update the target request identifier to the identifier of the data write request.

13. The method according to any one of claims 1-12, characterized in that, The first data size is smaller than the preset size, and the Raft file includes a log file; The leader node writes the data write request to the leader node's Raft file, including: The target data is written to the log file according to the first data size and the first starting position; The method further includes: The disk block corresponding to the virtual disk file is determined, and the target data is written to the disk block corresponding to the virtual disk file.

14. A computer-readable storage medium comprising: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the data writing method as described in any one of claims 1 to 13.

15. A data writing device, applied to a distributed block storage system, the distributed block storage system conforming to the Raft protocol, characterized in that, include: The processing module is used to respond to a data write request received by the leader node. The leader node writes the data write request to the leader node's Raft file and sends a copy of the data write request to the corresponding multiple follower nodes. The Raft file has a first disk block mapping relationship with the leader node's local disk. The data write request includes the target data to be written, the first data size of the target data, and the first starting position of the persistent address of the target data in the Raft file. The determination module is used to respond to the majority response information received by the leader node, update the state in the leader node's state machine based on the majority response information, and determine the second starting position of the target write address of the target data in the virtual disk file. The majority response information consists of multiple response messages sent to the leader node by the majority of follower nodes, and the content of the multiple response messages is consistent. The virtual disk file corresponds to the leader node's state machine, and the virtual disk file has a second disk block mapping relationship with the leader node's local disk. The write module is used to exchange the first disk block mapping relationship with the second disk block mapping relationship to complete the writing of the target data.

16. The apparatus according to claim 15, characterized in that, The distributed block storage system further includes a user-space file system and a disk block mapping exchange interface; the device also includes: The exchange module is used to call the disk block mapping exchange interface through the user-space file system to record the replay position of the Raft file corresponding to this disk block mapping relationship exchange. The replay position includes a replica set identifier and a log replay index, wherein the replica set identifier and the log replay index are function parameters of the disk block mapping exchange interface.

17. The apparatus according to claim 16, characterized in that, The number of the plurality of follower nodes is N, and the majority response information is sent by M follower nodes, where M is greater than or equal to (N+1) / 2+1, and M and N are positive integers.

18. The apparatus according to claim 17, characterized in that, The Raft file includes a log file and auxiliary files; the device further includes: An execution module is configured to determine, based on the first starting position and the preset disk block size, the first data and the second data in the target data if the first data size is greater than or equal to a preset size. The execution module is further configured to write the first data to the first disk block corresponding to the log file and write the second data to the second disk block corresponding to the auxiliary file, so as to write the data write request to the Raft file of the leader node.

19. The apparatus according to claim 18, characterized in that, The first data includes the header data and / or the tail data of the target data. The data size of the header data and the tail data is smaller than the preset disk block size. The header data size is the difference between the preset disk block size and the first offset. The first offset is the remainder between the first starting position and the preset disk block size. The second data includes the middle part of the target data, and the size of the second data is a positive integer multiple of the preset disk block size.

20. The apparatus according to claim 19, characterized in that, The execution module is specifically used to determine the header data size based on the first starting position and the preset disk block size; The execution module is further configured to determine the difference between the first data size and the header data size as the remaining data size; The execution module is further configured to determine the number of disk blocks by the integer part of the quotient of the remaining data size and the preset disk block size; The execution module is further configured to determine the middle data size by multiplying the number of disk blocks by the preset disk block size. The execution module is further configured to obtain a first difference between the first data size and the remaining data size, and to determine the difference between the first difference and the middle data size as the tail data size; The execution module is further configured to determine the first data and the second data in the target data based on the size of the header data, the size of the middle data, and the size of the tail data.

21. The apparatus according to claim 20, characterized in that, The execution module is specifically used to divide the target data into header data, middle data, and tail data according to the size of the header data, the size of the middle data, and the size of the tail data; The execution module is further configured to determine the header data and / or the tail data as the first data; The execution module is further configured to determine the middle data as the second data.

22. The apparatus according to claim 21, characterized in that, The first disk block mapping relationship is the disk block mapping relationship between the auxiliary file and the local disk of the leader node; The execution module is specifically used to update the disk block mapping relationship between the virtual disk file and the local disk of the leader node to the first disk block mapping relationship; The execution module is further configured to update the disk block mapping relationship between the auxiliary file and the local disk of the leader node to the second disk block mapping relationship; The execution module is further configured to write the data in the disk block corresponding to the log file to the disk block corresponding to the virtual disk file.

23. The apparatus according to claim 22, characterized in that, The device includes: The verification module is used to obtain the target request identifier corresponding to the last disk block mapping relationship exchange; The verification module is further configured to, based on the target request identifier, if determine that the next request to be exchanged for disk block mapping is the data write request, then exchange the first disk block mapping relationship with the second disk block mapping relationship.

24. The apparatus according to claim 23, characterized in that, The device further includes: The parameter module is used to determine the first file information of the virtual disk file, the second file information of the Raft file, and the first data size. The first file information includes the file handle of the virtual disk file and the second starting position of the target write address of the target data in the virtual disk file. The second file information includes the file handle of the Raft file and the first starting position of the persistent address of the target data in the Raft file. The parameter module is further configured to exchange the first disk block mapping relationship and the second disk block mapping relationship through the disk block mapping exchange interface based on the first file information, the second file information, and the first data size.

25. The apparatus according to claim 24, characterized in that, The parameter module is specifically used to determine the first file information, the second file information, and the first data size as the function input parameters of the disk block mapping exchange interface, and to call the disk block mapping exchange interface through the user-mode file system to exchange the first disk block mapping relationship and the second disk block mapping relationship.

26. The apparatus according to claim 25, characterized in that, The function parameters of the disk block mapping exchange interface also include the identifier of the data write request; the device also includes: An update module is used to update the target request identifier to the identifier of the data write request.

27. The apparatus according to any one of claims 15-26, characterized in that, The first data size is smaller than the preset size, and the Raft file includes a log file; the device further includes: The judgment module is used to write the target data to the log file according to the first data size and the first starting position; The judgment module is further configured to determine the disk block corresponding to the virtual disk file and write the target data to the disk block corresponding to the virtual disk file.

28. A computing device, comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the computing device to perform the data writing method as described in any one of claims 1 to 13.

Images