Method, device, storage medium and program product for storing, reading and archiving multiple copies of data

Abstract

The embodiment of the invention provides a multi-copy data storage, reading and sealing method and device, a storage medium and a program product. In the embodiment of the invention, the storage nodes with the number larger than that of the copies are provided through a rapid fault-tolerant mechanism for storage fault tolerance, data are written into the copy nodes firstly, and when the failed nodes appear and are within the fault-tolerant range, the data are written into the candidate nodes, so that the success probability of multi-copy storage is improved, and the storage efficiency is improved. The low-speed fault-tolerant mechanism for sealing and storing the data unit does not need to be triggered within the fault-tolerant range, the probability of triggering the fault-tolerant mechanism is reduced, the access impact on a management node (such as an MS) is reduced, and therefore the data writing efficiency is improved, and the writing delay is reduced.

Description

(19) State Intellectual Property Office (12) Invention Patent Application (10) Application Publication Number (43) Application Publication Date (21) Application Number 202411895773.0 (22) Application Date 2024.12.20 (71) Applicant Cloud Intelligent Asset Holdings (Singapore) Pte Ltd Address 51 Bras Basa Road, Singapore, 1 Laizam Tower #03-06 (72) Inventor Zhu Jiaji (74) Patent Agency Beijing Taihe Jiusi Intellectual Property Agency Co., Ltd. 11610 Patent Attorney Zhang Ai Liu Ge (51) Int.Cl. G06F 3 / 06 (2006.01) (54) Invention Title Method, Device, Storage Medium and Program Product for Storing, Reading and Encapsulating Multiple Copy Data (57) Abstract This application provides a method, device, storage medium and program product for storing, reading and encapsulating multiple copy data. In this embodiment, a fast fault-tolerance mechanism is used to provide storage fault tolerance by providing more storage nodes than the number of replicas. Data is first written to the replica nodes. When a node fails but is within the fault tolerance range, data is written to the candidate node. This helps to improve the success probability of multi-replica storage. Within the fault tolerance range, there is no need to trigger the slow fault tolerance mechanism that triggers the sealing of data units, which reduces the probability of triggering the fault tolerance mechanism and reduces the access impact on management nodes (such as MS), thereby improving the efficiency of data writing and reducing write latency. Claims 4 pages, Description 24 pages, Drawings 5 ​​pages, CN 122261465 A 2026.06.23 CN 1 22 26 14 65 A 1. A method for storing multi-replica data, characterized in that it includes: determining target data to be stored and its corresponding target data unit, wherein the target data unit is a basic data unit supporting multi-replica storage; obtaining multiple storage nodes corresponding to the target data unit, wherein the multiple storage nodes include a first number of replica nodes and a second number of candidate nodes, wherein the first number is the number of replicas in multi-replica storage; writing the target data to the first number of replica nodes to perform multi-replica storage of the target data; if a replica node fails to write during the writing process, and the third number of the replica nodes that fail to write is less than or equal to the second number, then writing the target data to at least some of the candidate nodes in the second number of candidate nodes; if the number of candidate nodes that successfully write reaches the third number, returning a message indicating that the multi-replica storage of the target data is completed. 2. The method according to claim 1, wherein when the number of candidate nodes with successful writes reaches the third number, the method further comprises: replacing the replica nodes that failed to write among the first number of replica nodes with the candidate nodes with successful writes;Wherein, the missing data in the candidate node that was successfully written will be filled after the target data is written, and the missing data refers to the data that was already present in the target data unit that was missing on the candidate node. 3. The method according to claim 2, characterized in that it further includes: under the condition of satisfying the scanning conditions, scanning the metadata information of the first number of replica nodes currently corresponding to the target data unit; determining whether there is missing data in the replica node corresponding to the currently scanned metadata information based on the currently scanned metadata information; if so, sending data filling information to the replica node corresponding to the currently scanned metadata information, the data filling information including: the identification information of other storage nodes that store the missing data, so that the replica node corresponding to the currently scanned metadata information can obtain the missing data from the other storage nodes. 4. The method according to claim 3, wherein the metadata information includes: the starting write position information and hole position information corresponding to the target data unit, the hole position information including the hole start offset address and the hole end offset address information, and determining whether there is hole data in the replica node corresponding to the currently scanned metadata information based on the currently scanned metadata information includes: if the hole end offset address information is greater than the hole start offset address information, then it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information, the hole data including the data between the hole start offset address information and the hole end offset address information in the target data unit; and / or in the case that the target data unit is sealed, if the starting write position information is less than the sealing offset address information of the target data unit, then it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information, the hole data including the data between the starting write position information and the sealing offset address information. 5. The method according to claim 1, characterized in that it further comprises: if the number of candidate nodes successfully written does not reach the third number, sending a fault tolerance request to the management node, the fault tolerance request being used to request the management node to seal the target data unit and allocate a new data unit, the new data unit corresponding to at least a first number of new replica nodes; and writing the target data to the first number of new replica nodes corresponding to the new data unit. 6. The method according to claim 5, characterized in that the management node seals the target data unit through the following steps: determining the sealing address range information corresponding to the target data unit on the first number of replica nodes based on the offset address information of the target data in the target data unit; sealing the target data unit based on the sealing address range information.7. The method according to any one of claims 1-6, characterized in that, after writing the target data to the first number of replica nodes, it further includes: counting the number of write success messages returned by the first number of replica nodes; if the number of received write success messages is less than the first number, determining that there are replica nodes that have failed to write, and calculating the difference between the first number and the number of received write success messages as the third number. 8. The method according to any one of claims 1-6, characterized in that, writing the target data to at least some of the candidate nodes in the second number of candidate nodes includes: if the third number is less than the second number, determining the void data and its quantity and / or size existing in each of the second number of candidate nodes, wherein the void data refers to data already present in the missing target data unit on the candidate node; selecting a third number of candidate nodes from the second number of candidate nodes based on the quantity and / or size of the void data existing in each of the second number of candidate nodes; and writing the target data to the third number of candidate nodes. 9. The method according to claim 8, characterized in that, selecting a third number of candidate nodes from the second number of candidate nodes based on the number and / or size of the void data existing in each of the second number of candidate nodes, includes: selecting the third number of candidate nodes from the second number of candidate nodes based on the number and / or size of the void data existing in each of the second number of candidate nodes, combined with the time interval between each of the second number of candidate nodes and its last time as a replica node. 10. A method for storing multi-replica data, characterized in that it is applicable to storage nodes in a storage system, comprising: receiving target data to be stored and address offset information of the target data in its corresponding target data unit; obtaining metadata information corresponding to the target data unit, the metadata information including the current starting write position information and hole position information corresponding to the target data unit, the hole position information describing whether the storage node has hole data and the position information of the hole data in the target data unit if it exists; the hole data refers to data that is missing in the target data unit on the storage node; writing the target data into the target data unit according to the address offset information, and updating the starting write position information according to the address offset information; and updating the hole position information when the current starting write position determined according to the address offset information is greater than the starting write position information in the metadata information before the update. 11. The method according to claim 10, characterized in that updating the hole position information when the current starting write position determined according to the address offset information is greater than the starting write position information in the metadata information before the update comprises:Determine whether the current start write position information determined based on the address offset information is greater than the start write position information in the metadata information before the update; if so, use the start write position information in the metadata information as the hole start position information in the hole position information, and use the current start write position information as the hole end position information in the hole position information to update the hole position information. 12. A method for reading multi-replica data, characterized in that it is applicable to storage nodes in a storage system, the method comprising: receiving a read request, the read request being used to request reading target data from a target data unit on the storage node, and the read request including address offset information of the target data in the target data unit; obtaining metadata information corresponding to the target data unit, the metadata information including the current start write position information and hole position information corresponding to the target data unit, the hole position information describing whether the storage node has hole data and, if so, the position information of the hole data in the target data unit; the hole data referring to data that is missing from the target data unit on the storage node; reading the target data according to the address offset information, the start write position information and the hole position information, and returning the requesting end of the read request. 13. The method according to claim 12, characterized in that reading the target data according to the address offset information, the start write position information, and the hole position information includes: determining whether the target data is existing data in the target data unit according to the start write position information and the address offset information; if so, determining whether the target data includes hole data in the target data unit according to the hole position information and the address offset information; if not, reading the target data from the target data unit according to the address offset information. 14. The method according to claim 13, characterized in that it further comprises: if included, returning a read failure message to the requesting end so that the requesting end can read the target data from other storage nodes; or, if included, sending a data filling request to the management node in the storage system, the data filling request including hole location information so that the management node can return data filling information, the data filling information including: identification information of other storage nodes storing the hole data so that the storage node can obtain the hole data from the other storage nodes to fill the target data unit; reading the target data from the target data unit after the hole data is filled according to the address offset information. 15. A method for reading multi-copy data, characterized in that it comprises:The method receives a read request sent by a service node, the read request being used to request the reading of target data in a target data unit; if the target data unit is not sealed and a write operation is being performed in the target data unit, it determines whether the initiator of the write operation is the service node; if so, it returns the identification information of a first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes. 16. The method according to claim 15, further comprising: if the target data unit has been sealed, returning the identification information of a first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes; or if the target data unit is not sealed and no write operation is currently being performed in the target data unit, after sealing the target data unit, returning the identification information of a first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes. 17. An electronic device, comprising: a memory and a processor; the memory for storing a computer program; the processor, coupled to the memory, for executing the computer program to implement the steps of the method according to any one of claims 1-9, 10-11, 12-14, and 15-16. 18. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, the processor causes the processor to implement the steps of the method according to any one of claims 1-9, 10-11, 12-14, and 15-16. 19. A computer program product comprising a computer program / instructions, characterized in that, when the computer program / instructions are executed by a processor, the processor causes the processor to implement the steps of the method according to any one of claims 1-9, 10-11, 12-14, and 15-16. Claims 4 / 4 Page 5 CN 122261465 A Method, apparatus, storage medium and program product for storing, reading and sealing multiple replica data Technical Field

[0001] This application relates to the field of cloud computing technology, and in particular to a method, apparatus, storage medium and program product for storing, reading and sealing multiple replica data. Background Art

[0002] In distributed storage systems, a combination of multiple replicas and append-only storage is usually used.Storage methods are used to ensure the performance of written data. Multiple replicas refer to storing multiple copies of data at the data block (Chunk) level on different storage nodes during the data storage process to ensure the reliability and availability of the data blocks; append-only storage means that only new data can be appended during data storage, and existing data cannot be modified or deleted to ensure data consistency and integrity.

[0003] In the process of data storage using a combination of multiple replicas and append-only storage, if the storage node responsible for storing data replicas fails, a fault-tolerance mechanism is needed to resolve the failure. One such fault-tolerance mechanism includes: if a storage node fails during multiple replica writes to any Chunk, it reports to the MetaServer (MS); the MS seals the Chunk in read-only mode, then creates a new Chunk, allocates multiple new storage nodes to the new Chunk, and then notifies the data writer to perform multiple replica writes to the multiple storage nodes corresponding to the new Chunk.

[0004] As can be seen from the above, once the above fault tolerance mechanism is triggered, the data write latency will increase. Especially when the abnormal storage node stores a large number of chunk replicas, the MS may need to process a large number of chunk sealing and creation operations in a short period of time, which can easily affect the service quality and stability of the MS. Therefore, a solution is urgently needed to reduce the probability of the above fault tolerance mechanism being triggered. Summary of the Invention

[0005] Several aspects of this application provide a method, device, storage medium and program product for storing, reading and sealing multi-replica data, so as to reduce the probability of triggering the fault tolerance mechanism, improve the efficiency of data writing and reduce the write latency.

[0006] This application embodiment provides a method for storing multiple replica data, including: determining target data to be stored and its corresponding target data unit, wherein the target data unit is a basic data unit that supports multiple replica storage; obtaining multiple storage nodes corresponding to the target data unit, wherein the multiple storage nodes include a first number of replica nodes and a second number of candidate nodes, wherein the first number is the number of replicas in the multiple replica storage; writing the target data to the first number of replica nodes to perform multiple replica storage on the target data; if a replica node fails to write during the writing process, and the third number of replica nodes that fail to write is less than or equal to the second number, then writing the target data to at least some of the candidate nodes in the second number of candidate nodes; if the number of candidate nodes that successfully write reaches the third number, returning a message indicating that the multiple replica storage of the target data is completed.

[0007] This application embodiment also provides a method for storing multiple replica data, applicable to storage nodes in a storage system, including: receiving target data to be stored and address offset information of the target data in its corresponding target data unit;Obtain the metadata information corresponding to the target data unit. The metadata information includes the current start write position information and hole position information corresponding to the target data unit. The hole position information describes whether there is hole data in the storage node and, if so, the location information of the hole data in the target data unit. Hollow data refers to the missing data in the target data unit on the storage node. Write the target data into the target data unit according to the address offset information and update the start write position information according to the address offset information. If the start write position determined by the address offset information is greater than the start write position information in the metadata information before the update, update the hole position information.

[0008] This application embodiment also provides a method for reading multiple replica data, applicable to storage nodes in a storage system. The method includes: receiving a read request, the read request being used to request reading target data from a target data unit on the storage node, and the read request including address offset information of the target data in the target data unit; obtaining metadata information corresponding to the target data unit, the metadata information including the current start write position information and hole position information corresponding to the target data unit, the hole position information describing whether there is hole data on the storage node and the position information of the hole data in the target data unit if it exists; hole data refers to data that already exists in the target data unit that is missing on the storage node; reading the target data according to the address offset information, the start write position information and the hole position information, and returning the requesting end of the read request.

[0009] This application embodiment also provides a method for reading multiple replica data, including: receiving a read request sent by a service node, the read request being used to request reading target data in a target data unit; if the target data unit is not sealed and a write operation is being performed in the target data unit, determining whether the initiator of the write operation is a service node; if so, returning the identification information of a first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes.

[0010] This application embodiment also provides an electronic device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the multiple replica data storage method and the multiple replica data reading method provided in this application embodiment.

[0011] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the steps in the multiple replica data storage method and the multiple replica data reading method provided in this application embodiment.

[0012] This application embodiment also provides a computer program product, including a computer program / instruction, which, when executed by a computer program...When the sequence / instruction is executed by the processor, it causes the processor to implement the steps in the multi-replica data storage method and multi-replica data reading method provided in the embodiments of this application.

[0013] In the embodiments of this application, a storage fault tolerance mechanism is provided to provide more than the number of replicas for storage fault tolerance. Data is first written to the replica nodes. When a failed node occurs and it is within the fault tolerance range, data is written to the candidate node. This is beneficial to improve the success probability of multi-replica storage. Within the fault tolerance range, there is no need to trigger the slow fault tolerance mechanism that triggers the sealing of data units, reducing the probability of triggering the fault tolerance mechanism, reducing the access impact on the management node (such as MS), thereby improving the efficiency of data writing and reducing the write latency. Brief Description of the Drawings

[0014] The accompanying drawings described herein are used to provide a further understanding of this application and constitute a part of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application. In the accompanying drawings:

[0015] FIG1 is a schematic diagram of the structure of a cloud storage system provided by an exemplary embodiment of the present application;

[0016] FIG2 is a schematic diagram of the process of storing multiple copies of data provided by an exemplary embodiment of the present application;

[0017] FIG3a is a schematic diagram of the process of storing multiple copies of data provided by an exemplary embodiment of the present application;

[0018] FIG3b is a schematic diagram of the process of storing another multiple copies of data provided by an exemplary embodiment of the present application; Specification 2 / 24 pages 7 CN 122261465 A

[0019] FIG3c is a schematic diagram of the process of reading multiple copies of data provided by an exemplary embodiment of the present application;

[0020] FIG3d is a schematic diagram of the process of reading another multiple copies of data provided by an exemplary embodiment of the present application;

[0021] FIG4 is a schematic diagram of the structure of a storage device for multiple copies of data provided by an exemplary embodiment of the present application;

[0022] FIG5 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. Detailed Description

[0023] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.Relevant laws, regulations, and standards of relevant countries and regions are provided, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0025] In view of the above-mentioned need for a solution to reduce the probability of the above-mentioned fault tolerance mechanism being triggered, in this application embodiment, for ease of distinction and description, the fault tolerance mechanism provided in this application embodiment is called the fast fault tolerance mechanism. The fast fault tolerance mechanism provides storage fault tolerance by providing more than the number of storage nodes for replicas. Data is first written to the replica nodes. When a node fails and is within the fault tolerance range, data is written to the candidate node. This is beneficial to improve the success probability of multi-replica storage. Within the fault tolerance range, there is no need to trigger the slow fault tolerance mechanism that triggers the sealing of data units, reducing the probability of triggering the fault tolerance mechanism, reducing the access impact on management nodes (such as MS), thereby improving the efficiency of data writing and reducing the writing latency.

[0026] Optionally, when the number of write failures exceeds the fault tolerance range, the slow fault tolerance mechanism is adopted to trigger the sealing of data units and apply for new data units to ensure the correctness and availability of data writing. Among them, the fast fault tolerance mechanism and the slow fault tolerance mechanism complement each other, reducing the dependence on and impact on management nodes (e.g., MS), improving the overall service stability, and reducing the overall engineering implementation complexity.

[0027] Optionally, when the fast fault tolerance mechanism writes data to candidate nodes, it allows temporary data gaps in the candidate nodes, that is, the candidate nodes are missing existing data in the target data unit. The availability of the data fault tolerance mechanism is guaranteed by the gap data, and subsequent candidate nodes can fill the gap data to obtain complete replica data.

[0028] Further, in the absence of write failure, in terms of read and write amplification, for the data included in the data unit, a first number of replicas are stored. If the first number is represented by "W" (W≥1 and is a positive integer), the write amplification ratio is W:1. When reading data, 1 copy of data is read from W copies of data, and the read amplification ratio is 1:1, reducing the consumption of storage resources. Within the fault tolerance range, the write amplification ratio is less than N:1, where N is the total number of replicas of the data unit, N>W, and is a positive integer.

[0029] The following describes in detail a solution provided by an embodiment of this application with reference to the accompanying drawings.

[0030] FIG1 is a schematic diagram of the structure of a cloud storage system provided by an exemplary embodiment of this application. As shown in FIG1, the system includes: a service node 101, a management node 102, and multiple storage nodes 103. Optionally, the cloud storage system includes a service layer and a persistence layer. The service layer includes the service node 101, and the persistence layer includes the management node 102 and multiple storage nodes 103.

[0031] Optionally, the storage system also includes a computing node and a cloud storage access node carried on the computing node.

[0032] In this embodiment, a cloud storage access node is deployed on the computing node. The cloud storage access node can be described as page 3 / 24 of the specification.8 CN 122261465 A The service processes of a cloud storage system are deployed on compute nodes to provide access services to cloud computing instances (e.g., virtual machines or containers) running on these nodes. Specifically, the cloud storage access node can present cloud storage disks provided by the cloud storage system to the cloud computing instances through virtualization technology; users can send input / output (IO) read / write requests to the cloud computing instances, and the cloud computing instances send IO read / write requests to the cloud storage access node; the cloud storage access node can receive IO read / write requests from the cloud computing instances to the cloud storage disks, send the IO read / write requests to the backend storage cluster (e.g., service layer and persistence layer) corresponding to the cloud storage disks, and return the response results to the cloud computing instances, such as virtual machines or containers, after receiving the response results from the storage clusters. The cloud storage access node can interact with the cloud computing instances through the virtualization manager on the compute node that manages the cloud computing instances. The virtualization manager can be a Virtual Machine Monitor (VMM), also known as a Hypervisor. Optionally, the Hypervisor or VMM can be implemented as a virtual operating system emulator (Quick EMUlator, QEMU) or cloud computing and platform virtualization software, but is not limited thereto.

[0033] The service node 101 provides data interaction services for the cloud storage disk, for example, it is responsible for handling data read and write requests from upper-layer services, such as persisting data to storage node 103 or reading data from storage nodes. For example, upper-layer services may include, but are not limited to, block storage services, file storage services, and object storage services. Optionally, the service node 101 includes a software development kit (SDK). The service node 101 interacts with the management node 102 and storage node 103 through the SDK. The number of service nodes 101 can be one or more. In Figure 1, the example is that the number of service nodes 101 is multiple.

[0034] The management node 102 is responsible for managing the metadata of the data unit. Optionally, the management node 102 can provide management and control services for the data unit, such as being responsible for the creation or deletion of the data unit. A data unit is a basic form of data organization in a storage system. A data unit can exist independently and maintain its integrity during processing, transmission, or storage. Data units can include, but are not limited to, physical blocks, data chunks, pages, files, and sectors. Preferably, a data unit is implemented as a data chunk.

[0035] Storage node 103 is used to persist the data included in the data unit.

[0036] In specific deployment implementation, management node 102 and storage node 103 can be deployed together on a physical device (e.g., the same storage device), or management node 102 and storage node 103 can be deployed on different physical devices. Figure 1 illustrates an example where management node 102 and storage node 103 are deployed on different physical devices.

[0037] In this embodiment, the implementation of service node 101, management node 102, and multiple storage nodes 103 is not limited. Figure 1 illustrates an example where service node 101 is a data server (Block Server, BS), management node 102 is a metadata server (Meta Server, MS), and storage node 103 is a block server (Chunk Server, CS), but it is not limited to this.

[0038] In this embodiment, for ease of distinction and description, the storage of multiple replicas of data for a target data unit is used as an example. The target data unit is a basic data unit that supports multiple replica storage. For an introduction to the target data unit, please refer to the aforementioned introduction to data units, which will not be repeated here. Among them, multiple replicas refer to storing multiple copies of data units (e.g., data blocks, chunks) on different storage nodes during the data storage process to ensure the reliability and availability of the data. It should be noted that the target data unit includes multiple replicas, and different replicas are stored on different storage nodes. The "target data unit included on the storage node" described in the following process actually refers to the storage node storing the replica corresponding to the target data unit.

[0039] In one storage method, in addition to following the multiple replica principle, the data storage of the data unit also includes the append-only storage principle. Append-only storage means that when storing data for the data unit, only new data can be appended, and existing data in the data unit cannot be modified or deleted to ensure the consistency and integrity of the data.

[0040] In one storage method, the target data unit can be written by a service node. If the service node restarts or disconnects, the target data unit can be automatically sealed by the management node and no more data can be written to the target data unit.

[0041] In this embodiment, the service node 101 can determine the target data to be stored and its corresponding target data unit. The implementation method for determining the target data to be stored and its corresponding target data unit is not limited. The target data to be stored can be provided to the service node 101 by a computing node. The service node 101 receives the target data and determines the target data unit corresponding to the target data. For example, the service node 101 can request the target data from the management node 102.The management node 102 allocates a target data unit for the target data and returns multiple storage nodes corresponding to the target data unit to the service node 101, as shown in step ① of Figure 1. These multiple storage nodes include a first number of replica nodes and a second number of candidate nodes, as shown in step ① of Figure 1. The first number of replica nodes are the storage nodes from which the target data is written by default. Typically, if data is being stored for the target data unit for the first time, the first number of storage nodes can be randomly selected from the multiple storage nodes corresponding to the target data unit as the first number of replica nodes; if data is not being stored for the target data unit for the first time, the first number of storage nodes from the previous successful storage of the target data unit are used as the first number of replica nodes. Other storage nodes besides the first number of replica nodes can be used as the second number of candidate nodes. The sum of the first and second numbers determines the number of storage nodes corresponding to the target data unit. For example, the target data unit corresponds to N storage nodes, where N is a positive integer and N≥1. N can be 5 or 7, etc., without limitation. The first quantity can be W, where W is a positive integer and W < N. The second quantity can be N - W. For example, N = 5 and W = 3.

[0042] In this embodiment, the service node can write target data to the first number of replica nodes, as shown in step ② of Figure 1. The first quantity is the replica data of the multi-replica storage, that is, if the target data is successfully written to the first number of storage nodes, it is considered that the multi-replica storage of the target data is successful. The second quantity represents the number of storage nodes that are allowed to tolerate write failures while ensuring multi-replica storage. For example, the second number of storage nodes are allowed to fail to write, thereby ensuring that the first number of replica data exists. It should be noted that the storage nodes that fail to write can include replica nodes that fail to write, or candidate nodes that fail to write. Based on this, the service node 101 can write target data to the first number of replica nodes to perform multi-replica storage of the target data. If the first number of replica nodes all successfully write the target data, it is considered that the multi-replica storage of the target data is completed, and the first number of replica nodes can return a message to the service node indicating that the multi-replica storage of the target data is completed.

[0043] During the process of writing target data to the first number of replica nodes, there may be cases where the target data writing fails. The reasons for target data writing failure may include, but are not limited to, the following: 1) a network failure between the service node and the replica nodes; 2) hardware failure, software failure, or resource exhaustion of the replica nodes; 3) permission issues, for example, the replica nodes rejecting write requests from the service node due to the service node lacking the necessary permissions.

[0044] In this embodiment, if there are replica nodes among the first number of replica nodes where writing fails, ifIf the third number of failed write replica nodes is less than or equal to the second number, then the target data is written to at least some of the candidate nodes among the second number of candidate nodes, as shown in step ③ of Figure 1. For example, the third number is K, where K is a positive integer and 1≤K≤N. For example, K can be 1, 2, or 5, which is not limited. Here, for ease of distinction and description, the number of failed write replica nodes is referred to as the third number.

[0045] The implementation method of writing the target data to at least some of the candidate nodes among the second number of candidate nodes is not limited. For example, a third number of candidate nodes are selected from the second number of candidate nodes, and the target data is written to the third number of candidate nodes. If there are candidate nodes that have failed to write among the third number of candidate nodes, then the candidate nodes for writing the target data are selected from the remaining candidate nodes. For another example, the target data can be written to all the second number of candidate nodes.

[0046] In this embodiment, if a candidate node successfully writes during the writing process, and the number of candidate nodes successfully written reaches a third number, the third number of candidate nodes successfully writing the target data, and the replica nodes successfully writing the target data can constitute a first number of replica data to determine that the multi-replica storage of the target data is completed. Furthermore, the service node can return a message to the upper-layer caller indicating that the multi-replica storage of the target data is completed. For example, the upper-layer caller can be a computing node, or various storage services deployed on the service node.

[0047] In this embodiment, by providing more storage nodes than the number of replicas for storage fault tolerance, data is first written to the replica nodes. When a failed node occurs and is within the fault tolerance range, data is written to the candidate nodes. This is beneficial to improving the success probability of multi-replica storage. Within the fault tolerance range, there is no need to trigger the above-mentioned fault tolerance mechanism that triggers the sealing of data units, reducing the probability of triggering the fault tolerance mechanism, thereby improving the efficiency of data writing and reducing writing latency.

[0048] In an optional embodiment, when the number of candidate nodes that have successfully written data reaches a third number, the replica nodes that failed to write data in the first number of replica nodes are replaced with candidate nodes that have successfully written data, so that new data is written to the replaced first number of replica nodes when multiple replica storage is performed on the target data unit next time.

[0049] Wherein, when multiple replica storage is performed on the target storage unit, data is first written to the first number of replica nodes. If there are replica nodes that failed to write data, then data is written to the second number of candidate nodes. When the first number of data replicas are successfully written, there is no need to write data to the second number of candidate nodes, which may result in gaps in the second number of candidate nodes. These gaps refer to missing data in the target data unit that is missing from the candidate nodes.

[0050] For example, data B1 is first written to a first number of replica nodes. If the first number of replica nodes successfully write data B1, then there is no need to write data B1 to a second number of candidate nodes. Data B2 is written to the first number of replica nodes. If there are replica nodes that fail to write data, then data B2 is written to at least some of the candidate nodes in the second number of candidate nodes. If there is a candidate node that successfully writes data B2, the candidate node is missing data B1, that is, the candidate node includes empty data, i.e., data B1.

[0051] Wherein, if there is empty data in the candidate node that has been successfully written, the empty data can be filled after the target data is written, so that the candidate node that has been successfully written includes all the data in the target data unit, so as to obtain the multi-replica data corresponding to the target data unit, thereby improving the accuracy and efficiency of subsequent data reading.

[0052] Optionally, how to determine whether there is gapped data in a storage node, and how to fill the gapped data when there is gapped data in a storage node, are problems that need to be solved. An exemplary implementation method X1 is provided below. Under the condition of satisfying the scanning conditions, the metadata information of the first number of replica nodes corresponding to the target data unit is scanned; based on the currently scanned metadata information, it is determined whether there is gapped data in the replica node corresponding to the currently scanned metadata information; if so, data filling information is sent to the replica node corresponding to the currently scanned metadata information. The data filling information includes: the identification information of other storage nodes storing gapped data, so that the replica node corresponding to the currently scanned metadata information can obtain the gapped data from other storage nodes. Wherein, when determining whether there is gapped data in a replica node, by setting reasonable scanning conditions, frequent and comprehensive checks on all replica nodes can be avoided, thereby saving resources and time. When it is determined that there is gapped data in a replica node, obtaining the missing gapped data from other storage nodes can ensure the integrity and availability of the target data unit.

[0053] Wherein, the executing entity of implementation method X1 is not limited. For example, implementation X1 can be implemented entirely on service node 101, or entirely on management node 102, or it can be implemented by service node 101 and management node 102 in combination with the specification page 6 / 24 11 CN 122261465 A, without limitation.

[0054] In the case where implementation X1 is implemented by service node 101 and management node 102 in combination, service node 101 can perform the following: when the scanning conditions are met, scan the metadata information of the first number of replica nodes corresponding to the target data unit; determine whether there is empty data in the replica node corresponding to the currently scanned metadata information based on the currently scanned metadata information; if so, notify management node 102, and management node 102 will notify the currently scanned metadata information of the first number of replica nodes.Data filling information is sent to the replica node corresponding to the information. The data filling information includes: identification information of other storage nodes that store void data, so that the replica node corresponding to the currently scanned metadata information can obtain void data from other storage nodes. Alternatively, if the scanning conditions are met, the management node 102 scans the metadata information of the first number of replica nodes currently corresponding to the target data unit; based on the currently scanned metadata information, it determines whether there is void data in the replica node corresponding to the currently scanned metadata information; if so, it notifies the service node 101, which then sends data filling information to the replica node corresponding to the currently scanned metadata information. The data filling information includes: identification information of other storage nodes that store void data, so that the replica node corresponding to the currently scanned metadata information can obtain void data from other storage nodes.

[0055] Wherein, the method of scanning the metadata information of the multiple replica nodes currently corresponding to the target data unit is not limited when the scanning conditions are met. For example, if the scanning condition is that the target data unit is sealed, then the metadata information of the multiple replica nodes currently corresponding to the target data unit is scanned every set time. For example, scanning conditions include, but are not limited to: the occurrence of a data storage event, a data storage completion event, and a replica node replacement operation. When the scanning conditions are met, the metadata information of multiple replica nodes corresponding to the target data unit is scanned.

[0056] The identification information of the storage node includes, but is not limited to: hostname, IP address, port number, and globally unique identifier (UUID). Other storage nodes storing void data can be replica nodes or candidate nodes; any storage node capable of reading void data is acceptable. The replica node corresponding to the currently scanned metadata information can receive data filling information sent by the management node 102, parse the data filling information, and obtain the identification information of other storage nodes. If the identification information of one other storage node is obtained, void data is directly requested from that other storage node based on that identification information. If the identification information of multiple other storage nodes and the health scores of multiple other storage nodes are obtained, the other storage node with the highest health score can be selected, and void data is requested from that other storage node based on its identification information.

[0057] The replica node corresponding to the currently scanned metadata information includes metadata information of the target data unit. This metadata information includes: hole location information, which describes whether there is hole data in the storage node and, if so, the location information of the hole data in the target data unit, i.e., the address of the hole data in the target data unit.Offset information. Multiple storage nodes corresponding to the target data unit store the target data unit according to the same address offset information. Therefore, other storage nodes can obtain the hole data from the currently stored target data unit based on the address offset information and provide it to the replica node. The replica node receives the hole data and fills the hole data into the target data unit according to the address offset information of the hole data.

[0058] Optionally, the metadata information included on the replica node includes not only the hole location information but also: the current starting write position information corresponding to the target data unit. The current starting write position information refers to the latest offset address information of the target data unit that has been persisted. When data needs to be persisted, persistence is defaulted to starting from this starting write position information.

[0059] The implementation method for determining whether there is hole data in the replica node corresponding to the currently scanned metadata information is not limited. An exemplary description follows. Instruction manual, page 7 / 24, 12 CN 122261465 A

[0060] Example D1: Determine whether the hole termination offset address information is greater than the hole start offset address information. If the hole termination offset address information is greater than the hole start offset address information, it indicates that the length of the hole is not 0. Then, it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information. The hole data includes the data between the hole start offset address information and the hole termination offset address information in the target data unit.

[0061] Example D2: When the target data unit is sealed, determine whether the start write position information is less than the sealing offset address information of the target data unit. If the start write position information is less than the sealing offset address information of the target data unit, it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information. The hole data includes the data between the start write position information and the sealing offset address information.

[0062] Example D3 includes both Example D1 and Example D2. That is, when the target data unit is sealed, if the hole termination offset address information is greater than the hole start address offset information, and the start write position information is less than the sealing offset address information of the target data unit, then it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information. The hole data includes the data between the hole start address offset information and the hole termination offset address information in the target data unit, and the data between the start write position information and the sealing offset address information.

[0063] Optionally, when the target data unit is sealed, if the hole termination offset address information is equal to the hole start address offset information, and the start write position information is greater than or equal to the sealing offset address information of the target data unit, then it is determined that there is no hole data in the replica node corresponding to the currently scanned metadata information, and the currently sealed target data unit...The element is a complete copy, meaning that the currently scanned copy node includes a complete copy.

[0064] Further optionally, the management node 102 can also determine at set intervals whether the number of complete copies corresponding to the target data unit is less than a first number. If the number of complete copies corresponding to the target data unit is less than the first number, a copy completion task for the target data unit is initiated. The set interval can include, but is not limited to, 1 minute, 1 day, 1 month, or 6 months. The copy completion task can be initiated to candidate nodes first. If the candidate nodes cannot execute the copy completion task, a new storage node is allocated to the target data unit, and the copy completion task is executed through the new storage node. When the management node 102 initiates a copy completion task to a candidate node, it can send data completion information to the candidate node. The data completion information includes the identification information of the copy node to which the complete copy belongs. The candidate node can obtain the complete data of the target data unit from the copy node to which the complete copy belongs or complete its own missing data according to the identification information to obtain a complete copy, and notify the management node 102 that the data completion task has been completed.

[0065] Further optionally, when initiating a replica replenishment task, multiple storage nodes may simultaneously replenish replicas to obtain complete replicas, or previously faulty or abnormal replica nodes may recover and have complete data. This may result in the number of replicas of the target data unit exceeding the first number, thus wasting resources. Based on this, the management node 102 can also determine whether the number of complete replicas corresponding to the target data unit is greater than the first number. If the number of complete replicas corresponding to the target data unit is greater than the first number, in order to save storage space, the management node 102 selects replicas to be reclaimed from the complete replicas and performs reclamation processing on these replicas to obtain the first number of complete replicas, thereby efficiently utilizing storage resources. Specifically, the performance information of each storage node to which a complete replica belongs can be determined based on the load information, network information, etc., and the storage nodes that need to be reclaimed for the complete replicas can be selected from them in order of performance information from low to high.

[0066] In an optional embodiment, the service node 101 can monitor the number of candidate nodes that have successfully written data. If the number of candidate nodes that have successfully written data does not reach a third number, it indicates that the target data unit cannot obtain a first number of replicas. Then, a fault tolerance request is sent to the management node 102. This fault tolerance request requests the management node 102 to seal the target data unit and allocate a new data unit. Wherein, if the target data unit cannot obtain a first number of replicas, the fault tolerance mechanism that triggered the sealing of the data unit will be returned to ensure the correctness and availability of the storage service. Specification 8 / 24 pages 13 CN 122261465 A

[0067] For example, the fault tolerance request includes the identification information of the target data unit and the data unit creation information. Management node102. The management node 102 seals the target data unit according to the identification information of the target data unit, and creates a new data unit and its corresponding multiple storage nodes, including a first number of replica nodes and a second number of candidate nodes. The management node 102 can provide the identification information of the new data unit and its multiple storage nodes to the service node 101. The service node 101 writes the target data to the first number of new replica nodes corresponding to the new data unit. For the specific implementation of writing the target data, please refer to the above description of writing the target data to the first number of replica nodes corresponding to the target data unit, which will not be repeated here.

[0068] Optionally, the implementation of the management node sealing the target data unit is not limited. The following is an example: the management node 102 determines the sealing address range information corresponding to the target data unit on the first number of replica nodes according to the offset address information of the target data in the target data unit; and seals the target data unit according to the sealing address range information.

[0069] Where the number of candidate nodes successfully written to the target data has not reached the third number, it means that the number of storage nodes successfully written to the target data has not reached the first number. It can be considered that the multi-copy storage of the target data has not been completed. During the sealing process of the target data unit, the target data cannot be sealed. Therefore, sealing the successfully written data in the target data unit before the target data is written can ensure the consistency and integrity of this part of the data. Even if the subsequent operation fails, the sealed data is still reliable. For example, the starting offset address information of the target data can be used as the sealing termination address information of the target data unit, and the sealing start address information of the target data unit can be used as the starting address information of the target data unit. For example, the starting address information can be 0. The sealing address range information can be determined from the sealing start address information and sealing termination address information of the target data unit. Wherein, the address information can also be randomly selected from the starting offset address information of the target data in the target data unit as the sealing termination address information, which is not limited.

[0070] For example, the target data unit corresponds to 5 storage nodes, 3 replica nodes, and 2 candidate nodes. The target data unit successfully writes data F1, data F2, and data F3 to 3 replica nodes. When writing data F4, only 2 of the 5 storage nodes successfully write the data. Since data F4 has fewer than 3 replicas, data F4 cannot be sealed. Instead, data F1, data F2, and data F3 are sealed. That is, the sealed target data unit includes data F1, data F2, and data F3. The sealed target data unit corresponds to 3 replica nodes.

[0071] In an optional embodiment, after writing the target data to a first number of replica nodes, if the replica node successfully writes the target data, it returns a write success message to the service node 101. Optionally, if the target data is not successfully written,A write failure message can be returned to service node 101, or no message can be returned. Based on this, after writing the target data to the first number of replica nodes, service node 101 can also count the write success messages returned by the first number of replica nodes; if the number of received write success messages is less than the first number, it is determined that there are replica nodes that have failed to write, and the difference between the first number and the number of received write success messages is calculated as the third number. In this way, by counting the write success messages, the service node can clearly know which replicas have been successfully written and which have not yet been completed, which helps to monitor the health status of the system in real time and take timely measures to deal with potential problems.

[0072] For example, after the service node writes the target data to the first number of replica nodes, it can set a deadline for receiving write success messages, and after the deadline is reached, count the number of received write success messages. The deadline is a set time interval after writing the target data to the first number of replica nodes, and the set time can include, but is not limited to: 30s, 1min or 2min.

[0073] In an optional embodiment, the implementation method of the service node 101 writing target data to at least some of the candidate nodes among the second number of candidate nodes is not limited. When the third number is less than the second number, the void data and its quantity and / or size existing in each of the second number of candidate nodes are determined. Void data refers to the data already existing in the missing target data unit on the candidate node. The size of the void data refers to the size of the void data, which may specifically be the length information of the offset address corresponding to the void data. According to the quantity and / or size of the void data existing in each of the second number of candidate nodes, a third number of candidate nodes are selected from the second number of candidate nodes. The target data is written to the third number of candidate nodes. Among them, selecting a suitable third number of candidate nodes according to the size or quantity of void data enables the system to better adapt to different workloads and environmental changes.

[0074] Among them, the implementation method of the service node 101 determining the void data and its quantity and / or size existing in each of the second number of candidate nodes is not limited. For example, service node 101 can obtain metadata information of a second number of candidate nodes, and determine whether there is any void data in each candidate node and the number of void data based on the metadata information. If void data exists, the size of the void data is determined based on the void location information in the metadata information.

[0075] It should be noted that the number of void data included in a storage node is not limited; for example, it can be one or more. When a candidate node includes multiple void data, the size information of the void data corresponding to the candidate node can be the sum of the size information of the multiple void data.

[0076] The implementation method of selecting a third number of candidate nodes from the second number of candidate nodes based on the number and / or size of the hole data in each of the second number of candidate nodes is not limited.

[0077] For example, considering that the hole data needs to be filled later, in order to reduce the amount of data to fill the hole data, candidate nodes with smaller hole data sizes are selected. For example, the third number of candidate nodes are selected from the second number of candidate nodes in order of increasing hole data size.

[0078] For another example, considering that the hole data needs to be filled later, in order to reduce the number of interactions between filling the hole data and other storage nodes, candidate nodes with smaller hole data are selected. For example, the third number of candidate nodes are selected from the second number of candidate nodes in order of increasing hole data size.

[0079] For yet another example, based on the number of hole data in the candidate nodes, candidate nodes with a number of hole data less than a set hole number threshold are selected. If the number of selected candidate nodes is greater than the third number, the third number of candidate nodes are selected from them in order of increasing hole data size.

[0080] For example, based on the size of the hole data of the candidate nodes, candidate nodes whose hole data size is smaller than a set hole size threshold are selected. If the number of selected candidate nodes is greater than a third number, then a third number of candidate nodes are selected from them in ascending order of the number of hole data.

[0081] Optionally, the implementation of selecting a third number of candidate nodes from the second number of candidate nodes based on the number and / or size of the hole data of each of the second number of candidate nodes is not limited. The following is a specific implementation: based on the number and / or size of the hole data of each of the second number of candidate nodes, combined with the time interval between the second number of candidate nodes and their last time as replica nodes, a third number of candidate nodes are selected from the second number of candidate nodes. Among them, when selecting storage nodes based on the time interval, storage nodes with a longer time interval can be selected to ensure that the workload of each node is relatively balanced and to avoid some nodes bearing too many tasks for a long time, resulting in performance degradation. Alternatively, storage nodes with a shorter time interval can be selected to reduce the hole data and reduce the resource consumption for filling the hole data later.

[0082] For example, if, based on the number of void data present in each of the second number of candidate nodes, candidate nodes with a void data number less than a set void number threshold are selected, and if the number of selected candidate nodes is greater than a third number, then the time interval between the selected candidate node and its last time as a replica node is determined, and a third number of candidate nodes are selected from the selected candidate nodes in ascending order of time interval.

[0083] As another example, if, based on the size of the void data present in each of the second number of candidate nodes, void data is selected... (The rest of the text appears to be a reference to a document or specification, possibly related to a document or specification, and is not translated.)If the size of the hole data is smaller than the set hole size threshold, and the number of selected candidate nodes is greater than the third number, then the time interval between the selected candidate node and its last time as a replica node is determined, and the third number of candidate nodes are selected from the selected candidate nodes in ascending order of time interval.

[0084] For example, if the size of the hole data is smaller than the set hole size threshold and the number of hole data is less than the set hole number threshold, and the number of selected candidate nodes is greater than the third number, then the time interval between the selected candidate node and its last time as a replica node is determined, and the third number of candidate nodes are selected from the selected candidate nodes in ascending order of time interval.

[0085] The following is an exemplary description of the process of multi-replica data storage. In Figure 2, taking a target data unit corresponding to 5 storage nodes as an example, the 5 storage nodes are represented as storage node R1, storage node R2, storage node R3, storage node R4 and storage node R5. Among them, the 3 replica nodes are storage node R1, storage node R2 and storage node R3, and the 2 candidate nodes are storage node R4 and storage node R5. Using the metadata information of the target data units included on each storage node<MaxSeq,LagBeginSeq,LagEndeq> Taking an example, MaxSeq represents the starting write position information corresponding to the target data unit, LagBeginSeq and LagEndeq represent the hole position information, with LagBeginSeq being the hole start position information and LagEndeq being the hole end position information. When no data is written to the target data unit, the metadata information of storage nodes R1 to R5 is all <0,0,0>. MaxSeq, LagBeginSeq, and LagEndeq are illustrated by sequence numbers, which reflect offset address information. This is illustrated using the example of one sequence number for each data item, but is not limited to this.

[0086] In the first round, the service node writes data Q1 to the target data unit. Data Q1 is written to the three replica nodes of the service node, namely storage nodes R1, R2, and R3, and all writes are successful. At this time, the metadata information of storage nodes R1, R2, and R3 is all <1,0,0>.

[0087] In the second round, the service node writes data Q2 into the target data unit. Data Q2 is also written into the three replica nodes of the service node, namely storage nodes R1, R2, and R3, and all writes are successful. At this time, the metadata information of storage nodes R1, R2, and R3 is <2,0,0>.

[0088] In the third round, the service node writes data Q3 into the target data unit. Data Q2 is written into the three replica nodes of the service node, namely storage nodes R1, R2, and R3, and all writes are successful.Data Q3 is written to storage nodes R1, R2, and R3, and all writes are successful. At this time, the metadata information of storage nodes R1, R2, and R3 is <3,0,0>.

[0089] In the fourth round, the service node writes data Q4 to the target data unit. Data Q4 is written to the three replica nodes of the service node, namely storage nodes R1, R2, and R3. Among them, storage nodes R1 and R2 successfully write data Q4, while storage node R3 fails, and the write of data Q4 fails. The service node writes the target data to the candidate node, such as storage node R4, and the write is successful. At this time, storage node R4 has empty data, which includes data Q1, data Q2, and data Q3. At this time, the three replica nodes can be updated to storage nodes R1, R2, and R4. At this time, the metadata information of storage nodes R1 and R2 is <4,0,0>. The metadata information of storage node R3 is <3, 0,0>, and the metadata information of storage node R4 is <4,0,3>.

[0090] In the fourth round, the service node writes data Q5 into the target data unit. Data Q5 is written into the three replica nodes of the service node, namely storage node R1, storage node R2, and storage node R4. Among them, storage node R1 and storage node R2 successfully write data Q5, while storage node R4 fails to write. At the same time, storage node R3 recovers from the failure and can be used as a candidate node. The service node selects one of storage node R3 and storage node R5. Storage node R3 includes missing data, i.e., missing data Q4, while storage node R5 includes missing data, which includes data Q1, data Q2, data Q3, and data Q4. The number of missing data in storage node R3 is less than the number of missing data in storage node R5. The service node selects storage node R3 to write data Q5. Update the three replica nodes to storage nodes R1, R2, and R3. The metadata information for storage nodes R1 and R2 is <5,0,0>. The metadata information for storage node R3 is <5,3,4>, and the metadata information for storage node R4 is <4,0,3>. It should be noted that a storage node can include one or more empty data points. When a storage node includes multiple empty data points, the metadata information can be represented as follows: <MaxSeq , (LagBeginSeq1,LagEndeq1),(LagBeginSeq2,LagEndeq2) ,…,(LagBeginSeqT,LagEndeqT)>. Wherein, (LagBeginSeq1, LagEndeq1) represents the first hole data, (LagBeginSeq2, LagEndeq2) represents the second hole data, (LagBeginSeqT, LagEndeqT) represents the Tth hole data, T is a positive integer, and T≥1.

[0091] In the embodiments of this application, when the service node 101 writes target data to the storage node, the storage node can be any replica node or candidate node, and the service node 101 can provide the target data to be stored and its location information in the target data unit to the storage node. The implementation method of the storage node writing target data is not limited. The following is an exemplary implementation method of the storage node storing target data.

[0092] Specifically, the storage node 103 receives the target data to be stored and the address offset information of the target data in its respective target data unit; obtains the metadata information corresponding to the target data unit, which includes the starting write position information and hole position information corresponding to the target data unit. The hole position information describes whether there is hole data in the storage node and, if so, the position information of the hole data in the target data unit; writes the target data into the target data unit according to the address offset information, and updates the starting write position information according to the address offset information to obtain the updated metadata information; and updates the hole position information when the starting write position information determined according to the address offset information is greater than the starting write position information in the metadata information before the update. By maintaining the hole position information in the metadata information, the position of the hole data can be accurately and efficiently known, facilitating subsequent filling of the hole data.

[0093] Optionally, if the starting write position information is equal to the starting write position information in the metadata information before the update, it indicates that there is no hole data in the target data unit, and therefore, it is not necessary to update the hole position information; updating the starting write position information in the metadata information is sufficient.

[0094] The address offset information includes the current start write position information and the current end write position information. The address offset information can be parsed to obtain the current start write position information and the current end write position information.

[0095] Optionally, the implementation method of updating the start write position information based on the address offset information is not limited. An exemplary implementation method Y1 is provided below, in which target data is written into the target data unit of the storage node. At this time, the metadata information of the target data unit can be updated, using the current end write position information as the start write position information in the metadata information to update the metadata information. By setting the end position of the previous write operation as the start position of the next write, the continuity and integrity of the data writing process are ensured, and data fragmentation is avoided.

[0096] Optionally, the implementation of updating hole position information is not limited to the case where the current starting write position determined according to the address offset information is greater than the starting write position information in the metadata information. A specific implementation Y2 is provided below: determining whether the current starting write position information is greater than the starting write position information in the metadata before update; if yes, taking the starting write position information in the metadata information as the hole start position information in the hole position information, and taking the current starting write position information as the hole end position information in the hole position information, so as to update the hole position information in the metadata information. Wherein, this method can automatically detect data holes caused by various reasons (such as write interruption, failure or concurrency conflict), accurately identify the specific range of the hole, and provide clear guidance for subsequent repair operations.

[0097] It should be noted that the execution order of implementation Y1 and implementation Y2 is not limited. Implementation Y1 may be executed first, and then implementation Y2, or implementation Y2 may be executed first, and then implementation Y1. Description 12 / 24 pages 17 CN 122261465 A

[0098] A scenario-based implementation for a storage node to write target data is provided below.

[0099] The service node 101 sends target data and its address offset information to the storage node, wherein the target data and its address offset information can be represented as <BeginSeq ,EndSeq ,Data> through a newly added data writing protocol, wherein BeginSeq and EndSeq are respectively the current starting write position and the current ending write position included in the address offset information, and Data is the target data. Further, the service node also provides the identification information of the target data unit to the storage node.

[0100] The storage node receives the target data and its address offset information, as well as the identification information of the target data unit. Obtain the metadata information of the target data unit according to the identification information of the target data unit, where the metadata information is represented as <MaxSeq,LagBeginSeq,LagEndeq>, wherein for the detailed introduction of MaxSeq, LagBeginSeq and LagEndeq, reference can be made to the foregoing embodiments, and details are not repeated herein.

[0101] Wherein, if LagBeginSeq<LagEndSeq, it indicates that the target data unit lacks data from position LagBeginSeq to position LagEndSeq, and there is a data hole in the target data unit.

[0102] a. If MaxSeq==BeginSeq and LagBeginSeq==LagEndSeq, it indicates that the target data unit had complete data before, then the target data can be written into the target data unit from position BeginSeq to position EndSeqand update MaxSeq=EndSeq, then return a write success message to the service node 101. Wherein, "==" is a comparison operator, which is used to compare whether two values are equal, and "=" is an assignment operator. If G1=G2, it means assigning the value of G2 to the variable G1.

[0103] b. If MaxSeq<BeginSeq and LagBeginSeq==LagEndSeq, it indicates that the target data unit does not participate in the latest data writing, and there is currently no hole data in the target data unit. At this time, hole data can be reserved between the position MaxSeq and the position BeginSeq, and the target data is written between the position BeginSeq and the position EndSeq of the target data unit. At this time, the hole position information is updated as: LagBeginSeq=MaxSeq, LagEndSeq=BeginSeq, and the start writing position information in the metadata information is updated as: MaxSeq=EndSeq, then a write success message is returned to the service node 101.

[0104] Wherein, after the storage node fills the hole data, both LagBeginSeq and LagEndSeq can be set to 0.

[0105] c. If MaxSeq>BeginSeq, it indicates that the target data has been written into the target data unit, and case c will not occur under normal circumstances. Because under the condition that the service node 101 ensures the increment and data invariance characteristics of append-only storage, it will not write data again at the already written position. Therefore, if MaxSeq>BeginSeq, the storage node returns a notification message of "position occupied, write failed" to the service node.

[0106] The service node counts the write success messages from the first number of replica nodes, if:

[0107] a. If all of the first number of replica nodes return write success messages, the service node updates and records MaxSeq=EndSeq of the first number of replica nodes in its memory, and returns a message of completing multi-copy storage of the target data to an upper-layer caller (e.g., a compute node).

[0108] b. If a third number of replica nodes fail to write or timeout, and the third number is less than or equal to the second number, the service node may select the third number of candidate nodes from the second number of candidate nodes and send write requests for the target data to the third number of candidate nodes. If all of the third number of candidate nodes return write success messages, the service node updates and records MaxSeq=EndSeq of the replicas that eventually succeed in writing in its memory, and returns a message of completing multi-copy storage of the target data to an upper-layer caller (e.g., a compute node).

[0109] If it is found after multiple retries that the number of candidate nodes that succeed in writing does not reach the third number, the service node no longer writes the target data into the target data unit, and the service node may apply to the management node for sealing the target data unit,And apply for a new data unit to rewrite the target data. The sealing address range of the target data unit can be 0- (Page 13 / 24 of the specification, 18 CN 122261465 A BeginSeq).

[0110] The method of sealing the target data unit is not limited. In addition to the above situation, if the size of the target data unit reaches the upper limit threshold, the service node will no longer continue to write data to the target data unit, and will apply to the management node to seal the target data unit. The sealing position is the position of the last successful write, i.e., EndSeq. The upper limit threshold can include, but is not limited to: 128M, 256M and 512M, etc.

[0111] In an optional embodiment, the service node can read the target data from the target data unit. It should be noted that the target data written into the target data unit in the above process may not be the same as the target data read in this embodiment. For example, in the above process, data J1, data J2 and data J3 are written into the target data unit. In this embodiment, any one or more of data J1, data J2 and data J3 can be read. In this embodiment, reading target data is used as an example for explanation; the reading process for other data is the same or similar.

[0112] The specific multi-replica data reading process is described below:

[0113] The service node initiates a read request for the target data in the target data unit. The read request is used to request the reading of the target data in the target data unit from the storage node (i.e., the replica node) corresponding to the target data unit, and sends the read request to the storage node.

[0114] The storage node receives the read request, which includes the address offset information of the target data in the target data unit; obtains the metadata information corresponding to the target data unit, which includes the current start write position information and hole position information of the target data unit; reads the target data in the target data unit according to the address offset information, start write position information and hole position information, and returns it to the service node. For a detailed description of the metadata information, please refer to the foregoing embodiments, which will not be repeated here. The address offset information may include the start offset address and data length information. By combining the address offset information, start write position information and hole position information, the system can accurately locate the specific location of the target data and avoid misreading or omission. Preventing erroneous reading: Ensuring that only actual data areas are read, excluding any empty data portions, improves the accuracy of the reading results.

[0115] Optionally, the implementation method for reading target data based on address offset information, start write position information, and empty position information is not limited. One implementation method for reading target data based on address offset information, start write position information, and empty position information includes: determining whether the target data is a target based on the start write position information and address offset information.The data already exists in the data unit; if the target data is already existing in the target data unit, then based on the hole location information and address offset information, it is determined whether the target data includes the hole data in the target data unit. If the target data does not include the hole data, then the target data is read from the target data unit based on the address offset information. Among them, before reading, it is verified whether the target data exists and whether it contains the hole data to ensure that the data returned to the service node is complete and consistent. This implementation not only improves the accuracy and efficiency of reading, but also enhances the reliability, high availability and fault tolerance of the system.

[0116] Further optionally, if the target data includes the hole data, the service node can be notified of the reading failure, so that the service node can read the target data from other storage nodes, or the current storage node can fill the hole data and then read the target data after filling. The following is an exemplary description.

[0117] Example L1, if the target data includes the hole data, then a reading failure message is returned to the service node so that the requesting end can read the target data from other storage nodes. By detecting and identifying the case of containing the hole data, it is ensured that incomplete or damaged data is not returned to the service node, thus improving the accuracy of the reading result.

[0118] Example L2: If the target data includes gapped data, a data filling request is sent to the management node. The data filling request includes gap location information so that the management node can return data filling information. The data filling information includes: identification information of other storage nodes that store gapped data; the storage node obtains the gapped data from the other storage node's specification (page 14 / 24, CN 122261465 A) based on the identification information of the other storage nodes to fill the target data unit; and reads the target data from the target data unit after the gapped data is filled based on the address offset information. By filling the gapped data, the integrity of the target data unit is ensured, and incomplete or damaged data is avoided from being received by the service node.

[0119] The following is an example of the reading process on the storage node.

[0120] When a service node reads target data from a target data unit, it sends a read request to the storage node corresponding to the target data unit. The read request can be represented as (ChunkID, offset, length), where ChunkID is the identifier of the target data unit, offset is the starting offset address of the target data, and length is the size of the target data. The address offset information of the target data includes offset and length.

[0121] The storage node receives the read request and, based on the ChunkID, queries the metadata information of the replica corresponding to the target data unit. For example,<MaxSeq,LagBeginSeq,LagEndSeq>

[0122] aa. If offset + length > MaxSeq, it means that the replica does not include the target data, and the error message "data exceeds replica MaxSeq" is returned, so that the service node can retry reading other replicas.

[0123] b. If offset + length ≤ MaxSeq, and the intervals (offset, offset + length) and (LagBeginSeq, LagEndSeq) intersect, the error message "target data includes gapped data" is returned, and the service node needs to retry reading other replicas or wait for the gapped data to be filled before reading.

[0124] c. If offset + length ≤ MaxSeq, and the intervals (offset, offset + length) and (LagBeginSeq, LagEndSeq) do not intersect, it is determined that the replica includes the target data to be read, and the storage node reads the target data from the replica of the target data unit and returns it to the service node. In an optional embodiment, the service node sends the read request to the management node, and the management node returns the storage node that can be read, so that the service node reads the target data from the storage node.

[0125] The management node can receive read requests sent by the service node. These read requests are used to request the reading of target data from the target data unit. If the target data unit is not sealed and a write operation is being performed on it, the management node determines whether the initiator of the write operation is the service node. If so, the writer of the target data is allowed to read the target data, and the management node returns the identification information of the first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes. When the service node itself is the initiator of the write operation, it directly provides the identification information of multiple replica nodes, enabling the service node to immediately read data from other replica nodes, reducing waiting time. The service node can attempt to read in parallel based on the provided identification information of multiple replica nodes, further improving reading speed and efficiency.

[0126] The management node maintains the status information of each data unit. For example, the status information of a data unit includes the replica location of the data unit (e.g., which storage node it is stored in), whether the data unit is sealed, etc. The management node can obtain the status information of the target data unit and parse out whether the target data unit is sealed.

[0127] Optionally, if the initiator of the write operation is not the service node itself, in order to avoid data reading errors, other service nodes are not allowed to read the data being written during the data writing process, and a notification message of refusal to read is returned to that service node.

[0128] Optionally, if the target data unit has not been sealed, and no write operation is currently being performed on the target data unit, this situation usually occurs when the writer of the target data unit (i.e., the service node) has not had time to actively seal the target data unit.If so, after sealing the target data unit, the management node returns the identification information of the first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes. After confirming that there is no write operation at present, the target data unit is sealed directly to ensure that the subsequent read operation is based on the latest and stable data state. The sealing action ensures the data synchronization of all replica nodes and avoids the data inconsistency problem caused by concurrent writes.

[0129] The method by which the management node seals the target data unit is not limited. The following is an example, including: querying the metadata information of multiple storage nodes corresponding to the target data unit, namely the metadata information of the first number of replica nodes and the second number of candidate nodes; obtaining the starting write position information (e.g., MaxSeq) of each storage node from the metadata information; selecting the first number (e.g., W) of starting write position information in descending order of the starting write position information, denoted as MaxSeq i, 1≤i≤W, where MaxSeq1>=MaxSeq2>=...>=MaxSeqW; if a storage node is abnormal, its starting write position information is set to infinity; if MaxSeqW is infinity, it means that there are not enough abnormal replica nodes, and the target data unit cannot be sealed temporarily, and the management node returns a "data is temporarily abnormal" message to the service node to refuse reading; if MaxSeqW is not infinity, the management node uses MaxSeqW as the sealing termination position information of the target data unit and seals the target data unit with this sealing termination position information.

[0130] Optionally, if the target data unit has been sealed, the system returns the identification information of the first number of replica nodes corresponding to the target data unit to the service node so that the service node can read the target data from the first number of replica nodes. If the target data unit has been sealed, the system directly returns the identification information of the replica nodes, ensuring that the read data is confirmed to be consistent and stable. The sealed data unit no longer accepts new write requests, avoiding the risk of data being modified during the reading process.

[0131] Further optionally, the management node can perform performance scoring on the first number of replica nodes based on information such as the data integrity or health status of each replica node, and sort the first number of replica nodes in descending order of performance scores to obtain a replica node list, and return the replica node list and the address information of each replica node to the service node. The service node can select a replica node from the replica node list to read the target data. For example, it can select randomly, or select the one that is sorted first, which is not limited.

[0132] In addition to providing a system embodiment, this application also provides a method for storing multiple replica data, which is described below.The process of the multi-copy data storage method provided in the embodiments of this application will be described.

[0133] As shown in FIG3a, the multi-copy data storage method includes:

[0134] 301a. Determining the target data to be stored and the target data unit to which it belongs, wherein the target data unit is a basic data unit that supports multi-copy storage;

[0135] 302a. Obtaining multiple storage nodes corresponding to the target data unit, wherein the multiple storage nodes include a first number of replica nodes and a second number of candidate nodes, wherein the first number is the number of replicas in the multi-copy storage;

[0136] 303a. Writing the target data to the first number of replica nodes to perform multi-copy storage of the target data;

[0137] 304a. If a replica node fails to write during the writing process, and the third number of replica nodes that fail to write is less than or equal to the second number, then writing the target data to at least some of the candidate nodes in the second number of candidate nodes;

[0138] 305a. If a candidate node succeeds to write during the writing process, and the number of candidate nodes that succeed to write reaches the third number, returning a message indicating that the multi-copy storage of the target data is completed.

[0139] In an optional embodiment, when the number of candidate nodes that have successfully written data reaches a third number, the method further includes: replacing the replica nodes that have failed to write data among the first number of replica nodes with candidate nodes that have successfully written data; wherein, the empty data in the candidate nodes that have successfully written data will be filled after the target data is written, and the empty data refers to the data that is already present in the target data unit that is missing on the candidate node.

[0140] Optionally, the method further includes: when the scanning conditions are met, scanning the metadata information of the first number of replica nodes that currently correspond to the target data unit; determining whether there is empty data in the replica node corresponding to the currently scanned metadata information based on the currently scanned metadata information; if so, sending data filling information to the replica node corresponding to the currently scanned metadata information on page 16 / 24 of the specification 21 CN 122261465 A, the data filling information includes: the identification information of other storage nodes that store empty data, so that the replica node corresponding to the currently scanned metadata information can obtain the empty data from other storage nodes.

[0141] Further optionally, the metadata information includes: the starting write position information and hole position information corresponding to the target data unit, the hole position information including the hole start offset address and the hole end offset address information, and determining whether there is hole data in the replica node corresponding to the currently scanned metadata information based on the currently scanned metadata information, including: if the hole end offset address information is greater than the hole start offset address information, then it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information, the hole data including the hole start offset address in the target data unit.The data between the information and the hole termination offset address information; and / or, in the case that the target data unit is sealed, if the starting write position information is less than the sealing offset address information of the target data unit, then it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information, and the hole data includes the data between the starting write position information and the sealing offset address information.

[0142] In an optional embodiment, the method further includes: if the number of candidate nodes that have been successfully written does not reach a third number, then a fault tolerance request is sent to the management node, the fault tolerance request is used to request the management node to seal the target data unit and allocate a new data unit, the new data unit corresponds to at least a first number of new replica nodes; and write the target data to the first number of new replica nodes corresponding to the new data unit.

[0143] Optionally, the method further includes: determining the sealing address range information corresponding to the target data unit on the first number of replica nodes according to the offset address information of the target data in the target data unit; and sealing the target data unit according to the sealing address range information.

[0144] In an optional embodiment, after writing target data to a first number of replica nodes, the method further includes: counting the number of write success messages returned by the first number of replica nodes; if the number of received write success messages is less than the first number, determining that there are replica nodes that have failed to write, and calculating the difference between the first number and the number of received write success messages as a third number.

[0145] In an optional embodiment, writing target data to at least some of the candidate nodes in a second number of candidate nodes includes: if the third number is less than the second number, determining the void data and its quantity and / or size existing in each of the second number of candidate nodes, where void data refers to data already present in the missing target data unit on the candidate node; selecting a third number of candidate nodes from the second number of candidate nodes based on the quantity and / or size of the void data existing in each of the second number of candidate nodes; and writing target data to the third number of candidate nodes.

[0146] Optionally, selecting a third number of candidate nodes from the second number of candidate nodes based on the number and / or size of the void data existing in each of the second number of candidate nodes includes: selecting a third number of candidate nodes from the second number of candidate nodes based on the number and / or size of the void data existing in each of the second number of candidate nodes, combined with the time interval between each of the second number of candidate nodes and its last time as a replica node.

[0147] This application embodiment also provides another method for storing multi-replica data, which is applicable to storage nodes in a storage system, as shown in FIG3b. The method includes:

[0148] 301b, receiving the target data to be stored and the address offset information of the target data in its respective target data unit;

[0149] 302b. Obtain the metadata information corresponding to the target data unit. The metadata information includes the current starting write position information and hole position information corresponding to the target data unit. The hole position information describes whether there is hole data in the storage node and the position information of the hole data in the target data unit if it exists. Hollow data refers to the data that is missing in the target data unit on the storage node. Specification 17 / 24 pages 22 CN 122261465 A

[0150] 303b. Write the target data into the target data unit according to the address offset information, and update the starting write position information according to the address offset information.

[0151] 304b. If the starting write position determined according to the address offset information is greater than the starting write position information in the metadata information before the update, update the hole position information.

[0152] In an optional embodiment, updating the starting write position information according to the address offset information includes: using the current termination write position information determined according to the address offset information as the starting write position information in the metadata information to update the metadata information.

[0153] Optionally, if the current starting write position determined based on the address offset information is greater than the starting write position information in the metadata information before the update, the hole position information is updated, including: determining whether the current starting write position determined based on the address offset information is greater than the starting write position information in the metadata information before the update; if so, the starting write position information in the metadata information is used as the hole starting position information in the hole position information, and the current starting write position information is used as the hole ending position information in the hole position information, so as to update the hole position information.

[0154] This application embodiment also provides a method for reading multiple replica data, applicable to storage nodes in a storage system, as shown in FIG3c. The method includes:

[0155] 301c, receiving a read request, the read request being used to request reading target data from a target data unit on the storage node, and the read request including address offset information of the target data in the target data unit;

[0156] 302c, obtaining metadata information corresponding to the target data unit, the metadata information including the starting write position information and hole position information corresponding to the target data unit, the hole position information describing whether there is hole data in the storage node and the position information of the hole data in the target data unit if it exists; hole data refers to the data already existing in the target data unit that is missing on the storage node;

[0157] 303c, reading the target data according to the address offset information, the starting write position information and the hole position information, and returning the requesting end of the read request.

[0158] In an optional embodiment, reading target data based on address offset information, start write position information, and hole position information includes: determining whether the target data is a target data unit based on the start write position information and address offset information.The method further includes: if the data already exists in the target data unit, then based on the hole location information and address offset information, it is determined whether the target data includes the hole data in the target data unit; if not, the target data is read from the target data unit based on the address offset information.

[0159] Optionally, the method further includes: if it includes the hole data, then a read failure message is returned to the requesting end so that the requesting end can read the target data from other storage nodes; or, if it includes the hole data, a data filling request is sent to the management node in the storage system, the data filling request including the hole location information so that the management node can return data filling information, the data filling information including: the identification information of other storage nodes that store the hole data so that the storage node can obtain the hole data from other storage nodes to fill the target data unit; and the target data is read from the target data unit after the hole data is filled based on the address offset information.

[0160] This application embodiment also provides another method for reading multi-replica data. This method is applicable to the management node in the storage system, as shown in FIG3d. The method includes:

[0161] 301d, receiving a read request sent by a service node, the read request being used to request reading target data in the target data unit;

[0162] 302d, if the target data unit is not sealed and a write operation is being performed in the target data unit, determining whether the initiator of the write operation is a service node;

[0163] 303d, if so, returning the identification information of the first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes.

[0164] In an optional embodiment, the method further includes: if the target data unit has been sealed, returning the identification information of a first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes; or if the target data unit has not been sealed and no write operation has been performed in the target data unit, then after sealing the target data unit, returning the identification information of the first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes.

[0165] The detailed implementation methods and beneficial effects of each step in the multi-replica data storage method and multi-replica data reading method provided in the embodiments of this application have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0166] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can also be executed by different devices. For example, the executing entity for steps 301 to 303 can be device A; or, for example, the executing entity for steps 301 and 302 can be device A, and the executing entity for step 303 can be device B; and so on.

[0167] In addition, some processes described in the above embodiments and figures include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 301, 302, etc., are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent the order of events, nor do they limit "first" and "second" to different types.

[0168] Figure 4 is a schematic diagram of the structure of a storage device for multiple copy data provided by an exemplary embodiment of this application. As shown in Figure 4, the device includes: a determination module 41, an acquisition module 42, a writing module 43, and a return module 44.

[0169] Determining module 41 is used to determine the target data to be stored and its corresponding target data unit, wherein the target data unit is a basic data unit that supports multi-replica storage;

[0170] Obtaining module 42 is used to obtain multiple storage nodes corresponding to the target data unit, wherein the multiple storage nodes include a first number of replica nodes and a second number of candidate nodes, wherein the first number is the number of replicas in multi-replica storage;

[0171] Writing module 43 is used to write the target data to the first number of replica nodes to perform multi-replica storage on the target data;

[0172] Writing module 43 is used to write the target data to at least some of the candidate nodes in the second number of candidate nodes if a replica node fails to write during the writing process, and the third number of replica nodes that fail to write is less than or equal to the second number;

[0173] Returning module 44 is used to return a message indicating that the multi-replica storage of the target data is completed if a candidate node that succeeds to write during the writing process, and the number of candidate nodes that succeeds to write reaches the third number.

[0174] In an optional embodiment, when the number of candidate nodes that succeed to write reaches the third number, the device further includes: a replacement module. The replacement module is used to replace the replica nodes that failed to write among the first number of replica nodes with candidate nodes that successfully wrote; wherein, the empty data in the candidate nodes that successfully wrote will be filled after the target data is written, and the empty data refers to the data that is already in the target data unit that is missing on the candidate node.

[0175] Optionally, the device further includes a scanning module and a sending module. The scanning module is used to scan the metadata information of the first number of replica nodes corresponding to the target data unit when the scanning conditions are met; the determining module is used to determine whether there is empty data in the replica node corresponding to the currently scanned metadata information based on the currently scanned metadata information; the sending module is used to send data filling to the replica node corresponding to the currently scanned metadata information if there is empty data.The specification page 19 / 24, CN 122261465 A information, the data filling information includes: the identification information of other storage nodes storing void data, so that the replica node corresponding to the currently scanned metadata information can obtain void data from other storage nodes.

[0176] Further optionally, the metadata information includes: the starting write position information and void position information corresponding to the target data unit, the void position information includes the void start offset address and void end offset address information, the determination module is specifically used for: if the void end offset address information is greater than the void start offset address information, then it is determined that there is void data in the replica node corresponding to the currently scanned metadata information, the void data includes the data between the void start offset address information and the void end offset address information in the target data unit; and / or in the case that the target data unit is sealed, if the starting write position information is less than the sealing offset address information of the target data unit, then it is determined that there is void data in the replica node corresponding to the currently scanned metadata information, the void data includes the data between the starting write position information and the sealing offset address information.

[0177] In an optional embodiment, the sending module is configured to send a fault tolerance request to the management node if the number of candidate nodes successfully written does not reach a third number. The fault tolerance request is used to request the management node to seal the target data unit and allocate a new data unit, wherein the new data unit corresponds to at least a first number of new replica nodes; and the writing module is configured to write the target data to the first number of new replica nodes corresponding to the new data unit.

[0178] Optionally, the device further includes: a sealing module. A determining module is configured to determine the sealing address range information corresponding to the target data unit on the first number of replica nodes based on the offset address information of the target data in the target data unit; the sealing module is configured to seal the target data unit based on the sealing address range information.

[0179] In an optional embodiment, the device further includes: a statistics module. After writing the target data to the first number of replica nodes, the statistics module is used to count the write success messages returned by the first number of replica nodes; the determination module is used to determine that there are replica nodes that have failed to write if the number of received write success messages is less than the first number, and calculate the difference between the first number and the number of received write success messages as the third number.

[0180] In an optional embodiment, the writing module is specifically used to: determine the void data and its quantity and / or size existing in each of the second number of candidate nodes when the third number is less than the second number, where void data refers to the data already present in the missing target data unit on the candidate node; select a third number of candidate nodes from the second number of candidate nodes according to the quantity and / or size of the void data existing in each of the second number of candidate nodes; and write the target data to the third number of candidate nodes.

[0181] Optionally, the writing module is specifically used to: select a third number of candidate nodes from the second number of candidate nodes based on the number and / or size of the void data existing in each of the second number of candidate nodes, combined with the time interval between each of the second number of candidate nodes and the last time it served as a replica node.

[0182] This application embodiment also provides a method for storing multi-replica data, corresponding to a storage node in a storage system. The device includes: a receiving module, an acquiring module, a writing module, and an updating module.

[0183] A receiving module is used to receive the target data to be stored and the address offset information of the target data in its respective target data unit;

[0184] An obtaining module is used to obtain the metadata information corresponding to the target data unit. The metadata information includes the starting write position information and the hole position information corresponding to the target data unit. The hole position information describes whether there is hole data in the storage node and the position information of the hole data in the target data unit if it exists. Hollow data refers to the data that is already in the target data unit that is missing on the storage node;

[0185] A writing module is used to write the target data into the target data unit according to the address offset information; an updating module is used to update the starting write position information according to the address offset information; and when the starting write position determined according to the address offset information is greater than the starting write position information in the metadata information before the update, the hole position information is updated.

[0186] In an optional embodiment, the updating module is specifically used to: use the ending write position information determined according to the address offset information as the starting write position information in the metadata information to update the metadata information.

[0187] Optionally, the update module is specifically used to: determine whether the current start write position information determined based on the address offset information is greater than the start write position information in the metadata information before the update; if so, then the start write position information in the metadata information is used as the hole start position information in the hole position information, and the current start write position information is used as the hole end position information in the hole position information, so as to update the hole position information.

[0188] This application embodiment also provides a multi-copy data reading device, corresponding to a storage node in a storage system, the device including:

[0189] a receiving module, used to receive a reading request, the reading request is used to request to read target data in a target data unit from the storage node, and the reading request includes the address offset information of the target data in the target data unit;

[0190] an acquisition module, used to acquire the metadata information corresponding to the target data unit, the metadata information includes the current start write position information and hole position information corresponding to the target data unit, the hole position information describes whether there is hole data in the storage node and the position information of the hole data in the target data unit when it exists; hole data refers to the storage nodeThe missing target data unit already contains the data;

[0191] The reading module is used to read the target data according to the address offset information, the start write position information and the hole position information, and return the requesting end of the reading request.

[0192] Optionally, the reading module is specifically used to: determine whether the target data is existing data in the target data unit according to the start write position information and the address offset information; if so, determine whether the target data includes the hole data in the target data unit according to the hole position information and the address offset information; if not, read the target data from the target data unit according to the address offset information.

[0193] Optionally, the device further includes a return module and a sending module. The return module, if included, returns a read failure message to the requesting end so that the requesting end can read the target data from other storage nodes; or, the sending module, if included, sends a data filling request to the management node in the storage system, the data filling request including hole location information so that the management node can return data filling information, the data filling information including: identification information of other storage nodes storing hole data so that the storage node can obtain the hole data from other storage nodes to fill the target data unit; the read module is also used to: read the target data from the target data unit after the hole data has been filled according to the address offset information.

[0194] This application embodiment also provides another multi-replica data reading device, which includes: a receiving module, a judging module, and a returning module;

[0195] The receiving module is used to receive a reading request sent by a service node, the reading request being used to request reading target data in a target data unit;

[0196] The judging module is used to judge whether the initiator of the write operation is a service node if the target data unit is not sealed and a write operation is being performed in the target data unit;

[0197] The returning module is used to return the identification information of a first number of replica nodes corresponding to the target data unit to the service node if yes, so that the service node can read the target data from the first number of replica nodes.

[0198] In an optional embodiment, the return module is further configured to: if the target data unit has been sealed, return the identification information of the first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes; or if the target data unit has not been sealed and no write operation has been performed in the target data unit, then after sealing the target data unit, return the identification information of the first number of replica nodes corresponding to the target data unit to the service node, so that the service node can read the target data from the first number of replica nodes.

[0199] Regarding the above-mentioned storage device and reading device for multiple replica data provided in the embodiments of this application.The detailed implementation methods and beneficial effects of each step in the device provided in the previous embodiments have been described in detail and will not be elaborated here.

[0200] FIG5 is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of the present application. As shown in FIG5, the device includes: a memory 54 and a processor 55.

[0201] The memory 54 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device.

[0202] The processor 55 is coupled to the memory 54 and is used to execute the computer programs in the memory 54 to execute the storage method and reading method of multiple copies of data described in the previous embodiments. For details, please refer to the descriptions in the previous embodiments, which will not be repeated here.

[0203] The detailed implementation methods and beneficial effects of each step in the device shown in FIG5 provided in the embodiments of the present application have been described in detail in the previous embodiments and will not be elaborated here.

[0204] Further, as shown in FIG5, the electronic device also includes: a communication component 56, a display 57, a power supply component 58, an audio component 59, and other components. Figure 5 only schematically shows some components and does not mean that the electronic device only includes the components shown in Figure 5. In addition, the components within the dashed boxes in Figure 5 are optional components, not mandatory components, and the specific components depend on the product form of the electronic device. The electronic device in this embodiment can be implemented as a desktop computer, laptop computer, smartphone, or IoT (Internet of Things) device and smart wearable device (such as smartwatch, smart bracelet) and other terminal devices, or it can be a conventional server, cloud server or server array and other server-side devices. If the electronic device in this embodiment is implemented as a desktop computer, laptop computer, smartphone and other terminal devices, it may include the components within the dashed boxes in Figure 5; if the electronic device in this embodiment is implemented as a conventional server, cloud server or server array and other server-side devices, it may not include the components within the dashed boxes in Figure 5.

[0205] Accordingly, this application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by the electronic device in the above-described multi-copy data storage method and multi-copy data reading method embodiments.

[0206] Accordingly, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, cause the processor to implement the steps of the above-described methods for storing and reading multiple copies of data, which can be executed by an electronic device.

[0207] The above-described memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM).Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), Magnetic Storage, Flash Memory, Disk, or Optical Disk.

[0208] The above-described communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as WiFi, 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0209] The above-mentioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touch screen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touch, swipe, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of the touch or swipe action, but also the duration and pressure associated with the touch or swipe operation.

[0210] The above-mentioned power supply component provides power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0211] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone.Microphone (MIC): When the device containing the audio component is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signals may be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0212] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application may take the form of a computer program product implemented on one or more computer-readable storage media (including, but not limited to, disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0213] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart and / or block diagram, and combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart and / or one or more blocks of the block diagram.

[0214] These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more blocks of the flowchart and / or one or more blocks of the block diagram.

[0215] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0216] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0217] Memory may include non-persistent memory in a computer-readable medium, random access memory (RAM), etc.Pages 23 / 24 28 CN 122261465 A Access Memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0218] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology for information storage. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0219] It should also be noted that the terms “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, product, or apparatus that includes the element.

[0220] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application. Specification 24 / 24 pages 29 CN 122261465 A Figure 1 Specification Drawings 1 / 5 pages 30 CN 122261465 A Figure 2 Specification Drawings 2 / 5 pages 31 CN 122261465 A Figure 3a Figure 3b Specification Drawings 3 / 5 pages 32 CN 122261465 A Figure 3c Figure 3d Figure 4Description Drawings Page 4 / 5 33 CN 122261465 A Figure 5 Description Drawings Page 5 / 5 34 CN 122261465 A Abstract Embodiments of the present application provide a method, device, storage medium and program product for storing, reading and archiving multi-copy data. In the embodiments of the present application, fast fault-tolerance mechanism is provided to perform storage fault tolerance for the number of copy storage nodes exceeding the fault-tolerance range. Data is first written to the copy nodes; when a node fails and the failure is within the fault-tolerance range, data is written to a candidate node, which is beneficial to improving the success probability of multi-copy storage. Since the slow fault-tolerance mechanism that triggers archiving of data units does not need to be triggered within the above fault-tolerance range, the probability of triggering the fault-tolerance mechanism is reduced, the access impact on management nodes (such as MS) is lowered, thereby improving data writing efficiency and reducing write latency.

Claims

1. A method for storing multiple copies of data, characterized in that, include: Determine the target data to be stored and its corresponding target data unit, wherein the target data unit is a basic data unit that supports multiple replica storage; Obtain multiple storage nodes corresponding to the target data unit, the multiple storage nodes including a first number of replica nodes and a second number of candidate nodes, the first number being the number of replicas in the multi-replica storage; Write the target data to the first number of replica nodes to perform multi-replica storage of the target data; If a replica node fails to write during the writing process, and the third number of the failed replica nodes is less than or equal to the second number, then the target data is written to at least some of the candidate nodes among the second number of candidate nodes. If the number of candidate nodes that have been successfully written reaches the third number, a message indicating that the multi-copy storage of the target data is complete is returned.

2. The method according to claim 1, characterized in that, If the number of candidate nodes with successful writes reaches the third number, the method further includes: Replace the replica nodes that failed to write among the first number of replica nodes with the candidate nodes that successfully wrote; wherein, the empty data in the candidate nodes that successfully wrote will be filled after the target data is written, and the empty data refers to the data that is missing in the target data unit on the candidate node.

3. The method according to claim 2, characterized in that, Also includes: If the scanning conditions are met, scan the metadata information of the first number of replica nodes corresponding to the target data unit. Based on the currently scanned metadata information, determine whether there is any missing data in the replica node corresponding to the currently scanned metadata information; If present, data filling information is sent to the replica node corresponding to the currently scanned metadata information. The data filling information includes: identification information of other storage nodes that store the void data, so that the replica node corresponding to the currently scanned metadata information can obtain the void data from the other storage nodes.

4. The method according to claim 3, characterized in that, The metadata information includes: the starting write position information and hole position information corresponding to the target data unit. The hole position information includes the hole start offset address and the hole end offset address information. Based on the currently scanned metadata information, it is determined whether there is hole data in the replica node corresponding to the currently scanned metadata information, including: If the hole termination offset address information is greater than the hole start address offset information, it is determined that there is hole data in the replica node corresponding to the currently scanned metadata information. The hole data includes the data between the hole start address offset information and the hole termination offset address information in the target data unit. and / or If the target data unit is sealed, and the starting write position information is less than the sealing offset address information of the target data unit, then it is determined that there is void data in the replica node corresponding to the currently scanned metadata information. The void data includes the data between the starting write position information and the sealing offset address information.

5. The method according to claim 1, characterized in that, Also includes: If the number of candidate nodes that have successfully written data does not reach the third number, a fault tolerance request is sent to the management node. This fault tolerance request requests the management node to seal the target data unit and allocate a new data unit, where the new data unit corresponds to at least a first number of new replica nodes. Write the target data to the first number of new replica nodes corresponding to the new data unit.

6. The method according to claim 5, characterized in that, The management node encapsulates the target data unit through the following steps: Based on the offset address information of the target data in the target data unit, determine the sealing address range information of the target data unit on the first number of replica nodes; The target data unit is sealed according to the sealed address range information.

7. The method according to any one of claims 1-6, characterized in that, After writing the target data to the first number of replica nodes, the process further includes: The number of write success messages returned by the first number of replica nodes are counted. If the number of successful write messages received is less than the first number, it is determined that there are replica nodes that have failed to write, and the difference between the first number and the number of successful write messages received is calculated as the third number.

8. The method according to any one of claims 1-6, characterized in that, Writing the target data to at least a portion of the candidate nodes among the second number of candidate nodes includes: If the third quantity is less than the second quantity, determine the void data and its quantity and / or size of each of the second quantity of candidate nodes, wherein the void data refers to the data that is missing in the target data unit on the candidate node; Based on the number and / or size of the void data in each of the second number of candidate nodes, a third number of candidate nodes are selected from the second number of candidate nodes; Write the target data to the third number of candidate nodes.

9. The method according to claim 8, characterized in that, Based on the number and / or size of the holes in each of the second number of candidate nodes, a third number of candidate nodes are selected from the second number of candidate nodes, including: Based on the number and / or size of the void data in each of the second number of candidate nodes, and combined with the time interval between each of the second number of candidate nodes and its last time as a replica node, the third number of candidate nodes are selected from the second number of candidate nodes.

10. A method for storing multiple copies of data, characterized in that, Storage nodes applicable to storage systems include: Receive the target data to be stored and the address offset information of the target data in its respective target data unit; Obtain metadata information corresponding to the target data unit. The metadata information includes the current start write position information and hole position information corresponding to the target data unit. The hole position information describes whether there is hole data in the storage node and the position information of the hole data in the target data unit if it exists. Hollow data refers to the missing data in the target data unit that is already there. The target data is written into the target data unit according to the address offset information, and the starting write position information is updated according to the address offset information; and If the starting write position determined based on the address offset information is greater than the starting write position information in the metadata information before the update, the hole position information is updated.

11. The method according to claim 10, characterized in that, If the starting write position determined based on the address offset information is greater than the starting write position information in the metadata information before the update, the hole position information is updated, including: Determine whether the current start write position information determined based on the address offset information is greater than the start write position information in the metadata information before the update; If so, the starting write position information in the metadata information is used as the starting position information of the hole in the hole position information, and the current starting write position information is used as the ending position information of the hole in the hole position information, so as to update the hole position information.

12. A method for reading multiple copies of data, characterized in that, The method, applicable to storage nodes in a storage system, includes: A read request is received, the read request being used to request the reading of target data from the target data unit on the storage node, and the read request including the address offset information of the target data in the target data unit; Obtain metadata information corresponding to the target data unit. The metadata information includes the current start write position information and hole position information corresponding to the target data unit. The hole position information describes whether there is hole data in the storage node and the position information of the hole data in the target data unit if it exists. Hollow data refers to the missing data in the target data unit that is already there. Based on the address offset information, the starting write position information, and the hole position information, the target data is read, and the requesting end of the read request is returned.

13. The method according to claim 12, characterized in that, Based on the address offset information, the starting write position information, and the hole position information, the target data is read, including: Based on the starting write position information and the address offset information, it is determined whether the target data is existing data in the target data unit; If so, then based on the cavity location information and the address offset information, determine whether the target data includes the cavity data in the target data unit; If not included, the target data is read from the target data unit according to the address offset information.

14. The method according to claim 13, characterized in that, Also includes: If included, a read failure message is returned to the requesting end, so that the requesting end can read the target data from other storage nodes; or, If included, a data filling request is sent to the management node in the storage system. The data filling request includes hole location information so that the management node can return data filling information. The data filling information includes: identification information of other storage nodes that store the hole data so that the storage node can obtain the hole data from the other storage nodes to fill the target data unit. The target data is read from the target data unit after the hole data is filled based on the address offset information.

15. A method for reading multiple copies of data, characterized in that, include: Receive a read request sent by the service node, the read request being used to request the reading of target data in the target data unit; If the target data unit is not sealed and a write operation is being performed in the target data unit, determine whether the initiator of the write operation is the service node; If so, the identification information of the first number of replica nodes corresponding to the target data unit is returned to the service node, so that the service node can read the target data from the first number of replica nodes.

16. The method according to claim 15, characterized in that, Also includes: If the target data unit has been sealed, the identification information of the first number of replica nodes corresponding to the target data unit is returned to the service node so that the service node can read the target data from the first number of replica nodes; or If the target data unit is not sealed and no write operation is currently being performed on the target data unit, then after sealing the target data unit, the identification information of the first number of replica nodes corresponding to the target data unit is returned to the service node, so that the service node can read the target data from the first number of replica nodes.

17. An electronic device, characterized in that, include: Memory and processor; The memory is used to store a computer program; the processor, coupled to the memory, is used to execute the computer program to implement the steps of the method according to any one of claims 1-9, 10-11, 12-14, and 15-16.

18. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 1-9, 10-11, 12-14, and 15-16.

19. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1-9, 10-11, 12-14, and 15-16.

Images