Cloud storage system, data read / write method, device, storage medium, and program product

Abstract

Provided in the embodiments of the present disclosure are a cloud storage system, a data read / write method, a device, a storage medium, and a program product. In the embodiments of the present disclosure, provided is a highly available cross-availability-zone cloud storage disk. On the basis of cross-availability-zone disaster recovery requirement information, data storage layout information for the cloud storage disk is generated, and on the basis of the data storage layout information, a data read / write operation is performed on persistent storage nodes across availability zones. The data storage layout information can embody a multi-copy and erasure coding fused redundant storage method, which is used by the cloud storage disk, and by means of which data storage is highly available across availability zones, so as to reduce a data redundancy ratio; furthermore, a better read / write performance is provided and elastic requirements for availability zones are met, a cross-availability-zone network bandwidth is reduced, and the storage cost is reduced.

Description

Cloud storage system, data reading and writing method, device, storage medium and program product

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on November 8, 2023, with application number 202311484532.2 and application name “Cloud Storage System, Data Reading and Writing Method, Device and Storage Medium”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of cloud computing technology, and in particular to a cloud storage system, a data reading and writing method, a device, a storage medium, and a program product. Background Art

[0003] Currently, mainstream cloud vendors have launched disaster-tolerant storage services such as object storage, file storage, and databases. These storage services are deployed within the same availability zone and utilize multiple replicas or erasure coding (EC) for disaster recovery. With the advancement of cloud computing technology, cross-availability zone disaster recovery for critical cloud services is becoming a growing concern for cloud users. Many users hope to use cloud hard drives to quickly implement service disaster recovery architectures.

[0004] Therefore, some cloud vendors have launched cross-AZ cloud drive disaster recovery solutions. This involves deploying storage clusters in two availability zones. When creating a highly available cross-AZ cloud drive, users create a cloud drive in each storage cluster in the two zones. The two cloud drives are independent and back up each other, and master-slave synchronous replication technology ensures state synchronization between the two cloud drives. While this cross-AZ cloud drive disaster recovery solution can ensure data consistency and achieve cross-AZ disaster recovery, it also requires high overall redundancy.

[0005] Summary of the Invention

[0006] Various aspects of the present disclosure provide a cloud storage system, a data reading and writing method, a device, a storage medium, and a program product to reduce storage redundancy ratio and achieve high-availability disaster recovery across availability zones.

[0007] An embodiment of the present disclosure provides a cloud storage system, comprising: a storage service management node, a storage service node, a persistence management node, and multiple persistence storage nodes, wherein the multiple persistence storage nodes are distributed across multiple availability zones. The storage service management node is configured to create a cloud storage disk across N availability zones for a computing node and allocate storage service nodes to the cloud storage disk, so that the allocated storage service nodes respond to read and write requests initiated by the computing node to the cloud storage disk, where N is greater than or equal to 2 and N is a positive integer. The cloud storage disk adopts a redundant storage method that combines multiple replicas with erasure coding, and the redundant storage method includes code sharding and multiple-replica data sharding. The persistence management node is configured to obtain cross-availability zone disaster recovery requirement information of the cloud storage disk and, based on the cross-availability zone disaster recovery requirement information, generate data storage layout information required for reading and writing the cloud storage disk, the data storage layout information being used to describe the storage method of the code sharding and multiple-replica data shards in the persistent storage nodes across the availability zones. The storage service node is configured to respond to read and write requests initiated by the computing node to the cloud storage disk and, based on the data storage layout information, perform data read and write operations on the persistent storage nodes across the availability zones.

[0008] An embodiment of the present disclosure also provides a data reading and writing method, which is applicable to a storage service management node in a cloud storage system. The method includes: creating a cloud storage disk across N availability zones for a computing node, where N ≥ 2, and N is a positive integer; the cloud storage disk adopts a redundant storage method that integrates multiple copies and erasure codes, and the redundant storage method includes code sharding and multiple copies of data sharding; allocating a storage service node to the cloud storage disk, so that the allocated storage service node performs data reading and writing operations on persistent storage nodes across availability zones based on data storage layout information, where the data storage layout information is generated by a persistent management node in the cloud storage system based on the cross-availability zone disaster recovery requirement information of the cloud storage disk.

[0009] The disclosed embodiments also provide a data reading and writing method applicable to a persistent management node in a cloud storage system, the method comprising: obtaining cross-availability zone disaster recovery requirement information of a cloud storage disk; the cloud storage disk spans N availability zones, and adopts a redundant storage method that integrates multiple copies and erasure codes, the redundant storage method including coding shards and multiple copies of data shards, N≥2, and N is a positive integer; based on the cross-availability zone disaster recovery requirement information, generating data storage layout information required for reading and writing the cloud storage disk, the data storage layout information being used to describe the storage method for coding shards and multiple copies of data shards in persistent storage nodes across availability zones; providing the data storage layout information to a storage service node, so that the allocated storage service node performs data reading and writing operations on the persistent storage nodes across availability zones based on the data storage layout information.

[0010] An embodiment of the present disclosure also provides a data reading and writing method, which is applicable to a storage service node in a cloud storage system, the method comprising: receiving data storage layout information required for reading and writing a cloud storage disk sent by a persistent management node in the cloud storage system, the data storage layout information being generated by the persistent management node based on disaster recovery requirement information across availability zones of the cloud storage disk; responding to read and write requests initiated by a computing node to the cloud storage disk, and performing data reading and writing operations on persistent storage nodes across availability zones based on the data storage layout information; wherein the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies with erasure codes, the redundant storage method including code sharding and multiple copies of data sharding, N≥2, and N is a positive integer.

[0011] An embodiment of the present disclosure also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement each step in the data reading and writing method provided in the embodiment of the present disclosure.

[0012] The embodiments of the present disclosure further provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps of the data reading and writing method provided by the embodiments of the present disclosure.

[0013] The embodiments of the present disclosure further provide a computer program product, including a computer program, which, when executed by a processor, implements the steps in the data reading and writing method provided in the embodiments of the present disclosure.

[0014] In an embodiment of the present disclosure, a highly available cloud storage disk across availability zones is provided. Data storage layout information for the cloud storage disk is generated based on cross-availability zone disaster recovery requirements. Data read and write operations are performed on persistent storage nodes across availability zones based on this data storage layout information. This data storage layout information can reflect the redundant storage method of the cloud storage disk, which combines multiple replicas with erasure coding, achieving high availability of data storage across availability zones and reducing data redundancy. Furthermore, it provides improved read and write performance and the flexibility required by availability zones, reducing cross-availability zone network bandwidth and saving storage costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0016] FIG1 is a schematic diagram of the structure of a cloud storage system provided by an exemplary embodiment of the present disclosure;

[0017] FIG2a is a flow chart of a data reading and writing method provided by an exemplary embodiment of the present disclosure;

[0018] FIG2 b is a flow chart of another data reading and writing method provided by an exemplary embodiment of the present disclosure;

[0019] FIG2c is a flow chart of another data reading and writing method provided by an exemplary embodiment of the present disclosure;

[0020] FIG3a is a schematic structural diagram of a data reading and writing device provided by an exemplary embodiment of the present disclosure;

[0021] FIG3 b is a schematic structural diagram of another data reading and writing device provided by an exemplary embodiment of the present disclosure;

[0022] FIG3c is a schematic structural diagram of another data reading and writing device provided by an exemplary embodiment of the present disclosure;

[0023] FIG4 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present disclosure.

[0025] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0026] In this embodiment, an Availability Zone (AZ) refers to a physical area in the same region where the power supply and network are independent of each other. The network latency between cloud computing instances in the same availability zone is smaller, and the user access speed is faster. Here, a region refers to the geographical area where the data center is located, usually divided according to the city where the data center is located. A region is usually composed of multiple availability zones. Within the same region, the availability zones are interconnected through the intranet and are connected using low-latency links; fault isolation can be achieved between the availability zones, that is, if one availability zone fails, it will not affect the normal operation of other availability zones. The availability zones in different regions are completely isolated.

[0027] With the development of cloud computing technology, cross-availability zone disaster recovery for critical cloud services has become a growing concern for cloud users. Many users hope to use cloud hard drives to quickly implement service disaster recovery architectures. They also hope that cross-availability zone cloud hard drive disaster recovery solutions can not only achieve cross-availability zone disaster recovery while ensuring data consistency, but also have a low redundancy ratio.

[0028] In view of the above needs, the embodiments of the present disclosure provide a cloud storage disk with high availability across availability zones. Based on the disaster recovery demand information across availability zones, data storage layout information for the cloud storage disk is generated. Based on the data storage layout information, data read and write operations are performed on persistent storage nodes across availability zones. The data storage layout information can reflect the redundant storage method of multiple copies and erasure codes adopted by the cloud storage disk. By optimizing data synchronization and layout, high availability of data storage across availability zones is achieved, and the data redundancy ratio is reduced. Furthermore, better read and write performance and elastic requirements of availability zones are provided, network bandwidth across availability zones is reduced, and storage costs are saved.

[0029] A solution provided by an embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0030] Figure 1 is a schematic diagram of the structure of a cloud storage system provided by an exemplary embodiment of the present disclosure. As shown in Figure 1 , the system includes a storage service management node 101, a storage service node 102, a persistence management node 103, and multiple persistent storage nodes 104. Optionally, as shown in Figure 1 , the cloud storage system also includes a computing node 105 and a cloud storage access node 106 hosted on the computing node 105.

[0031] In this embodiment, a cloud storage access node 106 is deployed on computing node 105. Cloud storage access node 106 can be deployed on computing node 105 as a service process of the cloud storage system, providing access to the cloud storage system to cloud computing instances (e.g., virtual machines or containers) running on computing node 105. Specifically, cloud storage access node 106 can use virtualization technology to present a cloud storage disk provided by the cloud storage system to the cloud computing instance. A user can send input / output (IO) read / write requests to the cloud computing instance, and the cloud computing instance will send the IO read / write requests to cloud storage access node 106. Cloud storage access node 106 can receive the IO read / write requests from the cloud computing instance for the cloud storage disk, send the IO read / write requests to the backend storage cluster corresponding to the cloud storage disk, and return the response to the cloud computing instance (e.g., virtual machine or container) after receiving the response from the storage cluster. Cloud storage access node 106 can interact with the cloud computing instance through the virtualization manager on computing node 105, which is responsible for managing the cloud computing instance. The virtualization manager may be a virtual machine monitor (VMM), or may be called a hypervisor. Optionally, the hypervisor or VMM may be implemented as a virtual operating system emulator (Quick EMUlator, QEMU) or VMware, but is not limited thereto.

[0032] In this embodiment, the backend cluster corresponding to the cloud storage disk includes: a service layer and a persistence layer, as shown in Figure 1. The service layer includes: a storage service management node 101 and a storage service node 102. The storage service management node 101 is responsible for managing the metadata of the cloud storage disk and controlling the cloud storage disk. For example, it can create, delete, mount, and unmount cloud storage disks, and schedule which storage service node 102 is responsible for processing I / O requests for the cloud storage disk. Another example is specifying data synchronization and layout strategies for the cloud storage disk, where the data synchronization and layout strategies can reflect the correspondence between persistent files and storage service nodes. For example, the data synchronization and layout strategies can be data storage layout information. For details, see later. The storage service node 102 provides data interaction services for the cloud storage disk. For example, it is responsible for processing I / O requests from the cloud storage access node 106. Optionally, the storage service node 102 can also be responsible for mapping the logical address information of the corresponding cloud storage disk to the corresponding persistent file and switching and scheduling. For example, the storage service node 102 stores the data to be stored in the user's I / O request in the persistence layer of the cloud storage disk and records the corresponding data mapping relationship for subsequent reading.

[0033] Among them, the persistence layer includes: a persistence management node 103 and multiple persistent storage nodes 104. The persistence management node 103 is responsible for managing the metadata of the cloud storage disk persistence layer, and optionally provides management and control services for persistent files, for example, responsible for the creation or deletion of persistent files, and for example, maintaining the correspondence between persistent files and storage service nodes 102. Furthermore, the persistence management node 103 can also persist data on the corresponding storage service node 102 according to the data synchronization and layout strategy specified by the service layer. The storage service node 102 is responsible for responding to the persistence operation and subsequent read access of the corresponding data. Furthermore, the persistence layer can provide a software development kit (SDK) to the storage service node 102 of the service layer to facilitate the storage service node 102 to create, delete, read and write files in the persistence layer, so as to complete the read and write operations of the user IO.

[0034] In this embodiment, the implementation of the storage service management node 101, storage service node 102, persistence management node 103, and multiple persistence storage nodes 104 is not limited. In Figure 1, the storage service management node 101 is a cloud disk management server (Block Master, BM), the storage service node 102 is a cloud disk data server (Block Server, BS), the persistence management node 103 is a metadata server (Meta Server, MS), and the persistence storage node is a chunk server (Chunk Server, CS) as an example, but the present invention is not limited to this.

[0035] In this embodiment, the deployment method of the storage service management node 101, storage service node 102, persistence management node 103 and multiple persistent storage nodes 104 is not limited. In terms of deployment, in order to meet the disaster recovery of the availability zone (that is, when any availability zone is completely unavailable, the user's cloud storage disk can still perform read and write operations normally). The storage service management node 101 and the persistence management node 103 are deployed in multiple availability zones and achieve high availability (Highly Available, HA). The storage service node 102 and the persistent storage node 104 are deployed in each availability zone. When the availability zone is unavailable, the storage service management node 101 and the persistence management node 103 will schedule and schedule new requests to the normal storage service node 102 and persistent storage node 104. From the perspective of the availability zone, each availability zone can include: the storage service management node 101, one or more storage service nodes 102, the persistence management node 103 and at least one persistent storage node 102; further including: one or more computing nodes 105. In terms of specific deployment implementation, the storage service management node 101 can be deployed together with the storage service node 102 on a physical device (e.g., the same storage device), or the storage service management node 101 and the storage service node 102 can be deployed on different physical devices. The persistence management node 103 can be deployed together with the persistent storage node 104 on a physical device (e.g., the same storage device), or the persistence management node 103 and the persistent storage node 104 can be deployed on different physical devices.

[0036] In FIG1 , a plurality of persistent storage nodes 102 are distributed in three availability zones, and each availability zone includes three persistent storage nodes, a storage service management node, a storage service node, and a persistent management node. The three availability zones are: availability zone AZ1, availability zone AZ2, and availability zone AZ3. Availability zone AZ1 includes: storage service nodes. Availability zone AZ1 includes: storage service nodes 102 (BS1 and BS2 respectively), storage service management node 101 (such as BM1), persistent management node 103 (such as MS1), persistent storage node 104 (CS 1, CS2 and CS3); Availability Zone AZ2 also includes: storage service nodes 102 (BS3 and BS4 respectively), storage service management node 101 (such as BM2) and persistent management node 103 (such as MS2), persistent storage nodes 104 (such as CS4, CS5 and CS6); Availability Zone AZ3 also includes: storage service nodes 102 (BS5 and BS6 respectively), storage service management node 101 (such as BM3) and persistent management node 103 (such as MS3), persistent storage nodes 104 (CS7, CS8 and CS9 respectively). In Figure 1, data reading and writing on the data plane are represented by "solid lines", and the service node BS2 providing read and write services is used as an example for illustration. The interaction of metadata and management information on the control plane is represented by "dashed lines", where BM1 controls BS, and the interaction of control information between BM1, BM2 and BM3, and the interaction of control information between MS1, MS2 and MS3 are used as examples for illustration. Other data plane and control plane processes are not shown in Figure 1.

[0037] The following describes the detailed functions of the storage service management node 101, the storage service node 102, the persistence management node 103 and the persistent storage node 104 in the embodiment of the present disclosure.

[0038] In this embodiment, storage service management node 101 is responsible for creating a cloud storage disk spanning N availability zones for compute node 105, where N ≥ 2 and N is a positive integer. Furthermore, storage service management node 101 is responsible for allocating storage service nodes 102 to the cloud storage disk, so that the allocated storage service nodes 102 can respond to read and write requests initiated by compute node 105 to the cloud storage disk. The "spanning N availability zones" primarily means that the cloud storage disk can provide storage services for compute nodes in N availability zones, and the backend persistent storage nodes 104 corresponding to the cloud storage disk can be distributed across N availability zones.

[0039] The cloud storage disk is used to provide storage services for cloud computing instances on computing nodes, for example, to store various data generated by cloud computing instances on computing node 103. The cloud storage disk adopts a redundant storage method that combines multiple copies with erasure codes. The redundant storage method includes code sharding and multiple copies of data sharding. Multiple copies refer to copying the original data (e.g., data to be stored) into multiple copies for storage; erasure code is a data protection method that divides the original data into m data shards, encodes the m data shards to obtain n code shards, and stores the n code shards and m data shards. In the event of a hard disk failure, if any m shards (either data shards or code shards) can be obtained, the original data can be restored based on any m shards obtained. The redundant storage method that combines multiple copies with erasure codes is a redundant storage strategy that uses erasure codes to encode the original data (e.g., data to be stored) to obtain data shards and code shards, and performs multiple copies of the data shards and stores them across availability zones. Optionally, the encoding shards can be stored in a single copy, preferably a single copy across availability zones. Of course, the encoding shards can also be stored in multiple copies, which have a higher redundancy ratio than single copy storage.

[0040] In this embodiment, the cloud storage disk can use Logical Block Addressing (LBA) for addressing. The entire storage space is divided into multiple LBA intervals, and a certain amount of data is stored in these LBA intervals. The size of the LBA interval is not limited; for example, it can be 4GB, 8GB, or 16GB. The LBA address information can be mapped to the physical address information of the persistent storage node to enable reading and writing to the cloud storage disk.

[0041] Furthermore, the storage service management node 101 can also maintain metadata for the cloud storage disk. For example, when a tenant creates a cloud storage disk for a cloud computing instance on a compute node, they can provide the storage service management node 101 with information such as the tenant identifier, the required cloud storage disk size, and performance requirements. The storage service management node 101 then creates a cloud storage disk for the tenant's compute node based on the tenant's requirements and maintains metadata for the cloud storage disk. The metadata for the cloud storage disk can include, but is not limited to, disk identifier (diskid), disk size (disk size), tenant identifier (userid), creation time (create time), disk Input / Output Operations Per Second capacity (DIPS), disk throughput capacity, and disk status, and can be expressed as (diskid, disk size, userid, create time, disk IOPS capacity, disk throughput capacity, disk status).

[0042] Optionally, there are multiple storage service nodes 102, and the multiple storage service nodes 102 are distributed in different availability zones; the storage service management node 101 adopts a proximity scheduling strategy based on the availability zones to which the multiple storage service nodes 102 belong, and allocates storage service nodes 102 to the cloud storage disk, trying to ensure that the storage service nodes 102 that respond to the read and write requests of the cloud storage disk and the cloud storage access nodes 106 are in the same availability zone to improve the response speed.

[0043] In this embodiment, the persistence management node 103 can obtain cross-availability zone disaster recovery requirement information for the cloud storage disk. This cross-availability zone disaster recovery requirement information is the tenant's requirement information for the cloud storage disk's availability. For example, the cross-availability zone disaster recovery requirement information may include the number of availability zone failures or data shard failures that can be tolerated while ensuring that the cloud computing instance on the compute node can read and write data normally. Alternatively, the number of data shard failures that can be tolerated when a set number of availability zones are tolerated is also allowed. Based on the cross-availability zone disaster recovery requirement information, the persistence management node 103 generates the data storage layout information required for reading and writing to the cloud storage disk. The data storage layout information is used to describe how code shards and replicated data shards are stored in the cross-availability zone persistent storage nodes 104. Specifically, the data storage layout information reflects how code shards are stored on multiple persistent storage nodes 104 in N availability zones, and how replicated data shards are stored on multiple persistent storage nodes 104 in N availability zones. For example, when there are 3 availability zones, m data shards, and n coding shards, the data storage layout information can be: storing one replica in each of the two availability zones, the replica including: m data shards, and n coding shards distributed in the three availability zones; or, the data storage layout information can also be: storing one replica in each of the two availability zones, the replica including: m data shards, and storing n coding shards in another availability zone and any availability zone containing the replica.

[0044] In this embodiment, the storage service node 102 can obtain the data storage layout information provided by the persistence management node 103 from the persistence management node; in the case of a read and write request initiated by the computing node 105 to the cloud storage disk, it responds to the read and write request initiated by the computing node 105 to the cloud storage disk, and performs data read and write operations on the persistent storage nodes 104 across the availability zones based on the data storage layout information.

[0045] Optionally, the storage service node 102 may obtain the data storage layout information from the persistence management node 103 when first receiving a write request from the computing node 105 to the cloud storage disk. Alternatively, the storage service node 102 may obtain the data storage layout information from the persistence management node 103 when it is powered on and initialized.

[0046] In an embodiment of the present disclosure, a highly available cloud storage disk across availability zones is provided. Data storage layout information for the cloud storage disk is generated based on cross-availability zone disaster recovery requirements. Based on this data storage layout information, data read and write operations are performed on persistent storage nodes across availability zones. This data storage layout information can reflect the redundant storage method of the cloud storage disk, which combines multiple replicas with erasure coding, achieving high availability of data storage across availability zones and reducing the data redundancy ratio. Furthermore, this provides improved read and write performance and the elasticity requirements of availability zones, reduces cross-availability zone network bandwidth, and saves storage costs.

[0047] In an optional embodiment, the cross-availability zone disaster recovery requirement information includes: when M of the N availability zones are abnormal, the number of abnormal shards f that can be tolerated, that is, when M of the N availability zones are abnormal and f shards are abnormal in the non-abnormal availability zones, it is possible to ensure that the cloud storage disk can be read and written. The abnormal shards may include data shards and may also include encoding shards. Among them, 1≤M<N, f≥1, and M and f are both positive integers. In addition, M, N and f are all known values. For example, the cross-availability zone disaster recovery requirement information includes: when 1 of the 3 availability zones is abnormal, 2 abnormal shards can be tolerated, that is, when 1 of the 3 availability zones is abnormal and 2 shards are abnormal, normal reading and writing are still possible. Among them, the data storage layout information specifically includes: a storage mapping relationship between P*m+n shards and K persistent storage nodes, which means that P*m+n shards are stored in K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones. For example, they can be distributed in P availability zones, or in P+L availability zones. L is a positive integer. Preferably, L=1, and there is no limitation on this. Among them, 2≤P≤(N-1), n≥f, P, K, m and n are all positive integers, "*" represents multiplication operation, m and n are the number of data shards and coding shards in the erasure code, respectively, P represents the number of copies of m data shards, and different copies of m data shards are located in different availability zones.

[0048] In an optional embodiment, the P availability zones and K persistent storage nodes are selected based on meeting the following cross-availability zone disaster recovery requirements:

[0049] a) Ensure high availability across availability zones for reading and writing data on the cloud storage disk. In the worst-case scenario of data anomalies, even if M (e.g., M = 1) availability zones are anomalies and f (e.g., f is 1 or 2) data shards are unavailable, normal read and write access to the cloud storage disk is still guaranteed.

[0050] b) Ensure that cross-zone write traffic is minimized when writing data to the cloud storage disk. At the same time, data reads can be completed in the local zone, improving overall read and write performance and reducing cross-zone network bandwidth requirements.

[0051] c) When an availability zone or sharding anomaly occurs, the read data traffic across availability zones should be kept as small as possible.

[0052] d) Data redundancy is relatively low, and the data redundancy ratio is close to the number of copies P.

[0053] For example, to ensure that read data in two availability zones for user service disaster recovery is completed within the local availability zone, one replica can be placed in the local availability zone. At the same time, to meet the availability zone disaster recovery requirements, at least one replica is distributed in other availability zones, so the data redundancy ratio is close to 2.

[0054] Based on the above, an implementation method for generating data storage layout information required for reading and writing to a cloud storage disk specifically based on cross-availability zone disaster recovery demand information includes: determining the number n of coding shards and the number P of replicas based on the number N of available zones and the number f of abnormal shards; for example, a correspondence table between the number N of available zones, the number f of abnormal shards, the number n of coding shards, and the number P of replicas can be preset, and the correspondence maintained in the correspondence table can ensure that when anomalies occur in M ​​(e.g., M=1) available zones and f other shards are abnormal, read and write operations can still be performed normally; based on the number N of available zones and the number f of abnormal shards, the number n of coding shards and the number P of replicas can be determined from the correspondence table; the number m of data shards is preset, for example, m can be 12; in order to ensure that the read and write traffic across availability zones is as small as possible, data redundancy is relatively low, and disaster recovery service requirements are met, at least K persistent storage nodes are selected from P availability zones based on the number of shards to be stored P*m+n, P≤K≤(P*m+n). For example, a persistent storage node can be selected for each shard, or a persistent storage node can be selected for two shards, without limitation; different copies of the m data shards are located in different availability zones, ensuring that each of the P availability zones contains a copy, and data can be read locally. At least based on the principle that different copies of the m data shards are located in different availability zones, a storage mapping relationship between P*m+n shards and K persistent storage nodes is established.

[0055] The following provides a table of data storage layout information for different numbers of availability zones N and numbers of abnormal shards f, as shown in Table 1 on the next page. Among them, d1-dm are data shards, p1-p6 are bit-coded shards, and AZ1-AZ5 are availability zones. Table 1 uses the example of 2 replicas of m data shards. Furthermore, Table 1 also includes: the tolerable number of damages to any shard, the write traffic amplification factor across availability zones, and the read traffic amplification factor and data redundancy ratio across availability zones after any availability zone or shard is abnormal. Among them, the write traffic amplification factor across availability zones is the ratio of the write traffic across availability zones to the amount of user write data, the read traffic amplification factor across availability zones refers to the ratio of the read traffic across availability zones to the amount of user read data, and the data redundancy ratio refers to the ratio of the amount of user data to the amount of stored data.

[0056] Optionally, the implementation method of selecting K persistent storage nodes from at least P available zones based on the number of shards to be stored (P*m+n) is not limited and is described below with an example.

[0057] For example, according to the number of shards to be stored P*m+n, K persistent storage nodes are selected from P availability zones. For example, in Table 1, when P=2, N=2, and f=1,2,3, K persistent storage nodes are selected from the two availability zones of AZ1 and AZ2. Accordingly, an implementation method for establishing a storage mapping relationship between P*m+n shards and K persistent storage nodes based on at least the principle that different copies of m data shards are located in different availability zones includes: establishing a storage mapping relationship between P*m+n shards and K persistent storage nodes based on the principle that different copies of m data shards are located in different availability zones and n encoding shards are dispersedly stored in P availability zones.

[0058] For another example, based on the number of shards to be stored, P*m+n, K persistent storage nodes are selected from P+1 availability zones. For example, in Table 1, when P=2, N=3, and f=1, 2, and 3, K persistent storage nodes are selected from the three availability zones AZ1, AZ2, and AZ3; when P=2, N=4, 5, and f=1, K persistent storage nodes are selected from the three availability zones AZ1, AZ2, and AZ3. Accordingly, based on at least the principle that different replicas of the m data shards are located in different availability zones, the storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established as follows: based on the principle that different replicas of the m data shards are located in different availability zones, and the n encoding shards are dispersedly stored in P+1 availability zones, the storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established.

[0059] Optionally, when P = N-1, when a persistent storage node is selected for each data shard and coding shard, the number of persistent storage nodes K = (N-1)*m+n. Further optionally, when N = 3, K = 2*m+n. For example, when N = 3 and f = 2, AZ1 includes data shards d1-dm and coding shard p1, AZ2 includes data shards d1-dm and coding shard p2, and AZ3 includes coding shard p3. If a persistent storage node is selected for each data shard and coding shard, the number of K is 2*m+n.

[0060] Table 1 is as follows:

[0061] For another example, based on the number of shards to be stored, P*m+n, K persistent storage nodes are selected from P+2 availability zones. For example, in Table 1, when P=2, N=3, f=2,3 and N=5, f=2, K persistent storage nodes are selected from the four availability zones of AZ1, AZ2, AZ3 and AZ4. Accordingly, based on at least the principle that different copies of the m data shards are located in different availability zones, the storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established as follows: based on the principle that different copies of the m data shards are located in different availability zones and the n coding shards are dispersedly stored in P+2 availability zones, the storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established.

[0062] For another example, based on the number of shards to be stored, P*m+n, K persistent storage nodes are selected from P+3 availability zones. For example, in Table 1, when P=2, N=5, and f=3, K persistent storage nodes are selected from AZ1, AZ2, AZ3, AZ4, and AZ5 and these 5 availability zones. Accordingly, based on at least the principle that different copies of the m data shards are located in different availability zones, the storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established as follows: based on the principle that different copies of the m data shards are located in different availability zones, and the n coding shards are dispersedly stored in P+3 availability zones, the storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established.

[0063] In an optional embodiment, computing node 105 can implement read and write operations on the cloud storage disk through cloud storage access node 106. Storage service node 102 can respond to read and write requests initiated by computing nodes to the cloud storage disk and, based on the data storage layout information, perform data read and write operations on persistent storage nodes across availability zones. The following describes the processes for write and read operations.

[0064] Case 1: Write operation. Compute node 105 initiates a write request to the cloud storage disk, including the data to be stored. Storage service node 102 responds to the write request initiated by the compute node to the cloud storage disk and obtains the data to be stored. Using the erasure code, which contains m data shards and n encoding shards, the data to be stored is sharded and erasure-coded, resulting in m data shards and n encoding shards. Based on the storage mapping between the P*m+n shards and the K persistent storage nodes, as specified in the data storage layout information, the P*m data shards and n encoding shards are stored in the K persistent storage nodes.

[0065] Optionally, an implementation method of storing P*m data shards and n coding shards in K persistent storage nodes based on the storage mapping relationship between P*m+n shards and K persistent storage nodes included in the data storage layout information includes: storing the same copy of the m data shards in a persistent storage node located in the same availability zone, and the persistent storage node located in the same availability zone belongs to the K persistent storage nodes, and accordingly, P different copies are stored in P different availability zones; and dispersing the n coding shards to at least one persistent storage node among the K persistent storage nodes.

[0066] Further optionally, n coding fragments can be dispersed and stored in P available zones. For example, in Table 1, when P=2, N=2, and f=1,2,3, n coding fragments are dispersed in two available zones, AZ1 and AZ2. Alternatively, n coding fragments can be dispersed and stored in P+1 available zones. For example, in Table 1, when P=2, N=3, and f=1,2,3, n coding fragments are dispersed in three available zones, AZ1, AZ2, and AZ3. In the case of P=2, N=4,5, and f=1, n coding fragments are dispersed in three available zones, AZ1, AZ2, and AZ3. The shards are distributed in the three availability zones of AZ1, AZ2, and AZ3. The n coded shards can also be distributed and stored in P+2 or P+3 availability zones. For example, in Table 1, when P=2, N=3, f=2,3 and N=5, f=2, the n coded shards are distributed in the four availability zones of AZ1, AZ2, AZ3, and AZ4. For another example, in Table 1, when P=2, N=5, f=3, the n coded shards are distributed in AZ1, AZ2, AZ3, AZ4, and AZ5 and these five availability zones.

[0067] In either case, the storage service node provides the corresponding data shard or encoding shard to the corresponding persistent storage node, which then stores the corresponding data shard or encoding shard. Optionally, after successfully storing the corresponding data shard or encoding shard, the K persistent storage nodes return a storage success message to the storage service node.

[0068] In an optional embodiment, K persistent storage nodes can provide multiple persistent files through virtualization technology, each persistent file corresponds to a stripe group, each stripe group includes multiple stripes, and the multiple stripes correspond to physical address information (such as 4K or 8K, etc.) on the K persistent storage nodes. For example, each stripe corresponds to a physical storage unit on a persistent storage node. Each stripe group is used to store P*m+n shards, for example, each stripe stores one shard. In other words, the file address information of each persistent file corresponds to the physical address information of the P*m+n shards on the K persistent storage nodes.

[0069] Based on this, when the K persistent storage nodes return storage success information to the storage service node after successfully storing the corresponding data shards or encoding shards, the storage service node 102 receives the storage success information returned by the K persistent storage nodes after successfully storing the data shards or encoding shards; based on the storage success message, the mapping relationship between the first LBA address information of the data to be stored in the cloud storage disk and the first file address information is recorded, where the first file address information corresponds to the physical address information of the shard corresponding to the data to be stored on the K persistent storage nodes. For example, the first file address information can be <persistent layer file name, offset address, data length>, and the offset address and data length, as well as the length of the shard, can determine the shard corresponding to the first file address information.

[0070] It should be noted that when storing the data to be stored in K persistent storage nodes, the data length of the data to be stored can be accumulated to a certain length threshold before being split and encoded to obtain P*m+n fragments, which are stored in the K persistent storage nodes. The length of the data to be stored in each write request may not be sufficient for splitting and encoding. For example, if the data length in the write request is 4K, splitting and encoding is performed when the data length of the data to be stored is 16K. Therefore, the data to be stored in the write request can be accumulated and split and encoded after reaching a certain data length, obtaining P*m+n fragments, which are stored in the K persistent storage nodes.

[0071] In an optional embodiment, the persistent management node can also monitor the operating status of the availability zones to which the K persistent storage nodes belong; when an abnormality is detected in the first availability zone, the first data shard that needs to be stored in the first availability zone is stored in other availability zones, and after the first availability zone is restored, the first data shard is migrated to the persistent storage node in the first availability zone; or, the first encoding shard that needs to be stored in the first availability zone is stored in other availability zones, and after the first availability zone is restored, the first encoding shard is migrated to the persistent storage node in the first availability zone; or, the first data shard and the first encoding shard that need to be stored in the first availability zone are stored in other availability zones, and after the first availability zone is restored, the first data shard and the first encoding shard are migrated to the persistent storage node in the first availability zone.

[0072] Case 2: Read operation. Compute node 105 initiates a read request to the cloud storage disk. Storage service node 102 responds to the read request initiated by the compute node to the cloud storage disk and obtains the second LBA address information of the data to be read on the cloud storage disk. Based on the second LBA address information and the pre-recorded mapping relationship between each LBA address information and file address information on the cloud storage disk, storage service node 102 determines the second file address information corresponding to the second LBA address information. Based on the second file address information and the data storage layout information, storage service node 102 obtains the physical address information of at least one shard corresponding to the data to be read on K persistent storage nodes. For example, the second file address information is <persistence layer file name, offset address, data length>. Based on the offset address and data length, combined with the data length of the shard, the data shard corresponding to the to-be-read data can be determined. Based on the storage mapping relationship between the P*m+n shards and the K persistent storage nodes in the data storage layout information, the storage location information of the P*m+n shards corresponding to the persistence layer file name on the K persistent storage nodes is determined. Based on the storage location information of the P*m+n shards corresponding to the persistence layer file name on the K persistent storage nodes, the physical address information of at least one data shard corresponding to the to-be-read data on the K persistent storage nodes is determined. Based on the physical address information of at least one data shard corresponding to the to-be-read data on the K persistent storage nodes, the at least one data shard corresponding to the to-be-read data is read from the corresponding persistent storage node.

[0073] In an optional embodiment, when the storage service node 102 reads at least one data shard corresponding to the data to be read from the corresponding persistent storage node based on the physical address information of at least one data shard corresponding to the data to be read on K persistent storage nodes, there may be an availability zone abnormality or a shard abnormality. When the number of abnormal availability zones is less than or equal to M and the number of abnormal shards in the non-abnormal availability zones is less than or equal to f, the storage service node can still read the data normally, which is explained below as an example.

[0074] Example A1: If the local availability zone where the storage service node is located is normal, then read at least one data shard from the persistent storage node in the local availability zone based on the physical address information of at least one data shard corresponding to the data to be read on the persistent storage node in the local availability zone.

[0075] Example A2: If the local availability zone is abnormal and at least one data shard corresponding to the data to be read in a non-abnormal availability zone is normal, then read the at least one data shard from the persistent storage node in the non-abnormal availability zone based on the physical address information of the at least one data shard corresponding to the data to be read on the persistent storage node in the non-abnormal availability zone.

[0076] Example A3: If the local availability zone is abnormal and at least one data shard corresponding to the data to be read in a normal availability zone contains an abnormal data shard, then m normal shards are read from the normal availability zone, and the abnormal data shard is restored based on the m normal shards. At least one data shard corresponding to the data to be read is obtained from the m normal shards and the restored abnormal shard.

[0077] The following provides a scenario-based embodiment, taking a cloud storage disk across three AZs as an example to describe the data storage layout information that meets the aforementioned cross-availability zone disaster recovery requirements (ad).

[0078] Specific parameters: Number of availability zones N = 3, with availability zones AZ1, AZ2, and AZ3. Number of data shards: m = 12; number of shard anomalies f = 2 that can be tolerated when one AZ is abnormal; availability zones AZ1 (primary) and AZ2 (backup) where user services are deployed; data storage layout information for data shard d and encoding shard p:

[0079] AZ1:d1,d2,d3,d4,d5,d6,d7,d8,d9,d10,d11,d12,p1;

[0080] AZ2:d1,d2,d3,d4,d5,d6,d7,d8,d9,d10,d11,d12,p2;

[0081] AZ3:p3.

[0082] The following describes how data storage layout information meets cross-availability zone disaster recovery requirements (ad):

[0083] 1. When one AZ is abnormal and f=2 shards are abnormal, normal reading and writing are still possible.

[0084] Assume that AZ1 is abnormal, and AZ2's shards dx, dy (x≠y, and x, y∈{1-12}) are also abnormal. If the data of dz is read and dz is not equal to dx or dy, then the corresponding shard dz is read directly from AZ2; if dz is equal to dx or dy, then dz is regenerated through erasure coding (12 normal shards are required). At this time, AZ2 includes 11 normal shards (AZ2 has a total of 13 shards, two abnormal shards, and normal shards 13-2=11), and AZ3 has 1 normal shard. Therefore, obtain AZ3's shard p3, and reconstruct the data shard of dz based on the normal 11 shards and shard p3.

[0085] 2. For new write requests, when an AZ is abnormal, the persistence management node 103 constructs new data storage layout information, temporarily stores write requests for the abnormal AZ in other AZs, and after the abnormal AZ returns to normal, migrates the data shards that need to be written to the abnormal AZ in other AZs to the restored AZ to meet disaster recovery requirements. For example, if AZ1 is abnormal, the persistence management node 103 notifies the storage service node to store data shards d1-d12 that need to be written to AZ1 in AZ2 and AZ3 respectively, p1 and p2 in AZ2, and p3 in AZ3. When AZ1 recovers, d1-d12 in AZ3 are migrated back to AZ1, and p1 in AZ2 is migrated back to AZ1.

[0086] 3. The data storage layout information enables the user's read request in AZ1 / AZ2 to be completed locally. For write requests, the traffic across availability zones is 13 shards in AZ2 (12 data shards and 1 encoding shard) and 1 encoding shard in AZ3. The amount of user data is 12 data shards, and the traffic across availability zones is (13+1) / 12=1.17 times the amount of user data (minimum value 1 times). The data storage layout information of 3 copies (each of the 3 availability zones stores one copy) requires the traffic to be written to the other 2 availability zones, that is, the traffic across availability zones is twice the amount of user data. The data storage layout information provided by the embodiment of the present disclosure has better performance in cross-availability zone write traffic than the data layout strategy of 3 copies.

[0087] 4. When an abnormality occurs in a data shard of AZ1, the corresponding data read request will be sent to AZ2. Therefore, the data traffic amplification rate across availability zones is 1. In the data encoding and layout scheme of EC (12, 9), there are 3 availability zones, 12 data shards for each replica, and 9 encoding shards. Each AZ includes 4 data shards and 3 encoding shards. For example, if a data shard in AZ1 is abnormal, there are 3 data and 3 encoding shards, a total of 6 normal shards. To construct the abnormal data shard, it is necessary to obtain 3 encoding shards from AZ2 and AZ3 respectively, and read 6 normal shards across availability zones to construct the abnormal shard. Therefore, the data read amplification across availability zones of the EC (12, 9) scheme is 6. Compared with the EC (12, 9) layout scheme, the data storage layout information provided by the embodiment of the present disclosure has better performance in data reading across availability zones.

[0088] 5. The data storage layout information provided by the embodiment of the present disclosure enables a data redundancy ratio of (12+12+3) / 12=2.25, which is reduced by (3-2.25) / 3=25% compared to the 3-copy data redundancy ratio (3 / 1).

[0089] In the embodiment of the present disclosure, the logical storage cluster deployed across availability zones has strong scalability (supporting a variable number of availability zones N), shielding the user from the details of internal data synchronization and disaster recovery, and presenting a logical cloud storage disk that is disaster-tolerant across N availability zones to the user. The user does not need to understand and maintain the replication logic between the master and slave availability zones and the binding relationship between the master and slave cloud storage disks. In addition, high availability of data across availability zones is achieved directly through the data consistency read and write mechanism of the underlying distributed storage, without the need for a master-slave replication mechanism, thereby reducing the overall complexity of the cloud storage system. Furthermore, by scheduling access locally and using multiple copies-erasure code fusion redundant encoding and layout methods for data reading and writing, it not only meets the requirements of better performance and elasticity of reading and writing, but also reduces the demand for network bandwidth across availability zones and maintains a low data redundancy ratio. In addition to providing a system embodiment, the embodiment of the present disclosure also provides a data reading and writing method. The process of the data reading and writing method provided by the embodiment of the present disclosure is described below.

[0090] FIG2a is a flow chart of a data reading and writing method provided by an exemplary embodiment of the present disclosure. The method is applicable to a storage service management node in a cloud storage system. As shown in FIG2a , the method includes:

[0091] 201a. Create a cloud storage disk across N availability zones for the compute node, where N ≥ 2 and N is a positive integer. The cloud storage disk uses a redundant storage method that combines multiple replicas with erasure coding. The redundant storage method includes code sharding and multiple replica data sharding.

[0092] 201b. Allocate storage service nodes to the cloud storage disk, so that the allocated storage service nodes perform data read and write operations on persistent storage nodes across availability zones based on the data storage layout information. The data storage layout information is generated by the persistent management node in the cloud storage system based on the cross-availability zone disaster recovery requirement information of the cloud storage disk.

[0093] In an optional embodiment, there are multiple storage service nodes, and the multiple storage service nodes are distributed in different availability zones; accordingly, the storage service nodes are allocated to the cloud storage disk, including: according to the availability zones to which the multiple storage service nodes belong, a proximity scheduling strategy is adopted to allocate storage service nodes to the cloud storage disk, and try to ensure that the storage service node that responds to the read and write requests of the cloud storage disk and the cloud storage access node that initiates the read and write requests are in the same availability zone to improve the response speed.

[0094] FIG2 b is a flow chart of another data reading and writing method provided by an exemplary embodiment of the present disclosure. The method is applicable to a persistent management node in a cloud storage system. As shown in FIG2 b , the method includes:

[0095] 201b. Obtain cross-availability zone disaster recovery requirement information for the cloud storage disk; the cloud storage disk spans N availability zones and uses a redundant storage method that combines multiple replicas with erasure coding. The redundant storage method includes code sharding and multiple replica data sharding, where N is a positive integer and is greater than or equal to 2.

[0096] 202b. Generate data storage layout information required for reading and writing to the cloud storage disk based on the cross-availability zone disaster recovery requirement information. The data storage layout information is used to describe how the encoding shards and the multiple replicas of the data shards are stored in the persistent storage nodes across the availability zones.

[0097] 203b. Provide the data storage layout information to the storage service node, so that the allocated storage service node performs data read and write operations on the persistent storage nodes across the available zones according to the data storage layout information.

[0098] In an optional embodiment, the cross-availability zone disaster recovery requirement information includes the number f of abnormal shards that can be tolerated when M availability zones among N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; wherein the data storage layout information specifically includes the storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; wherein m and n are the number of data shards and coding shards in the erasure code, respectively, P represents the number of copies of the m data shards, different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, and P, K, m and n are all positive integers.

[0099] In an optional embodiment, based on the cross-availability zone disaster recovery demand information, the data storage layout information required for reading and writing the cloud storage disk is generated, including: determining the number n of coding shards and the number P of replicas based on the number N of availability zones and the number f of abnormal shards, and the number m of data shards is preset; based on the number P*m+n of shards to be stored, selecting K persistent storage nodes from at least P availability zones, where P≤K≤(P*m+n); and establishing a storage mapping relationship between P*m+n shards and K persistent storage nodes based on the principle that different replicas of at least m data shards are located in different availability zones.

[0100] Optionally, based on the number of shards to be stored P*m+n, at least K persistent storage nodes are selected from P availability zones, including: based on the number of shards to be stored P*m+n, K persistent storage nodes are selected from P+1 availability zones; accordingly, based on the principle that at least different copies of the m data shards are located in different availability zones, a storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established, including: based on the principle that different copies of the m data shards are located in different availability zones and the n coding shards are dispersedly stored in P+1 availability zones, a storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established.

[0101] Further optionally, P=N-1, and K=(N-1)*m+n.

[0102] Further optionally, N=3, K=2*m+n.

[0103] FIG2c is a flow chart of another data reading and writing method provided by an exemplary embodiment of the present disclosure. The method is applicable to a storage service node in a cloud storage system. As shown in FIG2c , the method includes:

[0104] 201c. Receive data storage layout information required for reading and writing a cloud storage disk from a persistence management node in the cloud storage system. The data storage layout information is generated by the persistence management node based on cross-availability zone disaster recovery requirement information of the cloud storage disk.

[0105] 202c. Respond to read and write requests initiated by computing nodes to the cloud storage disk, and perform data read and write operations on persistent storage nodes across availability zones based on the data storage layout information; wherein the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies with erasure codes. The redundant storage method includes code sharding and multiple copies of data sharding, where N ≥ 2, and N is a positive integer.

[0106] In an optional embodiment, data storage layout information required for reading and writing the cloud storage disk is received from a persistent management node in the cloud storage system, including: when a write request initiated by a computing node to the cloud storage disk is received for the first time, the data storage layout information required for reading and writing the cloud storage disk is obtained from the persistent management node in the cloud storage system.

[0107] In an optional embodiment, the cross-availability zone disaster recovery demand information includes the number f of abnormal shards that can be tolerated when M availability zones in N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; the data storage layout information specifically includes the storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; where m and n are the number of data shards and coding shards in the erasure code, respectively, P represents the number of copies of the m data shards, and different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, and P, K, m and n are all positive integers. integer; accordingly, responding to read and write requests initiated by the computing node to the cloud storage disk, performing data read and write operations on the persistent storage nodes across the availability zones according to the data storage layout information, including: responding to write requests initiated by the computing node to the cloud storage disk to obtain the data to be stored; sharding and erasure coding the data to be stored using the number m of data shards and the number n of coding shards in the erasure code to obtain m data shards and n coding shards; and storing the P*m data shards and n coding shards in the K persistent storage nodes according to the storage mapping relationship between the P*m+n shards and the K persistent storage nodes included in the data storage layout information.

[0108] Optionally, according to the storage mapping relationship between P*m+n shards and K persistent storage nodes included in the data storage layout information, P*m data shards and n coding shards are stored in K persistent storage nodes, including: storing the same copy of the m data shards in a persistent storage node in the same availability zone among the K persistent storage nodes, and storing P different copies in persistent storage nodes in P different availability zones; and dispersing the n coding shards in at least one persistent storage node among the K persistent storage nodes.

[0109] Further optionally, K persistent storage nodes are distributed in P+1 availability zones, and the n coding shards are distributed and stored in at least one persistent storage node among the K persistent storage nodes, including: distributing and storing the n coding shards in P+1 availability zones.

[0110] In an optional embodiment, the method provided by the embodiment of the present disclosure also includes: receiving storage success information returned by K persistent storage nodes after successfully storing data shards or coding shards; based on the storage success message, recording the mapping relationship between the first LBA address information of the data to be stored in the cloud storage disk and the first file address information, the first file address information corresponding to the physical address information of the shard corresponding to the data to be stored on the K persistent storage nodes.

[0111] In an optional embodiment, the method provided by the embodiment of the present disclosure also includes: monitoring the operating status of the availability zones to which the K persistent storage nodes belong; when an abnormal first availability zone is detected, the first data shard and / or the first encoding shard that needs to be stored in the first availability zone is stored in other availability zones, and after the first availability zone is restored, the first data shard and / or the first encoding shard is migrated to the persistent storage node in the first availability zone.

[0112] In an optional embodiment, in response to a read and write request initiated by a computing node to a cloud storage disk, data read and write operations are performed on persistent storage nodes across availability zones according to the data storage layout information, including: in response to a read request initiated by a computing node to a cloud storage disk, obtaining second LBA address information of the data to be read in the cloud storage disk; determining second file address information according to the second LBA address information and a mapping relationship between each LBA address information and file address information in the pre-recorded cloud storage disk; obtaining physical address information of at least one data shard corresponding to the data to be read on K persistent storage nodes according to the second file address information and the data storage layout information; and reading at least one data shard corresponding to the data to be read from the corresponding persistent storage node according to the physical address information of at least one data shard corresponding to the data to be read on the K persistent storage nodes.

[0113] Optionally, based on the physical address information of at least one data shard corresponding to the data to be read on the K persistent storage nodes, read at least one data shard corresponding to the data to be read from the corresponding persistent storage node, including: when the number of abnormal availability zones is less than or equal to M and the number of abnormal shards in the non-abnormal availability zones is less than or equal to f, if the local availability zone where the storage service node is located is normal, then based on the physical address information of at least one data shard corresponding to the data to be read on the persistent storage node in the local availability zone, read at least one data shard from the persistent storage node in the local availability zone; if the local availability zone is abnormal and in the non-abnormal availability zone If at least one data shard corresponding to the data to be read in the abnormal availability zone is normal, then at least one data shard is read from the persistent storage node in the non-abnormal availability zone based on the physical address information of the at least one data shard corresponding to the data to be read on the persistent storage node in the non-abnormal availability zone; if the local availability zone is abnormal and there is an abnormal data shard in at least one data shard corresponding to the data to be read in the non-abnormal availability zone, then m normal shards are read from the non-abnormal availability zone, and the abnormal data shard is restored based on the m normal shards; and at least one data shard corresponding to the data to be read is obtained from the m normal shards and the restored abnormal shard.

[0114] The detailed implementation and beneficial effects of each step in the method shown in Figures 2a to 2c provided in the embodiments of the present disclosure have been described in detail in the aforementioned embodiments and will not be elaborated on here.

[0115] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 201a to 203a can be device A; for another example, the execution entity of steps 201a and 202a can be device A, and the execution entity of step 203a can be device B; and so on.

[0116] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The sequence numbers of the operations, such as 201a, 202a, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0117] FIG3a is a schematic structural diagram of a data reading and writing device provided by an exemplary embodiment of the present disclosure. The device corresponds to a storage service management node in a cloud storage system. As shown in FIG3a , the device includes: a creation module 31a and an allocation module 32a.

[0118] Creation module 31a is used to create a cloud storage disk across N availability zones for the computing node, where N ≥ 2 and N is a positive integer; the cloud storage disk adopts a redundant storage method that combines multiple copies with erasure coding, and the redundant storage method includes code sharding and multiple copies of data sharding;

[0119] The allocation module 32a is used to allocate storage service nodes to the cloud storage disk, so that the allocated storage service nodes can perform data read and write operations on persistent storage nodes across availability zones based on the data storage layout information. The data storage layout information is generated by the persistent management node in the cloud storage system based on the cross-availability zone disaster recovery requirement information of the cloud storage disk.

[0120] In an optional embodiment, there are multiple storage service nodes, and the multiple storage service nodes are distributed in different availability zones; the allocation module is specifically used to: adopt a proximity scheduling strategy based on the availability zones to which the multiple storage service nodes belong, to allocate storage service nodes to the cloud storage disk, and try to ensure that the storage service node that responds to the read and write requests of the cloud storage disk and the cloud storage access node that initiates the read and write requests are in the same availability zone to improve the response speed.

[0121] FIG3 b is a schematic structural diagram of another data reading and writing device provided by an exemplary embodiment of the present disclosure, corresponding to a persistence management node in a cloud storage system. The device includes: an acquisition module 31 b , a generation module 32 b , and a providing module 33 b .

[0122] Acquisition module 31b is used to obtain cross-availability zone disaster recovery requirement information of the cloud storage disk; the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies with erasure coding. The redundant storage method includes code sharding and multiple copies of data sharding, where N ≥ 2, and N is a positive integer;

[0123] A generation module 32b is used to generate data storage layout information required for reading and writing to the cloud storage disk based on the cross-availability zone disaster recovery requirement information. The data storage layout information is used to describe how the encoding shards and the multiple replicas of the data shards are stored in the persistent storage nodes across the availability zones.

[0124] The providing module 33b is used to provide the data storage layout information to the storage service node, so that the allocated storage service node can perform data read and write operations on the persistent storage nodes across the available zones according to the data storage layout information.

[0125] In an optional embodiment, the cross-availability zone disaster recovery requirement information includes the number f of abnormal shards that can be tolerated when M availability zones among N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; wherein the data storage layout information specifically includes the storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; wherein m and n are the number of data shards and coding shards in the erasure code, respectively, P represents the number of copies of the m data shards, different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, and P, K, m and n are all positive integers.

[0126] In an optional embodiment, the generation module is specifically used to: determine the number n of coding shards and the number P of replicas based on the number N of available zones and the number f of abnormal shards, where the number m of data shards is preset; select K persistent storage nodes from at least P available zones based on the number P*m+n of shards to be stored, where P≤K≤(P*m+n); establish a storage mapping relationship between P*m+n shards and K persistent storage nodes based on the principle that different replicas of m data shards are located in different available zones.

[0127] Optionally, the generation module is specifically used to: select K persistent storage nodes from P+1 availability zones based on the number of shards to be stored P*m+n; establish a storage mapping relationship between P*m+n shards and K persistent storage nodes based on the principle that different copies of m data shards are located in different availability zones and n encoding shards are dispersedly stored in P+1 availability zones.

[0128] Further optionally, P=N-1, and K=(N-1)*m+n.

[0129] Further optionally, N=3, K=2*m+n.

[0130] FIG3 c is a schematic structural diagram of another data reading and writing device provided by an exemplary embodiment of the present disclosure. The device corresponds to a storage service node in a cloud storage system and includes a receiving module 31 c and a reading and writing module 32 c.

[0131] Receiving module 31c, configured to receive data storage layout information required for reading and writing cloud storage disks from a persistence management node in the cloud storage system. The data storage layout information is generated by the persistence management node based on the cross-availability zone disaster recovery requirement information of the cloud storage disks;

[0132] The read / write module 32c is used to respond to the read / write requests initiated by the computing node to the cloud storage disk, and perform data read / write operations on the persistent storage nodes across the availability zones according to the data storage layout information; wherein, the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies with erasure codes. The redundant storage method includes code sharding and multiple copies of data sharding, N ≥ 2, and N is a positive integer.

[0133] In an optional embodiment, the receiving module is specifically used to: when receiving a write request initiated by a computing node to a cloud storage disk for the first time, obtain data storage layout information required for reading and writing the cloud storage disk from a persistent management node in the cloud storage system.

[0134] In an optional embodiment, the cross-availability zone disaster recovery requirement information includes the number f of abnormal shards that can be tolerated when M availability zones out of N availability zones are abnormal, where 1≤M<N, f≥1, and both M and f are positive integers; the data storage layout information specifically includes the storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; where m and n are the number of data shards and code shards in the erasure code, respectively, and P represents the number of copies of the m data shards, and different copies of the m data shards are located in different availability zones. , and 2≤P≤(N-1), n≥f, P, K, m and n are all positive integers; the read-write module is specifically used to: respond to the write request initiated by the computing node to the cloud storage disk to obtain the data to be stored; use the number m of data shards and the number n of coding shards in the erasure code to shard and perform erasure code encoding on the data to be stored, and obtain m data shards and n coding shards; according to the storage mapping relationship between the P*m+n shards and the K persistent storage nodes included in the data storage layout information, store the P*m data shards and n coding shards in the K persistent storage nodes.

[0135] Optionally, the read-write module is specifically used to: store the same copy of m data shards in a persistent storage node located in the same availability zone among K persistent storage nodes, and store P different copies in persistent storage nodes in P different availability zones; and store n coding shards in a dispersed manner in at least one persistent storage node among the K persistent storage nodes.

[0136] Further optionally, the K persistent storage nodes are distributed in P+1 availability zones, and the read-write module is specifically used to disperse and store the n encoding fragments in the P+1 availability zones.

[0137] In an optional embodiment, the data reading and writing device provided by the embodiment of the present disclosure also includes: a recording module; a receiving module, used to receive storage success information returned by K persistent storage nodes after successfully storing data shards or encoding shards; a recording module, used to record the mapping relationship between the first LBA address information of the data to be stored in the cloud storage disk and the first file address information based on the storage success message, and the first file address information corresponds to the physical address information of the shard corresponding to the data to be stored on the K persistent storage nodes.

[0138] In an optional embodiment, the data reading and writing device provided by the embodiment of the present disclosure also includes: a monitoring module, a storage module and a migration module; the monitoring module is used to monitor the operating status of the availability zones to which the K persistent storage nodes belong; the storage module is used to store the first data shard and / or the first coding shard that needs to be stored in the first availability zone in other availability zones when an abnormality is detected in the first availability zone, and the migration module is used to migrate the first data shard and / or the first coding shard to the persistent storage node in the first availability zone after the first availability zone is restored.

[0139] In an optional embodiment, the read-write module is specifically used to: respond to a read request initiated by a computing node to the cloud storage disk, and obtain the second LBA address information of the data to be read in the cloud storage disk; determine the second file address information based on the second LBA address information and the mapping relationship between each LBA address information and file address information in the pre-recorded cloud storage disk; obtain the physical address information of at least one data shard corresponding to the data to be read on K persistent storage nodes based on the second file address information and data storage layout information; read at least one data shard corresponding to the data to be read from the corresponding persistent storage node based on the physical address information of at least one data shard corresponding to the data to be read on the K persistent storage nodes.

[0140] Optionally, the read-write module is specifically used to: when the number of abnormal availability zones is less than or equal to M and the number of abnormal shards in the non-abnormal availability zone is less than or equal to f, if the local availability zone where the storage service node is located is normal, then according to the physical address information of at least one data shard corresponding to the data to be read on the persistent storage node in the local availability zone, read at least one data shard from the persistent storage node in the local availability zone; if the local availability zone is abnormal and at least one data shard corresponding to the data to be read in the non-abnormal availability zone is normal, then according to the physical address information of at least one data shard corresponding to the data to be read on the persistent storage node in the non-abnormal availability zone, read at least one data shard from the persistent storage node in the non-abnormal availability zone; if the local availability zone is abnormal and there is an abnormal data shard in at least one data shard corresponding to the data to be read in the non-abnormal availability zone, read m normal shards from the non-abnormal availability zone, and restore the abnormal data shard based on the m normal shards; and obtain at least one data shard corresponding to the data to be read from the m normal shards and the restored abnormal shards.

[0141] The detailed implementation and beneficial effects of each step in the apparatus shown in FIG. 3 a - FIG. 3 c provided in the embodiment of the present disclosure have been described in detail in the aforementioned embodiments and will not be elaborated on here.

[0142] FIG4 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. The electronic device corresponds to a storage service management node in a cloud storage system. As shown in FIG4 , the device includes a memory 44 and a processor 45 .

[0143] The memory 44 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method used to operate on the electronic device.

[0144] A processor 45 is coupled to the memory 44 and is used to execute a computer program in the memory 44 to: create a cloud storage disk across N availability zones for a computing node, where N ≥ 2, and N is a positive integer; the cloud storage disk adopts a redundant storage method that integrates multiple copies and erasure codes, and the redundant storage method includes code sharding and multiple copies of data sharding; and allocates storage service nodes to the cloud storage disk so that the allocated storage service nodes perform data read and write operations on persistent storage nodes across availability zones based on data storage layout information, where the data storage layout information is generated by a persistent management node in the cloud storage system based on the cross-availability zone disaster recovery requirement information of the cloud storage disk.

[0145] In an optional embodiment, there are multiple storage service nodes, and the multiple storage service nodes are distributed in different availability zones; when the processor 45 allocates storage service nodes to the cloud storage disk, it is specifically used to: adopt a proximity scheduling strategy based on the availability zones to which the multiple storage service nodes belong, and allocate storage service nodes to the cloud storage disk, and try to ensure that the storage service node that responds to the read and write requests of the cloud storage disk and the cloud storage access node that initiates the read and write requests are in the same availability zone to improve the response speed.

[0146] Furthermore, as shown in Figure 4, the electronic device also includes other components such as a communication component 46, a display 47, a power supply component 48, and an audio component 49. Figure 4 only schematically illustrates some components, which does not mean that the electronic device only includes the components shown in Figure 4. In addition, the components within the dotted box in Figure 4 are optional components, not mandatory components, and the specific requirements may depend on the product form of the electronic device. The electronic device of this embodiment can implement a server-side device such as a conventional server, a cloud server, or a server array.

[0147] The present disclosure also provides an electronic device corresponding to a persistent management node in a cloud storage system. The implementation structure of the electronic device is the same as or similar to the implementation structure of the electronic device shown in FIG4 , and can be implemented with reference to the structure of the electronic device shown in FIG4 . The difference between the electronic device provided in this embodiment and the electronic device in the embodiment shown in FIG4 is mainly that the functions implemented by the processor executing the computer program stored in the memory are different. For the electronic device provided in this embodiment, its processor executes the computer program stored in the memory, which can be used to: obtain cross-availability zone disaster recovery demand information of the cloud storage disk; the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies and erasure codes, the redundant storage method includes code sharding and multiple copies of data sharding, N ≥ 2, and N is a positive integer; based on the cross-availability zone disaster recovery demand information, generate data storage layout information required for reading and writing the cloud storage disk, the data storage layout information is used to describe the storage method of the code sharding and the multiple copies of data sharding in the persistent storage nodes across the availability zones; provide the data storage layout information to the storage service node, so that the assigned storage service node can perform data read and write operations on the persistent storage nodes across the availability zones according to the data storage layout information.

[0148] In an optional embodiment, the cross-availability zone disaster recovery requirement information includes the number f of abnormal shards that can be tolerated when M availability zones among N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; wherein the data storage layout information specifically includes the storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; wherein m and n are the number of data shards and coding shards in the erasure code, respectively, P represents the number of copies of the m data shards, different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, and P, K, m and n are all positive integers.

[0149] In an optional embodiment, when the processor generates the data storage layout information required for reading and writing the cloud storage disk based on the cross-availability zone disaster recovery demand information, it is specifically used to: determine the number n of coding shards and the number P of replicas based on the number of availability zones N and the number of abnormal shards f, where the number m of data shards is preset; based on the number P*m+n of shards to be stored, select K persistent storage nodes from at least P availability zones, where P≤K≤(P*m+n); and establish a storage mapping relationship between P*m+n shards and K persistent storage nodes based on the principle that different replicas of m data shards are located in different availability zones.

[0150] Optionally, when the processor selects K persistent storage nodes from at least P availability zones based on the number of shards to be stored P*m+n, the processor is specifically used to: select K persistent storage nodes from P+1 availability zones based on the number of shards to be stored P*m+n; accordingly, when the processor establishes a storage mapping relationship between P*m+n shards and K persistent storage nodes based at least on the principle that different copies of m data shards are located in different availability zones, the processor is specifically used to: establish a storage mapping relationship between P*m+n shards and K persistent storage nodes based on the principle that different copies of m data shards are located in different availability zones and n coding shards are dispersedly stored in P+1 availability zones.

[0151] Further optionally, P=N-1, and K=(N-1)*m+n.

[0152] Further optionally, N=3, K=2*m+n.

[0153] The present disclosure also provides an electronic device corresponding to a storage service node in a cloud storage system. The implementation structure of the electronic device is the same as or similar to the implementation structure of the electronic device shown in FIG4 , and can be implemented with reference to the structure of the electronic device shown in FIG4 . The difference between the electronic device provided in this embodiment and the electronic device in the embodiment shown in FIG4 is mainly that the functions implemented by the processor executing the computer program stored in the memory are different. For the electronic device provided in this embodiment, its processor executes the computer program stored in the memory, and can be used to: receive data storage layout information required for reading and writing a cloud storage disk sent by a persistent management node in the cloud storage system, the data storage layout information is generated by the persistent management node based on the cross-availability zone disaster recovery requirement information of the cloud storage disk; respond to read and write requests initiated by a computing node to the cloud storage disk, and perform data read and write operations on persistent storage nodes across availability zones according to the data storage layout information; wherein the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies with erasure codes. The redundant storage method includes code sharding and multiple copies of data sharding, N≥2, and N is a positive integer.

[0154] In an optional embodiment, when the processor receives the data storage layout information required for reading and writing the cloud storage disk sent by the persistence management node in the cloud storage system, it is specifically used to: when receiving the write request initiated by the computing node to the cloud storage disk for the first time, obtain the data storage layout information required for reading and writing the cloud storage disk from the persistence management node in the cloud storage system.

[0155] In an optional embodiment, the cross-availability zone disaster recovery demand information includes the number f of abnormal shards that can be tolerated when M availability zones in N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; the data storage layout information specifically includes the storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; where m and n are the number of data shards and coding shards in the erasure code, respectively, P represents the number of copies of the m data shards, and different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, and P, K, m and n are all positive integers. number; when the processor responds to the read and write requests initiated by the computing node to the cloud storage disk and performs data read and write operations on the persistent storage nodes across the availability zones according to the data storage layout information, it is specifically used to: respond to the write requests initiated by the computing node to the cloud storage disk to obtain the data to be stored; use the number m of data shards and the number n of coding shards in the erasure code to shard and perform erasure code encoding processing on the data to be stored to obtain m data shards and n coding shards; according to the storage mapping relationship between the P*m+n shards included in the data storage layout information and the K persistent storage nodes, store the P*m data shards and n coding shards in the K persistent storage nodes.

[0156] Optionally, when the processor stores P*m data shards and n coding shards in K persistent storage nodes according to the storage mapping relationship between P*m+n shards and K persistent storage nodes included in the data storage layout information, it is specifically used to: store the same copy of the m data shards in a persistent storage node in the same availability zone among the K persistent storage nodes, and store P different copies in persistent storage nodes in P different availability zones; and store the n coding shards in a dispersed manner in at least one persistent storage node among the K persistent storage nodes.

[0157] Further optionally, K persistent storage nodes are distributed in P+1 availability zones. Then, when the processor disperses and stores n coding shards in at least one persistent storage node among the K persistent storage nodes, it is specifically used to: disperse and store the n coding shards in P+1 availability zones.

[0158] In an optional embodiment, the processor is also used to: receive storage success information returned by K persistent storage nodes after successfully storing data shards or encoding shards; based on the storage success message, record the mapping relationship between the first LBA address information of the data to be stored in the cloud storage disk and the first file address information, and the first file address information corresponds to the physical address information of the shard corresponding to the data to be stored on the K persistent storage nodes.

[0159] In an optional embodiment, the processor is also used to: monitor the operating status of the availability zones to which the K persistent storage nodes belong; when an abnormality is detected in the first availability zone, the first data shard and / or the first encoding shard that needs to be stored in the first availability zone is stored in other availability zones, and after the first availability zone is restored, the first data shard and / or the first encoding shard is migrated to the persistent storage node in the first availability zone.

[0160] In an optional embodiment, when the processor responds to a read and write request initiated by a computing node to a cloud storage disk and performs data read and write operations on persistent storage nodes across availability zones according to the data storage layout information, the processor is specifically used to: respond to a read request initiated by a computing node to the cloud storage disk, and obtain the second LBA address information of the data to be read in the cloud storage disk; determine the second file address information according to the second LBA address information and the mapping relationship between each LBA address information and the file address information in the pre-recorded cloud storage disk; obtain the physical address information of at least one data shard corresponding to the data to be read on the K persistent storage nodes according to the second file address information and the data storage layout information; and read at least one data shard corresponding to the data to be read from the corresponding persistent storage node according to the physical address information of at least one data shard corresponding to the data to be read on the K persistent storage nodes.

[0161] In an optional embodiment, when the processor reads at least one data shard corresponding to the data to be read from the corresponding persistent storage node according to the physical address information of at least one data shard corresponding to the data to be read on K persistent storage nodes, the processor is specifically used to: when the number of abnormal availability zones is less than or equal to M and the number of abnormal shards in the non-abnormal availability zones is less than or equal to f, if the local availability zone where the storage service node is located is normal, then read at least one data shard from the persistent storage node in the local availability zone according to the physical address information of at least one data shard corresponding to the data to be read on the persistent storage node in the local availability zone; if the local availability zone is normal, If the local availability zone is abnormal and at least one data shard corresponding to the data to be read in the non-abnormal availability zone is normal, then at least one data shard is read from the persistent storage node in the non-abnormal availability zone based on the physical address information of the at least one data shard corresponding to the data to be read on the persistent storage node in the non-abnormal availability zone; if the local availability zone is abnormal and there is an abnormal data shard in at least one data shard corresponding to the data to be read in the non-abnormal availability zone, then m normal shards are read from the non-abnormal availability zone, and the abnormal data shard is restored based on the m normal shards; and at least one data shard corresponding to the data to be read is obtained from the m normal shards and the restored abnormal shard.

[0162] The detailed implementation and beneficial effects of each step in the electronic device provided by the embodiments of the present disclosure have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0163] Accordingly, an embodiment of the present disclosure further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by the session state synchronization device in the method embodiments shown in Figures 2a to 2c above.

[0164] Accordingly, an embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, can implement the steps that can be performed by the session state synchronization device in the method embodiments shown in Figures 2a to 2c above.

[0165] The above-mentioned memory can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0166] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0167] The above-mentioned display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0168] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0169] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0170] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0171] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0172] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0174] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0175] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0176] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0178] The above are merely examples of the present disclosure and are not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure are intended to be included within the scope of the claims of the present disclosure.

Claims

1. A cloud storage system, wherein: include: A storage service management node, a storage service node, a persistent management node, and multiple persistent storage nodes, wherein the multiple persistent storage nodes are distributed in multiple availability zones; The storage service management node is used to create a cloud storage disk across N availability zones for a computing node, and allocate a storage service node to the cloud storage disk, so that the allocated storage service node responds to the read and write requests initiated by the computing node to the cloud storage disk, N≥2, and N is a positive integer; the cloud storage disk adopts a redundant storage method that integrates multiple copies and erasure codes, and the redundant storage method includes code sharding and multiple copies of data sharding; The persistent management node is used to obtain the cross-availability zone disaster recovery requirement information of the cloud storage disk, and generate the data storage layout information required for reading and writing the cloud storage disk according to the cross-availability zone disaster recovery requirement information, wherein the data storage layout information is used to describe the storage method of the coding shards and the multi-copy data shards in the cross-availability zone persistent storage nodes; The storage service node is used to respond to the read and write requests initiated by the computing node to the cloud storage disk, and perform data read and write operations on the persistent storage nodes across the available zones according to the data storage layout information.

2. The system according to claim 1, wherein: The cross-availability zone disaster recovery demand information includes the number f of abnormal shards that can be tolerated when M availability zones among N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; The data storage layout information specifically includes a storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; wherein m and n are respectively the number of data shards and coding shards in the erasure code, P represents the number of copies of the m data shards, different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, P, K, m and n are all positive integers.

3. The system according to claim 2, wherein: When the persistence management node generates the data storage layout information required for reading and writing the cloud storage disk according to the cross-availability zone disaster recovery demand information, it is specifically used to: Determine the number n of encoding shards and the number P of replicas according to the number N of available zones and the number f of abnormal shards, wherein the number m of data shards is preset; According to the number of shards to be stored, P*m+n, select at least K persistent storage nodes from P availability zones, where P≤K≤(P*m+n); At least based on the principle that different copies of the m data shards are located in different availability zones, a storage mapping relationship between P*m+n shards and K persistent storage nodes is established.

4. The system according to claim 2 or 3, wherein: When the storage service node responds to the read and write request initiated by the computing node to the cloud storage disk and performs data read and write operations on the persistent storage nodes across the availability zones according to the data storage layout information, it is specifically used to: Responding to a write request initiated by the computing node to the cloud storage disk, and obtaining data to be stored; Using the number m of data fragments and the number n of coding fragments in the erasure code, fragmenting and erasure code encoding are performed on the data to be stored to obtain m data fragments and n coding fragments; According to the storage mapping relationship between the P*m+n shards and the K persistent storage nodes included in the data storage layout information, the P*m data shards and the n encoding shards are stored in the K persistent storage nodes.

5. The system according to any one of claims 2 to 4, wherein: When the storage service node responds to the read and write request initiated by the computing node to the cloud storage disk and performs data read and write operations on the persistent storage nodes across the availability zones according to the data storage layout information, it is specifically used to: In response to a read request initiated by the computing node to the cloud storage disk, obtaining second logical block addressing LBA address information of the data to be read in the cloud storage disk; Determine the second file address information according to the second LBA address information and the pre-recorded mapping relationship between each LBA address information and the file address information in the cloud storage disk; According to the second file address information and the data storage layout information, obtaining physical address information of at least one data shard corresponding to the to-be-read data on K persistent storage nodes; According to the physical address information of at least one data shard corresponding to the to-be-read data on the K persistent storage nodes, the at least one data shard corresponding to the to-be-read data is read from the corresponding persistent storage node.

6. A data reading and writing method, wherein: Applicable to a persistent management node in a cloud storage system, the method comprises: Obtain cross-availability zone disaster recovery demand information of a cloud storage disk; the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies with erasure codes, the redundant storage method includes code sharding and multiple copies of data sharding, N≥2, and N is a positive integer; Generate data storage layout information required for reading and writing the cloud storage disk according to the cross-availability zone disaster recovery demand information, wherein the data storage layout information is used to describe the storage method of the encoding shards and the multiple replicas of the data shards in the persistent storage nodes across the availability zones; The data storage layout information is provided to the storage service node, so that the allocated storage service node performs data read and write operations on the persistent storage nodes across the available zones according to the data storage layout information.

7. The method according to claim 6, wherein: The cross-availability zone disaster recovery demand information includes the number f of abnormal shards that can be tolerated when M availability zones among N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; The data storage layout information specifically includes a storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; wherein m and n are respectively the number of data shards and coding shards in the erasure code, P represents the number of copies of the m data shards, different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, P, K, m and n are all positive integers.

8. The method according to claim 6 or 7, wherein: According to the cross-availability zone disaster recovery requirement information, data storage layout information required for reading and writing the cloud storage disk is generated, including: Determine the number n of encoding shards and the number P of replicas according to the number N of available zones and the number f of abnormal shards, wherein the number m of data shards is preset; According to the number of shards to be stored, P*m+n, select at least K persistent storage nodes from P availability zones, where P≤K≤(P*m+n); At least based on the principle that different copies of the m data shards are located in different availability zones, a storage mapping relationship between P*m+n shards and K persistent storage nodes is established.

9. The method according to claim 8, wherein: According to the number of shards to be stored, P*m+n, select at least K persistent storage nodes from P availability zones, including: According to the number of shards to be stored, P*m+n, select K persistent storage nodes from P+1 availability zones; Accordingly, based on at least the principle that different copies of the m data shards are located in different availability zones, a storage mapping relationship between the P*m+n shards and the K persistent storage nodes is established, including: Based on the principle that different copies of m data shards are located in different availability zones, and n encoding shards are dispersedly stored in P+1 availability zones, a storage mapping relationship between P*m+n shards and K persistent storage nodes is established.

10. A data reading and writing method, wherein: Applicable to a storage service node in a cloud storage system, the method comprises: Receiving data storage layout information required for reading and writing a cloud storage disk sent by a persistence management node in the cloud storage system, wherein the data storage layout information is generated by the persistence management node according to disaster recovery requirement information across availability zones of the cloud storage disk; In response to the read and write requests initiated by the computing nodes to the cloud storage disk, data read and write operations are performed on the persistent storage nodes across the availability zones according to the data storage layout information; Among them, the cloud storage disk spans N availability zones and adopts a redundant storage method that combines multiple copies with erasure codes. The redundant storage method includes coding shards and multiple copies of data shards, N≥2, and N is a positive integer.

11. The method according to claim 10, wherein: The cross-availability zone disaster recovery demand information includes the number f of abnormal shards that can be tolerated when M of the N availability zones are abnormal, where 1≤M<N, f≥1, and M and f are both positive integers; the data storage layout information specifically includes the storage mapping relationship between P*m+n shards and K persistent storage nodes, and the K persistent storage nodes are distributed in at least P availability zones; where m and n are the number of data shards and coding shards in the erasure code, respectively, P represents the number of copies of the m data shards, and different copies of the m data shards are located in different availability zones, and 2≤P≤(N-1), n≥f, and P, K, m and n are all positive integers; Responding to the read and write requests initiated by the computing nodes to the cloud storage disk, and performing data read and write operations on the persistent storage nodes across the available zones according to the data storage layout information, including: Responding to the write request initiated by the computing node to the cloud storage disk, obtaining the data to be stored; Using the number m of data fragments and the number n of coding fragments in the erasure code, fragmenting and erasure code encoding are performed on the data to be stored to obtain m data fragments and n coding fragments; According to the storage mapping relationship between the P*m+n shards and the K persistent storage nodes included in the data storage layout information, the P*m data shards and the n encoding shards are stored in the K persistent storage nodes.

12. The method according to claim 11, wherein: According to the storage mapping relationship between the P*m+n shards and the K persistent storage nodes included in the data storage layout information, the P*m data shards and the n encoding shards are stored in the K persistent storage nodes, including: The same copy of the m data shards is stored in a persistent storage node in the same availability zone among the K persistent storage nodes, and P different copies are stored in persistent storage nodes in P different availability zones; The n encoding fragments are stored in a distributed manner in at least one persistent storage node among the K persistent storage nodes.

13. The method according to any one of claims 11 to 12, wherein: Also includes: Receiving storage success information returned by the K persistent storage nodes after successfully storing the data shards or the encoding shards; According to the storage success message, a mapping relationship between the first LBA address information of the data to be stored in the cloud storage disk and the first file address information is recorded, where the first file address information corresponds to the physical address information of the shards corresponding to the data to be stored on the K persistent storage nodes; and / or Monitor the operating status of the availability zones to which the K persistent storage nodes belong; When an abnormal first availability zone is detected, the first data shard and / or the first encoding shard that needs to be stored in the first availability zone will be stored in other availability zones, and after the first availability zone is restored, the first data shard and / or the first encoding shard will be migrated to a persistent storage node in the first availability zone.

14. The method according to any one of claims 10 to 12, wherein: Responding to the read and write request initiated by the computing node to the cloud storage disk, and performing data read and write operations on the persistent storage nodes across the available zones according to the data storage layout information, including: In response to a read request initiated by the computing node to the cloud storage disk, obtaining second logical block addressing LBA address information of the data to be read in the cloud storage disk; Determine the second file address information according to the second LBA address information and the pre-recorded mapping relationship between each LBA address information and the file address information in the cloud storage disk; According to the second file address information and the data storage layout information, obtaining physical address information of at least one data shard corresponding to the to-be-read data on K persistent storage nodes; According to the physical address information of at least one data shard corresponding to the to-be-read data on the K persistent storage nodes, the at least one data shard corresponding to the to-be-read data is read from the corresponding persistent storage node.

15. The method according to claim 14, wherein: According to the physical address information of at least one data shard corresponding to the to-be-read data on the K persistent storage nodes, reading at least one data shard corresponding to the to-be-read data from the corresponding persistent storage node includes: When the number of abnormal availability zones is less than or equal to M and the number of abnormal shards in the non-abnormal availability zones is less than or equal to f, if the local availability zone where the storage service node is located is normal, then read at least one data shard from the persistent storage node in the local availability zone according to the physical address information of at least one data shard corresponding to the data to be read on the persistent storage node in the local availability zone; If the local availability zone is abnormal and at least one data shard corresponding to the to-be-read data in the non-abnormal availability zone is normal, then read at least one data shard from the persistent storage node in the non-abnormal availability zone according to the physical address information of the at least one data shard corresponding to the to-be-read data on the persistent storage node in the non-abnormal availability zone; If the local availability zone is abnormal and there is an abnormal data shard in at least one data shard corresponding to the data to be read in the non-abnormal availability zone, m normal shards are read from the non-abnormal availability zone, and the abnormal data shard is restored according to the m normal shards; at least one data shard corresponding to the data to be read is obtained from the m normal shards and the restored abnormal shard.

16. A data reading and writing method, wherein: Applicable to a storage service management node in a cloud storage system, the method comprises: A cloud storage disk across N availability zones is created for the computing node, where N ≥ 2 and N is a positive integer; the cloud storage disk adopts a redundant storage method that combines multiple copies with erasure codes, and the redundant storage method includes code sharding and multiple copies of data sharding; A storage service node is allocated to the cloud storage disk so that the storage service node can perform data read and write operations on persistent storage nodes across availability zones according to data storage layout information. The data storage layout information is generated by a persistent management node in the cloud storage system according to the disaster recovery requirement information across availability zones of the cloud storage disk.

17. An electronic device, wherein: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method of any one of claims 6 to 9, claims 10 to 15 and claim 16.

18. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the processor is caused to implement the steps of the method as claimed in any one of claims 6 to 9, claims 10 to 15 and claim 16.

19. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the steps of the method described in any one of claims 6 to 9, claims 10 to 15 and claim 16 are implemented.

Images