Cache type cloud storage system, data reading and writing method, device, storage medium, and program product

Abstract

Embodiments of the present disclosure provide a cache type cloud storage system, a data reading and writing method, a device, a storage medium, and a program product. The embodiments of the present disclosure provide a high-performance and highly elastic cache type storage disk, allowing a user to mount and use the cache type storage disk on any computing node, and supporting a storage-compute separation architecture, such that elastic resource scaling can be realized, reducing the idle waste of computing and storage resources. In addition, using a one-hop data reading and writing protocol to directly read and write at storage nodes helps to reduce the delay of data reading and writing. Meanwhile, data is stored in shards, and each data shard is stored on at least two storage nodes to achieve redundant storage of replicas. Compared with a local disk, the cache type storage disk has higher data reliability and availability, making data calculation and analysis more cost-effective and providing better user experience.

Description

Cache-type cloud storage system and data reading and writing method, device, storage medium and program product

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on November 8, 2023, with application number 202311489520.9 and application name “Cache-type cloud storage system and data reading and writing method, device and storage medium”, the entire contents of which are incorporated by reference in this disclosure. Technical Field

[0002] The present disclosure relates to the field of cloud computing technology, and in particular to a cache-type cloud storage system and a data reading and writing method, device, storage medium, and program product. Background Art

[0003] With the rapid migration of data to the cloud and the explosive growth of user data, cloud-native applications such as data warehouses, big data analytics, high-performance computing, and artificial intelligence (AI) computing are becoming increasingly popular. Persistent user data is typically stored in object storage or distributed file systems to ensure high availability, scalability, and reduce storage costs. However, object storage or distributed file systems often lack the data read and write performance to meet the demands of many real-time computing requirements, particularly regarding data read and write latency.

[0004] Currently, a common approach to accelerating data access is to add a solid-state drive (SSD) to the compute node as a local disk. This caches frequently accessed data or temporary data generated during computation, reducing data read and write latency and meeting real-time data read and write requirements. However, the use of local disks reduces the advantages of a storage-and-compute separation architecture, preventing elastic resource expansion and potentially leading to idle computing or storage resources. Furthermore, given the relatively high failure rate of local disks, they cannot effectively meet computing requirements in terms of data reliability and availability.

[0005] Summary of the Invention

[0006] Multiple aspects of the present disclosure provide a cache-type cloud storage system and data reading and writing methods, devices, storage media and program products, which are used to reduce the latency of data reading and writing while maintaining the advantages of the storage and computing separation architecture, and ensure that computing requirements are met in terms of data reliability and availability.

[0007] An embodiment of the present disclosure provides a cache-type cloud storage system, comprising: a cloud storage management node, multiple storage nodes, and a cloud storage access node deployed on a computing node, wherein the multiple storage nodes are interconnected with the cloud storage management node and the cloud storage access node respectively; the cloud storage management node is used to create a cache-type cloud storage disk for the computing node, and to allocate at least two storage nodes to the same data shard stored in the cache-type cloud storage disk; the cloud storage access node is used to write the first data in the first data shard into the cache-type cloud storage disk when the computing node writes the first data in the first data shard, and to read the second data in the second data shard from any storage node corresponding to the second data shard when the computing node reads the second data in the second data shard from the cache-type cloud storage disk; and the multiple storage nodes are used to provide storage space for the allocated data shards.

[0008] An embodiment of the present disclosure also provides a data reading and writing method, which is applied to a cloud storage access node in a cache-type storage system. The cloud storage access node is deployed on a computing node. The computing node has a cache-type cloud storage disk. The same data shard stored in the cache-type cloud storage disk is distributed to at least two storage nodes in the cache-type storage system. The method includes: when the computing node writes first data in a first data shard to its cache-type cloud storage disk, the first data is written to at least two storage nodes corresponding to the first data shard; when the computing node reads second data in a second data shard from the cache-type cloud storage disk, the second data is read from any storage node corresponding to the second data shard.

[0009] An embodiment of the present disclosure also provides a data writing method, which is applied to a first storage node in a cache storage system, the cache storage system corresponds to a cache cloud storage disk created for a computing node, the first storage node corresponds to a first data shard stored in the cache cloud storage disk, and the first data shard corresponds to at least two storage nodes in the cache storage system, the method comprising: receiving a write request and first data, the first data being data in the first data shard; writing the first data to a local storage medium according to the write request; and, in a case where the first storage node is a newly allocated storage node corresponding to the first data shard, sending a copy request to a non-newly allocated storage node corresponding to the first data shard, so as to copy the missing data in the first data shard from the non-newly allocated storage node.

[0010] An embodiment of the present disclosure also provides a data reading method, which is applied to a second storage node in a cache storage system, the cache storage system corresponds to a cache cloud storage disk created for a computing node, the second storage node corresponds to a second data shard stored in the cache cloud storage disk, and the second data shard corresponds to at least two storage nodes in the cache storage system. The method includes: receiving a read request, the read request being used to request reading the second data in the second data shard; when the data in the second data shard is missing in the local storage medium, and the missing data includes at least the second data, obtaining the second data from other storage nodes corresponding to the second data shard, and returning the second data to the sender of the read request.

[0011] An embodiment of the present disclosure also provides a data processing method, which is applied to a cloud storage management node in a cache-type cloud storage system. The method includes: creating a cache-type cloud storage disk for a computing node, where the cache-type cloud storage disk is used to store at least one data shard; allocating at least two storage nodes in the cache-type cloud storage system to the same data shard; recording data shard allocation information, where the data shard allocation information includes at least identification information of at least one data shard and identification information of at least two storage nodes corresponding to each data shard; and synchronizing the data shard allocation information to a cloud storage access node and corresponding storage node deployed on the computing node.

[0012] An embodiment of the present disclosure also provides an electronic device, comprising: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the data reading and writing method, data reading method, data writing method and data processing method provided in the embodiment of the present disclosure.

[0013] The embodiments of the present disclosure also provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor implements the steps in the data reading and writing method, data reading method, data writing method, and data processing method provided in the embodiments of the present disclosure.

[0014] The embodiments of the present disclosure also provide a computer program product, including a computer program, which, when executed by a processor, implements the steps in the data reading and writing method, data reading method, data writing method, and data processing method provided in the embodiments of the present disclosure.

[0015] In the embodiments of the present disclosure, a high-performance, highly elastic cache-type cloud storage disk is provided, allowing users to mount and use the cache-type cloud storage disk on any computing node, supporting a storage-computing separation architecture, realizing elastic resource expansion, and reducing idle waste of computing and storage resources; in addition, a one-hop data read and write protocol is adopted to directly read and write to the storage node, which is conducive to reducing the delay of data reading and writing. At the same time, by sharding the data, each data shard is stored on at least two storage nodes, which has higher data reliability and availability than the local disk, making it more cost-effective in data calculation and analysis, and providing a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0017] FIG1a is a schematic structural diagram of a cache-type cloud storage system provided by an exemplary embodiment of the present disclosure;

[0018] FIG1b is an interactive flow chart of a data writing process provided by an exemplary embodiment of the present disclosure;

[0019] FIG1c is an interactive flow chart of a data reading process provided by an exemplary embodiment of the present disclosure;

[0020] FIG2 is a flow chart of a data reading and writing method provided by an exemplary embodiment of the present disclosure;

[0021] FIG3 is a flow chart of a data writing method provided by an exemplary embodiment of the present disclosure;

[0022] FIG4 is a flow chart of a data reading method provided by an exemplary embodiment of the present disclosure;

[0023] FIG5 is a flow chart of a data processing method provided by an exemplary embodiment of the present disclosure;

[0024] FIG6a is a schematic structural diagram of a data reading and writing device provided by an exemplary embodiment of the present disclosure;

[0025] FIG6 b is a schematic structural diagram of a data writing device provided by an exemplary embodiment of the present disclosure;

[0026] FIG6c is a schematic structural diagram of a data reading device provided by an exemplary embodiment of the present disclosure;

[0027] FIG6 d is a schematic structural diagram of a data processing device provided by an exemplary embodiment of the present disclosure;

[0028] FIG7 is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the technical solutions of the present disclosure will be clearly and completely described below in conjunction with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present disclosure.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In response to the above technical problems, in an embodiment of the present disclosure, a design of a high-performance cache cloud storage disk is provided, which can help various computing systems on the cloud with high real-time requirements (such as Online Transaction Processing (OLTP), data warehouses, data lake computing, big data computing, high-performance computing, AI computing, etc.) to provide a high-performance, highly elastic, on-demand cache cloud storage disk. Various user data (including but not limited to: persistent data, frequently accessed data or temporary data generated during computing, etc.) can be stored in the cache cloud storage disk. Compared with local disks, the cache cloud storage disk maintains the advantages of the computing separation architecture, can dynamically adjust the capacity and performance of the cache cloud storage disk, realize elastic scaling of resources, reduce idle waste of computing and storage resources, and have higher data reliability and availability than local disks, thereby making the above computing system more cost-effective and providing a better user experience in user data computing and analysis.

[0032] The technical solutions provided by various embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0033] Figure 1a is a schematic diagram of the structure of a cache-type cloud storage system provided by an exemplary embodiment of the present disclosure. The cache-type cloud storage disk system provided by an exemplary embodiment of the present disclosure is a cloud storage system that provides users with a cache-type cloud storage disk, offering data storage services via the cache-type cloud storage disk. Furthermore, compared to traditional storage systems, the cache-type cloud storage system of this embodiment utilizes a one-hop data read and write protocol and replica storage technology, which can reduce data read and write latency while meeting reliability and availability requirements. This system can meet the read and write latency performance requirements of various computing systems with high real-time requirements, and is therefore referred to as a cache-type cloud storage system. As shown in Figure 1a, the cache-type cloud storage system includes a cloud storage management node 101, multiple storage nodes 102, and a cloud storage access node 104 deployed on a computing node 103. The multiple storage nodes 102 are interconnected with the cloud storage management node 101 and the cloud storage access node 104, respectively. Figure 1a illustrates four storage nodes 102, but this is not limiting. In Figure 1a, the storage nodes are labeled BS1, BS2, BS3, and BS4. In addition, in FIG. 1 a , one computing node is used as an example, but the present invention is not limited thereto. The cache-type cloud storage system provided in this embodiment can provide cache-type storage services for each computing node 103 .

[0034] In this embodiment, a cloud storage access node 104 is deployed on computing node 103. Cloud storage access node 104 can be deployed on computing node 103 as a service process of the cached cloud storage system, providing access to the cached cloud storage system to cloud computing instances (e.g., virtual machines or containers) running on computing node 103. Specifically, cloud storage access node 104 can use virtualization technology to present a cached cloud storage disk provided by the cached cloud storage system to the cloud computing instance. A user can send input / output (IO) read / write requests to the cloud computing instance, and the cloud computing instance sends the IO read / write requests to cloud storage access node 104. Cloud storage access node 104 can receive the IO read / write requests from the cloud computing instance for the cached cloud storage disk, send the IO read / write requests to the end storage cluster corresponding to the cached cloud storage disk, and return the response to the cloud computing instance (e.g., virtual machine or container) after receiving the response from the storage cluster. Cloud storage access node 104 can interact with the cloud computing instance through the virtualization manager on computing node 103, which is responsible for managing the cloud computing instance. The virtualization manager may be a virtual machine monitor (VMM), or may be called a hypervisor. Optionally, the hypervisor or VMM may be implemented as a virtual operating system emulator (Quick EMUlator, QEMU) or VMware, but is not limited thereto.

[0035] The backend storage cluster includes a cloud storage management node 101 and multiple storage nodes 102. Optionally, the cloud storage management node 101 can be implemented as a cloud hard disk management server (BlockMaster, BM), and the storage node 102 can be implemented as a cloud hard disk data server (BlockServer, BS). The storage node 102 or BS is the carrier of the cache-type cloud storage disk required by the cloud computing instance on the computing node, responsible for data reading and writing and data replication / synchronization. As shown in Figure 1a, there is a one-hop network between the computing node 103 and the storage node 102. The computing node 103 only needs to store data on the storage node 102 through the cloud storage access node 104. The cloud storage management node 101 or BM manages the metadata of the cache-type cloud storage disk and is responsible for mapping and switching the corresponding cache-type cloud storage disk data segments to the corresponding BS. The BM and BS cooperate with each other to ensure the availability and reliability of the entire cache-type cloud storage system to meet application requirements through high availability (HA) technology. In specific deployment implementation, the BM and the BS can be deployed together on a physical device (such as the same storage device), or the BM and the BS can be deployed on different physical devices.

[0036] The following describes the detailed functions of the cloud storage management node 101, the storage node 102, and the cloud storage access node 104 in the embodiment of the present disclosure.

[0037] In this embodiment, cloud storage management node 101 is responsible for creating a cached cloud storage disk for compute node 103 and maintaining metadata for the cached cloud storage disk. For example, a tenant creating a cached cloud storage disk for a cloud computing instance on a compute node may provide cloud storage management node 101 with information such as the tenant's identifier, the required cached cloud storage disk size, and performance requirements. Cloud storage management node 101 then creates a cached cloud storage disk for the tenant's compute node based on the tenant's requirements and maintains metadata for the cached cloud storage disk. The metadata for the cached cloud storage disk may include, but is not limited to, disk identifier (diskid), disk size (disk size), tenant identifier (userid), creation time (create time), disk Input / Output Operations Per Second (DIPS) capacity (DIS), disk throughput capacity (DIS throughput), and disk status (DIS status), and may be represented as (diskid, disk size, userid, create time, disk IOPS capacity, disk throughput capacity, disk status). The cache-type cloud storage disk is used to provide storage services for the cloud computing instance on the computing node, for example, to store various data generated by the cloud computing instance on the computing node 103 .

[0038] In this embodiment, the cache-type cloud storage disk can use logical block addressing (LBA) for addressing. The entire storage space is divided into multiple LBA intervals, and a certain amount of data is stored in these LBA intervals. In this embodiment, based on the data dimension that the cache-type cloud storage disk can store, the data that the cache-type cloud storage disk can store is segmented to obtain at least one data segment. Each data segment includes data in one or more LBA intervals of the cache-type cloud storage disk. The size of the LBA interval corresponding to each data segment is not limited; for example, it can be an LBA interval of 4GB, 8GB, or 16GB. It should be noted that the data in the data segment and the data that the cache-type cloud storage disk can store include both data already stored in the cache-type cloud storage disk and data not yet stored in the cache-type cloud storage disk, as long as it is data that the cache-type cloud storage disk can store. The LBA address information can be mapped to the address information of the read / write request to implement reading and writing of the cache-type cloud storage disk. In this embodiment, the cloud storage management node 101 can dynamically change the division method between data shards and LBA intervals.

[0039] In this embodiment, in order to ensure that the reliability and availability of the cache-type cloud storage system can meet the application requirements, a data redundancy technology based on multiple copies is adopted. Specifically, the cloud storage management node 101 can allocate at least two storage nodes 102 to the same data shard, so as to store the data shard on at least two storage nodes, thereby ensuring the reliability and availability of the data. In Figure 1a, the data shard segment#1 is allocated to storage nodes BS1 and BS2, and the data shard segment#2 is allocated to storage nodes BS3 and BS4 as an example, but it is not limited to this. Among them, allocating storage nodes to data shards refers to establishing a mapping relationship between the logical block address corresponding to the data shard and the physical block address of the storage node, so as to store the data in the data shard on the corresponding physical block address on the storage node or read the data in the data shard from the physical block address.

[0040] In the embodiment of the present disclosure, the number of storage nodes 102 allocated to the same data shard is not limited, as long as there are no less than two storage nodes. For example, a set number of storage nodes, such as 2, 3 or 5, can be allocated to the same data shard by default. For another example, storage nodes are dynamically allocated to the same data shard, such as first allocating a certain number (such as 3) of storage nodes to the data shard, and then flexibly adjusting the number of storage nodes, such as increasing or decreasing the number of storage nodes. Regarding the embodiment of the present disclosure, the number of storage nodes corresponding to the same data shard can be flexibly determined based on the data reliability and availability of the computing node. For example, if the computing node does not have very high requirements for data reliability and availability, a relatively small number of storage nodes, such as 2, can be allocated to the same data shard to achieve dual-copy storage; if the computing node has high requirements for data reliability and availability, a relatively large number of storage nodes, such as 3, can be allocated to the same data shard to achieve triple-copy storage. Among them, the more data copies there are, the higher the data reliability and availability. In the disclosed embodiment, the number of storage nodes corresponding to the same data shard is determined based on the reliability and availability of the computing nodes, providing appropriate data reliability and availability. There is no need to pursue high requirements for data reliability and availability. For example, data reliability of about 4 9s can be achieved. This can simplify the design of cache-type cloud storage disks and the number of replicas required, and better improve the performance of the entire system in data reading and writing, such as reducing latency and storage costs. Further, in order to improve data reliability and availability, erasure codes can also be introduced to increase data redundancy. The longer the erasure code, the higher the data redundancy ratio, and the higher the data reliability and availability.

[0041] In the case of allocating at least two storage nodes to the same data shard, the method for selecting the at least two storage nodes is not limited. For example, attribute information such as load information, network status information, and geographic location information of each storage node can be obtained, and at least two storage nodes can be selected from the existing storage nodes based on at least one attribute information among the load information, network status information, and geographic location information of each storage node to be allocated to the same data shard. For example, the load information of the existing storage nodes is obtained, and at least two storage nodes whose load information is lower than a set load threshold (e.g., 30%, 50%, or 70%, etc.) are selected from the existing storage nodes to be allocated to the same data shard. For another example, the load information, network status information, and geographic location information of the existing storage nodes are obtained, and at least two storage nodes whose load information is lower than a set load threshold, whose network status information is greater than a set network status threshold (e.g., network delay is 50ms, 100ms, etc.), and whose distance to the computing node is less than a set distance threshold (e.g., 1km, 5km, or 10km, etc.) are selected, and then the at least two selected storage nodes are allocated to the same data shard.

[0042] In the embodiment of the present disclosure, in addition to dividing the cache-type cloud storage disk into data shards and allocating storage nodes to the data shards, the cloud storage management node 101 is also responsible for providing the correspondence between the data shards and the storage nodes to the cloud storage access node 104 and the corresponding storage node 102, so as to, on the one hand, let the storage node 102 know which data shards it is responsible for storing, and on the other hand, let the cloud storage access node 104 know which storage nodes 102 to read and write data in which data shards.

[0043] In an optional embodiment, the cloud storage management node 101 may record data shard allocation information, and the data shard allocation information includes at least: identification information of at least one data shard, and identification information of at least two storage nodes corresponding to each data shard. The identification information of the data shard and the identification information of the storage node may be any combination of letters, numbers, and symbols. For example, the identification information of the data shard may be segment#1 or segment#2, etc., and the identification information of the storage node may be the name of the storage node, the IP address, or any other information that can uniquely identify a storage node. In the embodiment of the present disclosure, the identification information of the storage node adopts the name of the storage node, such as BS1 or BS2, etc. For example, in the case where the cache-type cloud storage disk is divided into data shard segment#1 and data shard segment#2, the data shard allocation information may be expressed as: (segment#1: [<BS1,BS2,epoch=0> ]; segment#2:[<BS3,BS4,epoch=0> ), indicating that the storage nodes first allocated to data shard segment#1 are BS1 and BS2, and the storage nodes first allocated to data shard segment#2 are BS3 and BS4. The cloud storage management node 101 can synchronize the data shard allocation information to the cloud storage access node 104 and the at least two storage nodes 102 corresponding to the data shard, so that the cloud storage access node 104 and the at least two storage nodes 102 corresponding to the data shard read and write data based on the data shard allocation information.

[0044] Further optionally, in the embodiment of the present disclosure, it is allowed to reallocate part or all of the storage nodes for each data shard. To this end, the allocation replacement conditions that should be met for the reallocation of storage nodes for each data shard can be pre-set. Based on this, the cloud storage management node 101 can reallocate a new storage node to any data shard when any data shard meets the allocation replacement conditions, and update the identification information of the storage node corresponding to any data shard in the data shard allocation information to obtain new data shard allocation information. Among them, for any data shard, according to the allocation update conditions it meets, part of the storage nodes can be reallocated to the data shard (i.e., replacement of part of the storage nodes is achieved), or all of the storage nodes can be reallocated to the data shard (i.e., replacement of all storage nodes is achieved). For example, when data shard segment#1 meets the allocation replacement conditions, segment#1 can be reallocated from storage node BS1 to storage node BS5, and the data shard allocation information of data shard segment#1 is segment#1: [<BS1,BS2> ]Updated to segment#1:[<BS2,BS5> ]. After any data shard is assigned a new storage node, it may include newly assigned storage nodes and non-newly assigned storage nodes. In the above example, storage node BS5 is a newly assigned storage node, and storage node BS2 is a non-newly assigned storage node.

[0045] In the embodiments disclosed herein, the implementation method of the allocation replacement conditions satisfied by the data shards is not limited. Depending on the different allocation conditions satisfied by the data shards, the implementation method of the cloud storage management node 101 re-allocating a new storage node to any data shard may also vary. An exemplary description is provided below.

[0046] Example A1: The allocation replacement condition satisfied by a data shard is: a write failure occurs on the storage node to which the data shard is allocated.

[0047] When the cloud storage management node 101 receives a report from the cloud storage access node that a storage node has failed to write, it reassigns a new storage node to the data shard corresponding to the storage node where the write failure occurred to replace the storage node where the write failure occurred.

[0048] Example A2: The allocation replacement condition satisfied by a data shard is: the load of the storage node to which the data shard is allocated exceeds a set load threshold.

[0049] The cloud storage management node 101 can regularly obtain the load information of each storage node. When it is monitored that the load information of the storage node exceeds the set load threshold, it is considered that the data shards corresponding to the storage node that exceeds the set load threshold meet the allocation replacement conditions. A new storage node can be re-allocated to the data shards corresponding to the storage node whose load exceeds the set load threshold to replace the storage node whose load exceeds the set load threshold.

[0050] Example A3: A data shard's allocation change condition is that the storage node assigned to the data shard needs to be upgraded, for example, due to system software upgrades or hardware upgrades (such as computing resources, storage resources, and network resources). During the storage node upgrade, data cannot be read or written, so a new storage node can be reassigned to the data shard.

[0051] When a storage node needs to be upgraded, relevant personnel (such as system administrators or maintenance personnel) can send an upgrade request to the cloud storage management node 101. When the cloud storage management node 101 receives the storage node upgrade request, it considers that the data shards corresponding to the storage node to be upgraded meet the allocation replacement conditions and can re-allocate a new storage node for the data shards corresponding to the storage node to be upgraded to replace the storage node to be upgraded.

[0052] Furthermore, in application scenarios that allow replacement of storage nodes for data shards, in order to distinguish the latest storage nodes corresponding to the data shards, version information can be introduced in the data shard allocation information. The version information is used to identify whether the data shard allocation information is the latest version, and at the same time indicates which are the latest storage nodes corresponding to the data shards. In the embodiments disclosed herein, the implementation method of the version information is not limited. For example, epoch (literally translated as era, period) technology can be used, but not limited to, to indicate the modification time of the data shard allocation information, that is, version information. Based on this, in addition to the identification information of at least one data shard and the identification information of at least two storage nodes corresponding to each data shard, the data shard allocation information can also include version information of the data shard allocation information, such as the value of the epoch.

[0053] Taking the example of dividing the cache-type cloud storage disk into data shard segment#1 and data shard segment#2, and the cloud storage management node 101 initially allocates data shard segment#1 to storage nodes BS1 and BS2, and allocates data shard segment#2 to storage nodes BS3 and BS4, the data shard allocation information can be expressed as: (segment#1: [<BS1,BS2,epoch=0> ]; segment#2:[<BS3,BS4,epoch=0> ), indicating that the storage nodes allocated to data shard segment#1 for the first time are BS1 and BS2, and the data shard allocation information is synchronized to the storage nodes BS1, BS2 and the cloud storage access node 104; the storage nodes allocated to data shard segment#2 for the first time are BS3 and BS4, and the data shard allocation information is synchronized to the storage nodes BS3, BS4 and the cloud storage access node 104. As time goes by, assuming that a write failure occurs in storage node BS1, the cloud storage management node 101 re-assigns data shard segment#1 to storage node BS5, and replaces storage node BS1 with storage node BS5. The new data shard allocation information can be expressed as: (segment#1: [<BS2,BS5,epoch=1> ], indicating that the storage nodes allocated for data segment#1 for the second time are BS2 and BS5, and the new data segment allocation information is synchronized to the storage nodes BS2 and BS5 and the cloud storage access node 104.

[0054] In the disclosed embodiment, after cloud storage management node 101 creates a cached cloud storage disk for a computing node, divides the cached cloud storage disk into at least one data shard, and allocates at least two storage nodes to the same data shard, the foundation is laid for computing node 103 to use the cached cloud storage disk. Based on this, when a tenant's cloud computing instance (e.g., a virtual machine or container) needs to mount the cached cloud storage disk, cloud storage access node 104 provides the corresponding cached cloud storage disk identifier, tenant identifier, and cloud computing instance identifier to cloud storage management node 101 to request the mounting and use of the cached cloud storage disk. If cloud storage management node 101 finds that the cached cloud storage disk corresponding to the storage disk identifier is in an idle and available state, and that the storage disk identifier and tenant identifier pass authorization verification, it establishes a correspondence between the cloud computing instance and the cached cloud storage disk based on the cloud computing instance identifier and the storage disk identifier, updates the state of the cached cloud storage disk to an open state, and returns a mount success message to the cloud storage access node. At the same time, the cloud storage management node 101 returns metadata, LBA division information, and data shard allocation information of the cached cloud storage disk to the cloud storage access node 104 and the corresponding storage node 102, so that the computing node 103 can perform read and write operations on the cached cloud storage disk through the cloud storage access node 104. If the cloud storage management node 101 finds that the cached cloud storage disk corresponding to the storage disk identifier is in the enabled state, it indicates that the cached cloud storage disk cannot be mounted for the tenant's cloud computing instance.

[0055] In this embodiment, computing node 103 can implement read and write operations on a cached cloud storage disk through cloud storage access node 104. Computing node 103 can perform read and write operations on any data shard. For ease of distinction and description, this description will be based on an example of writing first data from a first data shard to a cached cloud storage disk and reading second data from a second data shard from the cached cloud storage disk. The first data shard can be any data shard, and the first data can be any data in the first data shard to be written to the cached cloud storage disk. Similarly, the second data shard can be any data shard, and the second data can be any data in the second data shard that has already been written to the cached cloud storage disk. Specifically, when computing node 103 writes the first data from the first data shard to the cached cloud storage disk, cloud storage access node 104 can write the first data to at least two storage nodes corresponding to the first data shard. Furthermore, when the computing node reads the second data from the second data shard from the cached cloud storage disk, the computing node can read the second data from any storage node corresponding to the second data shard. The multiple storage nodes 102 in the cache-type cloud storage system can provide storage space for the allocated data slices and cooperate with the cloud storage access node 104 to perform read and write operations on the data in the allocated data slices.

[0056] Specifically, the cloud storage access node 104 can receive a write request or a read request sent by a cloud computing instance on a computing node; determine a first data shard and its corresponding at least two storage nodes from the locally stored data shard allocation information based on the address information in the write request; send a write request and the first data to the at least two storage nodes corresponding to the first data shard, respectively, so as to write the first data to the at least two storage nodes corresponding to the first data shard; determine a second data shard and its corresponding at least two storage nodes from the locally stored data shard allocation information based on the address information in the read request; send a read request to any storage node corresponding to the second data shard, so as to enable any storage node corresponding to the second data shard to read and return the second data. Specifically, the read and write request includes an LBA address, and the LBA interval to which it belongs can be determined based on the LBA address, thereby determining the data shard corresponding to the LBA interval, and then determining the storage node corresponding to the data shard.

[0057] Further optionally, in the case where the data shard allocation information includes version information of the data shard allocation information, the cloud storage access node 104 will also obtain the version information of the data shard allocation information from the locally stored data shard allocation information. In the case of a write request, the write request, the first data, and the version information are respectively sent to at least two storage nodes corresponding to the first data shard, so as to write the first data to the at least two storage nodes corresponding to the first data shard; for any storage node corresponding to the first data shard, it can be determined whether the received version information is consistent with the version information in the locally stored data shard allocation information. If the two are consistent, the first data is written to the local storage medium according to the write request. In the case of a read request, a read request and the version information are sent to any storage node corresponding to the second data shard, so that any storage node corresponding to the second data shard reads and returns the second data; for any storage node corresponding to the second data shard, it can be determined whether the received version information is consistent with the version information in the locally stored data shard allocation information. If the two are consistent, the second data is read from the local storage medium according to the read request and returned to the cloud storage access node 104, which is then reported to the cloud computing instance by the cloud storage access node 104.

[0058] Furthermore, in some optional embodiments of the present disclosure, as shown in FIG1a, the local storage medium of the storage node 102 adopts a fusion storage medium, that is, it includes multiple storage media. First, these storage nodes are all non-volatile storage nodes that are powered off. In addition, the read and write speeds supported by the multiple storage media are different. Taking the storage node 102 using the first storage medium and the second storage medium as an example, the read and write speed supported by the first storage medium is greater than the read and write speed supported by the second storage medium. More specifically, the first storage medium is Intel Optane Persistent Memory (PMEM) and the second storage medium is a solid-state drive (SSD). In this way, when the storage node 102 reads and writes data, it can give priority to the first storage medium with a faster read and write speed. Then, in the background, the data in the first storage medium is written back to the second storage medium according to the usage level of the first storage medium and the data access popularity. While providing as much data storage space as possible for the computing node, storage costs are saved and data read and write latency is further reduced.

[0059] Specifically, a write request and first data sent by a cloud storage access node are received; the first data is written into a first storage medium, and the data position of the first data in the first storage medium is updated to a data position index, and a write success message is returned to the cloud storage access node; and when it is determined that the first data meets the write-back condition based on the usage level of the first storage medium and / or the access popularity of the first data, the first data is written back to a second storage medium, and the data position of the first data in the data position index is updated.

[0060] Taking the storage node's local storage medium as an example, where the storage node writes the first data to the local storage medium, the node specifically writes the first data to the PMEM, updates the data location of the first data in the PMEM to the data location index, and returns a write success message to the cloud storage access node. Furthermore, if the first data satisfies the write-back condition based on the PMEM usage level and / or the first data's access popularity, the node writes the first data back to the SSD and updates the data location of the first data in the data location index. Similarly, when the storage node reads the second data from the local storage medium, the node specifically reads the second data from the PMEM first. If the second data is read from the PMEM, the node returns the data to the cloud storage access node. If the second data cannot be read from the PMEM, the node reads the second data from the SSD and returns the data to the cloud storage access node. Correspondingly, the storage node can also determine the hot data based on the access popularity of each data in the background SSD, rewrite the hot data in the SSD into PMEM, and update the data position of the hot data in the data location index, thereby improving the reading efficiency of the hot data and reducing the delay in data reading.

[0061] The data writing process and data reading process in the cache-type cloud storage system according to the embodiment of the present disclosure will be described below with reference to the interactive flow charts shown in FIG. 1 b and FIG. 1 c .

[0062] Case B1: Data writing process, see FIG1b , the data writing process includes:

[0063] In this embodiment, description is made by taking the allocation of the first data slice to the storage node BS1 and the storage node BS2 as an example.

[0064] Step 10b: The cloud storage management node 101 creates a cached cloud storage disk for the computing node 103, divides the cached cloud storage disk into data slices, allocates storage node BS1 and storage node BS2 to each data slice, and records data slice allocation information.

[0065] In this embodiment, the data shard allocation information includes the identification information of at least one data shard, the identification information of the storage node BS1 and the storage node BS2, and the version information of the data shard allocation information (such as the epoch value). For example, the data shard allocation information is: (segment#1: [<BS1,BS2,epoch=0> ]; segment#2:[<BS3,BS4,epoch=0> ).

[0066] In steps 11b-13b, the cloud storage management node 101 synchronizes the data shard allocation information to the cloud storage access node 104, and the storage nodes BS1 and BS2 corresponding to the data shards.

[0067] Step 14b: The cloud computing instance on the computing node 103 sends a write request to the cloud storage access node 104 .

[0068] The write request is used to write the first data in the first data slice into the cache-type cloud storage disk, and the write request includes LBA address information to be written.

[0069] Step 15b: The cloud storage access node 104 receives the write request and determines the first data shard, the storage node BS1 and the storage node BS2 corresponding to the first data shard, and the version information from the locally stored data shard allocation information according to the LBA address information in the write request.

[0070] Specifically, the cloud storage access node 104 can determine the LBA interval to which the LBA address information belongs based on the LBA address information in the write request, and then determine the first data shard corresponding to the LBA interval based on the locally stored data shard allocation information, and then determine at least two storage nodes corresponding to the first data shard, as well as version information.

[0071] Steps 16b-17b: The cloud storage access node 104 sends a write request, the first data, and version information to the storage node BS1 and the storage node BS2 corresponding to the first data shard, respectively, to write the first data into the storage node BS1 and the storage node BS2.

[0072] Step 18b, when the storage node BS1 and the storage node BS2 corresponding to the first data shard receive the write request, the first data and the version information sent by the cloud storage access node, they identify whether the received version information is consistent with the version information in the locally stored data shard allocation information. If they are consistent, they write the first data into the local storage medium according to the write request.

[0073] For any storage node 102 corresponding to the first data shard, the first data may be successfully written to the local storage medium, or it may fail to successfully write the first data to the local storage medium due to a node failure or a local storage medium failure. In this embodiment, the example in which storage node BS1 fails to successfully write the first data to the local storage medium and BS2 successfully writes the first data to the local storage medium is used for illustration.

[0074] Step 19b: After successfully writing the first data into the local storage medium, the storage node BS2 returns a write success message to the cloud storage access node 104 .

[0075] The storage node BS1 does not return a write success message to the cloud storage access node 104 .

[0076] Step 20b: When the cloud storage access node 104 does not receive a write success message returned by the storage node BS1, it reports the write failure to the cloud storage management node 101, so that the cloud storage management node 101 reassigns a new storage node to the first data shard to replace the storage node where the write failure occurred.

[0077] The cloud storage access node 104 may not receive a write success message returned by any storage node corresponding to the first data shard in the following situations. For example, the storage node (such as BS1) fails to successfully write the first data into the local storage medium and returns a write failure message to the cloud storage access node 104; the cloud storage access node 104 receives the write failure message returned by the storage node, and then it is considered that the cloud storage access node 104 has not received a write success message returned by the storage node. For another example, the cloud storage access node 104 sets a waiting time, such as 50ms, 1s or 2s; the waiting time may start from the time the first data is sent to the storage node; within the waiting time, the cloud storage access node 104 does not receive a write success message returned by the storage node.

[0078] Step 21b: The cloud storage management node 101 reassigns a new storage node BS5 to the first data shard.

[0079] In this embodiment, it is assumed that the cloud storage management node 101 allocates a new storage node BS5 to the first data shard. The process of cloud storage management node 101 allocating storage node BS5 to the first data shard is the same or similar to the implementation method of allocating storage nodes to data shards in the previous embodiment, and will not be repeated here.

[0080] Step 22b: The cloud storage management node 101 updates the identification information and version information of the storage node corresponding to the first data shard in the data shard allocation information to obtain new data shard allocation information.

[0081] Among them, the updated data shard allocation information includes the identification information of the non-newly allocated storage node and the newly allocated storage node corresponding to the first data shard. For example, assuming that the first data shard is recorded as data shard Segment#1, then in the process of writing the first data in data shard Segment#1, a write failure occurs in storage node BS1 or storage node BS2 (assuming that storage node BS1 fails, storage node BS2 still writes successfully), then the cloud storage access node reports to the cloud storage management node that BS1 has a write failure, then the cloud storage management node allocates a new storage node (assuming it is storage node BS5) to replace BS1, and updates the data shard allocation information at the same time. For example, add new information to the original data shard allocation information, from the original segment#1:[<BS1,BS2,epoch=0> ]Changed to segment#1:[<BS1,BS2,epoch=0> ,<BS2,BS5,epoch=1> For another example, directly modify the original data shard allocation information, such as the original segment#1:[<BS1,BS2,epoch=0> ]Changed to segment#1:[<BS2,BS5,epoch=1> ].

[0082] Steps 23b-25b: The cloud storage management node 101 provides the new data shard allocation information to the cloud storage access node, storage node BS2, and storage node BS5.

[0083] Steps 26b-27b, the cloud storage access node 104 sends a write request, the first data and the new version information to the non-newly allocated storage node BS2 and the newly allocated storage node BS5 corresponding to the first data shard to write the first data to the non-newly allocated storage node BS2 and the newly allocated storage node BS5.

[0084] In steps 28b-29b, upon receiving the write request, first data and version information sent by the cloud storage access node, the non-newly allocated storage node BS2 and the newly allocated storage node BS5 identify whether the received version information is consistent with the version information in the locally stored data shard allocation information. If they are consistent, the first data is written to the local storage medium according to the write request.

[0085] In steps 30b-31b, the storage node BS2 and the storage node BS5 return a write success message to the cloud storage access node.

[0086] Optionally, before allocating a new storage node for the first data shard, the data on the first data shard is stored on a non-newly allocated storage node, and the data in the first data shard may be missing on the newly allocated storage node.

[0087] Therefore, the data writing process also includes:

[0088] Step 32b: The newly allocated storage node BS5 sends a replication request to the non-newly allocated storage node BS2 corresponding to the first data shard, to replicate the missing data in the first data shard from the non-newly allocated storage node BS2. The replication request is used to request the missing data in the first data shard.

[0089] Step 33b: After receiving the replication request, the non-newly assigned storage node BS2 returns the data of the missing first data slice to the newly assigned storage node BS5.

[0090] Case B2: Data reading process, see FIG1c , the data reading process includes:

[0091] In this embodiment, an example is given in which a first data slice is allocated to storage node BS1 and storage node BS2, storage node BS1 fails, and the storage node reallocated to the first data slice is storage node BS5.

[0092] Step 10c: The cloud storage management node 101 creates a cached cloud storage disk for the computing node 103, divides the cached cloud storage disk into data slices, allocates storage nodes BS2 and BS5 to each data slice, and records data slice allocation information.

[0093] In steps 11c-13c, the cloud storage management node 101 synchronizes the data shard allocation information to the cloud storage access node 104, and the storage nodes BS2 and BS5 corresponding to the data shards.

[0094] Step 14c: The cloud computing instance on the computing node 103 sends a read request to the cloud storage access node 104 .

[0095] The read request is used to read the second data in the second data slice from the cache-type cloud storage disk, and the read request includes LBA address information to be read.

[0096] Step 15c: The cloud storage access node receives the read request and determines the second data shard, the storage nodes BS2 and BS5 corresponding to the second data shard, and the version information from the locally stored data shard allocation information based on the LBA address information in the read request.

[0097] There is no restriction on which storage node corresponding to the second data shard the cloud storage access node 104 sends the read request to. For example, a storage node may be randomly selected from the storage nodes corresponding to the second data shard to send the read request. In another example, a storage node may be selected based on at least one of the load, network status, and geographic location information of the storage node corresponding to the second data shard to send the read request. The implementation method for selecting a storage node is the same or similar to the implementation method for allocating storage nodes to data shards in the aforementioned embodiment and will not be further described here. The following description will take the cloud storage access node sending a read request to storage node BS5 as an example.

[0098] Step 16c: The cloud storage access node 104 sends a read request and version information to the storage node BS5 corresponding to the second data shard, so as to read the second data of the second data shard from the local storage medium of the storage node BS5.

[0099] Step 17c: When receiving the read request and version information sent by the cloud storage access node, the storage node BS5 corresponding to the second data shard identifies whether the received version information is consistent with the version information in the locally stored data shard allocation information.

[0100] The second data in the second data slice is missing from the local storage medium of the storage node BS5.

[0101] Step 18c: When the received version information is consistent with the version information in the locally stored data shard allocation information, the storage node BS5 determines that the second data in the second data shard is missing from the local storage medium.

[0102] Step 19c: preferentially request the second data from the non-newly allocated storage node BS2 corresponding to the second data slice.

[0103] Step 20c: The storage node BS2 returns the second data to the storage node BS5.

[0104] Step 21c: The storage node BS5 receives the second data returned by the storage node BS2, and returns the second data to the cloud storage access node 104.

[0105] The storage node BS5 may also store the second data in a local storage medium.

[0106] Step 22c: The cloud storage access node 104 provides the second data to the cloud computing instance on the computing node 103 .

[0107] In the above and following embodiments of the present disclosure, the local storage medium of the storage node is not limited. In an optional embodiment, the storage medium of the storage node adopts two fused storage media. The first storage medium can be PMEM. PMEM has the characteristics of large capacity, fast read and write speed, low latency and persistence. Persistence means that the persistent memory has the same characteristics as a hard disk. After power failure and restart, the data in the persistent memory still exists. The second storage medium can be an SSD. Compared with PMEM, SSD has the advantage of low cost. Therefore, the local storage medium of the storage node can adopt a fused storage medium that combines PMEM and SSD. PMEM accelerates data read and write access, reduces the latency of computing node reading and writing data, and meets the real-time requirements of read and write performance; and then persists the data in PMEM to the SSD to save storage costs. Optionally, in order to better utilize the performance and service life of the SSD, the storage node divides the SSD into multiple AppendOnly (literally: can only add attributes) blocks (Chunks) to ensure data consistency. Among them, the block size can be the default 64MB, or modified to other sizes, such as 128MB. The storage node continuously adds (Append) the newly written data to the SSD to the Chunk and reclaims the Chunk space in the background. For example, the LBA address corresponding to the data stored in Chunk1 has been updated, and the data with the updated LBA address is written to the Chunk2 block, then the data in Chunk1 can be recycled.

[0108] In order to further accelerate access performance, the cloud storage management node 101 can store hot data in PMEM and cold data in SSD according to the access popularity of the data, so as to accelerate data access through PMEM.

[0109] Among them, any storage node assigned to any data shard can write each data in the data shard into the local storage medium. The storage node 102 maintains a data location index for each data in the data shard, and the data location index includes: the correspondence between the LBA address in the cache-type cloud storage disk and the local address (DataLocation) of the storage node. For example, the data location index can be expressed as: LBA->DataLocation. Among them, the local address can be a PMEM address or an SSD address. Under normal circumstances, the data in the storage node is first persisted in PMEM by default to increase the speed of writing data. Specifically, the data in the data shard is written into PMEM, and the data location of the data in PMEM is updated to the data location index, and a write success message is returned to the cloud storage access node 104.

[0110] When the data in the PMEM meets the write-back condition, the cloud storage management node 101 can write the data meeting the write-back condition from the PMEM back to the SSD to improve the utilization rate of the PMEM.

[0111] For example, based on the PMEM usage level, it is determined whether the data in the PMEM meets the write-back condition. The write-back condition is: the PMEM usage level exceeds a set usage level threshold, for example, the set usage level threshold accounts for a percentage of 90%, 95%, or 98% of the total PMEM. If the PMEM usage level exceeds the set usage level threshold, it is considered that the write-back condition is met, and part of the data in the PMEM can be written back from the PMEM to the SSD, and the data position of the data written back to the SSD in the data position index is updated.

[0112] For another example, determine whether the write-back condition is met based on the access heat of the data in PMEM. The write-back condition is: the access heat of the data in PMEM is lower than the set first access heat threshold, and the first access heat threshold can be an access number of 1,000, 50,000 or 100,000, etc., which is not limited to this. If the data in PMEM whose access heat is lower than the set first access heat threshold meets the write-back condition, the data whose access heat is lower than the set first access heat threshold can be written back from PMEM to SSD, and the data position of the first data in the data position index is updated. Alternatively, sort the access heat of each data in PMEM (such as, in the form of data blocks), take the set number of data with the lowest access heat as the data that meets the write-back condition, write the set number of data with the lowest access heat from PMEM back to SSD, and update the data position of the first data in the data position index.

[0113] For another example, based on the usage level of PMEM and the access heat of the data, it is determined whether the data in PMEM meets the write-back condition. There are two types of write-back conditions: the usage level of PMEM exceeds the set water level threshold, and the access heat of the first data is lower than the set first access heat threshold. Among them, when any one of the write-back conditions is met, the first data is written back from PMEM to the SSD, and the data position of the first data in the data location index is updated. Alternatively, when both conditions are met, it is considered that the data in PMEM meets the write-back condition; for example, when the usage level of PMEM exceeds the set usage level threshold, the access heat of each data in PMEM is further determined, and the data with access heat lower than the set first access heat threshold is written back from PMEM to the SSD, and the data position of the first data in the data location index is updated.

[0114] In an optional embodiment, for data in the SSD, data in the SSD whose access heat is higher than a set second access heat threshold is called third data, and the third data can be stored from the SSD to the PMEM to improve the access speed of the third data.

[0115] In the above and following embodiments of the present disclosure, the entire system includes multiple computing nodes, each of which belongs to multiple tenants. Each tenant may have one or more computing nodes, and one or more cloud computing instances may be deployed on the computing nodes. When a cached cloud storage disk is mounted for a tenant's cloud computing instance as described in the above embodiments, the cloud storage management node 101 may also allocate storage space to the tenant on a tenant-by-tenant basis from the storage space corresponding to the first storage medium of the storage node allocated to the tenant. The allocation method may be that the cloud storage management node may pre-allocate a fixed amount of storage space from the first storage medium to the target tenant to which the computing node belongs; or the storage space allocated to the target tenant from the first storage medium may be dynamically adjusted based on the data read and write latency performance achieved by the target tenant to which the computing node belongs when using the cached cloud storage system. The size of the allocated storage space and the specific allocation method are not limited. Below, the method of allocating PMEM space is described by taking the first storage medium as PMEM and allocating PMEM space to the target tenant from the PMEM of the storage node allocated to the target tenant as an example. The target tenant can be any tenant.

[0116] Example C1: Allocate a fixed-size PMEM space to the target tenant. Each storage node includes PMEM, and a fixed-size PMEM space can be pre-allocated from PMEM to the target tenant to which the computing node belongs. For example, the fixed-size PMEM can be a default size, such as 500M, 1GB, or 2GB. For another example, a human-computer interaction interface can be provided to the target tenant, and the target tenant can display the size of the PMEM space and the fee to be paid on the human-computer interaction interface. The larger the PMEM space, the higher the fee; in response to the target tenant's operation of selecting the PMEM space size, if the target tenant successfully pays, the target tenant is allocated a PMEM space of the size selected by the target tenant.

[0117] Example C2: Dynamically Allocating PMEM Space to a Target Tenant. When using a cache-based cloud storage system, the target tenant's data read and write latency performance can be improved. To meet the needs of this target tenant, the PMEM space allocated to the target tenant can be dynamically adjusted based on the data read and write latency performance achieved by the target tenant using the cache-based cloud storage disk. For example, if the target tenant's data read and write latency performance using the cache-based cloud storage disk is high, the amount of PMEM space allocated to the target tenant from PMEM can be increased to further improve the target tenant's data read and write latency performance. For another example, if the target tenant's data read and write latency performance using the cache-based cloud storage disk is low, the amount of PMEM space allocated to the target tenant from PMEM can be reduced, allowing other tenants to obtain more PMEM space and thus have the opportunity to achieve higher data read and write latency performance. In other words, based on the data read and write latency performance achieved by each tenant using the cache-based cloud storage disk, PMEM space can be preferentially allocated to tenants that can provide higher data read and write latency performance, thereby improving the performance and value of the entire cache-based cloud storage system.

[0118] Optionally, before dynamically allocating PMEM space to tenants, a dynamic resource allocation service can be published to tenants, allowing them to choose whether to use the dynamic resource allocation service. For example, a human-computer interaction interface can be provided to tenants, displaying a resource allocation service selection control on the interface. If this control is checked, it is considered that the target tenant has chosen to use the dynamic resource allocation service. If the target tenant chooses to use the dynamic resource allocation service, the PMEM space allocated to the target tenant from PMEM is dynamically adjusted based on the data read and write latency performance that the target tenant to whom the compute node belongs has achieved when using a cache-based cloud storage system.

[0119] Optionally, the tenant can submit a capacity expansion request for the cached cloud storage disk to the cloud storage management node 101, and the capacity expansion request includes the expanded capacity, such as 1GB, 5GB or 50GB. Based on the expanded capacity, the cloud storage management node 101 determines the newly expanded LBA space, divides the newly expanded LBA space into data slices, and allocates storage nodes to the data slices, and returns a successful expansion. For tenants' performance upgrades or downgrades, the cloud storage management node 101 updates the updated IOPS bandwidth limit to the cloud storage access node 104, and the cloud storage access node controls the read and write concurrency. For tenants' unloading requests, the cloud storage management node 101 notifies the corresponding cloud storage access node 104 to close the read and write request connection of the corresponding cached cloud storage disk, and updates the status of the cached cloud storage disk to an idle state.

[0120] Advantages over local SSDs:

[0121] 1. Storage and computing are separated. Tenants can mount and use cached cloud storage disks on any compute node, dynamically adjust the capacity and performance of cached cloud storage disks, and pay on demand. Large-scale pooling of storage media provides improved performance and capacity elasticity. The capacity of a single cached cloud storage disk can dynamically change between 1GB and 256TB (local disks can only be split to a limited extent, typically with a minimum of tens of GB and a maximum of 8TB, and the capacity cannot be dynamically changed). In terms of performance, a single cached cloud storage disk can reach up to 3 million IOPS and 15GB of throughput; while a single local disk typically has a maximum of 300,000 IOPS and 4GB of throughput.

[0122] 2. Through replica redundancy (each data shard corresponds to at least two storage nodes) and high availability design, it is superior to local disks in high reliability and high availability design, simplifying data access and high availability design. The average annual data reliability of local disks is less than 99.9% (or even lower than 99%), while the average annual data reliability of cache-type cloud storage disks can reach 99.99% or even higher.

[0123] 3. When a compute node fails over, the new compute node can immediately mount the cached cloud storage disk of the original compute node to access the original data, without causing a significant performance degradation or requiring a large reload of cached data. The steps for mounting a cached cloud storage disk on a new node are described above and will not be repeated here.

[0124] Advantages over standard cloud hard drives:

[0125] 1. The performance of cached cloud storage disks can be comparable to or even exceed that of local SSD disks. By using a high-performance network (Remote Direct Memory Access (RDMA) or self-developed high-performance network protocol) one-hop read and write protocol, integrated persistent memory (PMEM) and SSD storage media, it can provide 4K data read and write latency as low as 30 microseconds. Among them, the one-hop read and write protocol provided by the present disclosure is the minimum number of network interactions on the IO path, specifically refers to the data on the cloud computing instance being obtained from the storage node through one hop, shortening the transmission delay. The standard cloud hard disk has a higher latency, and it uses a two-hop protocol, that is, from the cloud computing instance to the storage node and then to the persistent storage, it needs to go through two network transmissions in the middle, which has a longer latency.

[0126] 2. Cache-type cloud storage disks do not require advanced features such as shared mounts, snapshots, and synchronous / asynchronous replication, which simplifies the design and copy number requirements of cache-type cloud storage disks (using at least two copies or shorter erasure codes), better improving performance and reducing storage costs.

[0127] FIG2 is a flow chart of a data reading and writing method provided by an exemplary embodiment of the present disclosure. The method is applied to a cloud storage access node in a cache-type storage system. The cloud storage access node is deployed on a computing node. The computing node has a cache-type cloud storage disk. The same data shard stored in the cache-type cloud storage disk is distributed to at least two storage nodes in the cache-type storage system. As shown in FIG2 , the method includes:

[0128] 201. When a computing node writes first data in a first data shard into its cache-type cloud storage disk, the computing node writes the first data into at least two storage nodes corresponding to the first data shard.

[0129] 202. When the computing node reads the second data in the second data shard from the cache-type cloud storage disk, the computing node reads the second data from any storage node corresponding to the second data shard.

[0130] In an optional embodiment, a cache-type cloud storage disk is used to store at least one data shard; the method provided by the embodiment of the present disclosure also includes: receiving data shard allocation information synchronized by a cloud storage management node in the cache-type storage system, the data shard allocation information including identification information of the at least one data shard and identification information of at least two storage nodes corresponding to each data shard.

[0131] In an optional embodiment, when a computing node writes the first data in a first data shard into its cache-type cloud storage disk, the first data is written into at least two storage nodes corresponding to the first data shard, respectively, including: receiving a write request sent by a cloud computing instance on the computing node; determining the first data shard and its corresponding at least two storage nodes from the locally stored data shard allocation information based on the address information in the write request; and sending a write request and the first data to the at least two storage nodes corresponding to the first data shard, respectively, so as to write the first data into the at least two storage nodes corresponding to the first data shard.

[0132] In an optional embodiment, the method provided by the embodiment of the present disclosure also includes: in the event that a write success message is not received from any storage node corresponding to the first data shard, reporting a write failure of the storage node to the cloud storage management node, so that the cloud storage management node can re-assign a new storage node to the first data shard to replace the storage node where the write failure occurred; in the event that the cloud storage management node re-assigns a new storage node to the first data shard, re-sending a write request and the first data to the non-newly assigned and newly assigned storage nodes corresponding to the first data shard to write the first data to the non-newly assigned and newly assigned storage nodes.

[0133] In an optional embodiment, re-sending a write request and the first data to the non-newly allocated and newly allocated storage nodes corresponding to the first data shard includes: receiving updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information including identification information of the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, and new version information corresponding to the updated data shard allocation information; according to the updated data shard allocation information, re-sending a write request, the first data and the new version information to the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, so as to write the first data to the non-newly allocated and newly allocated storage nodes; wherein the new version information is used for the non-newly allocated and newly allocated storage nodes to verify whether they are the legitimate storage nodes corresponding to the first data shard.

[0134] In an optional embodiment, when a computing node reads the second data in a second data shard from a cache-type cloud storage disk, the second data is read from any storage node corresponding to the second data shard, including: receiving a read request sent by a cloud computing instance on the computing node; determining the first data shard or the second data shard and at least two corresponding storage nodes from the locally stored data shard allocation information based on the address information in the read request; and sending a read request to any storage node corresponding to the second data shard, so that any storage node corresponding to the second data shard reads and returns the second data.

[0135] FIG3 is a flow diagram of a data writing method provided by an exemplary embodiment of the present disclosure. The method is applied to a first storage node in a cache storage system. The cache storage system corresponds to a cache cloud storage disk created for a computing node. The first storage node corresponds to a first data shard stored in the cache cloud storage disk. The first data shard corresponds to at least two storage nodes in the cache storage system. As shown in FIG3 , the method includes:

[0136] 301. Receive a write request and first data, where the first data is data in a first data shard.

[0137] 302. Write the first data into a local storage medium according to the write request;

[0138] 303. When the first storage node is a newly allocated storage node corresponding to the first data shard, send a copy request to a non-newly allocated storage node corresponding to the first data shard to copy the missing data in the first data shard from the non-newly allocated storage node.

[0139] In an optional embodiment, the method provided by the embodiment of the present disclosure also includes: receiving updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information at least including identification information of non-newly allocated and newly allocated storage nodes corresponding to the first data shard; based on the updated data shard allocation information, determining that the first storage node is the newly allocated storage node corresponding to the first data shard.

[0140] FIG4 is a flow chart of a data reading method provided by an exemplary embodiment of the present disclosure. The method is applied to a second storage node in a cache storage system. The cache storage system corresponds to a cache cloud storage disk created for a computing node. The second storage node corresponds to a second data shard stored in the cache cloud storage disk. The second data shard corresponds to at least two storage nodes in the cache storage system. As shown in FIG4 , the method includes:

[0141] 401. Receive a read request, where the read request is used to request reading second data in a second data slice.

[0142] 402. When data in the second data shard is missing in the local storage medium and the missing data includes at least the second data, obtain the second data from other storage nodes corresponding to the second data shard and return the second data to the sender of the read request.

[0143] In an optional embodiment, the method provided by the embodiment of the present disclosure also includes: receiving updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information at least including identification information of non-newly allocated and newly allocated storage nodes corresponding to the second data shard; and determining, based on the updated data shard allocation information, that the second storage node is the newly allocated storage node corresponding to the second data shard.

[0144] FIG5 is a flow chart of a data processing method provided by an exemplary embodiment of the present disclosure, which is applied to a cloud storage management node in a cache-type cloud storage system. As shown in FIG5 , the method includes:

[0145] 501. Create a cache-type cloud storage disk for a computing node, where the cache-type cloud storage disk is used to store at least one data shard.

[0146] 502. Allocate at least two storage nodes in the cache-type cloud storage system for the same data shard;

[0147] 503. Record data shard allocation information, where the data shard allocation information includes at least identification information of at least one data shard and identification information of at least two storage nodes corresponding to each data shard.

[0148] 504. Synchronize the data shard allocation information to the cloud storage access node and the corresponding storage node deployed on the computing node.

[0149] In an optional embodiment, the method provided by the embodiment of the present disclosure also includes: when any data shard meets the allocation replacement conditions, re-allocating a new storage node for any data shard, and updating the identification information of the storage node corresponding to any data shard in the data shard allocation information.

[0150] In an optional embodiment, when any data shard meets the allocation replacement conditions, a new storage node is re-allocated to any data shard, including at least one of the following operations: when a cloud storage access node reports that a storage node has a write failure, a new storage node is re-allocated to the data shard corresponding to the storage node where the write failure occurred to replace the storage node where the write failure occurred; when it is monitored that the load of the storage node exceeds a set load threshold, a new storage node is re-allocated to the data shard corresponding to the storage node where the load exceeds the set load threshold to replace the storage node where the load exceeds the set load threshold; when a storage node upgrade request is received, a new storage node is re-allocated to the data shard corresponding to the storage node that needs to be upgraded to replace the storage node that needs to be upgraded.

[0151] The detailed implementation and beneficial effects of each step in the method of this embodiment have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0152] It should be noted that the execution entity of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution entity of steps 201 to 203 can be device A; for another example, the execution entity of steps 201 and 202 can be device A, and the execution entity of step 203 can be device B; and so on.

[0153] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 201, 202, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0154] Figure 6a is a schematic diagram of the structure of a data reading and writing device provided by an exemplary embodiment of the present disclosure. This device corresponds to a cloud storage access node in a cache-type storage system. The cloud storage access node is deployed on a computing node, which has a cache-type cloud storage disk. The same data shard stored in the cache-type cloud storage disk is distributed to at least two storage nodes in the cache-type storage system. As shown in Figure 6a, the device includes a writing module 61a and a reading module 62a.

[0155] A writing module 61a is configured to write the first data in the first data shard to the cache-type cloud storage disk of the computing node, and to write the first data into at least two storage nodes corresponding to the first data shard;

[0156] The reading module 62a is configured to read the second data from any storage node corresponding to the second data shard when the computing node reads the second data in the second data shard from the cache-type cloud storage disk.

[0157] In an optional embodiment, a cache-type cloud storage disk is used to store at least one data shard; the device provided by the embodiment of the present disclosure also includes: a receiving module; the receiving module is used to receive data shard allocation information synchronized by the cloud storage management node in the cache-type storage system, and the data shard allocation information includes identification information of at least one data shard and identification information of at least two storage nodes corresponding to each data shard.

[0158] In an optional embodiment, the write module 61a is specifically used to: receive a write request sent by a cloud computing instance on a computing node; determine a first data shard and its corresponding at least two storage nodes from locally stored data shard allocation information based on address information in the write request; and send a write request and first data to the at least two storage nodes corresponding to the first data shard, respectively, so as to write the first data to the at least two storage nodes corresponding to the first data shard.

[0159] In an optional embodiment, the device provided by the embodiment of the present disclosure also includes: a reporting module and a sending module; the reporting module is used to report to the cloud storage management node that a write failure has occurred in the storage node when a write success message returned by any storage node corresponding to the first data shard is not received, so that the cloud storage management node can re-assign a new storage node to the first data shard to replace the storage node where the write failure has occurred; the sending module is used to re-send a write request and the first data to the non-newly assigned and newly assigned storage nodes corresponding to the first data shard when the cloud storage management node re-assigns a new storage node to the first data shard, so as to write the first data to the non-newly assigned and newly assigned storage nodes.

[0160] In an optional embodiment, the sending module is specifically used to: receive updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information including identification information of the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, and new version information corresponding to the updated data shard allocation information; according to the updated data shard allocation information, re-send a write request, the first data and the new version information to the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, so as to write the first data to the non-newly allocated and newly allocated storage nodes; wherein the new version information is used for the non-newly allocated and newly allocated storage nodes to verify whether they are the legitimate storage nodes corresponding to the first data shard.

[0161] In an optional embodiment, the reading module is specifically used to: receive a read request sent by a cloud computing instance on a computing node; determine the first data shard or the second data shard and its corresponding at least two storage nodes from the locally stored data shard allocation information based on the address information in the read request; send a read request to any storage node corresponding to the second data shard, so that any storage node corresponding to the second data shard reads and returns the second data.

[0162] Figure 6b is a structural schematic diagram of a data writing device provided by an exemplary embodiment of the present disclosure. The device is applicable to a first storage node in a cache storage system. The cache storage system corresponds to a cache cloud storage disk created for a computing node. The first storage node corresponds to a first data shard stored in the cache cloud storage disk. The first data shard corresponds to at least two storage nodes in the cache storage system. As shown in Figure 6b, the device includes: a transceiver module 61b and a write module 62b.

[0163] The transceiver module 61b is configured to receive a write request and first data, where the first data is data in a first data slice;

[0164] A writing module 62b, configured to write the first data into a local storage medium according to a write request;

[0165] The transceiver module 61b is used to send a copy request to the non-newly allocated storage node corresponding to the first data shard when the first storage node is a newly allocated storage node corresponding to the first data shard, so as to copy the missing data in the first data shard from the non-newly allocated storage node.

[0166] In an optional embodiment, the device provided by the embodiment of the present disclosure also includes: a determination module; a transceiver module for receiving updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information at least including identification information of non-newly allocated and newly allocated storage nodes corresponding to the first data shard; a determination module for determining that the first storage node is the newly allocated storage node corresponding to the first data shard based on the updated data shard allocation information.

[0167] Figure 6c is a structural schematic diagram of a data reading device provided by an exemplary embodiment of the present disclosure. The device is suitable for a second storage node in a cache storage system. The cache storage system corresponds to a cache cloud storage disk created for a computing node. The second storage node corresponds to a second data shard stored in the cache cloud storage disk. The second data shard corresponds to at least two storage nodes in the cache storage system. As shown in Figure 6c, the device includes: a transceiver module 61c and an acquisition module 62c.

[0168] The transceiver module 61c is configured to receive a read request, where the read request is configured to request reading of the second data in the second data slice;

[0169] The acquisition module 62c is used to obtain the second data from other storage nodes corresponding to the second data shard when the data in the second data shard is missing in the local storage medium and the missing data includes at least the second data, and return the second data to the sender of the read request.

[0170] In an optional embodiment, the device provided by the embodiment of the present disclosure also includes: a determination module; the transceiver module is also used to: receive updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information at least includes identification information of non-newly allocated and newly allocated storage nodes corresponding to the second data shard; the determination module is used to determine that the second storage node is the newly allocated storage node corresponding to the second data shard based on the updated data shard allocation information.

[0171] Figure 6d is a structural diagram of a data processing device provided by an exemplary embodiment of the present disclosure. The device corresponds to a cloud storage management node in a cache-type cloud storage system. As shown in Figure 6d, the device includes: a creation module 61d, an allocation module 62d, a recording module 63d and a synchronization module 64d.

[0172] A creation module 61d is configured to create a cache-type cloud storage disk for the computing node, where the cache-type cloud storage disk is configured to store at least one data shard;

[0173] An allocation module 62d is configured to allocate at least two storage nodes in the cache-type cloud storage system to the same data shard;

[0174] Recording module 63d, configured to record data shard allocation information, the data shard allocation information including at least identification information of at least one data shard and identification information of at least two storage nodes corresponding to each data shard;

[0175] The synchronization module 64d is used to synchronize the data shard allocation information to the cloud storage access node and the corresponding storage node deployed on the computing node.

[0176] In an optional embodiment, the device provided by the embodiment of the present disclosure also includes: an update module; the allocation module is also used to: re-allocate a new storage node to any data shard when any data shard meets the allocation replacement conditions; the update module is used to update the identification information of the storage node corresponding to any data shard in the data shard allocation information.

[0177] In an optional embodiment, the allocation module is specifically used to perform at least one of the following operations: upon receiving a report from a cloud storage access node that a storage node has a write failure, re-allocating a new storage node to the data shard corresponding to the storage node where the write failure occurred to replace the storage node where the write failure occurred; upon monitoring that the load of the storage node exceeds a set load threshold, re-allocating a new storage node to the data shard corresponding to the storage node where the load exceeds the set load threshold to replace the storage node where the load exceeds the set load threshold; upon receiving a storage node upgrade request, re-allocating a new storage node to the data shard corresponding to the storage node that needs to be upgraded to replace the storage node that needs to be upgraded.

[0178] The detailed implementation and beneficial effects of each step in the device of this embodiment have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0179] Figure 7 is a schematic diagram of the structure of an electronic device provided by another exemplary embodiment of the present disclosure. This electronic device is implemented as a cloud storage access node in a cache-type storage system. The cloud storage access node is deployed on a computing node, which has a cache-type cloud storage disk. The same data shard stored in the cache-type cloud storage disk is distributed to at least two storage nodes in the cache-type storage system. As shown in Figure 7, the device includes a memory 74 and a processor 75.

[0180] The memory 74 is used to store computer programs and can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device.

[0181] The memory 74 can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0182] The processor 75 is coupled to the memory 74 and is used to execute the computer program in the memory 74, so as to: when the computing node writes the first data in the first data shard to its cache-type cloud storage disk, write the first data to at least two storage nodes corresponding to the first data shard; when the computing node reads the second data in the second data shard from the cache-type cloud storage disk, read the second data from any storage node corresponding to the second data shard.

[0183] In an optional embodiment, the cache-type cloud storage disk is used to store at least one data shard; the processor 75 is also used to: receive data shard allocation information synchronized by the cloud storage management node in the cache-type storage system, the data shard allocation information including identification information of at least one data shard and identification information of at least two storage nodes corresponding to each data shard.

[0184] In an optional embodiment, when the computing node writes the first data in the first data shard into its cache-type cloud storage disk, the processor 75 writes the first data into at least two storage nodes corresponding to the first data shard, and is specifically used to: receive a write request sent by the cloud computing instance on the computing node; determine the first data shard and its corresponding at least two storage nodes from the locally stored data shard allocation information according to the address information in the write request; send a write request and the first data to the at least two storage nodes corresponding to the first data shard, respectively, so as to write the first data to the at least two storage nodes corresponding to the first data shard.

[0185] In an optional embodiment, the processor 75 is further used to: in the event that a write success message is not received from any storage node corresponding to the first data shard, report to the cloud storage management node that a write failure has occurred in the storage node, so that the cloud storage management node can re-assign a new storage node to the first data shard to replace the storage node where the write failure has occurred; in the event that the cloud storage management node re-assigns a new storage node to the first data shard, re-send the write request and the first data to the non-newly assigned and newly assigned storage nodes corresponding to the first data shard, so as to write the first data to the non-newly assigned and newly assigned storage nodes.

[0186] In an optional embodiment, when the processor 75 re-sends the write request and the first data to the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, it is specifically used to: receive the updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information including the identification information of the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, and the new version information corresponding to the updated data shard allocation information; according to the updated data shard allocation information, re-send the write request, the first data and the new version information to the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, so as to write the first data to the non-newly allocated and newly allocated storage nodes; wherein the new version information is used for the non-newly allocated and newly allocated storage nodes to verify whether they are the legitimate storage nodes corresponding to the first data shard.

[0187] In an optional embodiment, when the computing node reads the second data in the second data shard from the cache-type cloud storage disk, and reads the second data from any storage node corresponding to the second data shard, the processor 75 is specifically used to: receive a read request sent by the cloud computing instance on the computing node; determine the first data shard or the second data shard and its corresponding at least two storage nodes from the locally stored data shard allocation information according to the address information in the read request; and send a read request to any storage node corresponding to the second data shard, so that any storage node corresponding to the second data shard reads and returns the second data.

[0188] Furthermore, as shown in Figure 7, the electronic device also includes other components such as a communication component 76, a display 77, a power supply component 78, and an audio component 79. Figure 7 only schematically shows some components, which does not mean that the electronic device only includes the components shown in Figure 7. In addition, the components within the dotted box in Figure 7 are optional components, not mandatory components, and the specific requirements may depend on the product form of the electronic device. The electronic device of this embodiment can be implemented as a server-side device such as a conventional server, a cloud server, or a server array.

[0189] The present disclosure also provides an electronic device having an implementation structure that is the same as or similar to the implementation structure of the electronic device shown in FIG7 , and can be implemented with reference to the structure of the electronic device shown in FIG7 . The electronic device provided in this embodiment can be implemented as a first storage node in a cache-type storage system, the cache-type storage system corresponding to a cache-type cloud storage disk created for a computing node, the first storage node corresponding to a first data shard stored in the cache-type cloud storage disk, and the first data shard corresponding to at least two storage nodes in the cache-type storage system. The electronic device provided in this embodiment differs from the electronic device shown in FIG7 primarily in that the functions implemented by the processor executing the computer program stored in the memory are different. In the electronic device provided in this embodiment, the processor executes the computer program stored in the memory and can be configured to: receive a write request and first data, the first data being data in the first data shard; write the first data to a local storage medium in response to the write request; and, if the first storage node is a newly assigned storage node corresponding to the first data shard, send a copy request to a non-newly assigned storage node corresponding to the first data shard to copy the missing data in the first data shard from the non-newly assigned storage node.

[0190] In an optional embodiment, the processor is also used to: receive updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information at least including identification information of non-newly allocated and newly allocated storage nodes corresponding to the first data shard; and determine, based on the updated data shard allocation information, that the first storage node is the newly allocated storage node corresponding to the first data shard.

[0191] The present disclosure also provides an electronic device having an implementation structure that is the same as or similar to the implementation structure of the electronic device shown in FIG7 , and can be implemented with reference to the structure of the electronic device shown in FIG7 . The electronic device provided in this embodiment can be implemented as a second storage node in a cache-type storage system, the cache-type storage system corresponding to a cache-type cloud storage disk created for a computing node, the second storage node corresponding to a second data shard stored in the cache-type cloud storage disk, and the second data shard corresponding to at least two storage nodes in the cache-type storage system. The electronic device provided in this embodiment differs from the electronic device shown in FIG7 primarily in that the functions implemented by the processor executing the computer program stored in the memory are different. In the electronic device provided in this embodiment, the processor executes the computer program stored in the memory and can be used to: receive a read request for reading second data in the second data shard; and, if data in the second data shard is missing from the local storage medium, and the missing data includes at least the second data, retrieve the second data from other storage nodes corresponding to the second data shard, and return the second data to the sender of the read request.

[0192] In an optional embodiment, the processor is also used to: receive updated data shard allocation information synchronized by the cloud storage management node, the updated data shard allocation information at least including identification information of non-newly allocated and newly allocated storage nodes corresponding to the second data shard; and determine, based on the updated data shard allocation information, that the second storage node is the newly allocated storage node corresponding to the second data shard.

[0193] The disclosed embodiments also provide an electronic device having an implementation structure identical or similar to that of the electronic device shown in FIG. 7 , and can be implemented with reference to the structure of the electronic device shown in FIG. The electronic device provided in this embodiment can be implemented as a cloud storage management node in a cache-type cloud storage system. The electronic device provided in this embodiment differs from the electronic device shown in FIG. 7 primarily in that the functions implemented by the processor executing a computer program stored in the memory are different. In the electronic device provided in this embodiment, the processor executing the computer program stored in the memory can be used to: create a cache-type cloud storage disk for a computing node, the cache-type cloud storage disk being used to store at least one data shard; allocate at least two storage nodes in the cache-type cloud storage system to the same data shard; record data shard allocation information, the data shard allocation information including at least identification information of at least one data shard and identification information of at least two storage nodes corresponding to each data shard; and synchronize the data shard allocation information to the cloud storage access node deployed on the computing node and the corresponding storage node.

[0194] In an optional embodiment, the processor is further used to: when any data shard meets the allocation replacement condition, re-allocate a new storage node for any data shard, and update the identification information of the storage node corresponding to any data shard in the data shard allocation information.

[0195] In an optional embodiment, when any data shard satisfies the allocation replacement condition, the processor is used to perform at least one of the following operations when reallocating a new storage node to any data shard: upon receiving a report from the cloud storage access node that a storage node has a write failure, reallocating a new storage node to the data shard corresponding to the storage node where the write failure occurred, to replace the storage node where the write failure occurred; upon monitoring that the load of the storage node exceeds a set load threshold, reallocating a new storage node to the data shard corresponding to the storage node where the load exceeds the set load threshold, to replace the storage node where the load exceeds the set load threshold; upon receiving a storage node upgrade request, reallocating a new storage node to the data shard corresponding to the storage node that needs to be upgraded, to replace the storage node that needs to be upgraded. Accordingly, the embodiment of the present disclosure also provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by the electronic device in the above-mentioned method embodiments of Figures 2 to 5.

[0196] Accordingly, an embodiment of the present disclosure further provides a computer program product, including a computer program, which, when executed by a processor, can implement the steps that can be executed by an electronic device in the method embodiments of Figures 2 to 5 above.

[0197] The above-mentioned memory can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0198] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0199] The above-mentioned display includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor can not only sense the boundary of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0200] The power supply assembly provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0201] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as call mode, recording mode, and voice recognition mode, the microphone is configured to receive external audio signals. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0202] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, compact disc read-only memory (CD-ROM), optical storage, etc.) containing computer-usable program code.

[0203] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0204] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0205] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0206] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0207] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0208] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0209] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0210] The above are merely examples of the present disclosure and are not intended to limit the present disclosure. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure are intended to be included within the scope of the claims of the present disclosure.

Claims

1. A cache-type cloud storage system, wherein: include: A cloud storage management node, multiple storage nodes and a cloud storage access node deployed on a computing node, wherein the multiple storage nodes are interconnected with the cloud storage management node and the cloud storage access node respectively; The cloud storage management node is used to create a cache-type cloud storage disk for the computing node, and allocate at least two storage nodes to the same data slice stored in the cache-type cloud storage disk; The cloud storage access node is configured to, when the computing node writes first data in a first data shard into the cache-type cloud storage disk, write the first data into at least two storage nodes corresponding to the first data shard, and, when the computing node reads second data in a second data shard from the cache-type cloud storage disk, read the second data from any storage node corresponding to the second data shard; The multiple storage nodes are used to provide storage space for the allocated data slices.

2. The system according to claim 1, wherein: The number of storage nodes corresponding to the same data shard is determined based on the requirements of the computing nodes for data reliability and availability.

3. The system according to claim 1 or 2, wherein: The cache-type cloud storage disk is used to store at least one data shard; the cloud storage management node is also used to: record data shard allocation information, and synchronize the data shard allocation information to the cloud storage access node and the corresponding storage node; The data shard allocation information includes at least identification information of the at least one data shard and identification information of at least two storage nodes corresponding to each data shard.

4. The system according to claim 3, wherein: The cloud storage management node is also used for: When any data shard meets the allocation replacement condition, a new storage node is re-allocated to the any data shard, and identification information of the storage node corresponding to the any data shard in the data shard allocation information is updated.

5. The system according to claim 3 or 4, wherein: The cloud storage access node is specifically used for: Receiving a write request or a read request sent by the cloud computing instance on the computing node; Determine, according to the address information in the write request, a first data shard and at least two storage nodes corresponding thereto from the locally stored data shard allocation information; Sending the write request and the first data to at least two storage nodes corresponding to the first data shard respectively, so as to write the first data to the at least two storage nodes corresponding to the first data shard; or, Determine, according to the address information in the read request, a second data shard and at least two storage nodes corresponding thereto from the locally stored data shard allocation information; The read request is sent to any storage node corresponding to the second data shard, so that any storage node corresponding to the second data shard reads and returns the second data.

6. The system according to claim 5, wherein: The local storage media of the multiple storage nodes include: a first storage medium and a second storage medium, the read and write speed supported by the first storage medium is greater than the read and write speed supported by the second storage medium; any storage node corresponding to the first data shard is used to: Receiving the write request and the first data sent by the cloud storage access node; Writing the first data into a first storage medium, updating the data position of the first data in the first storage medium into a data position index, and returning a write success message to the cloud storage access node; and When it is determined that the first data meets the write-back condition according to the usage level of the first storage medium and / or the access popularity of the first data, the first data is written back to the second storage medium and the data position of the first data in the data position index is updated.

7. The system according to claim 6, wherein: The cloud storage access node is also used for: In the case of failing to receive a write success message returned by any storage node corresponding to the first data shard, reporting a write failure of the storage node to the cloud storage management node, so that the cloud storage management node re-assigns a new storage node to the first data shard to replace the storage node where the write failure occurred; In the case where the cloud storage management node reallocates a new storage node for the first data shard, the write request and the first data are resent to the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, so as to write the first data to the non-newly allocated and newly allocated storage nodes.

8. The system according to claim 7, wherein: The data shard allocation information also includes version information of the data shard allocation information; Before any storage node corresponding to the first data shard writes the first data into a local storage medium according to the write request, the storage node is further configured to: Receiving version information of the data allocation sharding information stored locally sent by the cloud storage access node; The received version information is compared with the version information in the data shard allocation information stored locally, and it is determined that the two are consistent.

9. The system according to any one of claims 5 to 8, wherein: Any storage node corresponding to the second data shard is used for: Receiving the read request sent by the cloud storage access node; When the data in the second data shard is missing in the local storage medium, and the missing data includes at least the second data, the second data is obtained from a non-newly allocated storage node corresponding to the second data shard, and the second data is returned to the cloud storage access node.

10. The system according to any one of claims 6 to 8, wherein: The cloud storage management node is also used for: Allocate a fixed-size storage space in advance from the first storage medium for the target tenant to which the computing node belongs; or Dynamically adjust the storage space allocated to the target tenant from the first storage medium according to the data read and write latency performance obtained by the target tenant to which the computing node belongs when using the cache-type cloud storage system.

11. A data reading and writing method, wherein: A cloud storage access node applied to a cache storage system, wherein the cloud storage access node is deployed on a computing node, the computing node has a cache cloud storage disk, and the same data slice stored in the cache cloud storage disk is allocated to at least two storage nodes in the cache storage system, the method comprising: When the computing node writes the first data in the first data shard into its cache-type cloud storage disk, the first data is written into at least two storage nodes corresponding to the first data shard respectively; When the computing node reads the second data in the second data shard from the cache-type cloud storage disk, the second data is read from any storage node corresponding to the second data shard.

12. The method according to claim 11, wherein: The cache-type cloud storage disk is used to store at least one data shard; the method further comprises: receiving data shard allocation information synchronized by a cloud storage management node in the cache-type storage system, wherein the data shard allocation information comprises identification information of the at least one data shard and identification information of at least two storage nodes corresponding to each data shard; Accordingly, when the computing node writes the first data in the first data shard into its cache-type cloud storage disk, the first data is written into at least two storage nodes corresponding to the first data shard, respectively, including: receiving a write request sent by the cloud computing instance on the computing node; Determine, according to the address information in the write request, a first data shard and at least two storage nodes corresponding thereto from the locally stored data shard allocation information; The write request and the first data are respectively sent to at least two storage nodes corresponding to the first data shard, so as to write the first data into the at least two storage nodes corresponding to the first data shard.

13. The method according to claim 12, wherein: Also includes: In the case of failing to receive a write success message returned by any storage node corresponding to the first data shard, reporting a write failure of the storage node to the cloud storage management node, so that the cloud storage management node re-assigns a new storage node to the first data shard to replace the storage node where the write failure occurred; In the case where the cloud storage management node reallocates a new storage node for the first data shard, the write request and the first data are resent to the non-newly allocated and newly allocated storage nodes corresponding to the first data shard, so as to write the first data to the non-newly allocated and newly allocated storage nodes.

14. The method according to any one of claims 12 to 13, wherein: When the computing node reads the second data in the second data shard from the cache-type cloud storage disk, reading the second data from any storage node corresponding to the second data shard includes: Receiving a read request sent by the cloud computing instance on the computing node; Determine, according to the address information in the read request, the first data shard or the second data shard and at least two storage nodes corresponding thereto from the locally stored data shard allocation information; The read request is sent to any storage node corresponding to the second data shard, so that any storage node corresponding to the second data shard reads and returns the second data.

15. A data writing method, wherein: A method for applying to a first storage node in a cache storage system, wherein the cache storage system corresponds to a cache cloud storage disk created for a computing node, wherein the first storage node corresponds to a first data shard stored in the cache cloud storage disk, and wherein the first data shard corresponds to at least two storage nodes in the cache storage system, wherein the method comprises: receiving a write request and first data, where the first data is data in the first data slice; Writing the first data into a local storage medium according to the write request; and In the case where the first storage node is a newly allocated storage node corresponding to the first data shard, a replication request is sent to a non-newly allocated storage node corresponding to the first data shard, so as to replicate the first data shard from the non-newly allocated storage node. Click to copy the missing data in the first data shard.

16. A data reading method, wherein: A second storage node is applied to a cache storage system, the cache storage system corresponds to a cache cloud storage disk created for a computing node, the second storage node corresponds to a second data slice stored in the cache cloud storage disk, the second data slice corresponds to at least two storage nodes in the cache storage system, the method comprising: receiving a read request, where the read request is used to request to read the second data in the second data slice; When the data in the second data shard is missing in the local storage medium, and the missing data includes at least the second data, the second data is obtained from other storage nodes corresponding to the second data shard, and the second data is returned to the sender of the read request.

17. A data processing method, wherein: Applied to a cloud storage management node in a cache-type cloud storage system, the method comprises: Creating a cache-type cloud storage disk for a computing node, wherein the cache-type cloud storage disk is used to store at least one data shard; Allocating at least two storage nodes in the cache-type cloud storage system to the same data slice; Recording data shard allocation information, wherein the data shard allocation information includes at least identification information of the at least one data shard and identification information of at least two storage nodes corresponding to each data shard; The data shard allocation information is synchronized to the cloud storage access node and the corresponding storage node deployed on the computing node.

18. The method according to claim 17, wherein: Also includes at least one of the following operations: Upon receiving a report from the cloud storage access node that a storage node has a write failure, reassigning a new storage node to the data shard corresponding to the storage node where the write failure occurred, to replace the storage node where the write failure occurred; When it is detected that the load of a storage node exceeds a set load threshold, a new storage node is reassigned to the data shard corresponding to the storage node whose load exceeds the set load threshold to replace the storage node whose load exceeds the set load threshold; When a storage node upgrade request is received, a new storage node is reassigned to the data shard corresponding to the storage node that needs to be upgraded to replace the storage node that needs to be upgraded.

19. An electronic device, wherein: include: Memory and processor; The memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program to implement the steps in the method of any one of claims 11-14, claim 15, claim 16 and claims 17-18.

20. A computer-readable storage medium storing a computer program, wherein: When the computer program is executed by a processor, the processor is caused to implement the steps in the method of any one of claims 11 to 14, claim 15, claim 16, and claims 17 to 18.

21. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the steps in the method described in any one of claims 11 to 14, claim 15, claim 16 and claims 17 to 18 are implemented.

Images