Data writing method and computing device

Abstract

The invention relates to the technical field of storage, in particular to a data writing method and computing device.The method comprises the steps that in response to data writing operation, multiple sets of data copies corresponding to data to be written are generated; wherein each group of data copies comprises a plurality of data strips; if a target storage node corresponding to the to-be-written data contains a multi-magnetic-arm disk, selecting a corresponding candidate storage unit for each data strip from a plurality of storage units corresponding to the multi-magnetic-arm disk contained in the target storage node; wherein the candidate storage units corresponding to different data strips of the same data copy belong to different multi-magnetic-arm disks. According to the method, physical isolation of a plurality of data strips in the same data copy can be realized, and the problem of disk heating of a multi-magnetic-arm disk is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, and in particular to a data writing method and a computing device. BACKGROUND

[0002] With the rapid development of high-performance computing (HPC), big data, artificial intelligence (AI), and the like, the demand for data storage of enterprises is growing explosively, and higher requirements are put forward for the timeliness of data processing. Under this background, storage technology evolves towards large capacity and high performance. Multi-actuator hard disks (Multi-Actuator HDDs) are widely used because they can balance storage capacity and read-write performance. Compared with traditional single-actuator hard disks (Single-Actuator HDDs), the storage performance of Multi-Actuator HDDs of the same capacity is better, which significantly reduces the storage cost of high-load scenarios such as HPC and big data analysis.

[0003] However, the traditional data writing method is mainly designed for traditional single-actuator hard disks. In a single-actuator hard disk, one disk corresponds to one logical volume / disk (LUN / Disk). In the data read-write process, the traditional algorithm selects candidate disks based on disk capacity, performance, and the like, and writes data to the traditional single-actuator hard disk, thereby balancing the load and capacity among storage nodes and within nodes.

[0004] However, when this traditional data writing method is directly applied to a multi-actuator hard disk environment, many problems will occur. A significant feature of a multi-actuator hard disk is that one disk can correspond to multiple LUNs / Disk (Disk: LUN / Disk = 1: N). Therefore, when performing data read-write operations, the multi-actuator hard disk is prone to local hot disk phenomenon, which has poor reliability. SUMMARY

[0005] Embodiments of the present application provide a data writing method and a computing device to solve the problem that a multi-actuator hard disk is prone to hot disk during data read-write.

[0006] In a first aspect, the embodiments of the present application provide a data writing method applied to a computing device, the computing device being in communication connection with a distributed storage system; the distributed storage system comprising a plurality of storage nodes, each storage node comprising a plurality of multi-armed disks, each multi-armed disk comprising a plurality of storage units; the method comprising: in response to a data writing operation, generating a plurality of groups of data replicas corresponding to the data to be written; wherein each group of data replicas comprises a plurality of data strips; if a target storage node corresponding to the data to be written comprises a multi-armed disk, selecting a corresponding candidate storage unit for each data strip from a plurality of storage units corresponding to the multi-armed disk comprised by the target storage node; wherein the candidate storage units corresponding to different data strips of the same data replica belong to different multi-armed disks.

[0007] Based on this, the candidate storage units corresponding to the plurality of data strips in the same data replica all belong to different multi-armed disks, and the plurality of data strips in the same data replica can be written to different multi-armed disks, thereby realizing physical isolation of the plurality of data strips in the same data replica, and obviously reducing the occurrence of simultaneous reading and writing of different storage units of the same multi-armed disk when performing data reading and writing operations, thereby effectively solving the hot disk problem of the multi-armed disk.

[0008] In a possible implementation, selecting a corresponding candidate storage unit for each data strip from a plurality of storage units corresponding to the multi-armed disk comprised by the target storage node comprises: obtaining identifiers of all storage units corresponding to the target storage node, the identifiers being used to mark the multi-armed disk to which the storage unit belongs; determining a preset storage strategy; based on the preset storage strategy, sorting all storage units corresponding to the target storage node to form a first candidate table; and determining the candidate storage unit corresponding to each data strip based on the identifiers and the first candidate table. Based on the preset storage strategy, sorting all storage units corresponding to the target storage node to form a first candidate table comprises: in a case where the preset storage strategy is a capacity selection strategy, obtaining capacity utilization rates of all storage units corresponding to the target storage node; and sorting all storage units corresponding to the target storage node in an order from low to high capacity utilization rates to form the first candidate table.

[0009] In this way, the disks can be selected by capacity selection strategy in the order of utilization rates, which can not only ensure that the data strips of the same data replica are stored in different multi-armed disks to avoid the hot disk problem, but also preferentially use storage units with low utilization rates, thereby taking into account reliability and balance.

[0010] In a possible implementation, the all storage units corresponding to the target storage node are sorted based on the preset storage strategy to form the first candidate table, and the method further includes: in a case where the preset storage strategy is a performance selection strategy, obtaining input / output (IO) bandwidth or IO operations per second (IOPS) of the all storage units corresponding to the target storage node; and sorting the all storage units corresponding to the target storage node in descending order of the IO bandwidth or in descending order of the IOPS to form the first candidate table.

[0011] In this way, the storage unit with better performance can be preferentially selected when the disk is selected, and both the hot disk and the speed of data writing and subsequent access can be improved, thereby adapting to a high-performance requirement scenario.

[0012] In a possible implementation, the all storage units corresponding to the target storage node are sorted based on the preset storage strategy to form the first candidate table, and the method further includes: in a case where the preset storage strategy is a reliability selection strategy, determining whether a storage unit of each multi-arm disk corresponding to the target storage node is faulty; and sorting the all storage units corresponding to the target storage node in ascending order of the number of faulty storage units of the multi-arm disk to which the storage unit belongs to form the first candidate table.

[0013] In this way, by preferentially selecting the multi-arm disk with few faulty storage units, and by combining the identifier to ensure that the data stripes of the same data copy are dispersed on different disks, physical isolation is implemented. Both the hot disk problem and the use of the storage resource with high reliability through the sorting mechanism are reduced, thereby improving data persistence and system availability.

[0014] In a possible implementation, the candidate storage unit of each data stripe is determined based on the identifier and the first candidate table, and the method includes: for each data stripe in each group of data copies storing data, based on the identifier, selecting, in the first candidate table, a first storage unit that does not belong to the same multi-arm disk as the first storage unit as the candidate storage unit of the current data stripe; and the first storage unit refers to the candidate storage unit corresponding to the other data stripe belonging to the same data copy as the current data stripe, so that the candidate storage units corresponding to different data stripes of the same data copy belong to different multi-arm disks.

[0015] In this way, the storage unit that does not belong to the same disk as the other data stripe of the same copy can be preferentially selected from the first candidate table based on the identifier, and strict physical isolation of the data stripes of the same data copy is ensured.

[0016] In a possible implementation, the method further includes: for each data stripe in each group of data replicas of the to-be-written data, selecting, in the first candidate table, a first storage unit that does not belong to a same storage node as a second storage unit as a candidate storage unit of the current data stripe, wherein the second storage unit refers to a candidate storage unit corresponding to another data stripe belonging to a same data replica as the current data stripe, so that the candidate storage units corresponding to different data stripes of a same data replica belong to different multi-armed disks.

[0017] In this way, the physical isolation of the data stripes of a same data replica can be achieved at the node level by selecting storage units of different storage nodes.

[0018] In a possible implementation, the method further includes: for each data stripe in each group of data replicas of the to-be-written data, selecting, in the first candidate table, a first storage unit that does not belong to a same multi-armed disk as a third storage unit as a candidate storage unit of the current data stripe based on the identifier, wherein the third storage unit refers to a candidate storage unit corresponding to any data stripe in a same data stripe group, so that the candidate storage units corresponding to the to-be-written data belong to different multi-armed disks.

[0019] In this way, the identifier can be used to ensure that all data stripes (including multiple groups of data replicas) of a same to-be-written data are dispersed in different multi-armed disks, to achieve the physical isolation of all data stripes and avoid the concentration of data on a single disk, thereby improving the storage stability.

[0020] In a possible implementation, the storage node further includes a plurality of single-armed disks, and each single-armed disk includes one storage unit; and in response to the data write operation, after the plurality of groups of data replicas corresponding to the to-be-written data are generated, the method further includes: if a target storage pool corresponding to the to-be-written data contains a single-armed disk, selecting, for each data stripe, a corresponding candidate storage unit from a storage unit corresponding to a single-armed disk contained in the target storage node; and the candidate storage units corresponding to different data stripes of a same data replica do not belong to a same single-armed disk.

[0021] In this way, for a single-armed disk storage node, different data stripes of a same data replica can be dispersed in different single-armed disks, to avoid the concentration of single-disk load or the invalidation of a replica due to a fault, adapt to the characteristics of a single-armed disk scenario, and ensure data availability and storage balance.

[0022] In a second aspect, an embodiment of the present application provides a data writing device, applied to a distributed storage system, the distributed storage system comprising a plurality of storage nodes, each of the storage nodes comprising a plurality of multi-armed disks, each of the multi-armed disks comprising a plurality of storage units; the device comprises: a redundancy module configured to generate a plurality of groups of data replicas corresponding to to-be-written data in response to a data writing operation of an application program; each of the groups of data replicas comprises a plurality of data strips; and a balancing module configured to, if a target storage node corresponding to the to-be-written data comprises a multi-armed disk, select a corresponding candidate storage unit for each data strip from a plurality of storage units corresponding to the multi-armed disk comprised by the target storage node; wherein the candidate storage units corresponding to different data strips of a same data replica belong to different multi-armed disks.

[0023] In a third aspect, an embodiment of the present application further provides a computing device, comprising: one or more processors; and a memory configured to store one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of the first aspect.

[0024] In a fourth aspect, an embodiment of the present application provides a chip, configured to execute the method of any one of the first aspect.

[0025] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, having stored therein computer-executable instructions that, when executed by a computer, implement the method of any one of the first aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a program product, comprising a computer program configured to implement the method of any one of the first aspect when executed by a processor. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 A system architecture diagram of a distributed storage system provided by an embodiment of the present application;

[0028] Figure 2 An application scenario diagram of a distributed storage system provided by an embodiment of the present application;

[0029] Figure 3 A first flowchart of a data writing method provided by an embodiment of the present application;

[0030] Figure 4 A schematic diagram of data splitting and replication provided by an embodiment of the present application;

[0031] Figure 5 A first data balancing schematic diagram provided by an embodiment of the present application;

[0032] Figure 6 is Figure 5 a contrast chart of

[0033] Figure 7 a second data equalization schematic diagram provided by the embodiment of the present application;

[0034] Figure 8 a third data equalization schematic diagram provided by the embodiment of the present application;

[0035] Figure 9 a fourth data equalization schematic diagram provided by the embodiment of the present application;

[0036] Figure 10 a fifth data equalization schematic diagram provided by the embodiment of the present application;

[0037] Figure 11 a sixth data equalization schematic diagram provided by the embodiment of the present application;

[0038] Figure 12 a second flow schematic diagram of the data writing method provided by the embodiment of the present application;

[0039] Figure 13 a seventh data equalization schematic diagram provided by the embodiment of the present application;

[0040] Figure 14 a structure schematic diagram of the data writing device provided by the embodiment of the present application;

[0041] Figure 15 a schematic diagram of a computing device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. In order to clearly describe the technical solutions in the embodiments of the present application, the first, second, etc. descriptions in the embodiments of the present application are only used for indicating and distinguishing the description objects, and do not have the order, and do not represent the special limitation of the number of devices in the embodiments of the present application, and cannot constitute any limitation on the embodiments of the present application.

[0043] Before introducing the technical solutions in the embodiments of the present application, the terms related to the embodiments of the present application are first introduced.

[0044] 1. Distributed storage system: a storage system that disperses data storage in multiple independent storage nodes, and the storage nodes can be physical servers or devices in a cluster.

[0045] 2、Multiple-Copy / Multiple-Copies algorithm: The Multiple-Copy algorithm is a strategy for ensuring data reliability in a distributed storage system, and is a data slice redundancy protection mechanism. The core idea of the Multiple-Copy algorithm is to store multiple copies of data in different storage locations, so that even if a copy of data is lost, other copies can still provide data access services.

[0046] Figure 1 The system architecture diagram of the distributed storage system provided by the embodiments of the present application is shown.

[0047] As shown in Figure 1 The embodiments of the present application provide a distributed storage system, which includes a plurality of storage nodes (Node). The storage node is a basic physical unit in the distributed storage system, and each storage node can be composed of one or more server hardware. From the aspect of form, the server can be a rack server or an entire cabinet server; from the aspect of performance, the server can be a general server, a GPU (graphics processing unit) server, or an AI (artificial intelligence) server.

[0048] Further, the server can be built-in with a plurality of storage medium interfaces (such as SATA, SAS, NVMe, etc.) for connecting hard disk storage devices. The hard disk storage devices can include multi-arm disks and single-arm disks. Specifically, the single-arm disk contains one arm (also known as a head arm) and a plurality of platters. Among them, the platter is the core storage component of the hard disk storage device, and is the physical medium that carries data. In the single-arm disk, all platters share one arm, and a plurality of heads are installed on the arm. Each platter has a corresponding head on its upper and lower surfaces. When the arm moves, all heads will move synchronously along the radial direction (i.e., all heads point to the same track position of each platter at the same time). In this way, the single-arm disk presents a single, indivisible logical storage unit to the upper layer application, i.e., a logical volume / disk (LUN / Disk).

[0049] Multi-armed disk includes multiple independent arms, each arm corresponds to a set of independent heads and disk surfaces, each arm can be independently moved, and the heads are independently operated. That is, the heads of different arms can access different tracks of different disk surfaces at the same time without synchronization, and the parallelism is higher. Based on this, the multi-armed disk presents multiple independently accessible logical storage units to the upper layer application, that is, multiple logical volumes / disk (LUN / Disk). These logical storage units can be regarded as multiple independent "sub-disks" by the upper layer application, and the upper layer application can simultaneously read and write these logical storage units through different interfaces or paths. Due to the parallel working characteristics of multi-armed, when the upper layer application processes high-concurrency data requests, the parallel IO capability of multi-armed can be fully utilized, and the data read / write efficiency is greatly improved, especially in the random read / write scenario, the performance advantage is more obvious.

[0050] In actual application, the number of storage nodes included in a distributed storage system, and the number of single-armed disks and / or multi-armed disks contained in each storage node, depend on actual configuration, and embodiments of the present application do not make specific limitation.

[0051] It should be further pointed out that the distributed storage system can realize the redundancy protection of data in the form of "multiple copies". Specifically, when data is written, the data can be divided into at least one data stripe, a data stripe contains multiple data strips, and then multiple groups of data copies are generated based on the data stripe, each group of data copies contains a corresponding plurality of data strips. For the storage of these data strips, embodiments of the present application will be described in detail below, and will not be repeated here.

[0052] Figure 2 An application scenario diagram of the distributed storage system provided by the embodiments of the present application is shown.

[0053] As shown in Figure 2 The embodiments of the present application also provide a client, which is an entrance for interacting with users or application programs, for converting the requirements of upper layer business (such as writing files, reading data, deleting records, etc.) into specific requests (such as API calls, protocol instructions, etc.) for the distributed storage system. Then, the client communicates with the distributed system through a network or a switch. After receiving the request of the client, the distributed storage system cooperates with multiple internal storage nodes to complete the processing and storage of data, and returns the result to the client. Alternatively, the client can directly generate explicit operation instructions based on the requirements of the upper layer business. These instructions not only contain data itself, but also carry specific storage strategy parameters, such as the number of copies, the selection of storage pool (Pool), the size of data strip, etc. At this time, the distributed storage system only needs to complete the write, read or delete operation based on the operation instruction, without the need to analyze or infer the requirements of the upper layer business again.

[0054] Further, the application of the client and the distributed storage system includes a deployment stage and an application stage. The deployment stage includes but is not limited to:

[0055] ① Client environment configuration, such as installing client software, configuring a communication protocol (such as S3, NFS) with the distributed storage system, establishing an identity authentication mechanism (such as a key, a token), and presetting a commonly used storage strategy template. The storage strategy template is defined in detail in the following embodiments of the present application, and will not be described here.

[0056] ② Hardware deployment of the distributed storage system, including storage node construction, storage node initialization, network switch building, storage pool creation, and configuration data striping rules.

[0057] The construction stage of the storage node is a physical preparation process at the hardware level, specifically referring to assembling various hardware components constituting the storage node into a runnable physical unit. For example, a rack-mounted server is selected as a node carrier, and basic computing components such as a central processing unit (CPU), a memory, and a motherboard are installed; a single-arm disk and / or a multi-arm disk is inserted; a network interface card (NIC) is connected to ensure that the node can access an external network; the assembled server is deployed to a data center cabinet (i.e., "on-rack"), and a power cord and a network cable are connected to complete physical fixation.

[0058] After hardware configuration, each storage node can include a certain number of single-arm disks and / or multi-arm disks. For example, Node 1 includes 12 single-arm disks and 12 multi-arm disks, Node 2 includes 12 single-arm disks and 12 multi-arm disks, and Node M includes 12 single-arm disks and 12 multi-arm disks. It can be seen that the disk configuration of each storage node can be the same, and the number of single-arm disks and multi-arm disks contained in each storage node is the same.

[0059] In the storage node initialization stage, the embodiment of the application can automatically scan and identify all disks (single-arm disks and / or multi-arm disks) in the storage node, and collect hardware information. For example, the disk type (single-arm / multi-arm), the world-wide unique name (WWN), the serial number (SN), the arm state (for multi-arm disks, the normal and fault states of each arm are recorded), and the LUN / Disk list corresponding to each disk are collected. For example, the LUN / Disk list entry of a single-arm disk can be: [LUN1 (Disk1): capacity 16 TB, normal state, corresponding to arm 0 (unique arm)]; the LUN / Disk list entry of a multi-arm disk can be: [LUN1 (Disk1): capacity 8 TB, normal state, corresponding to arm 0; LUN2 (Disk2): capacity 8 TB, normal state, corresponding to arm 1], wherein "LUN" represents a logical volume.

[0060] Subsequently, the embodiment of the application can write this information into a field of a system metadata table to form a mapping relationship of "disk-storage unit-storage node", which can be used as a basis for judging "the disk to which the storage unit belongs" and "whether the storage unit is faulty".

[0061] Subsequently, the network switch can be built to ensure that the communication link between the client and the node and between the nodes is smooth.

[0062] Further, in the storage pool creation stage, a plurality of single-arm disks or a plurality of multi-arm disks can be divided into the same storage pool. One storage pool can contain the disks of a plurality of storage nodes, and the disks on one storage node can also be divided into different storage pools. For example, according to the business type, Pool-A (multi-arm high-performance pool), Pool-B (single-arm capacity pool), Pool-C (multi-arm high-performance pool), etc. are divided, and the number of copies, the balancing strategy of the storage pool is set. The number of copies is also referred to as the multi-copy protection level, the multi-copy redundancy, or the multi-copy redundancy ratio. For example, Pool-A is set to 3 copies, Pool-B is set to 2 copies, or the number of copies can be set based on different businesses, for example, application program A is set to 3 copies, and application program B is set to 2 copies, which is not limited in the embodiment of the application. The embodiment of the application can write these configurations into the storage pool attribute field of the metadata table. Further, the balancing strategy is limited in detail in the following, which is not described herein.

[0063] Further, the embodiment of the application can also configure the data slice rule. For example, the default slice size is configured to be 4MB, 8MB or 16MB, so that data strips of 4MB, 8MB or 16MB can be obtained.

[0064] Further, the application stage of the client and the distributed storage system includes but is not limited to: the client receives the service request of the upper application in real time (such as the batch writing of the data set of the AI training platform, the real-time stream storage of the video monitoring system, the incremental data addition of the log system, etc.), and generates operation instructions dynamically according to the business characteristics. The distributed storage system writes data, reads data or deletes data based on the operation instructions.

[0065] Further, in order to solve the problem that the distributed storage system is prone to multi-arm disk hot disk, the embodiment of the application provides a data writing method, which can be applied to the data storage process of the client and the distributed storage system. Further, based on the data writing method, the disk can be selected through the balancing strategy in the data writing process, and the hot disk problem can be effectively avoided. In the embodiment of the application, the method can be executed by a computing device. The computing device can be a server hardware running a client. The server can be a rack server or a whole cabinet server in form. The server can be a general server, a GPU server or an artificial intelligence server in performance. The computing device can also be one of the storage nodes of the distributed storage system. In the following, the computing device is taken as an example of a server hardware running a client, and the data writing method can be built into the client in the form of a software module.

[0066] Specifically, the computing device can be in communication connection with the distributed storage system. In combination with the foregoing embodiment, the distributed storage system can include a plurality of storage nodes, and the storage node is composed of a plurality of multi-arm disks and / or a plurality of single-arm disks.

[0067] The multi-arm disk has two or more arms working simultaneously, and the plurality of arms can read or write data in parallel. Since the plurality of arms can independently process input / output (I / O) requests, one physical multi-arm disk can be split into a plurality of logical volumes (LUNs). For example, Disk A (multi-arm disk) is divided into two logical volumes (LUN1 and LUN2). Since these logical volumes are indistinguishable from independent physical disks in the eyes of the operating system or the application, that is, the multi-arm disk presents a plurality of logical volumes or disks (LUN / Disk) to the upper operating system or the upper application. Therefore, one multi-arm disk can include a plurality of logical volumes / disks (LUN / Disk), that is, the correspondence between the multi-arm disk and the LUN / Disk is disk:LUN / Disk=1:q(q≥2).

[0068] The single-arm disk includes one LUN / Disk, and the correspondence between the single-arm disk and the LUN / Disk is disk:LUN / Disk=1:1.

[0069] It can be understood that the "LUN / Disk" refers to a logical storage unit presented by a disk to an upper layer application. A single-armed disk presents a single and indivisible logical storage unit, i.e., a LUN / Disk, to the upper layer application. A multi-armed disk presents multiple independently accessible logical storage units, i.e., multiple LUNs / Disk, to the upper layer application.

[0070] It should be further noted that the LUN / Disk is collectively referred to as "storage unit" hereinafter, which is only for the convenience of description and does not constitute other limiting effects.

[0071] Referring to FIG. 1, Figure 1 For example, the distributed storage system can include M storage nodes composed of multi-armed disks, which are storage node 1 (Node 1), storage node 2 (Node 2), storage node 3 (Node 3),..., and storage node M (Node M). Each storage node can include two multi-armed disks, and each multi-armed disk can include two storage units. In addition, the distributed storage system can include K storage nodes composed of single-armed disks, which are storage node M+1 (Node (M+1)) to storage node M+K (Node (M+K)). Each storage node can include four single-armed disks, and each single-armed disk includes one storage unit.

[0072] The multi-armed disks in the plurality of storage nodes are divided into one or more storage pools, and / or the single-armed disks in the plurality of storage nodes are divided into one or more storage pools. It can be understood that the storage pool can be divided during the configuration stage of the client and the distributed storage system.

[0073] Referring to FIG. 1, Figure 1 For example, the M storage nodes composed of multi-armed disks can all belong to the same storage pool, and the K storage nodes composed of single-armed disks can all belong to the same storage pool.

[0074] Figure 3 A first flowchart of the data writing method provided by the embodiments of the present application.

[0075] As shown in FIG. 2, Figure 3 The data writing method provided by the embodiments of the present application can include the following steps S100-S200.

[0076] S100: In response to a data writing operation, a plurality of groups of data replicas corresponding to the to-be-written data are generated; each group of data replicas includes a plurality of data strips.

[0077] In the embodiments of the present application, the data writing operation can be triggered by an upper layer application program. When the client receives a write request from the upper layer application program, the process of generating data replicas can be triggered.

[0078] Specifically, the step S100 can include steps S101-S102.

[0079] S101: In response to a data write operation, split the data to be written into a set of data stripes.

[0080] In the embodiments of the present application, the set of data stripes can be referred to as a data stripe set. The specific splitting rule can be pre-set and can be obtained from a system metadata table. For example, the splitting rule is to form data stripes in 2MB slices. It can be understood that the splitting rule can be flexibly adjusted, and the embodiments of the present application do not make specific limitations thereto. Figure 4 A schematic diagram of data splitting and replication provided in the embodiments of the present application.

[0081] For example, as shown in Figure 4 The data to be written can be a 24MB log file. Then, the 24MB data is split into 6MB, and a data stripe set including four data stripes can be generated:

[0082] Data stripe 1 (Stripe_001): includes data stripe 1-data stripe 4 (Strip_001 to Strip_004).

[0083] It should be noted that in the embodiments of the present application, the step of data splitting can be performed by the client or by one of the distributed storage nodes.

[0084] S102: Generate a plurality of sets of data replicas corresponding to the set of data stripes; each set of data replicas includes a plurality of data stripes.

[0085] The embodiments of the present application can use the multi-copy technology to redundantly store the data stripe (a plurality of data stripes) into N copies (for example, 2-8 copies). For example, when the multi-copy redundancy is 3 copies, the embodiments of the present application can construct 2 identical copies based on the original data, and finally there are 3 copies of the original data+2 copies.

[0086] It should be noted that the first copy can be obtained after data splitting, that is, the copy is created based on the original data slice. Then, through the copying operation, N-1 copies can be obtained, and finally N sets of identical data replicas are obtained, each set of data replicas including a plurality of data stripes. The plurality of data stripes in each set of data replicas can completely express the data to be written.

[0087] In the embodiments of the present application, the step of replication can be performed by the client or by one of the distributed storage nodes.

[0088] The following are illustrated with 3 copies. Exemplary: Data Stripe 1 (Stripe_001) generates two groups of copies, Copy1 and Copy2, each group containing copies of Strip_001 to Strip_004.

[0089] For ease of description, the original data and the copied data are collectively referred to as multiple groups of data copies in the embodiments of the present application. In actual applications, the original data and the copied data can be distinguished by different first identifiers, which can be temporarily recorded in a metadata table. The first identifier is, for example, "Stripe ID + copy serial number + data stripe ID". The copy serial number of the original data can be "copy0". The first identifier of the first data stripe of the original data of Data Stripe 1 is Stripe_001_Copy0_Strip_001, and the first identifier of the first data stripe in the first copy of Data Stripe 1 is Stripe_001_Copy1_Strip_001. In this way, different data stripes in different copies can be distinguished.

[0090] It can be understood that in each data stripe, copy0, Copy1 and Copy2 are redundant copy data.

[0091] S200: If the target storage node corresponding to the to-be-written data contains a multi-arm disk, a corresponding candidate storage unit is selected for each data stripe from a plurality of storage units corresponding to the multi-arm disk contained in the target storage node; wherein the candidate storage units corresponding to different data stripes of the same data copy belong to different multi-arm disks.

[0092] It can be understood that the to-be-written data is generated by a corresponding application program. Further, the target storage node corresponding to the application program can be pre-configured. For example, in the initialization stage of the client and the distributed storage system, a specific application program can be bound to a plurality of target storage nodes, and the binding relationship can be written into the system metadata table. The plurality of target storage nodes can be aggregated into a target storage pool, and the plurality of target storage nodes can all be single-arm disks or all be multi-arm disks. Alternatively, a specific application program can be directly bound to part or all of the storage nodes in the target storage pool. The bound storage nodes can be referred to as target storage nodes, and the bound storage nodes can all be single-arm disks or all be multi-arm disks.

[0093] The binding relationship is, for example, "application program ID = APP001 → target storage pool ID = Pool-A, target storage pool ID = Pool-A → storage node list = [Node-01, Node-05, Node-09]". The binding relationship can be written into the "application-storage pool-storage node mapping" field of the system metadata table.

[0094] As can be seen, whether the target storage node corresponding to the data to be written uses a multi-arm disk or a single-arm disk can be determined during the configuration phase. Based on this, it can be determined whether the target storage node corresponding to the data to be written includes a multi-arm disk.

[0095] For example, if the "Real-time Trading System" is bound to Pool-A (a multi-arm high-performance pool), then all data write requests from this application will be directed to Pool-A by default, and the target storage node for the data to be written generated by this application will be a multi-arm disk. If the "Historical Data Archiving Application" is bound to Pool-B (a single-arm capacity pool), then all data write requests from this application will be directed to Pool-B by default, and the target storage node for this application will be a single-arm disk.

[0096] Based on step S200, this application embodiment can determine a corresponding candidate storage unit for each data stripe in multiple sets of data copies.

[0097] Furthermore, embodiments of this application can achieve "disk isolation" of the same data copy through a feasible load balancing strategy.

[0098] Figure 5 This is a schematic diagram of the first type of data balancing provided in the embodiments of this application.

[0099] like Figure 5 As shown in (a), taking a data replica (e.g., Stripe_001_Copy0) in a 3-replica scenario as an example, its four data stripes (Strip_001 to Strip_004) must correspond to four different multi-arm disks. In this embodiment, when selecting candidate storage units, the globally unique name (WWN) or serial number (SN) of the disk to which the storage unit belongs can be verified to ensure that:

[0100] The candidate storage unit of Strip_001 belongs to disk Disk 1 (WWN=W1);

[0101] The candidate storage unit of Strip_002 belongs to disk Disk 3 (WWN = W3, and W3 ≠ W1);

[0102] The candidate storage unit of Strip_003 belongs to disk Disk 5 (WWN = W5, and W5 ≠ W1, W3);

[0103] The candidate storage unit of Strip_004 belongs to disk Disk 6 (WWN = W6, and W6 ≠ W1, W3, W5).

[0104] In this way, by strictly verifying the uniqueness of WWN / SN, we can avoid storing multiple data stripes of the same data copy on the same multi-arm disk, eliminating the risk of "single disk failure causing the entire copy to fail" at the physical layer. More importantly, we can avoid the risk of hot disks on multi-arm disks and improve the balance of data storage.

[0105] Understandable Figure 5 Storage node 1, storage node 2 and storage node 3 shown in the figure all belong to the target storage node.

[0106] Figure 6 yes Figure 5 The comparison chart.

[0107] like Figure 6 As shown in (a), if data stripe Strip_001 and data stripe Strip_002 of the same data copy are not "disk isolated" but are stored in different storage units of the same multi-arm disk, then... Figure 6 As shown in (b), the multi-arm disk of storage node 2 obviously has a hot disk problem during data migration, data reading or data deletion.

[0108] like Figure 5 As shown in (b), based on the method provided in this application embodiment, Strip_001 to Strip_004 reside in different multi-arm disks, thus eliminating the hot disk problem during data migration, data reading, or data deletion. The data migration, data reading, or data deletion steps can be triggered by actions such as node expansion or modification of the replica count.

[0109] As described above, this application provides a data writing method, which includes: generating multiple sets of data copies corresponding to the data to be written in response to a data writing operation; wherein each set of data copies includes multiple data stripes; if the target storage node corresponding to the data to be written contains a multi-arm disk, then selecting corresponding candidate storage units for each data stripe from the multiple storage units corresponding to the multi-arm disk contained in the target storage node; wherein the candidate storage units corresponding to different data stripes of the same data copy belong to different multi-arm disks. This method can achieve physical isolation of multiple data stripes in the same data copy, effectively solving the hot disk problem of multi-arm disks.

[0110] Furthermore, after step S200, the following step S300 may also be included.

[0111] S300: Write each data stripe into the corresponding candidate storage unit respectively.

[0112] In actual applications, the embodiment of the present application can generate a set of distributed write instructions, and convert the mapping relationship between each data stripe and the candidate storage unit into executable operation instructions. Then, the operation instructions are respectively sent to the corresponding storage nodes through a switch or a network. After receiving the operation instructions, the storage nodes can parse the operation instructions and write the data stripe into the physical disk based on the operation instructions.

[0113] In some implementations, the storage node can submit a write confirmation to the metadata service, update the slice location information in the system metadata table, for example, "Stripe_001_Copy1_Strip_001→Node1 / Disk1 / LUN1", where Disk1 represents the first physical disk in Node 1, and LUN1 represents the first (logical) storage unit in the physical disk. Further, the embodiment of the present application provides three preset storage strategies based on the balancing strategy, which are capacity selection strategy, performance selection strategy and reliability selection strategy. The capacity selection strategy can measure each storage unit from the dimension of capacity utilization to form a first candidate table, and the first candidate table includes each candidate storage unit. The performance selection strategy can measure each storage unit from the performance of the storage node to form a first candidate table. The reliability selection strategy can sort each disk from the fault condition of the disk to form a first candidate table. Then, the candidate storage unit can be further determined based on the first candidate table.

[0114] Specifically, step S200 can include steps S201-S204.

[0115] S201: Obtain the identifiers WWN / SN of all storage units corresponding to the target storage node, which are used to mark the multi-arm disk to which the storage unit belongs.

[0116] The identifier WWN / SN can be obtained from the system metadata table. In a distributed storage system, each multi-arm disk can be pre-assigned a WWN (such as "5000CCA001234567") and written into the disk firmware and the system metadata table. Since a multi-arm disk can include multiple storage units (LUN / Disk), these LUN / Disk can share the WWN of the disk to which they belong. Therefore, by checking the WWN of the LUN / Disk, the multi-arm disk to which the LUN / Disk belongs can be directly determined (for example, the WWN of LUN / Disk1 and LUN / Disk2 is "5000CCA001234567", and they belong to the same multi-arm disk.

[0117] SN is a unique serial number assigned by the disk manufacturer to each physical disk (such as "ST1234567890"), usually printed on the disk shell or stored in the disk firmware, used to distinguish different disks at the physical level. Similar to the WWN, all storage units (LUN / Disk) of the same multi-armed disk are associated with the same SN, so the SN can also be used as a basis for determining the "storage unit belonging to the multi-armed disk".

[0118] S202: Determine a preset storage strategy, which is a capacity selection strategy, a performance selection strategy, or a reliability selection strategy.

[0119] The preset storage strategy is also called an IO read-write optimization strategy. Through the preset strategy, capacity balancing, load balancing, or reliability balancing can be achieved, data hotspots can be eliminated, and reliability can be improved. Specifically, the capacity selection strategy can be based on the capacity utilization rate of the disk, and the available capacity proportion of the disk is traversed to select the disk with the highest available capacity that meets the requirements. This strategy can greatly utilize storage space and is suitable for large-capacity backup, archiving, and other scenarios.

[0120] The performance selection strategy is based on the utilization rate of the disk, and the IO utilization rate of the disk is traversed to select the disk with the lowest utilization rate that meets the requirements. This strategy can improve system performance and is suitable for HPC or big data processing scenarios.

[0121] The reliability selection strategy preferentially selects disks without arm failures. This strategy can greatly improve system reliability and is suitable for production business scenarios.

[0122] Further, the preset storage strategy can be specifically read from the metadata table of the distributed storage system. It can be understood that the specific preset storage strategy to be used is already determined and fixed during the configuration phase of the target storage pool, and the specific configuration of the preset storage strategy is determined based on actual needs and circumstances, and the embodiments of the present application do not limit the specific configuration.

[0123] S203: Based on the preset storage strategy, sort all storage units corresponding to the target storage node to form a first candidate table.

[0124] The core purpose of this step is to filter and sort the storage units that meet the strategy requirements.

[0125] S204: Determine the candidate storage unit corresponding to each data stripe based on the identifier and the first candidate table.

[0126] The core purpose of this step is to match each data stripe with an appropriate storage unit.

[0127] Further, step S203 can include steps S2031-S2032.

[0128] S2031: In the case of a preset storage strategy being a capacity selection strategy, the capacity usage rates of all storage units corresponding to the target storage node are obtained.

[0129] The capacity usage rate is a core indicator for measuring the remaining storage resources of a storage unit, and can be measured in the following two ways:

[0130] ① The utilization rate of the storage unit is used as the measurement. The utilization rate refers to the proportion of the used capacity of the storage unit to the total capacity, which can be presented in percentage form. For example: the total capacity of a storage unit is 1 TB, and 300 GB has been used, so the utilization rate is 30%. At this time, the lower the utilization rate, the lower the capacity usage rate.

[0131] ② The available capacity percentage of the storage unit is used as the measurement. The available capacity percentage refers to the proportion of the remaining capacity of the storage unit to the total capacity, which can be presented in percentage form. For example: the total capacity of a storage unit is 1 TB, and 300 GB has been used, so the available capacity percentage is 70%. At this time, the greater the available capacity percentage, the lower the capacity usage rate.

[0132] S2032: The all storage units corresponding to the target storage node are sorted in order of capacity usage rate from low to high to form a first candidate table.

[0133] It can be understood that step S2032 can sort all storage units according to the capacity usage rate to generate the first candidate table. For example, if the available capacity percentage of storage unit A is 80% and the available capacity percentage of storage unit B is 60%, storage unit A will be placed before storage unit B.

[0134] It is worth noting that step S2032 can also include the following steps S2032-1-S2032-2.

[0135] S2032-1: The all storage units corresponding to the target storage node are sorted in order of capacity usage rate from low to high to obtain a first candidate table.

[0136] This step is a step of sorting all storage units.

[0137] S2032-2: The selected storage units are selected from the first candidate table to form a first candidate table; wherein the remaining capacity of the selected storage unit is greater than or equal to the size of the data strip.

[0138] It can be understood that this step is a step of screening the first candidate table. The selected storage units are selected from the first candidate table to form the first candidate table. The specific requirement is that the remaining capacity of the selected storage unit must be greater than or equal to the size of the data strip.

[0139] In some implementations, the screening steps of steps S2032-1-S2032-2 can not be performed. This is because, in actual storage, the size of a data strip is relatively small (such as 2 MB, 4 MB), and the storage unit in the target storage pool usually has a large total capacity (such as 100 GB, 1 TB or even larger). Even if the storage unit has been partially occupied, its remaining capacity is still much larger than the size of the data strip. For example, a storage unit has a total capacity of 1 TB, has been used for 800 GB, and has a remaining capacity of 200 GB. For a data strip of 2 MB, the remaining capacity (200 GB) is much larger than the size of the data strip, and no additional screening is required to meet the write-in requirement. Moreover, when the remaining capacity of the storage unit is small, the distributed storage system can trigger an alarm. For example, when the remaining capacity of the storage unit is less than a set capacity threshold (such as 10%), the distributed storage system can automatically trigger an alarm to notify the client that the remaining capacity of the storage unit is insufficient. Therefore, in actual applications, the step of performing the remaining capacity judgment on the first candidate table can be omitted to save resources.

[0140] Further, step S203 can further include steps S2033-S2034.

[0141] S2033: In a case where the preset storage strategy is a performance selection strategy, obtaining input / output (IO) bandwidth or IO operations per second (IOPS) of all storage units corresponding to the target storage node.

[0142] The IO bandwidth refers to the amount of data that can be transmitted by the storage unit in a unit of time, which can be in units of MB / s or GB / s. For example, the IO bandwidth of a storage unit is 100 MB / s, which means that the storage unit can transmit 100 MB of data per second. The IOPS refers to the number of IO operations that can be completed by the storage unit in a unit of time, which can be in units of “times / second”. For example, the IOPS of a storage unit is 1000 times / second, which means that the storage unit can complete 1000 IO operations per second.

[0143] In the embodiments of the present application, one of the IO bandwidth and the IOPS can be selected as a performance selection strategy measurement index. The specific selection of the index depends on the application scenario and business requirements, which are not limited in the embodiments of the present application.

[0144] S2034: The all storage units corresponding to the target storage node are sorted in descending order of IO bandwidth or in descending order of IOPS to form a first candidate table.

[0145] For example, the target storage pool can include the following storage units:

[0146] Storage unit A: IO bandwidth is 100 MB / s, IOPS is 1000 times / s;

[0147] Storage unit B: IO bandwidth is 80 MB / s, IOPS is 800 times / s;

[0148] Storage unit C: IO bandwidth is 120 MB / s, IOPS is 1200 times / s;

[0149] Then, when selecting IO bandwidth as the sorting basis, the sorting result is: C>A>B. When selecting IOPS as the sorting basis, the sorting result is: C>A>B.

[0150] It can be understood that this is only an exemplary introduction to the sorting process. In actual application process, the first candidate table is a storage table covering all target storage units in the target storage pool.

[0151] In some implementations, step S2034 can include steps S2034-1-S2034-2.

[0152] S2034-1: Sort all storage units corresponding to the target storage node according to the order of IO bandwidth from large to small, or according to the order of IOPS from high to low, to obtain a second candidate table;

[0153] S2034-2: Select the candidate storage unit from the second candidate table to form the first candidate table; wherein the remaining capacity of the candidate storage unit is greater than or equal to the size of the data stripe.

[0154] It can be understood that this step is a step of screening the second candidate table. The candidate storage unit is selected from the second candidate table to form the first candidate table. The specific screening requirement is that the remaining capacity of the candidate storage unit must be greater than or equal to the size of the data stripe.

[0155] Further, step S203 can also include steps S2035-S2036.

[0156] S2035: In the case that the preset storage strategy is a reliability selection strategy, determine whether the storage unit of each multi-arm disk corresponding to the target storage node exists a fault.

[0157] In the embodiments of the present application, the health status of each storage unit can be detected in real time or at a preset frequency. For example, the storage node in the distributed storage system can continuously collect disk health status data through self-monitoring, analysis and reporting technology (SMART), and monitor the threshold values of the key indicators of the storage unit, such as read-write response time, error count, read-write error rate, head seek time, and temperature, to determine whether the storage unit has failed.

[0158] The distributed storage system can record the failure status of the failed storage unit in the metadata table, including the failure time and / or the identifier WWN / SN of the failed storage unit.

[0159] S2036: Sort all the storage units corresponding to the target storage node in the order from few to many of the number of failed storage units in the multi-platter disk to which the storage unit belongs, to form a first candidate table.

[0160] The embodiments of the present application can count the number of failed storage units in each multi-platter disk corresponding to the target storage node. For example, if a multi-platter disk has two storage units, one of which has failed, the number of failed storage units of the multi-platter disk is one.

[0161] Subsequently, the embodiments of the present application can sort the storage units in the multi-platter disk corresponding to the target storage node in the order from few to many of the number of failed storage units in the multi-platter disk to which the storage unit belongs. For example, if multi-platter disk A has one failed storage unit, multi-platter disk B has two failed storage units, and multi-platter disk C has no failed storage unit, in the first candidate table, the storage units of multi-platter disk C are arranged at the front, the storage units of multi-platter disk A are arranged after the storage units of multi-platter disk C, and the storage units of multi-platter disk B are arranged after the storage units of multi-platter disk A.

[0162] In this way, based on the failure condition of the storage units corresponding to the target storage node, the multi-platter disk without failure (in the case of disk: LUN / Disk = 1: q, q LUN / Disk all have no failure) can be preferentially selected. If the storage unit has failed, the priority of the multi-platter disk corresponding to the failed storage unit is reduced through the first candidate table, and when there is no available storage unit to be selected, the multi-platter disk corresponding to the failed storage unit is selected.

[0163] In some implementations, step S2036 can include steps S2036-1-S2036-2.

[0164] S2036-1: Sort all storage units corresponding to the target storage node in ascending order of the number of storage units with faults in the multi-arm disk to obtain the third candidate list;

[0165] S2036-2: Select candidate storage units from the third candidate list to form the first candidate list; wherein the remaining capacity of the candidate storage units is greater than or equal to the size of the data stripe.

[0166] Understandably, this step involves filtering the third candidate table. Candidate storage units are selected from the third candidate table to form the first candidate table. Specifically, the remaining capacity of each candidate storage unit must be greater than or equal to the size of the data stripe.

[0167] Furthermore, this application embodiment, in conjunction with the accompanying drawings, provides a more detailed description of the steps for selecting disks using a balancing strategy combined with a preset storage strategy.

[0168] Figure 7 This is a schematic diagram of a second type of data balancing provided in an embodiment of this application.

[0169] like Figure 7 As shown, step S204 may include the following step S2041.

[0170] S2041: For each data stripe in each data copy of the data to be written, based on the identifier, select the first storage unit in the first candidate table that does not belong to the same multi-arm disk as the first storage unit as the candidate storage unit of the current data stripe; wherein, the first storage unit refers to the candidate storage unit corresponding to other data stripes that belong to the same data copy as the current data stripe, so that the candidate storage units corresponding to different data stripes of the same data copy belong to different multi-arm disks.

[0171] This step sequentially selects disks for each data stripe in each group of data replicas corresponding to the data to be written. Furthermore, this step can be used to achieve "disk isolation" between different data stripes of the same data replica. It ensures that different data stripes (Strip_001-Strip_004) of the same data replica (e.g., Stripe_001_Copy0) are stored on different multi-arm disks, improving data reliability.

[0172] For example, the currently processed data copy could be Stripe_001_Copy0, containing four data stripes (Strip_001-Strip_004). The first candidate table is sorted by capacity utilization, input / output IO bandwidth or IO operations per second (IOPS), or the number of faulty storage units. For example, the first candidate table contains LUN / Disk 1-LUN / Disk8.

[0173] First, allocate storage units for the first data stripe Strip_001.

[0174] For Strip_001, the first storage unit can be selected directly from the first candidate table header (e.g., LUN / Disk 1, belonging to Disk 1, WWN = W1), and marked as the "first storage unit". The "first storage unit" set is recorded as {LUN / Disk 1} (corresponding to the disk {Disk 1}).

[0175] Further, allocate isolated storage units for subsequent data stripes.

[0176] For Strip_002, the first storage unit with WWN ≠ W1 can be found by traversing the first candidate table header (e.g., LUN / Disk 3, belonging to Disk 2, WWN = W2). LUN / Disk 3 is allocated to Strip_002, and the "first storage unit" set is updated to {LUN / Disk 1, LUN / Disk 3} (corresponding to the disks {Disk 1, Disk 2}).

[0177] For Strip_003, traverse the first candidate table, exclude storage units with WWN = W1 and W2, and select LUN / Disk 5 (belonging to Disk 3, WWN = W3). Update the "first storage unit" set to {LUN / Disk 1, LUN / Disk 3, LUN / Disk 5}.

[0178] For Strip_004, traverse the first candidate table, exclude storage units with WWN = W1, W2, and W3, and select LUN / Disk 7 (belonging to Disk 4, WWN = W4).

[0179] In this way, the 4 data stripes of Stripe_001_Copy1 can be stored in different storage units on Disk 1-Disk 4, achieving disk isolation.

[0180] It should be noted that in the disk selection step of step S2041, only "disk isolation" can be performed between different data stripes of the same data copy, and "node isolation" is not performed between different data stripes of the same data copy. For example, as shown in FIG. 2B, the different data stripes Strip_001 and Strip_002 of the same data copy Stripe_001_Copy0 correspond to Disk 1 and Disk 2, respectively, and Disk 1 and Disk 2 belong to the same storage node 1. Figure 7

[0181] ​Furthermore, in the disk selection step of S2041, from the perspective of different data copies of the same data stripe, it is not necessary to perform "storage unit isolation," "disk isolation," or "storage node isolation" between different data copies of the same data stripe. For example, as Figure 7 As shown, for data replicas Stripe_001_Copy0 and Stripe_001_Copy1 of data stripe_001, the data stripe Stripe_001_Copy0_Strip_001 of data replica Stripe_001_Copy1 and the data stripe Stripe_001_Copy1_Strip_001' of data replica Stripe_001_Copy1 can both be stored on storage node 1. Specifically, they can both be stored on LUN / Disk 1, or they can be stored separately on LUN / Disk 1 and LUN / Disk 2 of the same multi-arm disk Disk1 (and...). Figure 7 (corresponding to the example shown), or, respectively, stored in different multi-arm disks of storage node 1.

[0182] As can be seen, the disk selection strategy in step S2041 ensures that different data stripes of the same data replica belong to different multi-arm disks, avoiding hot disk issues. At the same time, the disk selection strategy in step S2041 can relax other constraints, allowing data stripes of different data replicas to be written to the same multi-arm disk or storage node simultaneously, reducing resource waste, improving utilization, and balancing reliability, performance, and storage complexity.

[0183] Figure 8 The third data balancing diagram provided in this application embodiment.

[0184] like Figure 8 As shown, step S204 may include the following step S2042.

[0185] S2042: For each data stripe in each data replica of the data to be written, in the first candidate table, select the first storage unit that does not belong to the same storage node as the second storage unit as the candidate storage unit of the current data stripe; wherein, the second storage unit refers to the candidate storage unit corresponding to other data stripes that belong to the same data replica as the current data stripe, so that the candidate storage units corresponding to different data stripes of the same data replica belong to different multi-arm disks.

[0186] The step can select disks for each data strip in each group of data copies corresponding to the data to be written in sequence, and the step can be used to achieve "storage node isolation" between different data strips of the same data copy, thereby achieving "disk isolation". Ensure that different data strips (Strip_001-Strip_004) of the same data copy (such as Stripe_001_Copy1) are stored on different storage nodes, eliminate hot disk problems, and improve data reliability. At the same time, the data to be written can be balanced on the multi-arm disks between nodes, the multi-arm disks within the node.

[0187] For example, the current processing data copy can be Stripe_001_Copy1, which contains 4 data strips (Strip_001'-Strip_004'). The first candidate table has been sorted by capacity usage rate, input / output (IO) bandwidth or input / output operations per second (IOPS), or the number of failed storage units, and the first candidate table contains LUN / Disk 1-LUN / Disk8, for example.

[0188] First, assign a candidate storage unit to the first data strip.

[0189] For Strip_001', the first storage unit (such as LUN / Disk 1, the disk Disk1 belonging to it, located in Node 1) is selected from the head of the first candidate table, and is marked as the "second storage unit". The "second storage unit" set is recorded as {LUN / Disk1} (corresponding to the disk {Disk}).

[0190] Further, assign cross-node candidate storage units to subsequent data strips.

[0191] For Strip_002', the first storage unit not on storage node 1 (such as LUN / Disk 6, the disk Disk3 belonging to it, located in storage node 2) can be found by traversing the head of the first candidate table. Assign LUN / Disk 6 to Strip_002', and update the "second storage unit" set to {LUN / Disk 1, LUN / Disk 6} (corresponding to nodes {Node1, Node2}).

[0192] For Strip_003', all storage units located on storage node 1 or storage node 2 can be excluded, and LUN / Disk9 (the disk Disk 5 belonging to it, located in storage node 3) is selected, and the "second storage unit" set is updated to {LUN / Disk 1, LUN / Disk6, LUN / Disk 9} (corresponding to nodes {Node 1, Node 2, Node 3}).

[0193] For Strip_004', the cells of storage node 1, storage node 2 and storage node 3 can be excluded, and LUN / Disk 13 (the disk Disk 7 belonging to it, located in storage node 4) is selected.

[0194] In this way, the 4 data strips of Stripe_001_Copy1 are respectively stored in different multi-arm disks of 4 different storage nodes, realizing "double isolation" (cross-disk + cross-node).

[0195] It can be understood that, Figure 8 The storage node 1, the storage node 2, the storage node 3 and the storage node 4 shown in the figure all belong to the target storage node.

[0196] It should be noted that, in the disk selection step based on step S2042, X storage nodes will be involved in the disk selection process of different data strips of one data copy, and X is equal to the number of data strips in the same data copy.

[0197] Further, in the disk selection step of step S2042, "storage unit isolation", "disk isolation" and / or "storage node isolation" can not be made between different data copies of the same to-be-written data from the dimension of the same to-be-written data. That is, the data strips of different copies of the same to-be-written data (such as Stripe_001_Copy1 and Stripe_001_Copy2) can be stored in the same storage unit, the same multi-arm disk or the same storage node. For example, Strip_001' of Stripe_001_Copy1 is stored in LUN / Disk1 of Disk 1 of storage node 1, and Strip_001" of Stripe_001_Copy2 can be stored in the same LUN / Disk1 (corresponding to the example shown in Figure 8 The figure), or can be stored in LUN / Disk2 of Disk 1, or can be stored in LUN / Disk3 of Disk2 belonging to storage node 1.

[0198] This design not only guarantees reliability, but also further improves the flexible allocation space of storage resources, maximizes resource utilization, reduces cross-node traffic and expansion cost.

[0199] Figure 9 The fourth data balancing schematic diagram provided by the embodiment of the application is provided.

[0200] As Figure 9 shown, step S204 can include the following step S2043.

[0201] S2043: For each data stripe in each group of data copies of the data to be written, based on the identifier, in the first candidate table, a first storage unit not belonging to the same multi-armed disk as the third storage unit is selected as the candidate storage unit of the current data stripe; wherein the third storage unit refers to the candidate storage unit corresponding to any data stripe in any group of data copies, so that the candidate storage units corresponding to the data to be written all belong to different multi-armed disks.

[0202] This step can select disks for each data stripe in each group of data copies corresponding to the data to be written in sequence, and this step can be used to ensure that all data stripes of the same data to be written (regardless of which group of copies) are distributed on different multi-armed disks, in other words, the LUN / Disk of the same multi-armed disk does not appear in multiple data copies of the same data stripe. The "third storage unit" here covers the candidate storage unit of any data stripe in multiple groups of data copies of the same data to be written, that is, as long as any data stripe of the data to be written (regardless of the copy) has been stored on a certain disk in the first candidate table, the other data stripes cannot use this disk.

[0203] For example, the data stripe Stripe_001 can correspond to 3 groups of data copies (Copy0, Copy1, Copy2), and each group of data copies contains 4 data stripes (Strip_001, Strip_002, Strip_003, Strip_004). The first candidate table has been sorted by capacity usage rate, input / output (IO) bandwidth or IO operations per second (IOPS), or the number of failed storage units, for example, it contains 24 LUN / Disk belonging to Disk 1-Disk 12 (WWN is W1-W12, respectively).

[0204] First, assign a candidate storage unit to the first data stripe.

[0205] For Stripe_001_Copy0_Strip_001, the first storage unit (such as LUN / Disk1, belonging to Disk1, WWN=W1) is selected from the head of the first candidate table, marked as the "third storage unit", and the third storage unit set is {LUN / Disk1}.

[0206] Further, assign isolated storage units to subsequent data stripes.

[0207] For Stripe_001_Copy0_Strip_002: traverse the candidate table, select the first storage unit (e.g. LUN / Disk 3, Disk 2, WWN=W2) not belonging to Disk 1 as the candidate storage unit, and mark the storage unit as the "third storage unit". And the third storage unit set is updated as {LUN / Disk 1, LUN / Disk 3}.

[0208] By analogy, assign candidate storage units to each data stripe of Stripe_001_Copy0. At this time, the third storage unit set is updated as {LUN / Disk 1, LUN / Disk 3, LUN / Disk 5, LUN / Disk 9}.

[0209] Further, for Stripe_001_Copy1_Strip_001': exclude {W1, W2, W5} (because the data stripes of Copy0 of the same data stripe have occupied these disks), select LUN / Disk 8 (Disk 4, WWN=W4), at this time, the third storage unit set is updated as {LUN / Disk 1, LUN / Disk 3, LUN / Disk 5, LUN / Disk 8, LUN / Disk 9}.

[0210] It can be understood that in this example, the position of the storage unit LUN / Disk 9 in the first candidate table is ahead of the position of the storage unit LUN / Disk 8 in the first candidate table, so the data stripe Stripe_001_Copy0_Strip_004 can be stored in the storage unit LUN / Disk 9 in priority.

[0211] By analogy, this will not be repeated here.

[0212] In this way, the 12 data stripes of the to-be-written data can be respectively stored in the 12 multi-arm disks, realizing "cross-disk isolation of all data stripes of the same to-be-written data".

[0213] It can be understood that, Figure 9 The storage node 1, the storage node 2, the storage node 3 and the storage node 4 shown in FIG. 1 all belong to the target storage node.

[0214] It should be noted that, unlike S2041 which only isolates data strips of the same data copy, the "third storage unit" of S2043 covers all candidate storage units corresponding to all data strips of all data copies of the same to-be-written data. For example, if Copy1_Strip_001' occupies LUN / Disk 8, then all data strips belonging to Stripe_001 such as Copy1_Strip_002', Copy2_Strip_001", and the like cannot occupy LUN / Disk 8 any more, nor can they occupy LUN / Disk 7 which is on the same disk as LUN / Disk 8. By checking the WWN / SN of the storage unit, it is strictly ensured that "the same data strip has no repeated storage unit and no repeated multi-platter disk". In this way, the hot disk problem can be significantly eliminated, the data protection reliability is improved, and the data can be balanced among multi-platter disks between nodes and multi-platter disks within a node.

[0215] Through S2043, the embodiments of the present application can achieve more comprehensive disk isolation in the dimension of "the same to-be-written data (data strip)", further improve the data protection reliability, and avoid resource waste at the same time.

[0216] Figure 10 The fifth data balancing schematic diagram provided by the embodiments of the present application is provided.

[0217] As shown in (a) of FIG. 16, step S204 can include steps S2044-S2045. Figure 10

[0218] S2044: For each data strip in each group of data copies of the to-be-written data, the storage units belonging to the first storage node are filtered out from the first candidate table to form a second candidate table; wherein the first storage node is N of the plurality of target storage nodes, and N is equal to the number of copies of the data copy.

[0219] This step can select disks for each data strip in each group of data copies corresponding to all data strips in sequence, and this step can narrow the range of disk selection by limiting the range of storage nodes. For example, in a 3-copy mechanism, 3 target storage nodes (first storage nodes) can be bound.

[0220] In actual application, the selection of the first storage node can be combined with the number of copies and the resource state of the storage node. For example, the storage node with CPU usage rate ≤ a first threshold value (for example, 60%), memory usage rate ≤ a second threshold value (for example, 70%), and / or IOPS ≤ a third threshold value (for example, 80%) is preferentially selected.

[0221] ​Further, the embodiment of the present application can perform screening according to a preset threshold (such as CPU≤60%, memory≤70%, IOPS≤80%) to generate a "candidate node pool". In the screening process, the storage node can be screened based on the condition that the storage node meets any threshold, or the condition that the storage node meets all thresholds, and the embodiment of the present application does not make specific limitation.

[0222] For example, in the case of the number of copies equal to 3.

[0223] Node 1: CPU=55%≤60%, memory=65%≤70%, IOPS=75%≤80%, the storage node can pass the screening;

[0224] Node 2: CPU=58%≤60%, memory=68%≤70%, IOPS=72%≤80%, the storage node can pass the screening;

[0225] Node 3: CPU=52%≤60%, memory=60%≤70%, IOPS=70%≤80%, the storage node can pass the screening;

[0226] Node 4: CPU=65%>60%, memory=72%>70%, IOPS=85%>80%, all thresholds are out of limits, excluded.

[0227] Finally, the first storage node: Node 1, Node 2, Node 3, exclude Node 4.

[0228] Further, the embodiment of the present application can retain the storage unit "belonging to the first storage node (Node 1, Node 2, Node 3)" from the first candidate table to form a second candidate table.

[0229] S2045: From the second candidate table, select the first storage unit that does not belong to the same multi-arm disk as the third storage unit based on the identifier as the candidate storage unit of the current data stripe; wherein the third storage unit refers to the candidate storage unit corresponding to any data stripe in the plurality of data copies, so that the candidate storage unit corresponding to the to-be-written data belongs to different multi-arm disks.

[0230] This step can be within the scope of the second candidate table, further ensuring that "different data stripes of the same data stripe" are stored in different multi-arm disks, realizing the dual constraints of "node range fixed + disk isolation".

[0231] For example, data stripe Stripe_001 can correspond to 3 groups of data copies (Copy1-Copy3), each group of data copies includes 4 data slices (Slice_001-Slice_004), the first storage node is storage node 1, storage node 2, storage node 3, and the second candidate table includes: the storage units corresponding to storage node 1, storage node 2, and storage node 3.

[0232] In this example, the "third storage unit" refers to the data slice of all selected candidate storage units of the same data stripe (Stripe_001), regardless of the copy group, and the disk selection step is as follows:

[0233] For Stripe_001_Copy0_Slice_001: the first storage unit LUN / Disk 1 (Disk 1, WWN=W1) is selected from the head of the second candidate table and marked as the "third storage unit", at this time, the third storage unit set {LUN / Disk 1} can be generated.

[0234] For Stripe_001_Copy0_Slice_002: the first storage unit LUN / Disk 3 (Disk 2, WWN=W2) that does not belong to Disk 1 is selected as a candidate storage unit by traversing the second candidate table, and is marked as the "third storage unit", and the third storage unit set is updated to {LUN / Disk 1, LUN / Disk 3}.

[0235] Subsequent data slices: in this way, all data slices of Stripe_001 are finally allocated to different multi-armed disks, and are all located in the first storage node (Node 1-Node 3).

[0236] Continuing to refer to Figure 10 In (a), steps S2044-S2045 limit that the different data slices of the same to-be-written data are stored in different multi-armed disks of the first storage node, which avoids the multi-armed disk hot disk problem, improves data reliability, and the to-be-written data can be balanced among the multi-armed disks between nodes and the multi-armed disks within the node.

[0237] Further, step S2045 does not limit whether the different data slices of the same data copy are stored in the same storage node, for example, as shown in (a), Stripe_001_Copy0_Slice_001 and Stripe_001_Copy0_Slice_002 can exist in the same storage node 1 (Node 1), and Stripe_001_Copy0_Slice_001 and Stripe_001_Copy0_Slice_003 can exist in different storage nodes. Figure 10 Further, step S2045 does not limit whether the different data slices of the same data copy are stored in the same storage node, for example, as shown in (a), Stripe_001_Copy0_Slice_001 and Stripe_001_Copy0_Slice_002 can exist in the same storage node 1 (Node 1), and Stripe_001_Copy0_Slice_001 and Stripe_001_Copy0_Slice_003 can exist in different storage nodes.

[0238] With reference back to Figure 10 (b), step S2044 can further include the following S2046.

[0239] S2046: From the second candidate table, select the first storage unit which is not in the same first storage node as the fourth storage unit and in the same first storage node as the fifth storage unit but not in the same multi-arm disk as the fourth storage unit, as the candidate storage unit of the current data slice; wherein the fourth storage unit is the candidate storage unit corresponding to other data slices which do not belong to the same data copy as the current data slice; the fifth storage unit refers to the candidate storage unit corresponding to other data slices which belong to the same data copy as the current data slice, so that the candidate storage units corresponding to different data slices of the same data slice belong to different multi-arm disks.

[0240] This step forms a double constraint of "cross-copy group node isolation + intra-copy group node disk isolation" within the first storage node range defined in S2044. "Not in the same first storage node as the fourth storage unit" ensures that data slices of different data copies are dispersed in different nodes to avoid simultaneous damage of multiple copies caused by single node failure. "In the same first storage node as the fifth storage unit but not in the same multi-arm disk" ensures that data slices of the same copy are stored in different nodes to facilitate management of data of the same copy and avoid excessive load on a single disk.

[0241] For example, take data slice Stripe_001 as an example, which corresponds to 3 groups of data copies (Copy0-Copy2), each group containing 4 data slices, the first storage node is storage node 1, storage node 2, storage node 3, and the second candidate table contains: storage units corresponding to storage node 1, storage node 2, storage node 3.

[0242] The disk selection step is as follows (take the Copy1 copy group as an example):

[0243] For Stripe_001_Copy1_Slice_001': select LUN / Disk 9 (Disk 5, WWN=W5) in storage node 2 which is different from the fourth storage unit (if storage node 1 of Copy0 has been selected) as the candidate storage unit from the second candidate table. At this time, the candidate storage unit is the "fifth storage unit" reference item for data slices of the subsequent copy group, and also the "fourth storage unit" reference item for data slices of other copy groups.

[0244] For Stripe_001_Copy1_Slice_002', the candidate storage unit needs to be in the same node as the fifth storage unit (LUN / Disk 9, storage node 2) and different disks, and different from the fourth storage unit (if Copy0 has selected the storage unit of storage node 1). After traversal, LUN / Disk 11 (Disk 6, WWN = W6) of storage node 2 is selected, and the storage unit is added to the "fifth storage unit" set.

[0245] For Stripe_001_Copy1_Slice_003', the candidate storage unit needs to be in the same node as the fifth storage unit (LUN / Disk 9, LUN / Disk 11, both belong to storage node 2) and different disks (excluding Disk 5, Disk 6), and different from the storage unit selected by Copy0 and / or Copy2.

[0246] For Stripe_001_Copy1_Slice_004', the above rules are continued to ensure that the fourth storage unit is different from the fifth storage unit in the same node and different disks.

[0247] Other copy groups select disks according to the same logic. Finally, in all data strips of Stripe_001, the data strips of different copies belong to different nodes, and the data strips of the same copy are dispersed in different disks in the same node.

[0248] It should be noted that step S2046 is constrained by two constraints, that is, isolation across copy nodes to improve data reliability (single node failure does not affect the integrity of multiple copies), and isolation of the same copy disks to balance the node load. At the same time, the fixed range of the first storage node ensures the controllability of resource scheduling, avoids the efficiency loss caused by too large disk selection range, and thus forms an optimal balance among reliability, load balancing and resource utilization efficiency.

[0249] Figure 11 The sixth data balancing diagram provided by the embodiment of the application is shown in the following table.

[0250] As Figure 11 shown, step S204 can include steps S2047-S2048.

[0251] S2047: For each data strip in each group of data copies to be written, the storage units that do not belong to the same storage node as the sixth storage unit are filtered from the first candidate table to form a third candidate table; wherein the sixth storage unit refers to the candidate storage unit corresponding to the other data strip belonging to the same data copy as the current data strip.

[0252] The step can select disks for each data slice in each set of data copies corresponding to all data slices in sequence, and the step can further narrow the range of disk selection based on the first candidate table through "node isolation" to ensure that different data slices of the same data copy are distributed in different storage nodes, and then ensure that different data slices of the same data copy are distributed in different multi-arm disks. At this time, different data slices of the same data copy will be written to Y storage nodes, and Y is equal to the number of data slices in the same data copy.

[0253] In the process of selecting disks for Stripe_001_Copy1_Slice_001', the sixth storage unit set is empty, and the third candidate table is equal to the first candidate table.

[0254] In the process of selecting disks for Stripe_001_Copy1_Slice_002', the sixth storage unit includes the candidate storage unit corresponding to Stripe_001_Copy1_Slice_001', and then the third candidate table is the result of removing all storage units of the storage node to which the sixth storage unit belongs from the first candidate table, so as to realize "storage node isolation" between different data slices of the same data copy.

[0255] In the process of selecting disks for each data slice, the third candidate table is determined in the same way, which is not described here.

[0256] In some implementations, for the data copies corresponding to the same to-be-written data, in the process of allocating candidate storage units for the data slices of the first data copy, the finally selected candidate storage units belong to Y storage nodes, and Y is equal to the number of data slices in the same data copy. Then, in the process of allocating candidate storage units for other data copies of the same to-be-written data, the first candidate table can be limited to only include the storage units of the aforementioned Y storage nodes, and then the third candidate table is formed based on the limited first candidate storage unit. In this way, the cross-node data transmission overhead can be reduced by fixing the node range, and the constraint of "cross-node isolation of data slices of the same data copy" can be strictly followed in the limited nodes, achieving a balance between reliability and performance, which is specifically manifested as:

[0257] Taking "Copy0 of data slicing Stripe_001 has occupied Y=4 nodes (storage node 1, storage node 2, storage node 3, and storage node 4)" as an example:

[0258] Other data copies of the same data stripe (e.g., Stripe_001_Copy1): the first candidate table thereof is limited to only contain storage units of storage node 1, storage node 2, storage node 3, and storage node 4. When selecting disks for Slice_001'~Slice_004' of Stripe_001_Copy1, S2047 still performs node isolation, and Stripe_001_Copy1_Slice_001' needs to be selected from one of the storage node 1, storage node 2, storage node 3, and storage node 4 (e.g., storage node 1), and Stripe_001_Copy1_Slice_002' needs to be selected from the remaining three nodes (e.g., storage node 2). Finally, the four data stripes of Stripe_001_Copy1 are still in the four storage nodes of storage node 1, storage node 2, storage node 3, and storage node 4, but belong to different nodes, satisfying the "cross-node isolation of the same data copy".

[0259] S2048: In the third candidate table, the first storage unit that does not belong to the same multi-armed disk as the third storage unit is selected based on the identifier as the candidate storage unit of the current data stripe; wherein the third storage unit refers to the candidate storage unit corresponding to any data stripe in the plurality of data copies, so that the candidate storage units corresponding to different data stripes of the same data stripe belong to different multi-armed disks.

[0260] This step further ensures that "all data stripes of the same data stripe" do not share a multi-armed disk based on the third candidate table (node isolation has been completed), achieving "storage node + disk double isolation", significantly solving the hot disk problem, and the data to be written can be balanced among the multi-armed disks between nodes and the multi-armed disks within the node.

[0261] In summary, S2047 can be used to solve the problem of "data stripe across nodes of the same data copy", and S2048 can be used to solve the problem of "all data stripes across disks of the same data stripe", avoiding the influence of single disk failure on multiple data copies of data stripes.

[0262] For example, Stripe_001 can have three groups of data copies (Copy0, Copy1, and Copy2), each group having four data stripes. Then, the four data stripes of copy0 can be allocated as follows:

[0263] Slice_001→LUN / Disk 1 (storage node 1 / Disk1);

[0264] Slice_002→LUN / Disk 5 (storage node 2 / Disk3);

[0265] Slice_003→LUN / Disk 9 (Storage Node 3 / Disk5);

[0266] Slice_004→LUN / Disk 13 (Storage Node 4 / Disk7).

[0267] The third storage unit set is {LUN / Disk 1, LUN / Disk 5, LUN / Disk 9, LUN / Disk 13} (the same data slice has been selected disks).

[0268] The 4 data strips of copy1 can be allocated as:

[0269] Slice_001'→LUN / Disk 3 (Storage Node 1 / Disk2);

[0270] Slice_002'→LUN / Disk 7 (Storage Node 2 / Disk4);

[0271] Slice_003'→LUN / Disk 11 (Storage Node 3 / Disk6);

[0272] Slice_004'→LUN / Disk 15 (Storage Node 4 / Disk8).

[0273] By analogy, finally, the 12 data strips (3 copies x 4 data strips) of Stripe_001 are distributed in 12 different multi-armed disks, and the data strips of each group of data copies belong to 4 different storage nodes, while meeting the "node isolation" and "disk isolation".

[0274] From the above, the data writing method of the embodiment of the application can ensure that different data strips of the same data copy are distributed in different multi-armed disks, effectively avoiding the hot disk problem. Further, the method provided by the embodiment of the application can also limit different data strips (belonging to any group of copies) of the same to-be-written data to be distributed in different multi-armed disks, avoid the loss of multiple data strips of the same data slice caused by single-disk failure, further improve the hot disk phenomenon, and improve the reliability.

[0275] Figure 12 The second flowchart of the data writing method provided by the embodiment of the application.

[0276] Figure 13 The seventh data balancing diagram provided by the embodiment of the application.

[0277] As shown in Figure 12 and 13 , the step S100 can further include the following step S400.

[0278] S400: If the target storage node corresponding to the to-be-written data contains a single-arm disk, a corresponding candidate storage unit is selected for each data stripe from the storage unit corresponding to the single-arm disk contained in the target storage node; wherein the candidate storage units corresponding to different data stripes of the same data copy do not belong to the same single-arm disk.

[0279] It can be understood that the storage node can also include a plurality of single-arm disks, and each single-arm disk includes one storage unit; the single-arm disks in the plurality of storage nodes can be divided into one or more storage pools. The step is for the scenario in which the target storage node adopts a single-arm disk, and ensures data reliability and storage load balancing by implementing "disk isolation" on the data stripes of the same data copy.

[0280] For example, the Copy0 of the data stripe Stripe_001 includes four data stripes (Strip_001-Strip_004), and the disk is selected for Strip_001: a storage unit of any single-arm disk (such as the storage unit of Disk A) in the single-arm disk corresponding to the target storage node is selected, and the disk ID corresponding to the data stripe is A ("A" can be WWN or SN) at this time. The disk is selected for Strip_002: the disk with ID = A (Disk A) is excluded, and a disk such as Disk B is selected from the remaining disks, and the disk ID is B. The disk is selected for Strip_003: the disks with ID = A and B (Disk A and Disk B) are excluded, and a disk such as Disk C is selected from the remaining disks, and the disk ID is C. The disk is selected for Strip_004: the disks with ID = A, B, and C (Disk A, Disk B, and Disk C) are excluded, and a disk such as Disk D is selected from the remaining disks, and the disk ID is D.

[0281] In some implementations, the embodiments of the present application can also select a disk based on the balancing strategy in combination with the capacity selection strategy and the performance selection strategy. Specifically, step S400 can include steps S401-S404.

[0282] S401: A preset storage strategy is determined, and the preset storage strategy is a capacity selection strategy or a performance selection strategy.

[0283] S402: If the preset storage strategy is the capacity selection strategy, the capacity utilization rates of all storage units corresponding to the target storage node are obtained.

[0284] S403: All storage units corresponding to the target storage node are sorted in descending order of the capacity utilization rates to form a fourth candidate table.

[0285] S404: determine the candidate storage unit of each data stripe based on the fourth candidate table.

[0286] Further, the step S401 can further include the following steps S405-S407.

[0287] S405: in the case that the preset storage strategy is the performance selection strategy, acquire the input IO bandwidth or IOPS of all storage units corresponding to the target storage node;

[0288] S406: sort all storage units corresponding to the target storage node in descending order of IO bandwidth, or in descending order of IOPS, to form a fourth candidate table;

[0289] S407: determine the candidate storage unit of each data stripe based on the fourth candidate table.

[0290] Further, the step S404 or the step S407 can save different data stripes of the same data copy in the storage units of different single-arm disks.

[0291] Or, different data copies of the same to-be-written data can be saved in the same storage node, that is, a group of data copies are saved in one storage node, and further, different data stripes of the same data copy can be saved in different single-arm disks of the same storage node.

[0292] Or, Z data stripes of one data copy can be saved in Z single-arm disks. On this basis, all data stripes of the same data stripe are saved in different single-arm disks of the Z storage nodes.

[0293] From the above, it can be seen that the embodiment of the application provides a data writing method, which can isolate different data stripes of the same data copy in different single-arm disks, avoid the hot disk problem, and realize flexible allocation.

[0294] Further, the method is suitable for HPC, big data analysis, artificial intelligence and the like. Moreover, the method supports single-arm disk storage environment, multi-arm disk (≥2 arms) storage environment, and mixed storage environment of single-arm disk and multi-arm disk. The method can cover full life cycle operations such as data read and write, node expansion, and copy number modification. Moreover, the method can realize data balancing among LUNs / Disks in the multi-arm disk, and the balancing strategy can be flexibly configured based on dimensions such as capacity (such as preferentially selecting low utilization rate disks), performance (such as preferentially selecting high IOPS disks) or reliability (such as preferentially selecting fault-free disks).

[0295] Figure 14 The structure diagram of the data writing device provided by the embodiment of the application is shown.

[0296] As Figure 14 shown in the embodiments of the present application, a data writing device is provided, which is applied to a computing device, the computing device is in communication connection with a distributed storage system; the distributed storage system comprises a plurality of storage nodes, the storage node comprises a plurality of multi-armed disks, and the multi-armed disk comprises a plurality of storage units.

[0297] The device comprises a redundancy module 1101 configured to generate a plurality of groups of data copies corresponding to the data to be written in response to a data writing operation; each group of data copies comprises a plurality of data strips.

[0298] The balancing module 1102 is configured to select a corresponding candidate storage unit for each data strip from a plurality of storage units corresponding to the multi-armed disk contained in the target storage node if the target storage node corresponding to the data to be written contains a multi-armed disk; the candidate storage units corresponding to different data strips of the same data copy belong to different multi-armed disks.

[0299] In a possible implementation, the balancing module 1102 is specifically configured to obtain identifiers of all storage units corresponding to the target storage node, the identifier is used to mark the multi-armed disk to which the storage unit belongs; determine a preset storage strategy; sort all storage units corresponding to the target storage node based on the preset storage strategy to form a first candidate table; and determine the candidate storage unit corresponding to each data strip based on the identifier and the first candidate table.

[0300] In a possible implementation, the balancing module 1102 is further configured to, in the case that the preset storage strategy is a capacity selection strategy, obtain the capacity utilization of all storage units corresponding to the target storage node; sort all storage units corresponding to the target storage node in the order from low to high capacity utilization to form the first candidate table.

[0301] In a possible implementation, the balancing module 1102 is further configured to, in the case that the preset storage strategy is a performance selection strategy, obtain the input / output (IO) bandwidth or the number of IO operations per second (IOPS) of all storage units corresponding to the target storage node; sort all storage units corresponding to the target storage node in the order from large to small IO bandwidth or in the order from high to low IOPS to form the first candidate table.

[0302] In a possible implementation, the balancing module 1102 is further configured to: in the case that the preset storage strategy is a reliability selection strategy, determine whether a storage unit of each multi-platter disk corresponding to the target storage node exists a fault; and sort the storage units corresponding to the target storage node in ascending order of the number of faulty storage units of the multi-platter disks to which the storage units belong, to form a first candidate table.

[0303] In a possible implementation, the balancing module 1102 is further configured to: for each data stripe in each group of data replicas of the data to be written, select, in the first candidate table, a first storage unit that does not belong to the same multi-platter disk as the first storage unit as a candidate storage unit of the current data stripe based on the identifier; and wherein the first storage unit refers to the candidate storage units corresponding to other data stripes belonging to the same data replica as the current data stripe, so that the candidate storage units corresponding to different data stripes of the same data replica belong to different multi-platter disks.

[0304] In a possible implementation, the balancing module 1102 is further configured to: for each data stripe in each group of data replicas of the data to be written, select, in the first candidate table, a first storage unit that does not belong to the same storage node as the second storage unit as a candidate storage unit of the current data stripe based on the identifier; and wherein the second storage unit refers to the candidate storage units corresponding to other data stripes belonging to the same data replica as the current data stripe, so that the candidate storage units corresponding to different data stripes of the same data replica belong to different multi-platter disks.

[0305] In a possible implementation, the balancing module 1102 is further configured to: for each data stripe in each group of data replicas of the data to be written, select, in the first candidate table, a first storage unit that does not belong to the same multi-platter disk as the third storage unit as a candidate storage unit of the current data stripe based on the identifier; and wherein the third storage unit refers to the candidate storage units corresponding to any data stripe in any group of data replicas of the same data stripe, so that the candidate storage units corresponding to the data to be written belong to different multi-platter disks.

[0306] In a possible implementation, the storage node further includes a plurality of single-platter disks, and each single-platter disk includes one storage unit; and the balancing module 1102 is further configured to: if the target storage node corresponding to the data to be written includes a single-platter disk, select, for each data stripe, a corresponding candidate storage unit from the storage units corresponding to the single-platter disks included in the target storage node; and wherein the candidate storage units corresponding to different data stripes of the same data replica do not belong to the same single-platter disk.

[0307] Figure 15 A schematic diagram of a computing device is provided for the embodiments of the present application.

[0308] As shown in Figure 15 , the computing device 1200 can include a server, a terminal, and the like; the computing device 1200 includes one or more processors 1201 and a memory 1202. The memory 1202 is configured to store one or more programs. Wherein when the one or more programs are executed by the one or more processors 1201, the one or more processors 1201 implement the data writing method in the above embodiments.

[0309] Continuing to refer to Figure 15 , the computing device 1200 can further include a communications interface 1203 and a communications bus 1204.

[0310] Wherein the processor 1201, the memory 1202, and the communications interface 1203 complete the communication among each other through the communications bus 1204. The communications interface 1203 is used to communicate with network elements such as clients or other servers, and the like.

[0311] In some embodiments, the one or more processors 1201 are used to execute one or more programs 1205, and specifically can execute the related steps in the data writing method embodiments described above. Specifically, the program 1205 can include program code including computer executable instructions.

[0312] Exemplarily, the processor 1201 can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement some embodiments of the present application. The one or more processors that the computing device 1200 can include can be the same type of processors, like one or more CPUs; or can be different types of processors, like one or more CPUs and one or more ASICs.

[0313] In some embodiments, the memory 1202 is used to store one or more programs 1205. The memory 1202 can include a high-speed RAM memory, and can also include a non-volatile memory (NVM), for example at least one disk memory.

[0314] The program 1205 can specifically be invoked by the processor 1201 to cause the computing device 1200 to perform the data writing method operations.

[0315] Some embodiments of the present application provide a computer readable storage medium, which stores at least one executable instruction, which, when executed on the computing device 1200, causes the computing device 1200 to perform the data writing method in the above embodiments.

[0316] The executable instruction can be specifically used to cause the computing device 1200 to perform the data writing method operation.

[0317] For example, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0318] The beneficial effects that can be achieved by the readable storage medium provided by some embodiments of the present application can refer to the beneficial effects of the corresponding data writing method provided above, which will not be described here again.

[0319] It should be noted that, in the application, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0320] Each of the embodiments in the specification is described in a relevant manner, and the same or similar parts between each of the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the part of the method embodiment.

[0321] The logic and / or steps represented in the flowcharts and / or otherwise described herein, for example, can be considered as a sequence of executable instructions stored in a computer readable medium, which can be executed by an instruction execution system, apparatus or device, such as a computer-based system, processor, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or in conjunction with which the instructions can be executed.

[0322] For purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer readable medium include the following: an electrical connection having one or more wires, a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CD-ROM).

[0323] More specific examples (a non-exhaustive list) of the computer readable medium include the following: an electrical connection having one or more wires, a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CD-ROM).

[0324] Further, a computer readable medium can even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example by optically scanning the paper or other medium, then

[0325] In the above described embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and as in another embodiment, any of the following technologies known in the art, or a combination thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions upon an application of data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0326] The above embodiments are merely specific embodiments of the present application, and are not intended to limit the scope of protection of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the scope of protection of the present application.

Claims

1. A data write method, characterized by, The application is applied to a computing device connected with a distributed storage system; the distributed storage system comprises a plurality of storage nodes, and each storage node comprises a plurality of multi-armed disks, and each multi-armed disk comprises a plurality of storage units; The method comprises: In response to a data write operation, a plurality of groups of data replicas corresponding to the data to be written are generated; each group of data replicas comprises a plurality of data strips; If the target storage node corresponding to the data to be written comprises a multi-armed disk, a corresponding candidate storage unit is selected for each data strip from a plurality of storage units corresponding to the multi-armed disk comprised by the target storage node; the candidate storage units corresponding to different data strips of the same data replica belong to different multi-armed disks.

2. The data write method of claim 1, wherein, The method comprises: An identifier of all storage units corresponding to the target storage node is obtained, and the identifier is used to mark the multi-armed disk to which the storage unit belongs; A preset storage strategy is determined; All storage units corresponding to the target storage node are sorted based on the preset storage strategy to form a first candidate table; The candidate storage unit corresponding to each data strip is determined based on the identifier and the first candidate table.

3. The data write method of claim 2, wherein, The method comprises: In a case where the preset storage strategy is a capacity selection strategy, the capacity usage rate of all storage units corresponding to the target storage node is obtained; All storage units corresponding to the target storage node are sorted in an order from low to high according to the capacity usage rate to form the first candidate table.

4. The data write method of claim 2, wherein, The method further comprises: In a case where the preset storage strategy is a performance selection strategy, the input / output (IO) bandwidth or the number of IO operations per second (IOPS) of all storage units corresponding to the target storage node is obtained; All storage units corresponding to the target storage node are sorted in an order from large to small according to the IO bandwidth, or in an order from high to low according to the IOPS to form the first candidate table.

5. The data write method of claim 2, wherein, The method further comprises: In a case where the preset storage strategy is a reliability selection strategy, it is determined whether there is a fault in the storage unit of each multi-armed disk corresponding to the target storage node; All storage units corresponding to the target storage node are sorted in an order from few to many according to the number of storage units with faults in the multi-armed disk to form the first candidate table.

6. The data write method of claim 2, wherein, The method further comprises: The candidate storage unit corresponding to each data strip is determined based on the identifier and the first candidate table. For each data strip in each group of data copies of the data to be written, based on the identifier, in the first candidate table, the first storage unit not belonging to the same multi-armed disk as the first storage unit is selected as the candidate storage unit of the current data strip; wherein the first storage unit refers to the candidate storage unit corresponding to other data strips belonging to the same data copy as the current data strip, so that the candidate storage units corresponding to different data strips of the same data copy belong to different multi-armed disks.

7. The data write method of claim 2, wherein, The determination of the candidate storage unit corresponding to each data strip based on the identifier and the first candidate table comprises: For each data strip in each group of data copies of the data to be written, in the first candidate table, the first storage unit not belonging to the same storage node as the second storage unit is selected as the candidate storage unit of the current data strip; wherein the second storage unit refers to the candidate storage unit corresponding to other data strips belonging to the same data copy as the current data strip, so that the candidate storage units corresponding to different data strips of the same data copy belong to different multi-armed disks.

8. The data write method of claim 2, wherein, The determination of the candidate storage unit corresponding to each data strip based on the identifier and the first candidate table comprises: For each data strip in each group of data copies of the data to be written, based on the identifier, in the first candidate table, the first storage unit not belonging to the same multi-armed disk as the third storage unit is selected as the candidate storage unit of the current data strip; wherein the third storage unit refers to the candidate storage unit corresponding to any data strip in the plurality of data copies, so that the candidate storage units corresponding to the data to be written belong to different multi-armed disks.

9. The data write method of claim 1, wherein, The storage node further comprises a plurality of single-armed disks, and each single-armed disk comprises one storage unit; After generating the plurality of groups of data copies corresponding to the data to be written in response to the data write operation, the method further comprises: If the target storage node corresponding to the data to be written contains the single-armed disk, for each data strip, the corresponding candidate storage unit is selected from the storage unit corresponding to the single-armed disk contained in the target storage node; wherein the candidate storage units corresponding to different data strips of the same data copy do not belong to the same single-armed disk.

10. A computing device, comprising: Comprise: One or more processors; And Memory configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data write method according to any one of claims 1-9.

Images