Disk array initialization method and storage cluster

Abstract

The present application discloses a disk array initialization method and storage cluster, which relates to the field of computer technology. The method includes dividing a disk array into multiple particles, generating bitmap information based on the position and initialization status of each particle in the disk array, and assigning the multiple particles to storage nodes in the storage cluster. The storage nodes then perform initialization operations on each particle based on the position information and initialization status of the particles assigned in the bitmap information. This method solves the technical problem of low execution efficiency of disk array initialization tasks in related technologies and achieves the technical effect of improving the execution efficiency of disk array initialization tasks.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a disk array initialization method and a storage cluster. Background Art

[0002] In the storage field, RAID (Redundant Arrays of Independent Disks) technology is widely used to improve data storage performance and reliability. It primarily combines multiple independent disks into a single logical drive, utilizing striping, mirroring, or parity checking to achieve efficient data read and write operations and redundant protection. Before a RAID array is created and used, it typically requires an initialization process of writing zeros to ensure data consistency and optimal storage system performance. However, during this RAID initialization process, related RAID initialization methods primarily rely on a single storage node to perform the initialization task. This results in longer initialization times for larger RAID capacities, leading to low RAID initialization efficiency.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] The present application provides a disk array initialization method and a storage cluster, so as to at least solve the problem of low execution efficiency of disk array initialization tasks in related technologies.

[0005] The present application provides a disk array initialization method, which is applied to a target storage node, comprising: receiving bitmap information sent by a configuration node, wherein the bitmap information is used to indicate position information of a target particle allocated to the target storage node in the disk array and initialization status information of the target particle, the target storage node is located in a storage cluster, the storage cluster includes a configuration node and storage nodes, the storage nodes include the target storage node, the configuration node is used to divide the disk array into multiple particles according to a preset particle size, and the multiple particles are correspondingly allocated to the storage nodes, the storage node is used to perform an initialization operation on the allocated particles; and the initialization operation is performed on the target particle based on the position information of the target particle and the initialization status information of the target particle.

[0006] The present application also provides a storage cluster, comprising: a configuration node and a storage node, wherein the configuration node is used to divide a disk array into multiple particles according to a preset particle size, and allocate the multiple particles to each storage node; bitmap information is determined based on the particles allocated to each storage node, wherein the bitmap information is used to indicate the position information of the particles allocated to each storage node in the disk array and the initialization status of the particles allocated to each storage node; the storage node is used to receive the bitmap information sent by the configuration node; and perform an initialization operation on the particles allocated to the storage node based on the position information of the particles indicated by the bitmap information and the initialization status information of the particles.

[0007] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned disk array initialization methods when executing the computer program.

[0008] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned disk array initialization methods are implemented.

[0009] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned disk array initialization methods when executed by a processor.

[0010] Through this application, a disk array is divided into multiple particles, bitmap information is generated based on the position and initialization status of each particle in the disk array, and the multiple particles are correspondingly assigned to storage nodes in the storage cluster, thereby causing the storage nodes to perform initialization operations on each particle based on the position information and initialization status of the particles assigned in the bitmap information. Since the entire disk array is divided into multiple particles of the same size, the configuration node assigns these particles to each storage node in the storage cluster to perform initialization operations. This allows RAID initialization to be accelerated through parallel processing without increasing hardware costs, thereby improving the initialization efficiency of the entire storage cluster. Therefore, the technical problem of low efficiency in executing disk array initialization tasks in the related art can be solved, achieving the technical effect of improving the execution efficiency of disk array initialization tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1A hardware structure block diagram of a mobile terminal for a disk array initialization method provided in an embodiment of the present application;

[0013] Figure 2 A flowchart of a method for initializing a disk array provided in an embodiment of the present application;

[0014] Figure 3 The overall flow chart of disk array initialization provided in the embodiment of the present application;

[0015] Figure 4 This is a structural block diagram of the storage cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0017] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0018] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0019] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the disk array initialization method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0020] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure diagram of a mobile terminal for a disk array initialization method provided in an embodiment of the present application. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. The mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0021] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the disk array initialization method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0022] Transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0023] An embodiment of the present application provides a disk array initialization method, and the method is described in detail in conjunction with the execution flow of the disk array initialization method.

[0024] The following are explanations of the professional terms in the embodiments of this application:

[0025] RAID: Redundant Arrays of Independent Disks, also known as disk arrays, is a data storage technology that combines multiple physical disk drives into a logical unit to improve data transmission speed and data redundancy, thereby enhancing data security and system reliability.

[0026] RAID Strip: RAID striping refers to a method of dividing continuous data into data blocks of the same size and writing each piece of data to different disks in the disk array.

[0027] RAID Initialize: RAID initialization refers to the process of initializing and writing zero configurations to the RAID array before using it after the RAID array is created.

[0028] In this embodiment, a method for initializing a disk array is provided. Figure 2 A flowchart of a method for initializing a disk array provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the process includes the following steps:

[0029] Step S202: Receive bitmap information sent by the configuration node, where the bitmap information is used to indicate location information of a target granule assigned to a target storage node in the disk array and initialization status information of the target granule. The target storage node is located in a storage cluster, which includes a configuration node and storage nodes. The storage nodes include the target storage node. The configuration node is used to divide the disk array into multiple granules according to a preset granularity, and the multiple granules are assigned to storage nodes. The storage nodes are used to perform initialization operations on the assigned granules.

[0030] In embodiments of the present application, bitmap information is used to track and manage the initialization status of a RAID (i.e., disk array). This bitmap information can be a data structure, typically using binary notation to represent the initialization status of each particle (or data block) in the RAID. The bitmap information includes the location of each particle in the RAID and its initialization status. Using this bitmap information, the system can efficiently track and manage the allocation and initialization status of particles in the disk array. Furthermore, after a disk failure, it can quickly identify and locate particles requiring reconstruction for data recovery, thereby accelerating the RAID recovery process and ensuring efficient and reliable operation of the disk array.

[0031] In an embodiment of the present application, the target storage node may be any storage node in the storage cluster that performs initialization operations on a RAID (i.e., a disk array); the target particle may be a particle assigned to the target storage node by the configuration node.

[0032] For example, when a storage cluster initializes a RAID array, a configuration node in the storage cluster sets a granularity based on actual requirements, such as 16MB, 32MB, and 64MB. The RAID array (i.e., a disk array) is divided into multiple granules according to the preset granularity. Bitmap information for each granule is recorded in sequence based on the actual position of each granule in the RAID array. Each granule occupies one bit in the bitmap information, and each bit is initially assigned a value of 1 (i.e., indicating that the granule is being initialized). Simultaneously, the configuration node determines the storage node in the storage cluster that will perform the RAID initialization task and allocates each granule to each storage node performing the RAID initialization task. Optionally, each granule can be allocated sequentially according to the granule order, or as needed based on the actual needs of each storage node. After generating the bitmap information and completing the allocation of each granule, the configuration node sends the bitmap information to all storage nodes in the storage cluster that will perform the RAID initialization task. The target storage node then receives the bitmap information sent by the configuration node.

[0033] Step S204 : performing an initialization operation on the target particle based on the position information of the target particle and the initialization state information of the target particle.

[0034] Optionally, the execution entity of the above steps can be a background processor, or other devices with similar processing capabilities, or a machine that integrates at least an image acquisition device and a data processing device, wherein the image acquisition device may include a graphics acquisition module such as a camera, and the data processing device may include a computer, a mobile phone and other terminals, but is not limited to this.

[0035] Through the above steps, the disk array is divided into multiple particles, bitmap information is generated based on the position and initialization status of each particle in the disk array, and the multiple particles are correspondingly assigned to storage nodes in the storage cluster, thereby causing the storage nodes to perform initialization operations on each particle based on the position information and initialization status of the assigned particles in the bitmap information. Because the entire disk array is divided into multiple particles of equal size, the configuration node assigns these particles to each storage node in the storage cluster to perform initialization operations. This accelerates RAID initialization through parallel processing without increasing hardware costs, thereby improving the initialization efficiency of the entire storage cluster. This solves the technical problem of low efficiency in executing disk array initialization tasks in related technologies and improves the execution efficiency of disk array initialization tasks.

[0036] As an optional embodiment, an initialization operation is performed on the target particles based on the position information of the target particles and the initialization status information of the target particles, including: determining a first particle from the target particles according to the initialization status information, wherein the particle value of the first particle is a first value; determining a first sorting of each particle in the first particle based on the position information; and performing an initialization operation on each particle in the first particle in turn based on the first sorting.

[0037] In an embodiment of the present application, the first particle may be a particle whose particle value is the first value among the target particles, that is, a particle that performs an initialization operation among the target particles. The bitmap information represents the initialization state of each particle through the particle value of the particle. For example, when binary values ​​are used to represent particle status information, a particle value of 1 indicates that the particle is not performing an initialization operation, and a particle value of 0 indicates that the particle has already performed an initialization operation. In this case, the first value is 1.

[0038] In an embodiment of the present application, the first ordering may be the ordering of each particle in the first particle among the target particles. This first ordering may be determined based on the position information of each particle in the target particle in the disk array. For example, particles 1, 5, 9, and 13 in the disk array are target particles assigned to the target storage node. If the particle values ​​of the target particles are particles 5, 9, and 13 with the first value, that is, the first particles are particles 5, 9, and 13, then the first ordering is particle 5 → particle 9 → particle 13.

[0039] Based on the above, the target storage node identifies the first granule among the target granules that has not yet been initialized, i.e., the granule whose granule value is the first value. Then, based on the first ranking of the granules in the first granule, the target storage node sequentially performs the initialization operation on each granule in the first granule. By using bitmap information to perform initialization operations on each granule in the disk array, initialization operations can be performed more orderly, reducing errors and uncertainties during the initialization process, improving system stability and robustness, and reducing unnecessary computation and resource consumption, thereby improving overall processing efficiency and enhancing the resource utilization and task processing capabilities of the storage cluster.

[0040] As an optional implementation, performing an initialization operation on each particle in the first particle in sequence based on the first sorting includes: obtaining a target resource utilization rate of the target storage node at the current moment; determining a target rate corresponding to the target resource utilization rate from a target mapping relationship, wherein the target mapping relationship is used to indicate a relationship between the resource utilization rate and the rate at which the storage node performs the initialization operation, and the target rate is the rate at which the target storage node performs the initialization operation on the first particle within a preset time period; and performing the initialization operation on each particle in the first particle in sequence based on the target rate.

[0041] In an embodiment of the present application, the preset time period can be a time period of preset length, such as 3 minutes, 5 minutes, etc.; the current moment can be the moment for obtaining the resource utilization of the target storage node, and the current moment can be any moment. For example, the current moment can be the initial moment of each preset time period; the target resource utilization can be the resource utilization of the target storage node at the current moment, and the resource utilization can be the storage node's usage of resources, such as the storage node's cache utilization, CPU utilization, network bandwidth utilization, etc.

[0042] In an embodiment of the present application, the target mapping relationship may be a correspondence between resource utilization and the rate at which initialization operations are performed. In this mapping relationship, resource utilization may be the usage of a single resource or may include the usage of multiple resources, without limitation herein. Taking the cache utilization of a storage node as an example, the specific mapping relationship is shown in Table 1 below.

[0043] Table 1 Target mapping relationship

[0044]

[0045] In an embodiment of the present application, the target rate may be the rate at which the target storage node performs initialization operations on MiG particles in the first particle within a preset time period, as determined from the target mapping relationship based on the current target resource utilization. For example, if the preset time period is 5 minutes, and the cache utilization (i.e., target resource utilization) of the target storage node at the beginning of the 5-minute period (i.e., the current moment) is 15%, then the target rate is determined to be 512 MBps based on the target mapping relationship. This means that the target storage node performs initialization operations on each particle in the first particle at a rate of 512 MBps within the 5-minute period.

[0046] Through the above, the storage node's initialization rate is determined based on its resource utilization. The storage node can dynamically adjust the RAID initialization rate to match the current resource availability, thereby maximizing the use of existing resources and improving RAID initialization efficiency without increasing hardware costs. In a storage cluster environment, this adaptive initialization rate strategy allows each storage node to independently optimize the speed of its initialization tasks while maintaining the consistency and stability of the entire system, effectively improving RAID initialization efficiency.

[0047] As an optional implementation, an initialization operation is sequentially performed on each particle in the first particle based on the target rate, including: performing the following operations on each particle, wherein the particle currently being initialized is the current particle: dividing the current particle according to a preset stripe width to obtain multiple target stripes; determining a second ranking of each of the multiple target stripes in the current particle; performing the initialization operation on the multiple target stripes based on the second ranking; and, when it is determined that initialization of the current particle is complete, reporting initialization completion information to the configuration node, so that the configuration node updates the particle value of the current particle to the second value based on the initialization completion information.

[0048] In an embodiment of the present application, the current particle may be a particle currently performing an initialization operation; the target stripe may be a stripe obtained by dividing the current particle by the target storage node according to a preset stripe width, wherein the preset stripe width may be set according to actual conditions and is used to define the minimum unit into which data needs to be divided each time the disk array performs a write operation; the second sorting may be the sorting of each target stripe in the current particle.

[0049] Through the above, when performing initialization operations on the current granule, the target storage node divides the current granule according to the preset stripe width to obtain multiple target stripes. It then performs initialization operations on the current granule based on the second ranking of each target stripe within the current granule. Upon determining that initialization of the current granule is complete, the node reports initialization completion information to the configuration node, causing the configuration node to update the granule value of the current granule to the second value based on the initialization completion information. This granular operation (i.e., initializing each stripe within the granule one by one) and timely status feedback enable more efficient utilization of system resources, accelerate the overall initialization process, and enable the storage cluster to effectively monitor and manage RAID initialization progress, thereby maintaining a consistent global initialization state and ensuring efficient completion of initialization tasks within a limited time, thereby improving the efficiency and reliability of storage cluster initialization.

[0050] As an optional implementation, determining whether the initialization of the current particle is complete includes: generating a stripe bitmap based on multiple target strips, wherein the stripe bitmap is used to indicate initialization status information of the target stripes; and determining that the initialization of the current particle is complete when the stripe bitmap shows that the stripe values ​​of the multiple target strips are all the third value.

[0051] In an embodiment of the present application, a stripe bitmap may be information used to track and manage the initialization status of a particle. The stripe bitmap may be a data structure that typically uses different numerical values ​​to represent the different initialization states of each particle (or data block) in the RAID, such as a binary array. The stripe bitmap includes the location information of each target stripe in the current particle and the initialization state of the stripe. Each target stripe corresponds to a bit in the stripe bitmap, and the stripe value on each bit represents the initialization state of the target stripe. For example, if the stripe bitmap is a binary array, a 1 (i.e., stripe value) may be used to mark the target stripe as incompletely initialized, and a 0 (i.e., stripe value) may be used to mark the target stripe as completely initialized. Through the stripe bitmap, the current particle can determine the initialization state of each target stripe, further refining the management of the disk array's initialization state and providing more accurate status information, helping the target storage node determine which specific stripes require priority initialization when processing write IO requests.

[0052] In this embodiment of the present application, the third value may be a specific value used to mark the completion of initialization of the target stripe. The target storage node continuously updates the stripe bitmap during the initialization process for the current granule. Whenever a target stripe is initialized, the stripe value of the corresponding bit is updated from the pre-initialization value (e.g., 1) to the third value (e.g., 0). Initialization of the current granule is determined to be complete when the stripe bitmap indicates that the stripe values ​​of multiple target stripes are all the third value.

[0053] By using the stripe bitmap, the initialization status of multiple target stripes can be quickly determined, which helps to quickly identify and solve potential problems, reduce errors that may be caused by manual inspection, speed up the overall initialization process, and improve the reliability of the initialization process. At the same time, the initialization completion status can also be determined in a timely manner, thereby more effectively allocating and using system resources and avoiding unnecessary waiting or resource waste.

[0054] As an optional embodiment, the initialization operation is performed on each particle in the first particle in sequence based on the target rate. The above method also includes: receiving a write request for requesting to write the target data to the third particle, wherein the write request carries information about the third particle; determining the particle value of the third particle based on the information of the third particle; when the particle value of the third particle is the second value, writing the target data to the third particle; when the particle value of the third particle is the first value, determining the fourth particle that is currently performing the initialization operation; pausing the initialization operation on the first stripe, and performing the initialization operation on the third particle based on the target rate, so as to write the target data to the third particle after determining that the initialization of the third particle is completed; and when it is determined that all the target data is written into the third particle, continuing to perform the initialization operation on the fourth particle based on the target rate.

[0055] In an embodiment of the present application, the third particle may be a particle requesting to write target data; the fourth particle may be a particle that receives a write request requesting to write target data to the third particle, determines the initialization status of the third particle, and directly writes the target data to the third particle if the third particle has completed initialization at this time; if the third particle has not completed initialization at this time, at this time, the target storage node determines that the fourth particle is currently performing the initialization operation, and suspends the initialization operation on the fourth particle, and instead performs the initialization operation on the third particle first, and then, when it is determined that the third particle has completed initialization, writes the target data to the third particle, and after all the target data are written, continues to perform the initialization operation on the fourth particle.

[0056] Through the above, the target storage node can flexibly handle sudden write requests during RAID granule initialization, minimizing the impact on the initialization process while ensuring data write security. Furthermore, the target storage node can rationally allocate resources based on real-time demand, increasing the flexibility and adaptability of the target storage node during initialization, improving the overall efficiency of the storage cluster and user experience.

[0057] As an optional implementation, after determining the target rate corresponding to the target resource utilization from the target mapping, the method further includes: determining an expected completion time for the target storage node to perform the initialization operation on the first particle; and sending the expected completion time to the configuration node.

[0058] In the embodiment of the present application, the expected completion time of the target storage node performing the initialization operation on the first particle can be determined by the following formula:

[0059]

[0060] Where, The expected completion time; The number of target granules that have not yet been initialized among the target granules allocated to the target storage node; is the preset granularity size; is the target rate.

[0061] As an optional embodiment, after performing the initialization operation on each particle in the first particles in sequence based on the target rate, the above method further includes: after the preset time period ends, and when it is determined that the target particle has not completed the initialization operation, updating the target rate to perform the initialization operation on the second particle based on the updated target rate, wherein the second particle is a particle whose particle value is the first value after the preset time period ends, and the target particles include the second particle.

[0062] In the embodiment of the application, the second particle may be a particle that has not completed the initialization operation after the preset time period. After the preset time period, the target storage node checks whether all particles in the target particle have completed the initialization operation. If all have completed the initialization operation, the target storage node ends the initialization operation on the target particle. If there are still particles that have not completed the initialization, the target storage node determines the second particle that has not completed the initialization, and re-determines the rate for executing the initialization operation at the beginning of the next time period, updates the target rate, and then executes the initialization operation on the second particle based on the target rate.

[0063] Through the above, the target storage node can adaptively adjust the initialization rate according to the actual operation situation in real time, thereby improving resource utilization and initialization efficiency, thereby better balancing system resources and accelerating the RAID initialization process without increasing hardware costs.

[0064] As an optional implementation, the above method also includes: sending bitmap information to the first storage node, so that when the target storage node fails, the first storage node performs an initialization operation on the target particle based on the location information of the target particle and the initialization status information of the target particle; wherein the first storage node is a backup node of the target storage node, and the configuration node creates a mirror pair based on the storage nodes included in the storage cluster before sending the bitmap information to the target storage node, and the mirror pair includes a master node and a backup node.

[0065] In an embodiment of the present application, the first storage node can be a backup node of the target storage node, so as to perform initialization operations on the target particles assigned to the target storage node in the event of a failure of the target storage node. Before the storage cluster performs the initialization operation on the RAID, the configuration node creates a mirror pair in the storage cluster, and each mirror pair includes a master node and a backup node. The master node is responsible for processing normal initialization operations and read and write requests, while the backup node takes over its tasks when the master node fails to ensure high availability of the storage cluster and continuous access to data.

[0066] For example, consider a storage cluster consisting of four storage nodes, distributed across four mirrored pairs: mirrored pair 1 (storage node 1, storage node 3), mirrored pair 2 (storage node 3, storage node 2), mirrored pair 3 (storage node 2, storage node 4), and mirrored pair 4 (storage node 4, storage node 1). In each mirrored pair, the preceding storage node serves as the primary node, while the succeeding storage node serves as the primary node's backup node. If a storage node fails, its backup node temporarily takes over I / O requests from the failed storage node. Once the failed storage node recovers, the backup node no longer takes over I / O requests not belonging to it. Furthermore, a backup node will not initialize target granules assigned to the failed storage node until its own granules have completed initialization. If a backup node has completed initialization of all its own granules and the failed storage node has not recovered, the backup node will begin initializing the target granules assigned to the failed storage node and will not resume initialization until the failed storage node recovers. It should be noted that the above is merely a preferred embodiment; the number of primary and backup nodes in each mirrored pair can be adjusted based on actual circumstances and is not a limitation.

[0067] Through the above content, during the RAID initialization process, mirroring helps to parallelize the data initialization, so that when a storage node fails, it can automatically switch and the backup node continues to perform the initialization task, avoiding the interruption of the initialization process and improving the reliability and initialization efficiency of the storage cluster.

[0068] As an optional implementation, Figure 3 The overall flow chart of disk array initialization provided in the embodiment of the present application is as follows: Figure 3 As shown, the specific process is as follows:

[0069] S301: A storage cluster creates a mirror pair, and a storage node joins the mirror pair.

[0070] Configuration nodes in a storage cluster create mirrors. For example, a storage cluster with four storage nodes is divided into four mirror pairs: Mirror Pair 1 (Storage Node 1, Storage Node 3), Mirror Pair 2 (Storage Node 3, Storage Node 2), Mirror Pair 3 (Storage Node 2, Storage Node 4), and Mirror Pair 4 (Storage Node 4, Storage Node 1). In each mirror pair, the first node is the primary node, and the second node is the backup node. If a storage node (the target storage node) fails, its backup node (the first storage node) temporarily takes over I / O requests from the failed storage node. Once the failed storage node recovers, the backup node will no longer handle I / O requests not intended for it. Furthermore, the backup node will not initialize granules assigned to the failed storage node (the target granules) until its own granules have completed initialization. If the backup node has completed initialization of all its own granules and the failed storage node has not recovered, it will begin initializing the granules assigned to the failed node and will stop after the failed storage node recovers.

[0071] S302: The storage cluster creates a RAID (i.e., disk array), sets the RAID stripe width and granularity, splits the RAID overall space according to the granularity, generates a granularity bitmap table (i.e., bitmap information), and distributes it to each storage node in sequence.

[0072] When creating a RAID array in a storage cluster, the configuration node must set the RAID stripe width. The stripe width can be 128KB or 256KB, with 256KB being the default. The configuration node can also set the RAID granularity. Granularity sizes can be 16MB, 32MB, or 64MB, with 32MB being the default. If the configuration node does not set the RAID granularity, RAID initialization continues according to the existing process (that is, the configuration node initializes the RAID array). If the configuration node sets the RAID granularity, the storage cluster splits the RAID array into multiple granules based on the granularity and sequentially assigns the granules to each storage node. For example, granule 1 is assigned to storage node 1, granule 2 to storage node 2, granule 3 to storage node 3, granule 4 to storage node 4, granule 5 to storage node 1, and so on. The configuration node also generates a granule bitmap (i.e., bitmap information). Each granule occupies one bit in the granule bitmap, which is recorded sequentially. Each bit is initially assigned a value of 1 (the first value). The granule bitmap is then sent to all storage nodes. After that, a RAID initialization task is started, and the progress information of the task can be displayed on the task interface of the storage cluster, for example, "Running" can be displayed on the task interface.

[0073] S303 : Each storage node receives the particle bitmap information (ie, bitmap information) distributed by the storage configuration node, and generates a corresponding stripe bitmap for each particle.

[0074] After each storage node (including the target storage node) receives the particle bitmap information distributed by the configuration node, it splits the allocated particles (i.e., target particles) again according to the set RAID stripe width value (i.e., the preset stripe width) to generate a RAID stripe bitmap (i.e., stripe bitmap) for each particle. In the stripe bitmap, each stripe (i.e., target stripe) occupies one bit and is recorded in sequence. Each bit is initially assigned a value of 1.

[0075] S304: Start a 5-minute timer, adaptively set the RAID initialization rate, and report the expected completion time to the configuration node.

[0076] The storage node (i.e., the target storage node) starts a 5-minute timer (i.e., the preset time period), checks its own cache utilization (i.e., the target resource utilization) at the current moment, adaptively sets the execution rate of the RAID action (i.e., the target rate), calculates the expected completion time, and reports it to the configuration node. This includes:

[0077] When a storage node initializes a RAID granule (i.e., target granule), it creates an adaptive relationship (i.e., target mapping relationship) between the initialization rate and the storage node's cache utilization. The lower the storage node's cache utilization, the faster the storage node's RAID initialization rate. Conversely, the higher the storage node's cache utilization, the slower the storage node's RAID initialization rate. This adaptive relationship can be shown in Table 1 above.

[0078] In order to fully utilize the hardware resources of the storage node and improve the efficiency of RAID initialization, the storage node will automatically adjust the speed of RAID initialization (i.e., update the target rate) each time the 5-minute timer (i.e., the preset time period) starts (i.e., the current moment). For example, if the cache utilization of the storage node is 15% at moment 1, the storage node will adjust the RAID initialization speed to 512MBps. After 5 minutes, due to the addition of new IO services, the cache utilization of the storage node will be increased to 72%, so the node will adjust the RAID initialization speed to 8MBps. After another 5 minutes, because all IO services are completed, the cache utilization of the storage node will drop to 5%, so the node will adjust the RAID initialization speed to 1GBps. After another 5 minutes, if the cache utilization of the storage node is still below 10%, the node will adjust the RAID initialization speed to 1GBps. It should be noted that although the initialization speed before and after the subsequent changes are the same, there will be changes in the instructions issued and the execution actions. Expected completion time of the RAID action of the target storage node:

[0079]

[0080] Where, The expected completion time; The number of target granules that have not yet been initialized among the target granules allocated to the target storage node; is the preset granularity size; is the target rate.

[0081] Each storage node (including the target storage node) sends the expected completion time of the RAID action to the configuration node each time the rate changes. The configuration node selects the one with the longest expected completion time as the expected completion time of the entire RAID action and displays it.

[0082] S305: Initialize the stripe.

[0083] Perform an initialization write operation on the stripe of the particle (i.e., the first particle) whose value in the particle bitmap is 1 (i.e., the particle value is the first value), set the initialized stripe bit value in the stripe bitmap from 1 to 0 (i.e., update the stripe value to the third value), and after all stripes in the entire particle (i.e., the current particle) are initialized (i.e., the stripe values ​​of multiple target strips displayed in the stripe bitmap are all the third value), set the corresponding particle bit value in the particle bitmap (i.e., the bitmap information) from 1 to 0 (i.e., update the particle value of the current particle to the second value), and then perform an initialization operation on the particle with a value of 1 in the next particle bitmap.

[0084] S306, processing flow in different situations.

[0085] Case 1: There is no write IO request on the storage node;

[0086] The storage node (i.e., the target storage node) checks the received granule bitmap information (i.e., bitmap information). If the bit value in the granule bitmap is 0 (i.e., the granule value of the current granule is the second value), the granule corresponding to that bit (i.e., the current granule) is considered initialized and no further initialization is performed. Initialization of the next granule is then performed. If the bit value in the granule bitmap is 1 (i.e., the granule value of the current granule is the first value), the storage node initializes the stripes (i.e., the multiple target stripes) within the granule (i.e., the current granule) by writing 0s (i.e., performing initialization), setting the initialized stripe bits in the stripe bitmap from 1 to 0. After all stripes in the granule are initialized, the corresponding granule bits in the granule bitmap are set from 1 to 0, and initialization is performed on the next granule. After each stripe is initialized, the storage node (i.e., the target storage node) synchronizes the granule bitmap (i.e., bitmap information) and stripe bitmap information (i.e., stripe bitmap) for its initialization task with its backup node (i.e., the first storage node) in the mirror pair.

[0087] Case 2: The storage node has a write IO request, and the write IO request falls on an uninitialized stripe;

[0088] When a storage node (i.e., the target storage node) receives a write request (i.e., a write request) for an uninitialized stripe (i.e., the third granule has not yet completed initialization), it completes initialization of the current stripe (i.e., the first stripe in the fourth granule currently being initialized) and suspends subsequent stripe initialization operations (i.e., suspends initialization of the fourth granule). It then prioritizes initializing the stripe in the target area (i.e., the third granule) of the write request. After initialization is complete and the corresponding values ​​in the granule bitmap and stripe bitmap are changed, the write request is flushed to disk (i.e., the target data is written to the third granule). It then initializes subsequent stripes (i.e., stripes after the first stripe in the fourth granule) starting from the stripe location before the pause. Because sequential RAID initialization was interrupted, the storage node checks the received granule bitmap information, locates the granule with the first bit value 1, then checks the stripe bitmap information of the corresponding granule, locates the stripe with the stripe bitmap value 1 (i.e., the stripe before the RAID initialization task was paused), and resumes subsequent initialization operations.

[0089] Case 3: Storage node failure;

[0090] If a storage node (i.e., the target storage node) fails during RAID initialization, the backup node (i.e., the first storage node) in its mirror pair will aggregate the number of uninitialized granules in its own node (i.e., the first storage node) and the failed node (i.e., the target storage node) at the start of the next 5-minute timer (i.e., the preset time period), calculate the new expected completion time, and prioritize initializing the granules originally assigned to itself. After all granules are initialized, it will initialize the uninitialized granules assigned to the failed node.

[0091] Case 4: The failed node is restored to the storage cluster;

[0092] After the failed node recovers and returns to the cluster, it will synchronize cluster data information from the backup node in the mirror pair, obtain the granule (i.e., target granule) bitmap information and stripe bitmap information assigned to itself saved on the backup node, and compare them with the granule bitmap information and stripe bitmap information saved before the failure.

[0093] If the granule bitmap information obtained from the backup node is the same as the granule bitmap information saved before the failure, and the stripe bitmap information obtained from the backup node is the same as the stripe bitmap information saved before the failure, the failed node (i.e., the target storage node) believes that the backup node (i.e., the first storage node) did not initialize the granules assigned to the failed node during its failure. The failed node locates the granule with the first bit value 1 based on the granule bitmap information saved before the failure, and then checks the stripe bitmap information of the corresponding granule. After locating the stripe with the stripe bitmap value 1, it directly starts the 5-minute timer and continues the subsequent initialization operation. The backup node of the failed node will no longer count the number of granules assigned to the failed node when the next 5-minute timer starts to calculate the expected completion time. After the backup node completes the initialization of the granules assigned to itself, it will not initialize the granules assigned to the failed node.

[0094] If the backup node has initialized the particles assigned to the failed node when the failed node recovers, then when the failed node obtains the particle bitmap information and corresponding stripe bitmap information of the particles assigned to the failed node from the backup node, the backup node stops the RAID initialization action after initializing the current RAID stripe, and sends the latest particle bitmap information and stripe bitmap information to the failed node. After receiving them, the failed node compares them with the particle bitmap information and stripe bitmap information saved before the failure. If it is found that there are differences in the values ​​of the particle bitmap information and stripe bitmap information, it updates the particle bitmap information and stripe bitmap information saved before the failure based on the received particle bitmap information and stripe bitmap information, locates the first particle with a bit value of 1 according to the updated particle bitmap information, checks the stripe bitmap information of the corresponding particle, locates the stripe with a stripe bitmap of 1, directly starts the 5-minute timer, and continues with subsequent initialization operations.

[0095] Case 5: Deleting RAID when there is no storage node failure;

[0096] After the RAID deletion instruction is issued, the configuration node notifies each storage node to stop the RAID initialization action, delete the 5-minute timer, delete the received granule bitmap, delete the stripe bitmap generated by the node itself, and then the configuration node deletes the granule bitmap and deletes the RAID.

[0097] Scenario 6: After deleting the RAID when a storage node fails, the failed storage node is restored to the storage cluster.

[0098] If a storage node in the storage cluster is already faulty before the RAID deletion command is issued, the configuration node will notify each storage node to stop RAID initialization, delete the 5-minute timer, delete the received granular bitmap, and delete the node's self-generated stripe bitmap. The configuration node then deletes the granular bitmap and RAID. After the failed node returns to the cluster, it will synchronize cluster data with the backup node in the mirror pair. Upon discovering that there is no RAID information, it will delete the granular bitmap it received before the failure and the stripe bitmap it generated.

[0099] S307: After the 5-minute timer ends, it is determined whether all particles assigned to the storage node have been initialized.

[0100] After the 5-minute timer expires, the storage node (i.e., the target storage node) checks whether all the granules assigned to it (i.e., the target granules) have been initialized. If so, it returns a task completion message to the configuration node. If not, it sends its currently completed granule bitmap information and stripe bitmap information to the backup node in the mirror pair, then starts another 5-minute timer and continues RAID initialization (i.e., performs initialization on the second granule). This process continues until all the granules assigned to the node have been initialized.

[0101] S308: After receiving the initialization task completion information reported by all storage nodes, the configuration node processes the RAID initialization task.

[0102] The configuration node receives and records the initialization completion information of all particles assigned to itself sent by each storage node. If all storage nodes send initialization completion information, the configuration node considers that the initialization task of the RAID has been completed, and changes the RAID initialization task progress information to 100%. The task interface of the storage cluster no longer displays "Running" and instead displays the "Completed" task interface.

[0103] If the failed node has not recovered, its backup node will handle the RAID initialization task for the granules assigned to the failed node. After the backup node completes the initialization of the granules assigned to the failed node, it will send a message to the configuration node indicating that all granule initialization tasks have been completed. This message will contain a flag indicating that the completion message was sent on behalf of the failed node to complete the granule initialization. After receiving this message, the configuration node will also consider the RAID initialization task complete and change the RAID initialization task progress information to 100%. The task interface of the storage cluster will no longer display "Running" and will instead display the task interface as "Completed".

[0104] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0105] The embodiments of the present application also provide a storage cluster, which is used to implement the above embodiments and preferred implementations. Details already described are not repeated here. Although the storage cluster described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0106] Figure 4 The structural block diagram of the storage cluster provided in the embodiment of the present application is as follows: Figure 4 As shown, the storage cluster 40 includes a configuration node 402 and a storage node 404, wherein: the configuration node 402 is used to divide the disk array into multiple particles according to a preset particle size and allocate the multiple particles to each storage node; determine bitmap information based on the particles allocated to each storage node, wherein the bitmap information is used to indicate the position information of the particles allocated to each storage node in the disk array and the initialization status of the particles allocated to each storage node; the storage node 404 is used to receive the bitmap information sent by the configuration node; and perform an initialization operation on the particles allocated to the storage node based on the position information of the particles indicated by the bitmap information and the initialization status information of the particles.

[0107] As an optional implementation, the storage node 404 is further configured to perform the following operations: determining a first particle from particles assigned to the storage node according to the initialization state information, wherein a particle value of the first particle is a first value; determining a first ranking of each particle in the first particle based on the position information; and performing an initialization operation on each particle in the first particle in sequence based on the first ranking.

[0108] As an optional implementation, the storage node 404 is further configured to perform the following operations: obtaining a target resource utilization of the storage node at the current moment; determining a target rate corresponding to the target resource utilization from a target mapping relationship, wherein the target mapping relationship is used to indicate a relationship between the resource utilization and a rate at which the storage node performs initialization operations, and the target rate is a rate at which the storage node performs initialization operations on the first particle within a preset time period; and performing initialization operations on each particle in the first particle in sequence based on the target rate.

[0109] As an optional implementation, the storage node 404 is further configured to perform the following operations: perform the following operations on each particle, where the particle currently being initialized is the current particle: divide the current particle according to a preset stripe width to obtain multiple target stripes; determine a second ordering of each of the multiple target stripes in the current particle; perform an initialization operation on the multiple target stripes based on the second ordering; and, when it is determined that initialization of the current particle is complete, report initialization completion information to the configuration node, so that the configuration node updates the particle value of the current particle to the second value based on the initialization completion information.

[0110] As an optional implementation, the storage node 404 is further configured to perform the following operations: generating a stripe bitmap based on multiple target stripes, wherein the stripe bitmap is used to indicate initialization status information of the target stripes; and determining that initialization of the current particle is complete when the stripe bitmap shows that the stripe values ​​of the multiple target stripes are all the third value.

[0111] As an optional implementation, the storage node 404 is further configured to perform the following operations: after determining a target rate corresponding to a target resource utilization rate from a target mapping, determining an expected completion time for the storage node to perform an initialization operation on a first particle; sending the expected completion time to the configuration node; after performing the initialization operation on each particle in the first particle in sequence based on the target rate, after a preset time period ends, and if it is determined that the particle assigned to the storage node has not completed the initialization operation, updating the target rate, so as to perform the initialization operation on a second particle based on the updated target rate, wherein the second particle is a particle whose particle value is the first value after the preset time period ends, and the particles assigned to the storage node include the second particle.

[0112] As an optional implementation, the storage node 404 is further configured to perform the following operations: sending bitmap information to a backup node of the storage node, so that when the storage node fails, the backup node performs an initialization operation on the particles assigned to the storage node based on the location information of the particles assigned to the storage node and the initialization status information of the particles assigned to the storage node; wherein, before sending the bitmap information to the storage node, the configuration node creates a mirror pair based on the storage nodes included in the storage cluster, and the mirror pair includes a primary node and a backup node.

[0113] For descriptions of features in the embodiments corresponding to the storage cluster, please refer to the descriptions of the embodiments corresponding to the disk array initialization method, and will not be repeated here. Embodiments of the present application also provide an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to perform the steps in any of the above-mentioned disk array initialization method embodiments.

[0114] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps of any of the above-mentioned disk array initialization method embodiments when running.

[0115] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0116] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the computer program implements the steps of any of the above-mentioned disk array initialization method embodiments.

[0117] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned disk array initialization embodiments are implemented.

[0118] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] The above describes in detail the disk array initialization method and storage cluster provided by this application. This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only intended to help understand the method and core concept of this application. It should be noted that for ordinary technicians in this technical field, without departing from the principles of this application, various improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A method for initializing a disk array, characterized in that: Applied to the target storage node, including: Receive bitmap information sent by a configuration node, wherein the bitmap information is used to indicate location information of a target particle assigned to the target storage node in the disk array and initialization status information of the target particle, the target storage node is located in a storage cluster, the storage cluster includes the configuration node and multiple storage nodes, the multiple storage nodes include the target storage node, the configuration node is used to divide the disk array into multiple particles according to a preset particle size, and the multiple particles are correspondingly assigned to multiple storage nodes, and the storage node among the multiple storage nodes that performs an initialization task on the disk array is used to perform initialization operations on the assigned particles in parallel; performing an initialization operation on the target particle based on the position information of the target particle and the initialization state information of the target particle; The performing of the initialization operation on the target particle based on the position information of the target particle and the initialization state information of the target particle includes: determining a first particle from the target particles according to the initialization state information, wherein a particle value of the first particle is a first value; determining a first ranking of the first particles based on the position information; The initialization operation is performed on each of the first particles in sequence based on the first ranking.

2. The disk array initialization method according to claim 1, wherein: Performing the initialization operation on each of the first particles in sequence based on the first sorting includes: Obtaining the target resource utilization of the target storage node at the current moment; Determining a target rate corresponding to the target resource utilization from a target mapping relationship, wherein the target mapping relationship indicates a relationship between resource utilization and a rate at which a storage node performs an initialization operation, and the target rate is a rate at which the target storage node performs the initialization operation on the first particle within a preset time period; The initialization operation is sequentially performed on each of the first particles based on the target rate.

3. The disk array initialization method according to claim 2, characterized in that: The initialization operation is performed on each of the first particles in sequence based on the target rate, comprising: The following operations are performed on each particle, wherein the particle currently being initialized is the current particle: Dividing the current particle according to a preset stripe width to obtain a plurality of target stripes; determining a second ranking of each of the plurality of target bands in the current particle; performing an initialization operation on the plurality of target stripes based on the second sorting; When it is determined that the initialization of the current particle is completed, initialization completion information is reported to the configuration node, so that the configuration node updates the particle value of the current particle to a second value based on the initialization completion information.

4. The disk array initialization method according to claim 3, characterized in that: Determining that the current particle initialization is complete includes: generating a stripe bitmap based on the plurality of target stripes, wherein the stripe bitmap is used to indicate initialization state information of the target stripes; When the stripe bitmap shows that the stripe values ​​of the plurality of target stripes are all the third value, it is determined that the initialization of the current granule is completed.

5. The disk array initialization method according to claim 2, wherein: After determining the target rate corresponding to the target resource utilization from the target mapping, the method further includes: Determining an expected completion time of the target storage node performing the initialization operation on the first particle; sending the expected completion time to the configuration node; After performing an initialization operation on each of the first particles in sequence based on the target rate, the method further includes: After the preset time period ends and when it is determined that the target particle has not completed the initialization operation, the target rate is updated to perform the initialization operation on the second particle based on the updated target rate, wherein the second particle is a particle whose particle value after the preset time period ends is the first value, and the target particle includes the second particle.

6. The disk array initialization method according to claim 1, wherein: The method further comprises: Sending the bitmap information to a first storage node, so that the first storage node performs an initialization operation on the target particle based on the location information of the target particle and the initialization state information of the target particle when the target storage node fails; The first storage node is a backup node of the target storage node, and the configuration node creates a mirror pair based on the storage nodes included in the storage cluster before sending the bitmap information to the target storage node, wherein the mirror pair includes a primary node and a backup node.

7. A storage cluster, characterized in that: A storage cluster consists of a configuration node and multiple storage nodes, including: The configuration node is configured to divide the disk array into a plurality of granules according to a preset granularity, and to assign the plurality of granules to each of the plurality of storage nodes; determine bitmap information based on the granules assigned to each of the storage nodes, wherein the bitmap information is used to indicate position information of the granules assigned to each of the storage nodes in the disk array and an initialization status of the granules assigned to each of the storage nodes; The plurality of storage nodes are each configured to receive the bitmap information sent by the configuration node; wherein the storage node that performs the initialization task for the disk array among the plurality of storage nodes is further configured to concurrently perform initialization operations on the particles assigned to the storage node based on the particle position information and the particle initialization status information indicated by the bitmap information; The storage node is configured to perform the initialization operation on the particles assigned to the storage node in the following manner: determining a first particle from the assigned particles according to the initialization state information, wherein a particle value of the first particle is a first value; determining a first ranking of each particle in the first particles based on the position information; and performing the initialization operation on each particle in the first particles in sequence based on the first ranking.

8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the disk array initialization method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the disk array initialization method according to any one of claims 1 to 6 are implemented.

Images