Hard disk data writing method and device, electronic equipment and storage medium

Abstract

The invention relates to a hard disk data writing method and device, electronic equipment and a storage medium. The method comprises the following steps: if a data writing request is received, selecting a target object from a preset object pool; according to the target object, executing the following inspection steps: determining a target storage disk corresponding to the target object; according to the preset full-write threshold value, the preset adjustment threshold value, the used capacity and the total capacity of each target storage disk, determining the residual space of each target storage disk; if the residual space is smaller than a first preset space threshold value, obtaining the discarded number of the target objects; and if the discarding number is greater than or equal to a first preset number threshold, writing the target data indicated by the data writing request into each target storage disk. By applying the method and the device, the situation that the storage disk cannot be selected for a long time to affect foreground IO can be avoided, so that IO continuity is improved.

Description

Technical Field

[0001] This application relates to the field of computer technologies, and in particular, to a method, apparatus, electronic device, and storage medium for writing hard disk data. Background Art

[0002] To better utilize the writing performance of a hard disk, a common practice in the industry is to redirect (Redirect-On-Write, ROW) the data written to the hard disk pool objects together and aggregate them into large I / O (input / output), and then sequentially write them to the objects allocated by the hard disk pool. When writing to different storage disks in the hard disk pool, it will be hashed to different storage disks through a layout algorithm. However, due to the limitations of the layout algorithm or the imbalance of the I / O size, over time, the capacity ratios of the individual storage disks will eventually become unbalanced. As Figure 1 shown, the cluster includes 3 nodes, namely node1, node2, and node3. Each node includes 4 storage disks, where the storage disks corresponding to node1 are tgt0 to tgt3, the storage disks corresponding to node2 are tgt4 to tgt7, and the storage disks corresponding to node3 are tgt8 to tgt11. The object pool includes 3 objects, namely obj0, obj1, and obj2. The redundant copies of each object are hashed to different storage disks through a layout algorithm. In Figure 1 it, the redundant copies of obj0 are stored on tgt0, tgt4, and tgt8, the redundant copies of obj1 are stored on tgt1, tgt5, and tgt9, and the redundant copies of obj2 are stored on tgt2, tgt5, and tgt9. If the I / O of each object is the same, that is, the writing amounts of each object are the same, then the writing amounts of the two storage disks tgt5 and tgt9 corresponding to the two objects obj1 and obj2 at the same time will be higher than those of other storage disks. Over time, this will lead to an imbalance in the usage of each storage disk. When the number of storage disks with full capacity exceeds a certain proportion, the I / O will be interrupted because the layout algorithm cannot select a writable storage disk. Summary of the Invention

[0003] Embodiments of the present disclosure provide a method, apparatus, electronic device, and storage medium for writing hard disk data.

[0004] In a first aspect, an embodiment of the present disclosure provides a hard disk data writing method, including: in response to receiving a data writing request, selecting a target object from a preset object pool; according to the target object, performing the following checking steps: determining at least one target storage disk corresponding to the target object; obtaining the used capacity and total capacity of each target storage disk; according to the preset full write threshold, preset adjustment threshold, used capacity and total capacity of each target storage disk, determining the remaining space of each target storage disk; in response to determining that the remaining space is less than a first preset space threshold, obtaining the number of discarded target objects selected for the data writing request; in response to determining that the number of discarded objects is greater than or equal to a first preset quantity threshold, writing the target data indicated by the data writing request into at least one target storage disk.

[0005] In a second aspect, an embodiment of the present disclosure provides a hard disk data writing device, including: an object determination unit configured to determine a target storage disk from a hard disk in response to receiving a data writing request; a storage disk checking unit configured to perform checking steps through the following modules according to the target storage disk: a storage disk determination module configured to determine at least one target storage disk corresponding to the target object; a capacity obtaining module configured to obtain the used capacity and total capacity of each target storage disk; a space determination module configured to determine the remaining space of each target storage disk according to the preset full write threshold, preset adjustment threshold, used capacity and total capacity of each target storage disk; a number obtaining module configured to obtain the number of discarded target objects selected for the data writing request in response to determining that the remaining space is less than a first preset space threshold; a data writing module configured to write the target data indicated by the data writing request into at least one target storage disk in response to determining that the number of discarded objects is greater than or equal to a first preset quantity threshold.

[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory, a processor, a bus, and a computer program stored on the memory and executable on the processor, where the processor implements the hard disk data writing method described in the first aspect when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, on which a computer program is stored, and the computer program implements the hard disk data writing method described in the first aspect when executed by a processor.

[0008] By applying the technical solution of the present disclosure, when a data writing request is received, the target object can be first determined from the object pool, and then it is determined whether at least one target storage disk corresponding to the target object is full. If it is full, the discard quantity of the target object is obtained. When the discard quantity is greater than or equal to the first preset threshold, the target data in the data writing request is written into each target storage disk corresponding to the target object. Thus, when the discard quantity of the target object is excessive, the storage disk corresponding to the target object can be forcibly selected for writing, avoiding the situation that no storage disk can be selected for a long time and affecting the foreground IO, thereby improving the continuity of IO.

[0009] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0011] Figure 1 is a schematic diagram of an existing cluster topology structure;

[0012] Figure 2 is a schematic flowchart of an embodiment of the hard disk data writing method of the present disclosure;

[0013] Figure 3 is a schematic flowchart of another embodiment of the hard disk data writing method of the present disclosure;

[0014] Figure 4 is a schematic structural diagram of an embodiment of the hard disk data writing device of the present disclosure;

[0015] Figure 5 is a schematic structural diagram of an embodiment of the electronic device of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] It should be noted that the following detailed description is exemplary and is intended to provide further description of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present disclosure belongs.

[0017] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0018] In the case of no conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.

[0019] In order to make the technical solutions and advantages of the present disclosure clearer and more understandable, the following further describes the present disclosure in detail with reference to the accompanying drawings and specific embodiments.

[0020] Figure 2 The flowchart 200 of an embodiment of the hard disk data writing method of the present disclosure is shown. As Figure 2 shown, the hard disk data writing method of this embodiment may include the following steps:

[0021] Step 201, in response to receiving a data writing request, select a target object from a preset object pool..

[0022] In this embodiment, a data writing request may be received. The data writing request may be triggered by a user operation or triggered after a preset condition is met. The data writing request may include information such as target data, the identifier of the sender, and the data volume size. After receiving the data writing request, the target object may be determined from the object pool. Here, the hard disk may include multiple objects, and each object forms an object pool. Each object may correspond to multiple storage disks, and the capacity of each storage disk may be the same or different. When selecting the target object, the object corresponding to the identifier of the sender may be selected from the object pool according to the identifier of the sender. Alternatively, according to the type of target data indicated by the data writing request, the object storing the above type may be selected from the object pool.

[0023] Step 202, according to the target object, perform the following checking steps:

[0024] After determining the target object, the checking steps 2021 to 2024 may be performed:

[0025] Step 2021, determine at least one target storage disk corresponding to the target object.

[0026] After determining the target object, at least one target storage disk corresponding to the target object may be further determined. Specifically, according to the pre-established correspondence between the object and the storage disk, the target storage disk corresponding to the target object may be determined. The above correspondence may be established according to the redundancy type of the target storage disk through an existing algorithm.

[0027] Step 2022, obtain the used capacity and total capacity of each target storage disk.

[0028] After determining the target storage disks corresponding to the target object, the used capacity of the target storage disks can be further obtained. The used capacity refers to the capacity that has been used in the target storage disk. The total capacity refers to the capacity of all data that can be written into the target storage disk. These information can be obtained by querying the attribute information of the target storage disk through a specified instruction.

[0029] Step 2023, according to the preset full - write threshold, preset adjustment threshold, used capacity, and total capacity of each target storage disk, determine the remaining space of each target storage disk.

[0030] The preset full - write threshold and preset adjustment threshold of the target storage disk can also be obtained. Generally speaking, each storage disk will pre - set a reserved space. The available space of the storage disk is the total capacity of the storage disk minus the reserved space. The maximum value of the available space here is the preset full - write threshold. The preset adjustment threshold refers to the threshold for checking the storage disk when the storage disk is about to be full, that is, when the amount of data written is about to reach the preset full - write threshold. Specifically, if the reserved space of the storage disk is 20% and the available space is 90%, and the adjustment threshold is 3%, that is, a check is made when (available space - adjustment threshold) 90% - 3% = 87%. It should be noted that the preset full - write threshold and preset adjustment threshold can be specific capacity values or percentage values.

[0031] In this embodiment, after determining the used capacity of the target storage disk, the available space of the target storage disk can be determined by combining the preset full - write threshold, preset adjustment threshold, and total capacity. Specifically, it can be determined through the following calculation formula: available capacity = total capacity - preset full - write threshold - preset adjustment threshold.

[0032] Step 2024, in response to determining that the remaining space is less than the first preset space threshold, obtain the discard count of the target object selected for the data write request.

[0033] If the remaining space of the target storage disk is less than the first preset space threshold, it means that the capacity of the target storage disk is insufficient. In this case, the discard count of the target object selected for the data write request can be further obtained. In this embodiment, to ensure the integrity of data writing, as long as one of the target storage disks corresponding to the target object has insufficient capacity, the target object will be discarded. To further ensure that due to the excessive number of storage disks with insufficient capacity, it takes a long time to select a suitable storage disk, affecting the continuity of data writing, the discard count of the target object selected for the data write request can be obtained before selecting the storage disk for writing.

[0034] Step 2025, in response to determining that the discard count is greater than or equal to the first preset quantity threshold, write the target data indicated by the data write request into the above - mentioned at least one target storage disk.

[0035] If the discard count of the target object selected for the data write request is greater than or equal to the first preset quantity threshold, it indicates that a relatively large number of target objects are discarded. To ensure that the target data can be written to the storage disk in a timely manner, the target data in the data write request can be written to the respective target storage disks corresponding to the target object.

[0036] The hard disk data writing method provided by the above embodiments of the present disclosure first determines a target object from the object pool, and then determines whether at least one target storage disk corresponding to the target object is full. If it is full, the discard quantity of the target object is obtained. When the discard quantity is greater than or equal to the first preset threshold, the target data in the data write request is written to the respective target storage disks corresponding to the target object. Thereby, when the discard quantity of the target object is too large, the storage disk corresponding to the target object can be forcibly selected for writing, avoiding the situation that no storage disk can be selected for a long time and affecting the foreground IO, thereby improving the continuity of IO.

[0037] Continue to refer to Figure 3 , which shows the flow 300 of another embodiment of the hard disk data writing method according to the present disclosure. As Figure 3 shown, the method in this embodiment may include the following steps:

[0038] Step 301, in response to receiving a data write request, select a target object from a preset object pool.

[0039] Step 302, according to the target storage disk, perform the following inspection steps: determine at least one target storage disk corresponding to the target object; obtain the used capacity and total capacity of each target storage disk; determine the remaining space of each target storage disk according to the preset full write threshold, preset adjustment threshold, used capacity and total capacity of each target storage disk; in response to determining that the remaining space is less than the first preset space threshold, obtain the discard count of the target object selected for the data write request; in response to determining that the discard count is greater than or equal to the first preset quantity threshold, write the target data indicated by the data write request to the at least one target storage disk.

[0040] In some optional implementation manners of this embodiment, if it is determined that the remaining space of the target storage disk is less than the first preset space threshold, a preset mark can be set for the target object, indicating that the storage disk corresponding to the target object is about to be full. In this way, when selecting a target object from the object pool again later, the object with the preset mark set will not be selected.

[0041] In some alternative implementation manners of this embodiment, the preset full write threshold and the preset adjustment threshold exist in the form of percentages. When calculating the remaining space, the ratio of the used space to the total capacity can be calculated first. Then, according to the following formula: remaining space = 100% - preset full write threshold - preset adjustment threshold - used capacity ratio. Then, compare the remaining space with the first preset capacity threshold.

[0042] Step 303, in response to determining that the discard count is less than the first preset quantity threshold, discard the target object, and select other objects from the object pool as the new target object, and continue to execute the inspection step.

[0043] In this embodiment, if the discard count of the target object selected for the data write request is less than the first preset quantity threshold, in this case, when the capacities of the target storage disks corresponding to the target object are insufficient, the target object can be discarded. Further, select other objects from the object pool as the new target object, and continue to execute step 302 until the available capacities of the storage disks corresponding to the new target object are greater than the first preset space threshold, or the discard count of the target object is greater than or equal to the first preset quantity threshold and then stop.

[0044] Step 304, determine the overall remaining space of the hard disk; in response to determining that the overall remaining space is less than the second preset space threshold, stop the inspection step.

[0045] In this embodiment, the overall remaining space of the hard disk can also be obtained in real time or at intervals of a preset duration. Here, the attribute information of the hard disk can be read through a preset instruction, so as to obtain the overall remaining space of the hard disk. If the overall remaining space of the hard disk is less than the second preset space threshold, it means that the spaces of most of the storage disks in the hard disk are about to be full. In this case, it is not necessary to execute the inspection step 302 anymore, so as to avoid the situation that the target data in the data write request cannot be written.

[0046] Step 305, determine the number of storage disks with remaining space less than the first preset space threshold; in response to determining that the above number is greater than or equal to the second preset quantity threshold, stop the inspection step.

[0047] In this embodiment, the number of storage disks with remaining space less than the first preset space threshold can also be determined in real time or at intervals of a preset duration. Here, the remaining spaces of the storage disks in the hard disk can be counted. If the above number is greater than or equal to the second preset quantity threshold, it means that the spaces of most of the storage disks in the hard disk are about to be full. In this case, it is not necessary to execute the inspection step 302 anymore, so as to avoid the situation that the target data in the data write request cannot be written.

[0048] Step 306, after stopping the inspection step, directly write the target data to the storage disks corresponding to the target object.

[0049] After stopping the checking step 302, the target data can be directly written to the target storage disk.

[0050] The hard disk data writing method provided by the above embodiments of the present disclosure, by setting the adjustment threshold of the storage disk, discards in advance the objects that are about to be full, that is, checks the space of each storage disk in advance, selects a suitable storage disk for writing, ensures the balance of the usage of each storage disk, prevents the early interruption of IO caused by the excessive imbalance of the storage disk capacity, and at the same time adds an anti-fooling mechanism in the process of selecting objects, that is, when the number of discarded objects is too large, the storage disk corresponding to the target object is forcibly selected for writing, avoiding the risk of foreground IO interruption caused by the storage disk corresponding to some objects being full, and improving the continuity of IO.

[0051] Further referring to Figure 4 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a hard disk data writing device, and this device embodiment corresponds to Figure 2 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0052] As Figure 4 shown, the hard disk data writing device 400 of this embodiment includes: an object determination unit 401 and a storage disk checking unit 402. Among them, the storage disk checking unit 402 further includes: a storage disk determination module 4021, a capacity acquisition module 4022, a space determination module 4023, a number acquisition module 4024, and a data writing module 4025.

[0053] A storage disk determination unit, configured to determine at least one target storage disk corresponding to the target object;

[0054] The storage disk determination unit 401 is configured to determine a target storage disk from the hard disk in response to receiving a data writing request.

[0055] The storage disk checking unit 402 is configured to execute a checking step through the following modules according to the target storage disk:

[0056] The storage disk determination module 4021 is configured to determine at least one target storage disk corresponding to the target object.

[0057] The capacity acquisition module 4022 is configured to acquire the used capacity and the total capacity of each target storage disk.

[0058] The space determination module 4023 is configured to determine the remaining space of each target storage disk according to the preset full write threshold, preset adjustment threshold, used capacity, and total capacity of each target storage disk.

[0059] A quantity acquisition module 4024, configured to acquire the discard quantity of the target object selected for the data writing request in response to determining that the remaining space is less than the first preset space threshold.

[0060] A data writing module 4025, configured to write the target data indicated by the data writing request to at least one target storage disk in response to determining that the discard quantity is greater than or equal to the first preset quantity threshold.

[0061] In addition, in the technical solution of this application, an electronic device is also proposed.

[0062] Figure 5 The structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0063] As Figure 5 shown, the electronic device may include a processor 501, a memory 502, a bus 503, and a computer program stored on the memory 502 and executable on the processor 501. Among them, the processor 501 and the memory 502 complete mutual communication through the bus 503. When the processor 501 executes the computer program, the steps of the above method are implemented, for example, including: in response to receiving a data writing request, selecting a target object from a preset object pool; according to the target object, performing the following inspection steps: determining at least one target storage disk corresponding to the target object; acquiring the used capacity and total capacity of each target storage disk; determining the remaining space of each target storage disk according to the preset full write threshold, preset adjustment threshold, used capacity, and total capacity of each target storage disk; in response to determining that the remaining space is less than the first preset space threshold, acquiring the discard quantity of the target object selected for the data writing request; in response to determining that the discard quantity is greater than or equal to the first preset quantity threshold, writing the target data indicated by the data writing request to at least one target storage disk.

[0064] In addition, in an embodiment of the present disclosure, a non-transitory computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented, for example, including: in response to receiving a data writing request, selecting a target object from a preset object pool; according to the target object, performing the following inspection steps: determining at least one target storage disk corresponding to the target object; acquiring the used capacity and total capacity of each target storage disk; determining the remaining space of each target storage disk according to the preset full write threshold, preset adjustment threshold, used capacity, and total capacity of each target storage disk; in response to determining that the remaining space is less than the first preset space threshold, acquiring the discard quantity of the target object selected for the data writing request; in response to determining that the discard quantity is greater than or equal to the first preset quantity threshold, writing the target data indicated by the data writing request to at least one target storage disk.

[0065] In summary, in the technical solution of the present disclosure, when a data writing request is received, the target object can be first determined from the object pool, and then it is determined whether at least one target storage disk corresponding to the target object is full. If it is full, the discard quantity of the target object is obtained. When the discard quantity is greater than or equal to the first preset threshold, the target data in the data writing request is written into each target storage disk corresponding to the target object. Therefore, when the discard quantity of the target object is excessive, the storage disk corresponding to the target object can be forcibly selected for writing, avoiding the inability to select a storage disk for a long time and affecting the foreground IO, thereby improving the continuity of the IO.

[0066] The foregoing are only the preferred embodiments of the present disclosure and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A hard disk data writing method, comprising: In response to receiving a data writing request, selecting a target object from a preset object pool; According to the target object, performing the following checking steps: determining at least one target storage disk corresponding to the target object; obtaining the used capacity and total capacity of each target storage disk; According to the preset full - write threshold, preset adjustment threshold, the used capacity, and the total capacity of each target storage disk, determining the remaining space of each target storage disk; in response to determining that the remaining space is less than a first preset space threshold, obtaining the discard number of the target object selected for the data writing request; In response to determining that the discard number is greater than or equal to a first preset quantity threshold, writing the target data indicated by the data writing request into the at least one target storage disk.

2. The method according to claim 1, wherein, The method further comprises: In response to determining that the discard number is less than the first preset quantity threshold, discarding the target object, and selecting other objects from the object pool as new target objects, and continuing to perform the checking steps.

3. The method according to claim 2, wherein The method further comprises: In response to determining that the remaining space is less than the first preset space threshold, setting a preset mark for the target object.

4. The method according to claim 3, wherein, The method further comprises: Determining the overall remaining space of the hard disk; In response to determining that the overall remaining space is less than a second preset space threshold, stopping the checking steps.

5. The method according to claim 3, wherein, The method further comprises: Determining the number of storage disks with remaining space less than the first preset space threshold; In response to determining that the number is greater than or equal to a second preset quantity threshold, stopping the checking steps.

6. The method according to claim 4 or 5, wherein The method further comprises: After stopping the checking steps, directly writing the target data into each storage disk corresponding to the target object.

7. The method according to claim 1, wherein The method further comprises: Determining the redundancy type of each object in the object pool; For each object, according to the redundancy type of the object, allocating at least one corresponding storage disk for the object.

8. A hard disk data writing device, comprising: An object determination unit configured to, in response to receiving a data writing request, determine a target storage disk from the hard disk; A storage disk checking unit configured to perform checking steps according to the target storage disk through the following modules: A storage disk determination module configured to determine at least one target storage disk corresponding to the target object; A capacity acquisition module configured to obtain the used capacity and total capacity of each target storage disk; A space determination module configured to determine the remaining space of each target storage disk according to the preset full - write threshold, preset adjustment threshold, the used capacity, and the total capacity of each target storage disk; A number acquisition module configured to, in response to determining that the remaining space is less than a first preset space threshold, obtain the discard number of the target object selected for the data writing request; A data writing module configured to, in response to determining that the discard number is greater than or equal to a first preset quantity threshold, write the target data indicated by the data writing request into the at least one target storage disk.

9. An electronic device, comprising a memory, a processor, a bus, and a computer program stored on the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the hard disk data writing method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the hard disk data writing method according to any one of claims 1 to 7.

Images