Disk management method and device, computer equipment, storage medium and computer program product
By comprehensively analyzing the disk's read and write requests and task execution queue status, accurately identifying and processing disk IO status, the problems of misjudgment and resource waste in the existing technology are solved, the accuracy and stability of disk management are improved, and the read and write are ensured. The normal operation of the business.
Patent Information
- Application Number
- CN202411327821.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2025-01-14
AI Technical Summary
The existing disk management method determines slow disk based on abnormal parameters of the disk itself, which is easy to misjudgment, resulting in waste of resources and impact on read and write services, and lacks accurate disk status recognition and management.
By comprehensively considering the execution results of read and write requests of the target disk and the task execution queue status, accurately identify the IO status, and report the IO status when necessary, avoid the impact of batch reporting on read and write services, and promptly identify and handle the phenomenon of excessive disk IO in disks. .
It improves the accuracy and resource utilization and stability of disk management, avoids delays and resource waste in read and write services, and ensures the normal operation of the disk and read and write performance.
Smart Images
Figure CN119311450A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a disk management method, apparatus, computer equipment, storage medium and computer program product. Background Art
[0002] A hard disk drive (HDD) is a traditional data storage device that uses a rotating disk (platter) and a moving read / write head to store and retrieve data. In daily use of a hard disk, there may be a probabilistic delay in reading and writing data, and the delay often has a negative impact on normal read and write operations.
[0003] In the related art, during the disk management process, the abnormality of the disk's own parameter values can be used to determine whether the disk is a slow disk, and then the slow disk can be processed according to the predetermined slow disk processing strategy, for example: migrating the data of the slow disk to the solid-state hard disk, or moving infrequently used data to the slow disk to free up space on the fast disk, etc. However, the method of managing the slow disk by determining the abnormality of the disk's own parameter values does not mean that the disk cannot be used when the disk's own parameter values are abnormal. Therefore, if the disk is determined to be a slow disk based solely on its own parameter abnormality, many disks in the environment may be mistakenly identified as slow disks. Such a disk management method will affect normal read and write services and cause resource waste.
[0004] Therefore, how to accurately obtain the disk status to achieve precise disk management, thereby alleviating the impact of slow disks on read and write services and improving resource utilization is currently a focus of attention in the industry. Summary of the invention
[0005] Based on this, it is necessary to provide a disk management method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the recognition accuracy of disk status in response to the above technical problems.
[0006] In a first aspect, the present application provides a disk management method, the method comprising:
[0007] Get the request result from the target disk;
[0008] The IO status of the target disk is reported according to the obtained request result, the disk reporting status and the task execution queue status of the target disk.
[0009] By adopting the disk management method provided in the embodiment of the present application, the IO status of the target disk can be comprehensively determined based on the execution result of the target disk for the read and write request, the current disk reporting status of each disk, and the task execution queue of the target disk, and the corresponding IO status report of the target disk can be made. Since the IO status of the target disk is determined based on the execution result of the target disk for the read and write request and the task execution queue status of the target disk, the accuracy of the disk IO status report can be guaranteed. When reporting the disk IO status, the IO status reporting of other disks is also fully considered through the disk reporting status, which can avoid the impact of batch IO status reporting on the read and write business, and can improve the stability of the read and write business.
[0010] In one embodiment, reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk includes:
[0011] When the obtained request result is empty and the disk reporting status and the task execution queue status of the target disk meet the reporting conditions, the disk IO status abnormality is reported for the target disk.
[0012] Among them, the reporting conditions include: the disk reporting status of the target disk is that no disk IO status abnormality reporting is performed, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there are no other disks that report disk IO status abnormality during the monitoring period.
[0013] By adopting the disk management method provided by the embodiment of the present disclosure, when the disk reporting status of the target disk is that no disk IO status abnormality reporting is performed, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there are no other disks reporting IO status abnormality within the monitoring period, the disk IO status abnormality reporting can be performed, which can ensure accurate identification of disk IO abnormalities, timely report disk IO status abnormalities, and avoid the impact of batch disk IO status abnormality reporting on read and write tasks, thereby ensuring the smooth execution of read and write tasks.
[0014] In one embodiment, reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk includes:
[0015] When the obtained request result is not empty, if the existence duration of the first request task in the current task execution queue does not exceed the duration threshold, and the target disk has performed a disk IO status abnormality report, then the disk IO status recovery report is performed on the target disk.
[0016] By adopting the disk management method provided in the embodiment of the present application, the disk IO status recovery report of the target disk can be performed in time, so that the target disk can continue to participate in subsequent read and write tasks, and the target disk can be restored to normal in time, thereby improving resource utilization.
[0017] In one embodiment, reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk includes:
[0018] When the obtained request result is not empty, if the existence time of the first request task in the target task execution queue exceeds the time threshold, the disk IO status abnormality is reported for the target disk.
[0019] In an embodiment of the present application, when the disk completes the request task, the request task can also be identified as abnormal based on the length of time the request task has existed in the disk, and then the IO status abnormality of the target disk can be reported in time when an abnormality exists in the request task, thereby improving the accuracy of abnormality identification of the target disk.
[0020] In one embodiment, the reporting of abnormal disk IO status of the target disk includes:
[0021] Abnormal status information is reported to the management layer for the target disk, where the abnormal status information indicates that the target disk has an excessively long IO phenomenon and no longer performs data storage.
[0022] That is, the disk management method provided in the embodiment of the present application can timely monitor the disk's excessive IO phenomenon and report it, so that the target disk can be removed in time to avoid the impact of the excessive IO phenomenon of the target disk on the read and write performance and business.
[0023] In a second aspect, the present application provides a disk management method, the method comprising:
[0024] Receive reporting information for the target disk;
[0025] When the reported information is abnormal status information indicating that the target disk has an excessively long IO phenomenon, the data in the target disk is transferred to other disks, and no read or write request is delivered to the target disk.
[0026] The disk management method provided in the embodiment of the present application can enable the target disk to report abnormal disk IO status when the disk has excessively long IO, so that the management level can promptly remove the target disk where the disk has excessively long IO, thereby avoiding its impact on read and write performance and storage and access services, and preventing IO service flow interruption.
[0027] In one embodiment, the method further comprises:
[0028] When the reported information is recovery status information representing recovery of excessive disk IO of the target disk, determining a target read / write request, the target read / write request being a read / write request that triggers the target disk to report abnormal status information;
[0029] The area accessed by the target read / write request in the target disk is marked as an abnormal area, and the data in the abnormal area is transferred to other areas or other disks.
[0030] With the above disk management method, the target disk can report abnormal disk IO status when there is an excessively long disk IO phenomenon, so that the management can promptly remove the target disk where the excessively long disk IO phenomenon occurs. At the same time, after the disk IO status is restored, the disk IO status can also be restored, and the target disk can be restored in time, and its slow area can be marked, so that the slow area that affects the read and write performance is removed from the target disk, avoiding its impact on the read and write performance and on storage and access services, preventing the interruption of IO service flow, and improving resource utilization.
[0031] In a third aspect, the present application further provides a disk management device, the device comprising:
[0032] The acquisition module is used to obtain the request result from the target disk;
[0033] The reporting module is used to report the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk.
[0034] In one embodiment, the reporting module is further used to:
[0035] When the obtained request result is empty and the disk reporting status and the task execution queue status of the target disk meet the reporting conditions, the disk IO status abnormality is reported for the target disk.
[0036] Among them, the reporting conditions include: the disk reporting status of the target disk is that no disk IO status abnormality reporting is performed, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there are no other disks that report disk IO status abnormality during the monitoring period.
[0037] In one embodiment, the reporting module is further used for:
[0038] When the obtained request result is not empty, if the existence duration of the first request task in the current task execution queue does not exceed the duration threshold, and the target disk has performed a disk IO status abnormality report, then the disk IO status recovery report is performed on the target disk.
[0039] In one embodiment, the reporting module is further used for:
[0040] When the obtained request result is not empty, if the existence time of the first request task in the target task execution queue exceeds the time threshold, the disk IO status abnormality is reported for the target disk.
[0041] In one embodiment, the reporting module is further used for:
[0042] Abnormal status information is reported to the management layer for the target disk, where the abnormal status information indicates that the target disk has an excessively long IO phenomenon and no longer performs data storage.
[0043] In a fourth aspect, the present application further provides a disk management device, the device comprising:
[0044] A receiving module, used for receiving reporting information for a target disk;
[0045] The processing module is used to transfer the data in the target disk to other disks and no longer deliver read and write requests to the target disk when the reported information is abnormal status information indicating that the target disk has an excessively long IO phenomenon.
[0046] In one embodiment, the device further comprises:
[0047] A determination module, configured to determine a target read / write request when the reported information is recovery status information characterizing recovery of excessive disk IO of the target disk, wherein the target read / write request is a read / write request that triggers the target disk to report abnormal status information;
[0048] The marking module is used to mark the area accessed by the target read / write request in the target disk as an abnormal area, and transfer the data in the abnormal area to other areas or other disks.
[0049] In a fifth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements any of the above disk management methods when executing the computer program.
[0050] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above disk management methods when executed by a processor.
[0051] In a seventh aspect, the present application also provides a computer program product, including a computer program, which implements any of the above disk management methods when executed by a processor.
[0052] The above-mentioned disk management method, device, computer equipment, storage medium and computer program product can obtain the request result from the target disk, and report the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk. The disk management method provided by the embodiment of the present application can comprehensively determine the IO status of the target disk based on the execution result of the target disk for the read and write request, the disk reporting status of each disk at present and the task execution queue of the target disk, and report the corresponding IO status of the target disk. Since the IO status of the target disk is determined based on the execution result of the target disk for the read and write request and the task execution queue status of the target disk, the accuracy of the disk IO status reporting can be guaranteed, thereby improving the accuracy of disk management and improving resource utilization. When reporting the disk IO status, the IO status reporting of other disks is fully considered in combination with the disk reporting status, which can avoid the impact of batch IO status reporting on the read and write business, and improve the stability of the read and write business. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0054] FIG1 is a schematic diagram of a flow chart of a disk management method in one embodiment;
[0055] FIG2 is a schematic diagram of a disk management method in one embodiment;
[0056] FIG3 is a schematic diagram of a flow chart of a disk management method in another embodiment;
[0057] FIG4 is a schematic diagram of a flow chart of a disk management method in one embodiment;
[0058] FIG5 is a schematic diagram of a disk management method in one embodiment;
[0059] FIG6 is a block diagram of a disk management device in one embodiment;
[0060] FIG7 is a block diagram of a disk management device in one embodiment;
[0061] FIG. 8 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0063] In one embodiment, as shown in FIG1 , a disk management method is provided. This embodiment is described by taking the method applied to a terminal as an example. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps 102 to 104, wherein:
[0064] Step 102, obtaining the request result from the target disk.
[0065] In the embodiment of the present application, the target disk is read and written in asynchronous mode. Each disk has its own task request queue, and the first request to enter is processed first. Each disk has a corresponding thread to control the delivery of read and write IO (Input / Output) requests and the retrieval of results, which is completed by loop execution and checking, as shown in Figure 2.
[0066] For the target disk, when the task check for the target disk is executed in a loop, two basic operations need to be completed: IO request delivery and result retrieval, where IO requests include read requests and write requests. During the IO request delivery process, you can check whether there are requests to be delivered in the task request queue. If there are IO requests to be delivered, extract the IO request and deliver it to the target disk through io_submit, and record the IO request in the task execution queue of the target disk. During the result retrieval process, you can use io_getevents to obtain the completion status of the IO request in the task execution queue of the target disk, obtain the completed IO request and the request result of the IO request from the completed request, feedback the request result of the completed IO request to the requester through the callback interface, and delete the IO request from the task execution queue.
[0067] Step 104 , reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk.
[0068] In an embodiment of the present application, after obtaining the request result, the request result includes being empty or not empty, and the disk IO status can be determined based on the request result and the task execution queue status of the target disk. For example: when the request result is empty, and the task execution queue of the target disk is empty or there are no overdue and unprocessed request tasks, it can be determined that the target disk IO status is normal; or, when the request result is not empty, but there are overdue and unprocessed tasks in the task execution queue of the target disk, it can be determined that the target disk IO status is abnormal.
[0069] The disk reporting status may include the IO status reporting status of each disk, that is, including the IO status reporting status of the target disk and the IO status reporting status of other disks. When it is determined that the IO status of the target disk is abnormal, the IO status can be reported for the target disk. However, in order to avoid the situation where disks in the same stage report abnormal IO status in batches, it is possible to further determine whether there are other disks reporting abnormal IO status in the current time period in combination with the disk reporting status, or whether the target disk has reported abnormal disk IO status, and if no other disks report abnormal IO status in the current time period, or the target disk has never reported abnormal disk IO status, the target disk will be reported abnormal IO status.
[0070] The disk reporting status may be obtained by the target disk requesting the management layer when the obtained request result is empty; or the management layer may periodically send the disk reporting status to each disk. The method for obtaining the disk reporting status is not specifically limited in the embodiment of the present application.
[0071] The above disk management method can obtain the request result from the target disk, and report the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk. The disk management method provided by the embodiment of the present application can comprehensively determine the IO status of the target disk based on the execution result of the target disk for the read and write request, the disk reporting status of each disk at present and the task execution queue of the target disk, and report the corresponding IO status of the target disk. Since the IO status of the target disk is determined based on the execution result of the target disk for the read and write request and the task execution queue status of the target disk, the accuracy of the disk IO status reporting can be guaranteed, thereby improving the accuracy of disk management and improving resource utilization. When reporting the disk IO status, the IO status reporting of other disks is fully considered in combination with the disk reporting status, which can avoid the impact of batch IO status reporting on the read and write business, and improve the stability of the read and write business.
[0072] In an exemplary embodiment, in step 104, reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk may include the following steps:
[0073] When the obtained request result is empty, and the disk reporting status and the task execution queue status of the target disk meet the reporting conditions, the disk IO status abnormality is reported for the target disk.
[0074] Among them, the reporting conditions include: the disk reporting status of the target disk is that no disk IO status abnormality reporting is performed, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there is no other disk IO status reporting during the monitoring period.
[0075] In the embodiment of the present application, if the request result obtained for the target disk is empty, it indicates that the target disk may currently have a long delay in read and write requests due to an abnormal IO state, so the disk IO state can be further judged at this time. When the task execution queue of the target disk is empty, it can be indicated that the target disk currently has no read and write IO requests being executed, so it can be determined that the disk IO state is normal at this time, and the disk IO state abnormality report can be abandoned.
[0076] Alternatively, if the task execution queue of the target disk is not empty, but the existence time of the first request task in the task execution queue does not exceed the time threshold, it indicates that the target disk is executing the request task in the normal task execution queue, so it can be determined that the disk IO status is normal at this time, and the disk IO status abnormality report is abandoned. The time threshold can be a preset time, and the specific value is set by technicians in this field according to needs, for example, it is set to 30s.
[0077] Alternatively, if the target disk's disk reporting status indicates that it has already reported abnormal disk IO status, in order to avoid repeating the disk IO status report, the abnormal disk IO status report is abandoned regardless of whether the disk IO status is abnormal or not. Alternatively, if other disks have reported abnormal IO status in the monitoring period, in order to avoid multiple disks reporting abnormal disk IO status in batches, which would affect the read and write services, the abnormal disk IO status report is abandoned regardless of whether the disk IO status is abnormal or not, wherein the monitoring period is a preset time period, which is not specifically limited in the embodiments of the present application, and is set to 120s, for example.
[0078] In one embodiment, when other disks have reported abnormal disk IO status during the monitoring period, if the request result obtained for the target disk is empty at this time, and the existence time of the first request task in the task execution queue of the target disk exceeds the duration threshold, it can be further determined whether the existence time of the first request task exceeds the reporting duration threshold. If it exceeds the reporting duration threshold, the disk IO status abnormality report can be performed. That is, when the existence time of the first request task is greater than the duration threshold and less than the reporting duration threshold, the disk IO status abnormality report is abandoned, but when the existence time of the first request task is greater than or equal to the reporting duration threshold, the disk IO status abnormality report can be performed. In this way, while avoiding the disks from reporting abnormal disk IO status in batches, the disks with serious problems can be identified and the disk IO status abnormality reports can be made in time, so as to avoid such disks from participating in subsequent read and write tasks and causing more serious impact on the read and write tasks.
[0079] On the contrary, when the obtained request result is empty, and the disk reporting status and the task execution queue status of the target disk meet at least one of the following conditions, it can be considered that the target disk IO status is abnormal at this time, and the target disk can be reported for abnormal disk IO status, including: the disk reporting status of the target disk is that the disk IO status is not reported abnormally, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there are no other disks reporting IO status during the monitoring period. That is, the target condition can be composed of at least one of the above conditions, which can be selected and constructed by those skilled in the art based on needs, and this is not specifically limited in the embodiments of the present application.
[0080] By adopting the disk management method provided by the embodiment of the present disclosure, when the disk reporting status of the target disk is that no disk IO status abnormality reporting is performed, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there are no other disks reporting IO status abnormality within the monitoring period, the disk IO status abnormality reporting can be performed, which can ensure accurate identification of disk IO abnormalities and timely reporting of disk IO status abnormalities, and can also avoid the impact of batch disk IO status abnormality reporting on read and write tasks, thereby ensuring the smooth execution of read and write tasks.
[0081] In an exemplary embodiment, in step 104, reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk may also include:
[0082] When the obtained request result is not empty, if the existence time of the first request task in the current task execution queue does not exceed the time threshold, and the target disk has performed a disk IO status abnormality report, then the disk IO status recovery report is performed on the target disk.
[0083] In an embodiment of the present application, when the request result obtained for the target disk is not empty, if the existence duration of the first request task in the current task execution queue does not exceed the duration threshold, that is, the current disk IO status is normal, it can be determined whether the target disk has performed a disk IO status abnormality report, and when it is determined that the target disk has performed a disk IO status abnormality report, it is determined that the target disk has had a disk IO status abnormality and has now returned to normal, so the disk IO status recovery report can be performed on the target disk, so that the target disk can continue to participate in subsequent read and write tasks, so that the target disk can be restored to normal in time, thereby improving resource utilization.
[0084] In an exemplary embodiment, in step 104, reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk may also include:
[0085] When the obtained request result is not empty, if the existence time of the first request task in the target task execution queue exceeds the time threshold, the disk IO status abnormality is reported for the target disk.
[0086] In the embodiment of the present application, when the obtained request result is not empty, that is, there is a request task that is executed and the corresponding request result is obtained, the disk may also have a request task stuck, that is, there is a request task in the task execution queue that has not been completed, so the existence time of the first request task in the task execution queue can be checked. The first request task is the request task with the longest existence time in the task execution queue. If the existence time of the first request task exceeds the duration threshold, it can be considered that the request task is stuck, that is, the disk IO status is abnormal, and the IO status abnormality of the target disk can also be reported at this time.
[0087] That is, in an embodiment of the present application, when the disk completes a request task, it can also identify the exception of the request task based on the length of time the request task has existed in the disk, and then report the IO status exception of the target disk in a timely manner when an exception exists in the request task, thereby improving the accuracy of identifying the exception of the target disk.
[0088] In an exemplary embodiment, reporting the abnormal disk IO status of the target disk may include:
[0089] The target disk reports abnormal status information to the management layer, where the abnormal status information indicates that the target disk has excessive disk IO and no longer stores data.
[0090] In the embodiment of the present application, when it is determined that the target disk has an abnormal disk IO state, the abnormal state information can be reported to the management layer for the target disk. The abnormal state information is used to indicate that the target disk has an excessively long disk IO phenomenon. Excessively long disk IO means that the operation time is far beyond the normal or expected time, for example: reading / writing once takes tens of seconds or even hundreds of seconds. Excessively long disk IO is a disk defect. If the occurrence rate of excessively long disk IO is high, it will have a great impact on storage and access services, and may cause reduced read and write performance, or even interruption of read and write.
[0091] Therefore, in the embodiment of the present application, abnormal status information is reported to the management layer regarding the excessively long disk IO, indicating to the management layer that the target disk has an excessively long disk IO phenomenon and can no longer continue to store data, so that the management layer can promptly remove the target disk based on the abnormal status information, so that the target disk will no longer perform data storage in the future and will no longer perform parameter read and write tasks, thereby avoiding the impact on read and write performance and business.
[0092] That is, the disk management method provided in the embodiment of the present application can timely monitor the disk's excessive IO phenomenon and report it, so that the target disk can be removed in time to avoid the impact of the excessive IO phenomenon of the target disk on the read and write performance and business.
[0093] In one embodiment, as shown in FIG3 , a disk management method is provided. This embodiment is illustrated by applying the method to a terminal. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:
[0094] Step 302, receiving reporting information for the target disk;
[0095] Step 304: when the reported information is abnormal status information indicating that the target disk has an excessively long IO phenomenon, the data in the target disk is transferred to other disks, and no read or write request is delivered to the target disk.
[0096] In the embodiment of the present application, when the target disk detects that the disk IO status is abnormal, it reports the disk IO status abnormality (the specific process can be referred to the relevant description of the aforementioned embodiment, and will not be repeated in the embodiment of the present application). Exemplarily, when the target disk reports the disk IO status abnormality, it can send abnormal status information to the management layer, and the abnormal status information can include the identifier of the target disk, and can also include the identifier of the request task and the disk area accessed by the request task.
[0097] Upon receiving abnormal status information reported by the target disk indicating that the target disk has an extremely long disk IO phenomenon, the data in the target disk can be transferred to other disks, and then the target disk can be eliminated and no read or write requests will be delivered to the target disk. In other words, the target disk no longer participates in data reading and writing, thereby avoiding the impact of the extremely long disk IO phenomenon on the read and write performance.
[0098] With the above disk management method, the target disk can report abnormal disk IO status when the disk has excessive IO, so that the management can promptly remove the target disk where the disk has excessive IO, avoid its impact on read and write performance and storage and access services, and prevent IO service flow interruption.
[0099] In an exemplary embodiment, as shown in FIG. 4 , the method may further include:
[0100] Step 402, when the reported information is recovery status information representing recovery of excessive disk IO of the target disk, determining a target read / write request, the target read / write request being a read / write request that triggers the target disk to report abnormal status information;
[0101] Step 404: Mark the area accessed by the target read / write request in the target disk as an abnormal area, and transfer the data in the abnormal area to other areas or other disks.
[0102] In the embodiment of the present application, when the target disk detects that the disk IO status has recovered from abnormal to normal, it can report the disk IO status recovery (the specific process can be referred to the relevant description of the aforementioned embodiment, and will not be repeated in the embodiment of the present application). Exemplarily, when the target disk reports the disk IO status recovery, it can send recovery status information to the management layer, and the recovery status information can include the identifier of the target disk.
[0103] The management layer can respond to the recovery status information, obtain the abnormal status information previously reported by the target disk, and obtain the read and write request that triggered the target disk to report the abnormal status information as the target read and write request from the abnormal status information, and obtain the disk area accessed by the target read and write request, mark the disk area accessed by the target read and write request in the target disk as an abnormal area (or also called a slow area), and transfer the data in the abnormal area to other areas or other disks. Subsequently, the abnormal area will no longer participate in read and write tasks, that is, it will no longer store data.
[0104] With the above disk management method, the target disk can report abnormal disk IO status when there is an excessively long disk IO phenomenon, so that the management can promptly remove the target disk where the excessively long disk IO phenomenon occurs. At the same time, after the disk IO status is restored, the disk IO status can also be restored, and the target disk can be restored in time, and its slow area can be marked, so that the slow area that affects the read and write performance is removed from the target disk, avoiding its impact on the read and write performance and on storage and access services, preventing the interruption of IO service flow, and improving resource utilization.
[0105] In order to enable those skilled in the art to better understand the embodiments of the present application, the embodiments of the present application are described below through specific examples.
[0106] As shown in Figure 5, in the product architecture, the read and write disks are in asynchronous mode. Each disk has a corresponding thread to control the delivery of IO requests and the retrieval of request results, which is completed by looping execution checks. Each disk has its own task request queue, and the first request to enter is processed first, as shown in Figure 2.
[0107] When the program executes disk task checks in a loop, two basic operations need to be completed: IO request delivery and request result retrieval, including read and write requests. Collect read and write requests in the task request queue, check whether there are requests to be delivered in the task request queue, extract them and deliver them to the disk through io_submit, and record the request in the task execution queue. Obtain the completion status of the disk request through io_getevents, obtain the completed disk request, and obtain whether there is an error at present. Notify the upper-level requester of the result through the request callback interface, and remove the request from the task execution queue.
[0108] In the embodiment of the present application, after the disk reports excessive disk IO, the disk can be kicked out or service avoided in the product architecture management module.
[0109] When obtaining the disk request completion status, if the result is that no request is completed this time, that is, the request result is empty, you can enter the disk IO status abnormal reporting process; if a request is completed, you can enter the disk IO status recovery reporting process and the single request disk IO status abnormal reporting process.
[0110] Among them, the abnormal disk IO status reporting process may include: judging if the disk has reported abnormal disk IO status, or the disk status is unhealthy, or there is no task in the task execution queue of the disk, then the reporting process is terminated; if the existence time of the first request task in the task execution queue does not exceed the set threshold (for example: 30 seconds), then the reporting process is terminated; if other disks have reported abnormal disk IO status within the monitoring period (for example: 120 seconds), then the reporting process is terminated to avoid excessive impact on the environment. Otherwise, the disk IO status is reported abnormally for the disk, that is, the disk is reported to have excessive disk IO.
[0111] The disk IO status recovery reporting process may include: when the request result obtained this time is not empty, that is, when the disk has a request task completed, if the disk IO status abnormality report has been performed before, and there is no other request stuck on the current disk, the disk IO status recovery report is performed, that is, the disk IO is reported to have recovered. And the area accessed by the request that triggered the disk to report the IO status abnormality before is marked as a slow area, and all data in the area is transferred to other areas of the disk or other disks, and this slow area will not be used later.
[0112] The disk single request disk IO status abnormality reporting process may include: in each round, when a request is completed, there may be a request that has not been completed. Therefore, when a request is completed in each round, if the existence time of the first request in the task execution queue exceeds the set threshold, the disk IO status abnormality is reported for the request.
[0113] The disk management method provided in the embodiment of the present application can monitor whether the disk has an overlong IO by monitoring the request result and the task execution queue status of the target disk, and report the disk abnormality in time when the overlong IO of the disk is monitored, thereby avoiding the unpredictable impact of the defective disk on the normal read and write business as early as possible, and improving the monitoring of the defective disk. And the automatic monitoring and reporting of defective disks can kick out the defective disk in time, which has little impact on the read and write business and achieves a smooth transition. And it reduces the cost of manual maintenance and enhances product reliability. At the same time, when reporting the abnormal disk IO status, it limits the batch reporting of the abnormal disk IO status, avoiding the risk of paralysis of the read and write business. In the embodiment of the present application, the reported overlong IO of the disk can also be restored to minimize the impact on the read and write business and improve resource utilization.
[0114] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0115] Based on the same inventive concept, the embodiment of the present application also provides a disk management device for implementing the disk management method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more disk management device embodiments provided below can refer to the limitations of the disk management method above, and will not be repeated here.
[0116] In an exemplary embodiment, as shown in FIG6 , a disk management device is provided, including: an acquisition module 602 and a reporting module 604, wherein:
[0117] The acquisition module 602 is used to acquire the request result from the target disk;
[0118] The reporting module 604 is used to report the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk.
[0119] By using the disk management device provided in the embodiment of the present application, the IO status of the target disk can be comprehensively determined based on the execution result of the target disk for the read and write request, the current disk reporting status of each disk, and the task execution queue of the target disk, and the corresponding IO status report can be made for the target disk. Since the IO status of the target disk is determined based on the execution result of the target disk for the read and write request and the task execution queue status of the target disk, the accuracy of the disk IO status report can be guaranteed, thereby improving the disk management accuracy and improving resource utilization. In addition, when reporting the disk IO status, the IO status reporting of other disks is fully considered in combination with the disk reporting status, thereby avoiding the impact of batch IO status reporting on the read and write business, and improving the stability of the read and write business.
[0120] In one embodiment, the reporting module 604 is further used to:
[0121] When the obtained request result is empty and the disk reporting status and the task execution queue status of the target disk meet the reporting conditions, the disk IO status abnormality is reported for the target disk.
[0122] Among them, the reporting conditions include: the disk reporting status of the target disk is that no disk IO status abnormality reporting is performed, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there are no other disks that report disk IO status abnormality during the monitoring period.
[0123] In one embodiment, the reporting module 604 is further used to:
[0124] When the obtained request result is not empty, if the existence duration of the first request task in the current task execution queue does not exceed the duration threshold, and the target disk has performed a disk IO status abnormality report, then the disk IO status recovery report is performed on the target disk.
[0125] In one embodiment, the reporting module 604 is further used to:
[0126] When the obtained request result is not empty, if the existence time of the first request task in the target task execution queue exceeds the time threshold, the disk IO status abnormality is reported for the target disk.
[0127] In one embodiment, the reporting module 604 is further used to:
[0128] Abnormal status information is reported to the management layer for the target disk, where the abnormal status information indicates that the target disk has an excessively long IO phenomenon and no longer performs data storage.
[0129] In an exemplary embodiment, as shown in FIG. 7 , a disk management device is provided, including: a receiving module 702 and a processing module 704, wherein:
[0130] The receiving module 702 is used to receive the reporting information for the target disk;
[0131] The processing module 704 is used to transfer the data in the target disk to other disks and no longer deliver read and write requests to the target disk when the reported information is abnormal status information indicating that the target disk has an excessively long IO phenomenon.
[0132] The above-mentioned disk management device can report abnormal disk IO status when the disk has excessive IO, so that the management can promptly remove the target disk where the disk has excessive IO, avoid its impact on read and write performance and storage and access services, and prevent IO service flow interruption.
[0133] In one embodiment, the device further comprises:
[0134] A determination module, configured to determine a target read / write request when the reported information is recovery status information characterizing recovery of excessive disk IO of the target disk, wherein the target read / write request is a read / write request that triggers the target disk to report abnormal status information;
[0135] The marking module is used to mark the area accessed by the target read / write request in the target disk as an abnormal area, and transfer the data in the abnormal area to other areas or other disks.
[0136] Each module in the above disk management device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0137] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG8. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a disk management method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0138] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0139] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.
[0140] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0141] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0143] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0144] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0145] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A disk management method, characterized in that: The method comprises: Get the request result from the target disk; The IO status of the target disk is reported according to the obtained request result, the disk reporting status and the task execution queue status of the target disk.
2. The method according to claim 1, characterized in that The step of reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk includes: When the obtained request result is empty and the disk reporting status and the task execution queue status of the target disk meet the reporting conditions, the disk IO status abnormality is reported for the target disk. Among them, the reporting conditions include: the disk reporting status of the target disk is that no disk IO status abnormality reporting is performed, there is a request task in the task execution queue of the target disk, and the existence time of the first request task in the task execution queue has exceeded the time threshold, and there are no other disks that report disk IO status abnormality during the monitoring period.
3. The method according to claim 2, characterized in that The step of reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk includes: When the obtained request result is not empty, if the existence duration of the first request task in the current task execution queue does not exceed the duration threshold, and the target disk has performed a disk IO status abnormality report, then the disk IO status recovery report is performed on the target disk.
4. The method according to claim 1, characterized in that: The step of reporting the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk includes: When the obtained request result is not empty, if the existence time of the first request task in the target task execution queue exceeds the time threshold, the disk IO status abnormality is reported for the target disk.
5. The method according to claim 2, characterized in that: The reporting of abnormal disk IO status of the target disk includes: Abnormal status information is reported to the management layer for the target disk, where the abnormal status information indicates that the target disk has an excessively long IO phenomenon and no longer performs data storage.
6. A disk management method, characterized in that: The method comprises: Receive reporting information for the target disk; When the reported information is abnormal status information indicating that the target disk has an excessively long IO phenomenon, the data in the target disk is transferred to other disks, and no read or write request is delivered to the target disk.
7. The method according to claim 6, characterized in that The method further comprises: When the reported information is recovery status information representing recovery of excessive disk IO of the target disk, determining a target read / write request, the target read / write request being a read / write request that triggers the target disk to report abnormal status information; The area accessed by the target read / write request in the target disk is marked as an abnormal area, and the data in the abnormal area is transferred to other areas or other disks.
8. A disk management device, characterized in that: The device comprises: The acquisition module is used to obtain the request result from the target disk; The reporting module is used to report the IO status of the target disk according to the obtained request result, the disk reporting status and the task execution queue status of the target disk.
9. A disk management device, characterized in that: The device comprises: A receiving module, used for receiving reporting information for a target disk; The processing module is used to transfer the data in the target disk to other disks and no longer deliver read and write requests to the target disk when the reported information is abnormal status information indicating that the target disk has an excessively long IO phenomenon.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 or 6 to 7 are implemented.