Disk abnormality processing method, device, equipment and storage medium

Abstract

The present disclosure provides a disk exception processing method, device, equipment and storage medium, the method includes determining the abnormal disk where the abnormality occurs in response to an abnormal event, obtaining the storage data of the normal disk and the verification data of the verification disk; dividing the storage data into multiple block data of target size, and inputting the multiple block data into the first target area in sequence; the first target area is a static memory pre-allocated from the memory area for abnormal repair; in the first target area, the verification data and the single block data are used to calculate the abnormal block data at the same offset of the abnormal disk, perform abnormal repair on the abnormal block data, and obtain the next abnormal block data after the abnormal block data is repaired to perform abnormal repair until all abnormal block data are repaired. The method can flexibly and concisely perform the abnormal repair processing of basic input and output IO, complete the original IO request, and the host is not aware of it.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a disk exception processing method, device, equipment and storage medium. Background Art

[0002] With the development of electronic devices, the requirements of electronic devices for storage units are getting higher and higher. In the operation of electronic devices, disk reading anomalies often occur. Disk reading anomalies are anomalies generated when the disk reading and writing engine is working. For example, the hard disk is suddenly unplugged or offline, and the disk reading and writing engine cannot complete the operation; the hard disk internal processing anomalies, and the failure result is returned to the independent disk redundant array Raid. At this time, the disk reading and writing hardware engine will report the abnormal event to the abnormal event queue.

[0003] In the related art, the way to handle disk reading exceptions is usually to request the host to issue the system call input and output (IO) again, or to construct a new task list for abnormal retry, so that the hardware engine can bypass the reading operation of the bad disk according to the new IO mode. However, the above-mentioned exception handling methods all require the host to perceive, and there are problems such as complex implementation mechanism and high coupling. Summary of the invention

[0004] The present disclosure provides a disk exception processing method, device, equipment and storage medium to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present disclosure, a method for handling disk abnormalities is provided, the method comprising: In response to the abnormal event, an abnormal disk is determined to have an abnormality, and storage data of a normal disk and verification data of a verification disk are acquired; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; Dividing the stored data into a plurality of block data of a target size, and sequentially inputting the plurality of block data into a first target area; the first target area is a static memory pre-allocated from a memory area for abnormality repair; The abnormal block data at the same offset of the abnormal disk is calculated using the verification data and a single block data in the first target area, and the abnormal block data is repaired abnormally. After the abnormal block data is repaired, the next abnormal block data is obtained to perform abnormal repair until all the abnormal block data are repaired.

[0006] In one possible implementation, before responding to an abnormal event, the method further includes: Get the original basic input and output IO request; Using an engine to detect whether all the hard disk data corresponding to the original IO request is transmitted to a second target area; the second target area is a dynamic memory in the memory area; If not, report the abnormal event.

[0007] In one possible implementation, the step of determining the abnormal disk where the abnormality occurs in response to the abnormal event includes: In response to an abnormal event, using a redundant array of independent disks Raid to calculate the data that failed to be transmitted to obtain a calculation result, and determining the abnormal disk; The calculation result is transmitted to the second target area, so that the original IO request continues to be executed.

[0008] In one possible implementation, determining the abnormal disk where the abnormality occurs in response to the abnormal event further includes: Constructing a repair task list for repairing abnormal block data; wherein a single engine task repairs a single abnormal block data; the repair task list is used to describe the abnormal disk number and the abnormal block data offset information; Add the repair task order to the list to be repaired; Repairing the repair task orders in the to-be-repaired list one by one; The repair task list includes: a task header, a page table area and an engine behavior description.

[0009] In one possible implementation, the abnormality repairing of the abnormal block data, and obtaining the next abnormal block data for abnormality repairing after the abnormal block data is repaired, until all abnormal block data are repaired, includes: Repair the abnormal block data and obtain the repair result; If the repair result is successful, the next repair task order is obtained from the to-be-repaired list to repair the next abnormal block data until all abnormal block data are repaired; If the repair result is repair failure, determine whether the Raid group tolerance meets the standard, and re-repair the abnormal block data based on the determination result; whether the Raid group tolerance meets the standard is whether the number of abnormal disks meets the threshold.

[0010] In one possible implementation mode, the determining whether the tolerance of the Raid group meets the standard and re-repairing the abnormal block data according to the determination result includes: If the tolerance standard is met, the number of abnormal disks is re-determined, and the abnormal block data is repaired again according to the number of abnormal disks; If the tolerance standard is not met, the repair task ends.

[0011] In one possible implementation manner, the re-repairing of abnormal block data according to the number of abnormal disks includes: Rebuild the repair task list; the rebuilt repair task list avoids other abnormal disks except the abnormal disk currently being repaired; Repair the abnormal block data based on the reconstructed repair task order.

[0012] In one possible implementation manner, before determining the abnormal disk having an abnormality in response to the abnormal event, the method further includes: Poll to see if there are any reported abnormal events, and if so, determine the abnormal disk where the abnormality occurred.

[0013] In one possible implementation, after all abnormal block data are repaired, the following steps are also included: Repair the next abnormal disk that has abnormal disk reading; After all abnormal disks are repaired, continue to execute the original engine task order of the original IO request.

[0014] According to a second aspect of the present disclosure, a disk exception processing device is provided, the device comprising: An acquisition module, configured to determine the abnormal disk that has an abnormality in response to an abnormal event, and acquire storage data of a normal disk and verification data of a verification disk; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; A partitioning module, used for partitioning the stored data into a plurality of block data of a target size, and inputting the plurality of block data into a first target area in sequence; the first target area is a static memory pre-allocated from a memory area for abnormality repair; A repair module is used to calculate the abnormal block data at the same offset of the abnormal disk using the verification data and a single block data in the first target area, perform abnormal repair on the abnormal block data, and obtain the next abnormal block data for abnormal repair after the abnormal block data is repaired until all the abnormal block data are repaired.

[0015] In one embodiment, the device further comprises: The detection module is used to obtain the original basic input and output IO request; use the engine to detect whether the hard disk data corresponding to the original IO request is fully transmitted to the second target area; the second target area is the dynamic memory in the memory area; if not, report the abnormal event.

[0016] In one possible implementation, the acquisition module is specifically used to respond to an abnormal event, use an independent disk redundant array Raid to calculate the data that failed to be transmitted to obtain a calculation result, and determine the abnormal disk; transmit the calculation result to the second target area, so that the original IO request continues to be executed.

[0017] In one embodiment, the device further comprises: A construction module is used to construct a repair task list for repairing abnormal block data; wherein a single engine task repairs a single abnormal block data; the repair task list is used to describe the abnormal disk number and the abnormal block data offset information; the repair task list is added to a list to be repaired; the repair task lists in the list to be repaired are repaired one by one; the repair task list includes: a task header, a page table area and an engine behavior description.

[0018] In one possible implementation, the repair module is specifically used to repair the abnormal block data and obtain a repair result; if the repair result is a successful repair, the next repair task order is obtained from the list to be repaired to repair the next abnormal block data until all abnormal block data are repaired; if the repair result is a repair failure, it is determined whether the Raid group tolerance meets the standard, and the abnormal block data is repaired again according to the determination result; whether the Raid group tolerance meets the standard is whether the number of abnormal disks meets the threshold.

[0019] In one embodiment, the repair module includes a judgment unit, which is specifically used to redetermine the number of abnormal disks and re-repair the abnormal block data based on the number of abnormal disks if the tolerance standard is met; if the tolerance standard is not met, the repair task is terminated.

[0020] In one possible implementation, the judgment unit is specifically used to reconstruct the repair task list; the reconstructed repair task list avoids other abnormal disks except the abnormal disk currently being repaired; and the abnormal block data is repaired based on the reconstructed repair task list.

[0021] In one embodiment, the device further comprises: The polling module is used to poll whether there are any reported abnormal events, and if so, determine the abnormal disk where the abnormality occurs.

[0022] In one embodiment, the device further comprises: The execution module is used to repair the next abnormal disk with abnormal disk reading; after all abnormal disks are repaired, the original engine task list of the original IO request continues to be executed.

[0023] According to a third aspect of the present disclosure, there is provided an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0024] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure.

[0025] The disk exception processing method, device, equipment and storage medium disclosed in the present invention, in response to an abnormal event, Raid of the present application determines that an abnormal disk has an abnormality, obtains storage data of a normal disk and verification data of a verification disk; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; the storage data is divided into block data of multiple target sizes, and the multiple block data are sequentially input into a first target area; the first target area is a static memory pre-allocated from a memory area for abnormal repair; in the first target area, the verification data and a single block data are used to calculate the abnormal block data at the same offset of the abnormal disk, perform abnormal repair on the abnormal block data, and obtain the next abnormal block data after the abnormal block data is repaired to perform abnormal repair until all abnormal block data are repaired. The technical solution provided by the present application repairs the data in the abnormal disk after determining that there is an abnormal disk, and it repairs only fixed-size data each time by iterating multiple repairs, which consumes very little memory, can flexibly and concisely perform IO abnormal repair processing, complete the original IO request, and the host is not aware of it, ensuring that normal IO processing tasks and abnormal repair tasks will not compete for memory resources, and the coupling degree is low.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which: In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0028] Figure 1 The schematic diagram shows the implementation process of the disk abnormality processing method of the embodiment of the present disclosure. Figure 1 ; Figure 2 The schematic diagram shows the implementation process of the disk abnormality processing method of the embodiment of the present disclosure. Figure 2 ; Figure 3 The schematic diagram shows the implementation process of the disk abnormality processing method of the embodiment of the present disclosure. Figure 3 ; Figure 4The firmware module structure and firmware hardware interaction diagram of the embodiment of the present disclosure are shown; Figure 5 A schematic diagram showing the structure of a disk exception processing device according to an embodiment of the present disclosure is shown; Figure 6 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0030] IO exception means that the read and write IO sent by the host has an exception and cannot directly complete the user request. The main scope is the hardware exception that occurs in the hardware engine of the redundant array configuration module in the Raid during the execution of the task, such as resource application exception, Raid check calculation exception, and read and write disk engine exception.

[0031] Currently, the common IO exception handling mechanisms include the following two: (1) For IO that has not yet responded to the host's customizable completion queue (CQ), Raid can reply to the host with a special CQ to request the host to send a basic input / output (IO) request again. At this time, Raid is required to mark the relevant abnormal hard disk as an offline disk. The original read-modify-write (RMW) write IO mode will automatically switch to the reset configuration word (RCW) write IO mode, thereby bypassing the bad disk and completing the IO. However, this exception handling mechanism has poor performance and needs to consider the host IO timeout; the relevant retry operations are transferred to the host, which increases the host coupling and instability; some exception types do not support this method, such as solid-state disk (SSD) sector media errors and other exception types, because in this scenario the Raid system cannot mark the hard disk as offline.

[0032] (2) After an IO exception occurs, a new task list for abnormal retry is constructed to allow the hardware engine to bypass the read operation of the bad disk according to the new IO mode. For example, after an RMW exception occurs, an abnormal retry IO in the reset configuration word (RCW) mode is constructed, and then a task list is reconstructed. This task list can basically reuse the existing RCW task list. However, this exception handling mechanism may need to analyze the original IO request details when constructing a task list for abnormal retry, and the analysis process is relatively complicated; using the RCW task list to repair the RMW task list will make the internal processing flow of the new task list more cumbersome; the engine needs to apply for memory for Raid calculation. If the Raid group contains too many hard disks, more memory space needs to be applied. If the original IO task list has applied for more memory, this method of retrying with a new task list may result in the risk of memory application failure or memory resource competition.

[0033] The disk exception handling method provided by this application mainly focuses on the exceptions that occur when RAID reads and writes physical hard disks. This allows for a chance to retry when a disk read fails. The technical solution of this application can flexibly and concisely repair IO exceptions, complete the original IO request, and the host is not aware of it.

[0034] The following describes a disk exception processing method, device, equipment and medium provided by the present application in conjunction with the accompanying drawings.

[0035] like Figure 1 As shown, the present application provides a disk exception processing method, the method comprising: S101, in response to an abnormal event, determining an abnormal disk where an abnormality occurs, obtaining storage data of a normal disk and verification data of a verification disk; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; The abnormal events in this application are mainly disk reading abnormalities. Disk reading abnormalities are abnormalities generated when the disk reading and writing engine is working. The causes of the abnormality include: 1) The hard disk is suddenly unplugged or offline, and the disk reading and writing engine cannot complete the operation; 2) The internal processing of the hard disk is abnormal, and the failure result is replied to Raid. Among them, Raid receives an IO request (RMW mode) and first builds an original task list. The process of the original task list is: first apply for relevant memory space, read the disk (read the relevant user data and verification data in the hard disk into the Raid memory), then perform Raid verification calculation, and then write the disk (write the verification results calculated in the memory and user data to the hard disk); then release the memory. After the original task list is submitted to the hardware engine for execution, when the disk reading operation is abnormal, the original task list will suspend execution, that is, the original task list will suspend at the disk reading position; finally, the disk reading and writing hardware engine will report the abnormal event to the abnormal event queue.

[0036] After retrieving the reported abnormal events, the abnormal disk where the abnormality occurred can be determined; among them, when the original IO is executed, space will be requested in the memory managed by Raid. The meaning of the disk reading operation of the hardware engine is to transfer the hard disk data to the requested memory. The abnormal disk reading means that all data cannot be successfully transferred. The abnormal disk is the disk that cannot be read. It is understandable that if Figure 2 As shown, the disk array device Raid has multiple data disks, among which there may be multiple abnormal disks or one abnormal disk, and the data disk that can read the disk normally is a normal disk. The check disk can protect the data integrity and store the redundant information for data check and recovery.

[0037] S102, dividing the stored data into a plurality of block data of a target size, and sequentially inputting the plurality of block data into a first target area; the first target area is a static memory pre-allocated from a memory area for abnormality repair; Continue as Figure 2 As shown, because the stored data is large, if all of it is put into the memory to judge the abnormal data, the memory occupied is too large. Because the stored data is divided into block data with a fixed size in this application, the block data is transferred one by one to the static memory pre-allocated from the memory area for abnormal repair; the static memory segment pre-allocated in this application only repairs a small granularity each time, so the memory in this area can be controlled in a smaller range. For example, when the granularity is 4K, the Raid group supports a maximum of 32 disks, then only about memory space. It should be noted that the method provided in the present application needs to pre-allocate a section of static memory in the memory pool for Raid calculations, so as not to occupy the dynamic memory area. Since it is a small-grained data repair, parallel reuse can be ignored at this time, so the size of this section of static memory can be smaller. It can be understood that the method provided in the present application, if the pre-allocated static memory is large enough during the repair, parallel repair can be considered, that is, small-grained repairs can be performed on two "abnormal disk read operations" at the same time, or the granularity can be increased, that is, 8K data can be repaired each time.

[0038] In the present application, the memory area may be a memory pool. In the present application, a smaller static memory space is divided in the memory pool, which is specifically used for exception repair, so that the exception repair operation and the original IO processing operation can be performed simultaneously and in different memory spaces without competing for memory resources.

[0039] S103, calculating the abnormal block data at the same offset of the abnormal disk using the verification data and a single block data in the first target area, performing abnormal repair on the abnormal block data, and obtaining the next abnormal block data for abnormal repair after the abnormal block data is repaired until all abnormal block data are repaired.

[0040] It can be understood that the static memory segment pre-allocated in the present application only repairs a small granularity each time. The present application can calculate the abnormal block data at the same offset of the abnormal disk based on the verification data and the single block data in the static memory, repair the data here, and then repair the obtained abnormal block data in the same way after the repair. The repair is iterated in this way until all the abnormal data are repaired. That is to say, a single repair task order only repairs a small block of data of the original IO disk reading anomaly. When the anomaly is repaired, the firmware can control the original engine task order to continue execution, and finally complete the original host IO request. The host is not aware of the whole process.

[0041] The repair mechanism of the method of the present application is to repair multiple times through iterations, and only a fixed size of data is repaired each time, that is, each time the data is repaired through Raid calculation, only a small-grained data needs to be repaired.

[0042] The disk exception handling method provided in the present application can be implemented through Raid. After receiving an abnormal event, Raid first determines the abnormal disk where the abnormal event occurred, and allocates a smaller static memory space in the memory pool, which is dedicated to abnormal repair. Specifically, the data that failed to read is calculated by reading the data of the normal disk and the check disk, so as to repair the abnormal data. The technical solution completes the repair in a small-grained iterative manner, and only repairs a small block of the abnormal data each time. The next small block is repaired after the previous small block is repaired, and the repair is completed by iterating multiple times, thereby reducing the use of memory resources, so that the IO exception repair processing can be performed reasonably, flexibly and concisely, and the original IO request can be completed without letting the host perceive it.

[0043] In some embodiments, before responding to an abnormal event, the method further includes: Get the original basic input and output IO request; Using an engine to detect whether all the hard disk data corresponding to the original IO request is transmitted to a second target area; the second target area is a dynamic memory in the memory area; If not, report the abnormal event.

[0044] It should be noted that when the original IO is executed, space will be requested in the memory managed by Raid. The engine detects whether all the hard disk data is transferred to the dynamic memory in the memory area. An abnormal disk reading means that all data has not been successfully transmitted. At this time, an abnormal event is generated and the engine reports the abnormal event to Raid.

[0045] In some embodiments, the step of determining the abnormal disk where the abnormality occurs in response to the abnormal event includes: In response to an abnormal event, using a redundant array of independent disks Raid to calculate the data that failed to be transmitted to obtain a calculation result, and determining the abnormal disk; The calculation result is transmitted to the second target area, so that the original IO request continues to be executed.

[0046] After receiving an abnormal event, Raid can calculate the data that failed to be transmitted through the normal disk and the check disk, and determine which abnormal disk the data belongs to. At the same time, it can also write the calculation result directly into the original destination memory space of the data transmission, making it seem that the disk reading abnormality has never occurred, so that the host is not aware of it. Specifically, the memory area in this application includes pre-allocated static memory and dynamic memory. In this application, static memory will be used for abnormal repair, and dynamic memory will be used to execute the original IO request, which is not perceived by the host.

[0047] In some embodiments, determining the abnormal disk where the abnormality occurs in response to the abnormal event further includes: Constructing a repair task list for repairing abnormal block data; wherein a single engine task repairs a single abnormal block data; the repair task list is used to describe the abnormal disk number and the abnormal block data offset information; Add the repair task order to the list to be repaired; Repairing the repair task orders in the to-be-repaired list one by one; The repair task list includes: a task header, a page table area and an engine behavior description.

[0048] Continue as Figure 2 As shown, in this application, before repairing, for each abnormal block data repair, a repair task list needs to be constructed, all repair task lists are added to the list to be repaired, and the abnormal block data corresponding to the repair task list is repaired according to the list to be repaired. Among them, the repair task list includes a task header, a page table area, and an engine behavior description.

[0049] Among them, the task header includes basic information, such as the task order type, task order execution result and other fields; the page table area is used to indicate the memory address of the input data and output result of the Raid calculation, and specifically describes the memory page information associated with the task order. The hardware engine will access this area when working. For example, the disk reading engine accesses this field to move the hard disk data to the specified memory space; the engine behavior description includes fields that describe and control the work of the hardware engine, specifically fields that describe the engine behavior. The fields are sent to the hardware engine, and the hardware engine can work according to the information in the fields. The task order engine behavior description information will specifically specify which hard disk and location the disk reading engine starts reading from; the repair task order can also include other content, which is not limited in this application.

[0050] In some embodiments, performing abnormal repair on the abnormal block data, and obtaining the next abnormal block data for abnormal repair after the abnormal block data is repaired, until all abnormal block data are repaired, includes: Repair the abnormal block data and obtain the repair result; If the repair result is successful, the next repair task order is obtained from the to-be-repaired list to repair the next abnormal block data until all abnormal block data are repaired; If the repair result is repair failure, determine whether the Raid group tolerance meets the standard, and re-repair the abnormal block data based on the determination result; whether the Raid group tolerance meets the standard is whether the number of abnormal disks meets the threshold.

[0051] As a specific example, continue as Figure 2 As shown, first, we need to obtain the abnormal block data, and then access this field through the Raid verification calculation engine to determine the memory pages involved in the calculation and the memory pages of the calculation results. The input of the Raid calculation is 3 small blocks of memory data pages (static memory), and the calculation results are directly written into the memory dynamically requested by the original IO. Only a small block of results is calculated, which is equivalent to a single repair task order that only repairs a small block of data with abnormal original IO disk reading.

[0052] like Figure 3 As shown in the figure, after calculating the abnormal data, a repair task list is constructed and added to the list to be repaired. It is determined whether the current repair task is idle. If not, the repair task exits and waits for idleness. If the current state is idle, the repair task state is set to busy, and a repair task list to be repaired is taken from the list to be repaired. The repair offset offset=0 is initialized, and a single-granularity repair is initiated for the specified offset. Executing a single-granularity repair means repairing an abnormal block of data. After the repair is completed, the single-granularity repair result is submitted. If the single-granularity repair is successful, the offset is increased by 1. The hardware engine can only repair the next granularity after repairing one granularity. Therefore, the firmware initiates a small-granularity repair and the firmware processes the result asynchronously. After iterating to complete all granular repairs, if there is still an abnormal disk, continue to repair the next abnormal disk until all abnormal disks are repaired and set the repair task to idle. If a new abnormal disk reading event occurs during the small-granularity repair process, the small-granularity repair result is a failure, and the Raid group tolerance judgment needs to be performed first.

[0053] In some embodiments, the step of determining whether the tolerance of the Raid group meets the standard and re-repairing the abnormal block data according to the determination result includes: If the tolerance standard is met, the number of abnormal disks is re-determined, and the abnormal block data is repaired again according to the number of abnormal disks; If the tolerance standard is not met, the repair task ends.

[0054] The re-repairing of abnormal block data according to the number of abnormal disks includes: Rebuild the repair task list; the rebuilt repair task list avoids other abnormal disks except the abnormal disk currently being repaired; Repair the abnormal block data based on the reconstructed repair task order.

[0055] After all abnormal block data is repaired, it also includes: Repair the next abnormal disk that has abnormal disk reading; After all abnormal disks are repaired, continue to execute the original engine task order of the original IO request.

[0056] Specifically, the tolerance of the Raid group is judged as follows: if it meets the tolerance standard, small-grained repairs can be re-executed according to the number of new bad disks, which means that the constructed small-grained repair task list can also avoid the newly discovered bad disks, and subsequent small-grained repairs must be performed according to the number of new bad disks. If it does not meet the tolerance standard, for example, when the Raid is Raid5 and the second bad disk is found, the current repair task fails as a whole, and the original IO can no longer be repaired. You can only access the FIFO queue to be repaired to obtain the next task to be repaired, that is, to handle the next disk read exception.

[0057] In some embodiments, before determining the abnormal disk having an abnormality in response to the abnormal event, the method further includes: Poll to see if there are any reported abnormal events, and if so, determine the abnormal disk where the abnormality occurred.

[0058] Specifically, in the present application, Raid polls whether there are any reported abnormal events, so as to observe at any time whether there is any abnormal data that needs to be repaired, and when there is an abnormal event, determine the abnormal disk where the abnormality occurs.

[0059] The disk exception handling method provided by the present application divides a smaller static memory space in the memory pool, which is dedicated to exception repair, so as to avoid the competition of memory resources between the exception repair operation and the normal user IO processing operation; the exception repair of the present application is to calculate the data that failed to read through the method of "reading other normal disks" + "Raid verification calculation", and at the same time directly write the Raid calculation result into the destination memory space of the original data transmission, so as to create the effect that the disk reading exception seems to have never occurred; the repair method of the present application is to complete the repair through a small-grained iteration method in an independent background task (thread), that is, only a small block of the exception data is repaired each time, and the next small block is repaired after the previous small block is repaired, and the repair is completed by iterating multiple times, thereby reducing the use of memory resources. At the same time, since exception repair is usually not involved in the measurement of system performance indicators, small-grained repair is feasible.

[0060] like Figure 4 As shown, the method provided by the present application involves three firmwares, a module for polling abnormal events, a module for polling the end of task orders, and a module for repairing task status control. Among them, the module for polling abnormal events is responsible for processing task orders with abnormal disk reading. Task orders include two categories: task orders corresponding to original IO, the module for polling abnormal events needs to submit the disk reading operation to be repaired; task orders for small-grained repairs, marking failures in the task orders, and allowing the task orders to continue execution from the pause point. Among them, the module for polling the end of task orders is responsible for processing the results of small-grained repairs and forwarding the results; among them, the module for repairing task status control is responsible for iterative repair work, including: receiving the disk reading operation to be repaired, adding it to a FIFO queue to be repaired; receiving the repair results of a single granularity, and processing them separately for success and failure; when the module is idle, it obtains a disk reading abnormal operation from the queue and performs abnormal repair on it.

[0061] The disk exception handling method provided in the present application is that after the firmware detects the abnormal event reported by the hardware engine, it can build a new engine task list for abnormal repair, thereby completing data repair. When the abnormality is repaired, the firmware can control the original engine task list to continue execution, and finally complete the original host IO request. The host is not aware of the entire process.

[0062] The disk exception handling method provided in this application can simulate the effect that the disk reading operation seems to have no exception, without analyzing the original IO characteristics, and has high versatility; it consumes very little memory, can ensure that normal IO processing tasks and exception repair tasks do not compete for memory resources, and has low coupling.

[0063] like Figure 5 As shown, the present application provides a disk exception processing device, the device comprising: The acquisition module 501 is used to determine the abnormal disk that has an abnormality in response to the abnormal event, and acquire the storage data of the normal disk and the verification data of the verification disk; the verification data includes the data of the normal disk and the data within a preset time period before the abnormal disk fails; A division module 502 is used to divide the stored data into a plurality of block data of a target size, and input the plurality of block data into a first target area in sequence; the first target area is a static memory pre-allocated from a memory area for abnormality repair; The repair module 503 is used to calculate the abnormal block data at the same offset of the abnormal disk using the verification data and the single block data in the first target area, perform abnormal repair on the abnormal block data, and obtain the next abnormal block data for abnormal repair after the abnormal block data is repaired until all the abnormal block data are repaired.

[0064] The disk exception processing device provided by the present application has an acquisition module 501 that determines the abnormal disk where the exception occurs in response to an abnormal event, and obtains the storage data of the normal disk and the verification data of the verification disk; the verification data includes the data of the normal disk and the data within a preset time period before the abnormal disk fails; the division module 502 divides the storage data into block data of multiple target sizes, and inputs the multiple block data into a first target area in sequence; the first target area is a static memory pre-allocated from the memory area for exception repair; the repair module 503 calculates the abnormal block data at the same offset of the abnormal disk using the verification data and a single block data in the first target area, performs exception repair on the abnormal block data, and obtains the next abnormal block data after the abnormal block data is repaired to perform exception repair until all abnormal block data are repaired.

[0065] In some embodiments, the device further comprises: The detection module is used to obtain the original basic input and output IO request; use the engine to detect whether the hard disk data corresponding to the original IO request is fully transmitted to the second target area; the second target area is the dynamic memory in the memory area; if not, report the abnormal event.

[0066] In some embodiments, the acquisition module is specifically used to respond to an abnormal event, use independent disk redundant array Raid to calculate the data that failed to be transmitted to obtain a calculation result, and determine the abnormal disk; transmit the calculation result to the second target area, so that the original IO request continues to execute.

[0067] In some embodiments, the device further comprises: A construction module is used to construct a repair task list for repairing abnormal block data; wherein a single engine task repairs a single abnormal block data; the repair task list is used to describe the abnormal disk number and the abnormal block data offset information; the repair task list is added to a list to be repaired; the repair task lists in the list to be repaired are repaired one by one; the repair task list includes: a task header, a page table area and an engine behavior description.

[0068] In some embodiments, the repair module is specifically used to repair the abnormal block data and obtain a repair result; if the repair result is a successful repair, the next repair task order is obtained from the list to be repaired to repair the next abnormal block data until all abnormal block data are repaired; if the repair result is a repair failure, it is determined whether the Raid group tolerance meets the standard, and the abnormal block data is repaired again according to the determination result; whether the Raid group tolerance meets the standard is whether the number of abnormal disks meets the threshold.

[0069] In some embodiments, the repair module includes a judgment unit, which is specifically used to redetermine the number of abnormal disks and re-repair the abnormal block data based on the number of abnormal disks if the tolerance standard is met; if the tolerance standard is not met, the repair task is terminated.

[0070] In some embodiments, the judgment unit is specifically used to reconstruct the repair task list; the reconstructed repair task list avoids other abnormal disks except the abnormal disk currently being repaired; and the abnormal block data is repaired based on the reconstructed repair task list.

[0071] In some embodiments, the device further comprises: The polling module is used to poll whether there are any reported abnormal events, and if so, determine the abnormal disk where the abnormality occurs.

[0072] In some embodiments, the device further comprises: The execution module is used to repair the next abnormal disk with abnormal disk reading; after all abnormal disks are repaired, the original engine task list of the original IO request continues to be executed.

[0073] It should be noted that the disk exception handling device of the embodiment of the present application, since the principle of solving the problem by the disk exception handling device is similar to the aforementioned disk exception handling method, the implementation process, implementation principle, and beneficial effects of the disk exception handling device can all be referred to the description of the implementation process, implementation principle, and beneficial effects of the aforementioned method, and the repeated parts will not be repeated.

[0074] An embodiment of the present application provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform In response to the abnormal event, an abnormal disk is determined to have an abnormality, and storage data of a normal disk and verification data of a verification disk are acquired; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; Dividing the stored data into a plurality of block data of a target size, and sequentially inputting the plurality of block data into a first target area; the first target area is a static memory pre-allocated from a memory area for abnormality repair; The abnormal block data at the same offset of the abnormal disk is calculated using the verification data and a single block data in the first target area, and the abnormal block data is repaired abnormally. After the abnormal block data is repaired, the next abnormal block data is obtained to perform abnormal repair until all the abnormal block data are repaired.

[0075] The present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute In response to the abnormal event, an abnormal disk is determined to have an abnormality, and storage data of a normal disk and verification data of a verification disk are acquired; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; Dividing the stored data into a plurality of block data of a target size, and sequentially inputting the plurality of block data into a first target area; the first target area is a static memory pre-allocated from a memory area for abnormality repair; The abnormal block data at the same offset of the abnormal disk is calculated using the verification data and a single block data in the first target area, and the abnormal block data is repaired abnormally. After the abnormal block data is repaired, the next abnormal block data is obtained to perform abnormal repair until all the abnormal block data are repaired.

[0076] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0077] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0078] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0079] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0080] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as a disk exception handling method. For example, in some embodiments, the disk exception handling method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the disk exception handling method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the disk exception handling method in any other appropriate manner (e.g., by means of firmware).

[0081] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A disk exception processing method, characterized in that: The method comprises: In response to the abnormal event, an abnormal disk is determined to have an abnormality, and storage data of a normal disk and verification data of a verification disk are acquired; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; Dividing the stored data into a plurality of block data of a target size, and sequentially inputting the plurality of block data into a first target area; the first target area is a static memory pre-allocated from a memory area for abnormality repair; The abnormal block data at the same offset of the abnormal disk is calculated using the verification data and a single block data in the first target area, and the abnormal block data is repaired abnormally. After the abnormal block data is repaired, the next abnormal block data is obtained to perform abnormal repair until all the abnormal block data are repaired.

2. The method according to claim 1, characterized in that: Before responding to an abnormal event, it also includes: Get the original basic input and output IO request; Using an engine to detect whether all the hard disk data corresponding to the original IO request is transmitted to a second target area; the second target area is a dynamic memory in the memory area; If not, report the abnormal event.

3. The method according to claim 2, characterized in that The step of determining the abnormal disk where the abnormality occurs in response to the abnormal event includes: In response to an abnormal event, using a redundant array of independent disks Raid to calculate the data that failed to be transmitted to obtain a calculation result, and determining the abnormal disk; The calculation result is transmitted to the second target area, so that the original IO request continues to be executed.

4. The method according to claim 1, characterized in that: In response to the abnormal event, determining the abnormal disk where the abnormality occurs, further comprising: Constructing a repair task list for repairing abnormal block data; wherein a single engine task repairs a single abnormal block data; the repair task list is used to describe the abnormal disk number and the abnormal block data offset information; Add the repair task order to the list to be repaired; Repairing the repair task orders in the to-be-repaired list one by one; The repair task list includes: a task header, a page table area and an engine behavior description.

5. The method according to claim 4, characterized in that The abnormal block data is repaired abnormally, and after the abnormal block data is repaired, the next abnormal block data is obtained to perform abnormal repair until all abnormal block data are repaired, including: Repair the abnormal block data and obtain the repair result; If the repair result is successful, the next repair task order is obtained from the to-be-repaired list to repair the next abnormal block data until all abnormal block data are repaired; If the repair result is repair failure, determine whether the Raid group tolerance meets the standard, and re-repair the abnormal block data based on the determination result; whether the Raid group tolerance meets the standard is whether the number of abnormal disks meets the threshold.

6. The method according to claim 5, characterized in that The step of judging whether the tolerance of the Raid group meets the standard and re-repairing the abnormal block data according to the judgment result includes: If the tolerance standard is met, the number of abnormal disks is re-determined, and the abnormal block data is repaired again according to the number of abnormal disks; If the tolerance standard is not met, the repair task ends.

7. The method according to claim 6, characterized in that The re-repairing of abnormal block data according to the number of abnormal disks includes: Rebuild the repair task list; the rebuilt repair task list avoids other abnormal disks except the abnormal disk currently being repaired; Repair the abnormal block data based on the reconstructed repair task order.

8. The method according to claim 4, characterized in that Before determining the abnormal disk where the abnormality occurs in response to the abnormal event, the method further includes: Poll to see if there are any reported abnormal events, and if so, determine the abnormal disk where the abnormality occurred.

9. The method according to claim 1, characterized in that: After all abnormal block data is repaired, it also includes: Repair the next abnormal disk that has abnormal disk reading; After all abnormal disks are repaired, continue to execute the original engine task order of the original IO request.

10. A disk abnormality processing device, characterized in that: The device comprises: An acquisition module, configured to determine the abnormal disk that has an abnormality in response to an abnormal event, and acquire storage data of a normal disk and verification data of a verification disk; the verification data includes data of the normal disk and data within a preset time period before the abnormal disk fails; A partitioning module, used for partitioning the stored data into a plurality of block data of a target size, and inputting the plurality of block data into a first target area in sequence; the first target area is a static memory pre-allocated from a memory area for abnormality repair; A repair module is used to calculate the abnormal block data at the same offset of the abnormal disk using the verification data and a single block data in the first target area, perform abnormal repair on the abnormal block data, and obtain the next abnormal block data for abnormal repair after the abnormal block data is repaired until all the abnormal block data are repaired.

11. The device according to claim 10, characterized in that The device also includes: The detection module is used to obtain the original basic input and output IO request; use the engine to detect whether the hard disk data corresponding to the original IO request is fully transmitted to the second target area; the second target area is the dynamic memory in the memory area; if not, report the abnormal event.

12. The device according to claim 11, characterized in that The acquisition module is specifically used to respond to abnormal events, use independent disk redundant array Raid to calculate the data that failed to be transmitted to obtain calculation results, determine the abnormal disk; and transmit the calculation results to the second target area so that the original IO request continues to be executed.

13. The device according to claim 11, characterized in that The device also includes: A construction module is used to construct a repair task list for repairing abnormal block data; wherein a single engine task repairs a single abnormal block data; the repair task list is used to describe the abnormal disk number and the abnormal block data offset information; the repair task list is added to a list to be repaired; the repair task lists in the list to be repaired are repaired one by one; the repair task list includes: a task header, a page table area and an engine behavior description.

14. The device according to claim 13, characterized in that The repair module is specifically used to repair the abnormal block data and obtain a repair result; if the repair result is successful, the next repair task order is obtained from the to-be-repaired list to repair the next abnormal block data until all abnormal block data are repaired; If the repair result is repair failure, determine whether the Raid group tolerance meets the standard, and re-repair the abnormal block data based on the determination result; whether the Raid group tolerance meets the standard is whether the number of abnormal disks meets the threshold.

15. The device according to claim 14, characterized in that The repair module includes a judgment unit, which is specifically used to re-determine the number of abnormal disks and re-repair the abnormal block data according to the number of abnormal disks if the tolerance standard is met; if the tolerance standard is not met, the repair task is terminated.

16. The device according to claim 15, characterized in that The judgment unit is specifically used to reconstruct the repair task list; the reconstructed repair task list avoids other abnormal disks except the abnormal disk currently being repaired; and the abnormal block data is repaired based on the reconstructed repair task list.

17. The device according to claim 13, characterized in that The device also includes: The polling module is used to poll whether there are any reported abnormal events, and if so, determine the abnormal disk where the abnormality occurs.

18. The device according to claim 13, characterized in that The device also includes: The execution module is used to repair the next abnormal disk with abnormal disk reading; after all abnormal disks are repaired, the original engine task list of the original IO request continues to be executed.

19. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-9.

Images