Method, device and equipment for determining fault condition and computer readable storage medium

Abstract

The invention discloses a method, device and equipment for determining a fault condition and a computer readable storage medium, and belongs to the technical field of computers. The method comprises the following steps: in a PM extension scene, i.e., in a scene that a PM is connected with a plurality of disks, obtaining a first fault condition of the plurality of disks reported after determination of a driving layer, and a second fault condition of the plurality of disks determined by a business layer according to an average processing duration of each disk on an IO (Input / Output) instruction; and determining a third fault condition of the plurality of disks according to the first fault condition and the second fault condition. The third fault condition is determined by integrating the first fault condition and the second fault condition, so that the third fault condition has relatively high accuracy, and the security and reliability of data storage are ensured.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, device and computer-readable storage medium for determining a fault condition. Background Art

[0002] In computer technology, a disk is used to store data under the control of a controller. The controller can be connected to multiple disks through port multipliers (PMs). How to determine the failure of a disk in the presence of a PM is the key to ensuring the security and reliability of data storage. Summary of the invention

[0003] The present application provides a method, apparatus, device and computer-readable storage medium for determining a fault condition, so as to determine a fault condition of a disk in the presence of a PM. The technical solution provided by the present application includes the following aspects.

[0004] In a first aspect, a method for determining a fault condition is provided, in which a first fault condition of multiple disks determined and reported by a driver layer is obtained, and the multiple disks are connected to a PM. In addition, a second fault condition of the multiple disks is obtained, and the second fault condition is determined by a business layer based on an average processing time of input output (IO) instructions of each disk in the multiple disks. Thereafter, a third fault condition of the multiple disks is determined based on the first fault condition and the second fault condition.

[0005] In the PM expansion scenario, the present application obtains not only the first fault condition determined and reported by the driver layer, but also the second fault condition determined by the business layer based on the average processing time of each disk for IO instructions, so that a more accurate third fault condition can be determined by combining the first fault condition and the second fault condition, thereby improving the accuracy of determining the faulty disk and ensuring the security and reliability of data storage.

[0006] In one possible implementation, the first fault condition indicates that there is no faulty disk, or the first fault condition indicates that there is a faulty disk; for any one of the multiple disks, if any one of the disks is the first disk among the multiple disks whose time difference after the starting determination time is greater than or equal to the first threshold, then any one of the disks is a faulty disk; the time difference of any one of the disks is the difference between the first moment and the second moment, the first moment is the moment when the driver layer adds the first key frame to the controller's sending queue, and the controller is connected to the PM, and the second moment is the moment when the driver layer perceives that the controller has received the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

[0007] In this implementation, a more precise disk time difference is defined to accurately indicate the processing time of the disk for IO instructions, so that the first fault condition determined by the driver layer based on the disk time difference is more accurate, thereby ensuring the accuracy of the third fault condition subsequently determined based on the first fault condition.

[0008] In one possible implementation, the first fault condition indicates that there is no faulty disk, or the first fault condition indicates that there is at least one faulty disk; for any one of the multiple disks, if any one of the disks is a disk whose time difference among the multiple disks after the starting determined time is greater than or equal to the second threshold, then any one of the disks is a faulty disk; the time difference of any one of the disks is the difference between a third moment and a fourth moment, the third moment is the moment when the controller sends the first key frame added to the sending queue by the driver layer, and the controller is connected to the PM, and the fourth moment is the moment when the controller receives the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

[0009] In this implementation, the accuracy of the disk time difference is further improved, ensuring the accuracy of the first fault condition determined by the driver layer based on the disk time difference. In addition, this implementation also enables the first fault condition to indicate one or more faulty disks, making the first fault condition more flexible in indicating the faulty disk, which is conducive to improving the accuracy of the third fault condition subsequently determined based on the first fault condition.

[0010] In one possible implementation, the second fault condition indicates that there is no faulty disk, or the second fault condition indicates that there is at least one faulty disk; for any one of the multiple disks, if the average processing time of any one of the disks for IO instructions is greater than or equal to a third threshold, then any one of the disks is a faulty disk; wherein the average processing time of any one of the disks for IO instructions is a ratio between a first value and a second value, the first value being the total time for any one of the disks to process IO instructions within a reference time period, and the second value being the total number of IO instructions processed by any one of the disks within the reference time period.

[0011] Among them, the second fault condition determined by the business layer is not based on the single processing process of the disk to the IO instruction, but is based on the average statistics of multiple processing processes of the disk to the IO instruction, thereby avoiding the randomness of the single processing process of the disk to IO, making the second fault condition determined by the business layer more accurate, thereby ensuring the accuracy of the third fault condition subsequently determined based on the second fault condition.

[0012] In a possible implementation, after obtaining the first fault conditions of the multiple disks, the method further includes: controlling the driver layer to re-determine the fault conditions of the multiple disks.

[0013] If the driver layer is not controlled to redetermine the fault condition, the driver layer may only have a chance to obtain a first fault condition during one operation of the device where the driver layer is located. However, controlling the driver layer to redetermine the fault condition is conducive to the driver layer determining multiple fault conditions, and the multiple fault conditions can cooperate with the fault condition determined by the business layer, so that during one operation of the device where the driver layer is located, the accurate fault condition can be determined through the cooperation between the driver layer and the business layer for multiple times.

[0014] In one possible implementation, after determining the third fault condition of multiple disks based on the first fault condition and the second fault condition, the method also includes: if the third fault condition indicates that there is at least one faulty disk, deleting the disk indicated by the third fault condition from the multiple disks to obtain an updated disk group; determining the fault condition of the updated disk group; if the fault condition of the updated disk group indicates that there is at least one faulty disk, adding the faulty disk indicated by the third fault condition to the updated disk group to obtain a recovered disk group, and determining the fault condition of the recovered disk group.

[0015] If the fault condition of the updated disk group indicates that at least one faulty disk still exists in the disk group, it means that the faulty disk indicated by the third fault condition previously determined may not be accurate enough. Therefore, the deleted faulty disk indicated by the third fault condition can be re-added to the updated disk group to restore the updated disk group to the restored multiple disk groups, and then the fault conditions of the restored multiple disk groups are determined, which further improves the accuracy of determining the fault condition.

[0016] In one possible implementation, a third fault condition of multiple disks is determined based on a first fault condition and a second fault condition, including: if the first fault condition is different from the second fault condition, determining the third fault condition in any one of the following ways: determining the third fault condition based on at least one of a fourth fault condition and a fifth fault condition, the fourth fault condition being determined by controlling a driver layer, and the fifth fault condition being determined by controlling a business layer; taking the second fault condition as the third fault condition; taking a faulty disk randomly determined from multiple disks as a faulty disk indicated by the third fault condition; and determining the third fault condition according to a reference rule.

[0017] In the case where the first fault condition is different from the second fault condition, multiple methods are provided to comprehensively determine the third fault condition, thereby improving the flexibility and applicability of the method provided by the present application. Among them, in the case where the first fault condition is different from the second fault condition, the present application enables at least one of the driver layer and the business layer to re-detect the faulty disk and re-determine the fault condition, so as to determine the third fault condition according to the re-determined fault condition (i.e., the fourth fault condition and the fifth fault condition), thereby further eliminating contingency and improving the accuracy of the determined third fault condition.

[0018] In a possible implementation, the first fault condition is different from the second fault condition, including: the faulty disk indicated by the first fault condition is different from the faulty disk indicated by the second fault condition; the third fault condition is determined according to at least one of the fourth fault condition and the fifth fault condition, including: if the fifth fault condition is the same as the second fault condition, the second fault condition is taken as the third fault condition, because in this case the second fault condition is sufficiently reliable; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is the same first disk as the faulty disk indicated by the first fault condition, the first disk is taken as the faulty disk indicated by the third fault condition, because in this case the first disk is sufficiently reliable; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is different from the faulty disk indicated by the first fault condition, the third fault condition is determined according to the first setting method, and even if the same faulty disk is still not found after re-detection, other methods can be flexibly used to determine the third fault condition.

[0019] In one possible implementation, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is at least one faulty disk, and the second fault condition indicates that there is no faulty disk; the third fault condition is determined based on at least one of the fourth fault condition and the fifth fault condition, including: if the fifth fault condition indicates that there is no faulty disk, then determining that the third fault condition indicates that there is no faulty disk, that is, if the business layer still does not find the faulty disk after re-detection, the business layer can be used as the basis, and it can be directly assumed that there is no faulty disk; or, if the fifth fault condition indicates that there is at least one faulty disk, then determining the third fault condition based on the fifth fault condition and the first fault condition.

[0020] In one possible implementation, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is no faulty disk, and the second fault condition indicates that there is at least one faulty disk; the third fault condition is determined based on at least one of the fourth fault condition and the fifth fault condition, including: if the reference condition is met, the third fault condition is determined according to the second setting method, and the reference condition is met. The fourth fault condition indicates that there is no faulty disk, and the fifth fault condition indicates that there is at least one faulty disk, that is, after detecting the faulty disk again, the result obtained is the same as the previous one, that is, the driver layer did not detect the faulty disk but the business layer detected the faulty disk, then the third fault condition can be determined in a flexible manner; or, if the reference condition is not met, the third fault condition is determined based on the fourth fault condition and the fifth fault condition.

[0021] In one possible implementation, a third fault condition of multiple disks is determined based on the first fault condition and the second fault condition, including: if the faulty disk indicated by the first fault condition and the faulty disk indicated by the second fault condition include the same second disk, then the second disk is used as the faulty disk indicated by the third fault condition.

[0022] In this implementation, it is indicated that the second disk has a higher reliability, and thus the second disk can be used as the failed disk indicated by the third failure condition, making the third failure condition more reliable and accurate.

[0023] In a second aspect, a device for determining a fault condition is provided, the device comprising:

[0024] An acquisition module, used for acquiring a first fault condition of a plurality of disks, the first fault condition being reported after being determined by the driver layer, and the plurality of disks being connected to the PM;

[0025] The acquisition module is further used to acquire a second fault condition of the plurality of disks, where the second fault condition is determined by the business layer according to an average processing time of each disk in the plurality of disks for an IO instruction;

[0026] The determination module is used to determine a third fault condition of the plurality of disks according to the first fault condition and the second fault condition.

[0027] In one possible implementation, the first fault condition indicates that there is no faulty disk, or the first fault condition indicates that there is a faulty disk; for any one of the multiple disks, if any one of the disks is the first disk among the multiple disks whose time difference after the starting determination time is greater than or equal to the first threshold, then any one of the disks is a faulty disk; the time difference of any one of the disks is the difference between the first moment and the second moment, the first moment is the moment when the driver layer adds the first key frame to the controller's sending queue, and the controller is connected to the PM, and the second moment is the moment when the driver layer perceives that the controller has received the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

[0028] In one possible implementation, the first fault condition indicates that there is no faulty disk, or the first fault condition indicates that there is at least one faulty disk; for any one of the multiple disks, if any one of the disks is a disk whose time difference among the multiple disks after the starting determined time is greater than or equal to the second threshold, then any one of the disks is a faulty disk; the time difference of any one of the disks is the difference between a third moment and a fourth moment, the third moment is the moment when the controller sends the first key frame added to the sending queue by the driver layer, and the controller is connected to the PM, and the fourth moment is the moment when the controller receives the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

[0029] In one possible implementation, the second fault condition indicates that there is no faulty disk, or the second fault condition indicates that there is at least one faulty disk; for any one of the multiple disks, if the average processing time of any one of the disks for IO instructions is greater than or equal to a third threshold, then any one of the disks is a faulty disk; wherein the average processing time of any one of the disks for IO instructions is a ratio between a first value and a second value, the first value being the total time for any one of the disks to process IO instructions within a reference time period, and the second value being the total number of IO instructions processed by any one of the disks within the reference time period.

[0030] In a possible implementation, the device further includes: a control module, configured to control the driver layer to redetermine the failure conditions of the multiple disks.

[0031] In one possible implementation, the determination module is also used to, if the third fault condition indicates the existence of at least one faulty disk, delete the disk indicated by the third fault condition from multiple disks to obtain an updated disk group; determine the fault condition of the updated disk group; if the fault condition of the updated disk group indicates the existence of at least one faulty disk, add the faulty disk indicated by the third fault condition to the updated disk group to obtain a recovered disk group, and determine the fault condition of the recovered disk group.

[0032] In one possible implementation, a determination module is used to determine a third fault condition in any one of the following ways if the first fault condition is different from the second fault condition: determine the third fault condition based on at least one of a fourth fault condition and a fifth fault condition, the fourth fault condition being determined by controlling a driver layer, and the fifth fault condition being determined by controlling a business layer; use the second fault condition as the third fault condition; use a faulty disk randomly determined from a plurality of disks as the faulty disk indicated by the third fault condition; and determine the third fault condition according to a reference rule.

[0033] In one possible implementation, the first fault condition is different from the second fault condition, including: the faulty disk indicated by the first fault condition is different from the faulty disk indicated by the second fault condition; a determination module is used to, if the fifth fault condition is the same as the second fault condition, treat the second fault condition as the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is the same first disk as the faulty disk indicated by the first fault condition, treat the first disk as the faulty disk indicated by the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is different from the faulty disk indicated by the first fault condition, determine the third fault condition according to the first setting method.

[0034] In one possible implementation, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is at least one faulty disk, and the second fault condition indicates that there is no faulty disk; a determination module is used to determine that the third fault condition indicates that there is no faulty disk if the fifth fault condition indicates that there is no faulty disk; or, if the fifth fault condition indicates that there is at least one faulty disk, determine the third fault condition based on the fifth fault condition and the first fault condition.

[0035] In one possible implementation, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is no faulty disk, and the second fault condition indicates that there is at least one faulty disk; a determination module is used to determine the third fault condition according to the second setting method if a reference condition is met, and meeting the reference condition includes: the fourth fault condition indicates that there is no faulty disk, and the fifth fault condition indicates that there is at least one faulty disk; or, if the reference condition is not met, the third fault condition is determined according to the fourth fault condition and the fifth fault condition.

[0036] In a possible implementation, the determination module is configured to use the second disk as the failed disk indicated by the third fault condition if the failed disk indicated by the first fault condition and the failed disk indicated by the second fault condition include the same second disk.

[0037] In a third aspect, a device for determining a fault condition is provided, the device comprising a memory and a processor; at least one computer instruction is stored in the memory, and the at least one computer instruction is loaded and executed by the processor, so that the device for determining the fault condition implements the method for determining the fault condition provided by the first aspect or any possible implementation method of the first aspect.

[0038] Optionally, there are one or more processors and one or more memories.

[0039] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0040] In a fourth aspect, a computer-readable storage medium is provided, in which at least one computer instruction is stored, and the at least one computer instruction is loaded and executed by a processor to enable a computer to implement the method for determining a fault condition provided by the first aspect or any possible implementation method of the first aspect.

[0041] In a fifth aspect, a computer program or a computer program product is provided, the computer program or the computer program product comprising: computer instructions, which, when executed by a computer, cause the computer to execute a method for determining a fault condition provided by the first aspect or any possible implementation of the first aspect.

[0042] In a sixth aspect, a chip is provided, comprising a processor for calling and executing instructions stored in a memory, so that a computer equipped with the chip executes a method for determining a fault condition provided by the first aspect or any possible implementation of the first aspect.

[0043] In the seventh aspect, another chip is provided, comprising: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, the processor is used to execute the code in the memory, and when the code is executed, a computer equipped with the chip executes the method for determining a fault condition provided by the first aspect or any possible implementation of the first aspect.

[0044] It should be understood that the technical effects achieved by the technical solutions provided by the second to seventh aspects of the present application and the corresponding possible implementation methods can be referred to the above description of the technical effects achieved by the technical solutions provided by the first aspect and the corresponding possible implementation methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A schematic diagram of disk connection in a non-PM expansion mode provided in an embodiment of the present application;

[0046] Figure 2A schematic diagram of disk connection in a PM expansion mode provided in an embodiment of the present application;

[0047] Figure 3 A flowchart of host-disk interaction provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of a hardware architecture provided for an embodiment of the present application;

[0049] Figure 5 A schematic diagram of another hardware architecture provided for an embodiment of the present application;

[0050] Figure 6 A schematic diagram of a software architecture provided for an embodiment of the present application;

[0051] Figure 7 A flowchart of a method for determining a fault condition provided in an embodiment of the present application;

[0052] Figure 8 A flowchart of an interactive key frame provided in an embodiment of the present application;

[0053] Fig. 9 A schematic diagram of steps performed by a driver layer in a method for determining a fault condition provided in an embodiment of the present application;

[0054] Fig.10 A schematic diagram of steps performed by a business layer in another method for determining a fault condition provided in an embodiment of the present application;

[0055] Fig.11 A schematic diagram of the structure of a device for determining a fault condition provided in an embodiment of the present application;

[0056] Fig.12 A schematic diagram of a device for determining a fault condition provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0058] With the development of computer technology, there are more and more types of disks used for data storage, and serial advanced technology attachment (SATA) disk is one of them. Figure 1 As shown, the SATA disk includes a SATA controller and a disk. The SATA controller includes a SATA port (interface, I / F) that complies with the SATA port specification and is electrically connected to the disk. The electrical connection includes but is not limited to connection through a cable.

[0059] In some embodiments, see Figure 1 , each SATA port is directly connected to a disk through a cable. In this implementation mode (referred to as the standard mode or non-PM extension mode), each SATA port corresponds to one cable and one disk, and each disk has one SATA port. In other implementation modes, such as Figure 2 As shown, the SATA port is connected to the PM through a cable, and the PM is further connected to multiple disks, that is, each SATA port is indirectly connected to the disk through the cable and the PM. Under this implementation method (referred to as the PM expansion method), each SATA port corresponds to one cable, one PM and multiple disks, and multiple disks share one SATA port.

[0060] It can be seen that, compared with the non-PM expansion method, the PM expansion method increases the cost of PM, but it can save both the cost of SATA controllers and the cost of cables, and the cost savings are higher than the cost of adding PM, so the PM expansion method has lower costs and higher cost performance. In other words, when the number of SATA controllers and cables is constant, the PM expansion method can connect more disks, and when the number of disks to be connected is constant, the PM expansion method requires fewer SATA controllers and cables.

[0061] Taking the example that each SATA controller includes 4 SATA ports and each PM can be connected to 4 disks, when 16 disks need to be set up, according to the non-PM expansion method, 4 SATA controllers (including a total of 16 SATA ports) and 16 cables are required, while according to the PM expansion method, only 1 SATA controller (including a total of 4 SATA ports), 4 cables and 4 PMs are required, saving the cost of 3 SATA controllers and 12 cables, and the cost saved is higher than the cost of the additional 4 PMs.

[0062] Because the PM expansion method has a higher cost-effectiveness, SATA disks based on the PM expansion method can be used in a variety of products, such as products that require large-capacity storage (or products that require a large amount of data to be stored, a large storage space, and a large number of disks). Products that require large-capacity storage include but are not limited to: network video recorders (NVRs), storage products, and video surveillance all-in-ones. However, SATA disks based on the PM expansion method also have certain problems, which are explained below.

[0063] The way of interaction between a SATA host and a SATA device includes but is not limited to sending a frame information structure (FIS). The above description involves a SATA controller, a PM, and a disk. Between a SATA controller and a PM, the SATA controller is a SATA host and the PM is a SATA device. Between a PM and a disk, the PM is a SATA host and the disk is a SATA device. Among them, a FIS can have multiple FIS identifications (IDs). Different FIS IDs have different functions. In other words, the FIS ID is used to distinguish different functions of the FIS. For example:

[0064] When the FIS ID is 27H, the FIS can be expressed as FIS 0x27H, which is called register host to device FIS. It plays the role of sending commands to the drive, and the drive is the disk.

[0065] When the FIS ID is 34H, the FIS can be expressed as FIS 0x34H, which is called register device to host FIS, and its function is to respond status to the host.

[0066] When the FIS ID is 41H, the FIS can be expressed as FIS 0x41H, which is called direct memory access (DMA) setup FIS (DMA setup FIS);

[0067] When the FIS ID is 46H, the FIS can be expressed as FIS 0x46H, which is called data FIS and plays the role of transmitting data packets.

[0068] When the FIS ID is A1H, the FIS may be expressed as FIS 0xA1H, which is called a set device bits FIS, and serves to respond to a command completion for a native command queuing (NCQ) command.

[0069] Based on the interactive mode of sending FIS between SATA host and SATA device, the process of synchronously completing multiple instructions in multiple disks connected to SATA controller through PM can be realized. This process is also called NCQ mechanism. The number of instructions that can be synchronously completed through NCQ mechanism is as high as 32. In PM extension mode, PM can add multiple instructions that need to be completed to NCQ (also called first-party DMA (FPDMA) queue) in an orderly manner. The instructions that need to be completed are also called FPDMA queued commands, including but not limited to: read (read) FPDMA queued command for reading data stored on the disk, and write (write) FPDMA queued command for writing data to the disk for storage.

[0070] In addition, the SATA controller also sends FIS to the disk through PM to instruct the disk to add the instructions that need to be completed to the instruction queue (queue command). The process of the SATA controller sending FIS to the disk and the disk adding the instructions to the instruction queue is serial, that is, the SATA controller can send FIS to the next disk only after the SATA controller sends FIS to one disk and the disk adds the instructions to the instruction queue according to the FIS. After all FISs are sent and all the instructions that need to be completed are added to the instruction queue, the data transmission process between the SATA controller and the disk is parallel, that is, multiple disks can synchronously perform data transmission with the SATA controller according to the instructions that need to be completed, thereby realizing the process of synchronously completing multiple instructions in multiple disks connected to the SATA controller through PM, which can improve data transmission efficiency.

[0071] According to the FPDMA queued command protocol in SerialATA_Revision_3_5_Gold (a specification), multiple states are involved in the NCQ mechanism, including but not limited to DFPDMAQ1, DFPDMAQ2, DFPDMAQ12 and DI0, etc. Different states can be converted, as shown in Table 1 and Table 2 below.

[0072] Table 1

[0073]

[0074] Based on the various states shown in Table 1 and Table 2, combined Figure 3Taking the FPDMA queued instruction read FPDMA queuedcommand as an example, the process of the SATA controller reading data from the disk under the NCQ mechanism is explained.

[0075] Step 301, the SATA controller adds the read FPDMA queued command to the FPDMA queue (i.e., appending the command to the internal device command queue in Table 1) in the DFPDMAQ1 state (for adding the command to the queue). The read FPDMA queued command corresponds to a certain tag value, and the tag value is used to distinguish different instructions among the 32 instructions, so as to distinguish which instructions need to be executed immediately and which instructions need to wait for execution. Then, the SATA controller also writes the read FPDMA queued command with a certain tag value into the command register (write command register with read FPDMA queued command with certain tag, i.e., the storage tag value in Table 1). Then, the SATA controller sends FIS 0x27H (i.e., read FPDMA queued command) to the disk.

[0076] In step 302, if the disk successfully receives FIS 0x27H and adds the read FPDMA queued command to the command queue (i.e., the device in Table 1 successfully queues for commands), the disk enters the DFPDMAQ2 state (used to clear the busy state of the interface), indicating that the interface of the disk is no longer busy and can receive a new FIS 0x27H. In addition, the disk returns FIS 0x34H to the SATA controller (i.e., the transfer register device to host FIS in Table 1, and at the same time clears the BSY bit, the DRQ bit, and the interrupt bit to mark that the interface is ready for the next command).

[0077] Alternatively, if the disk fails to successfully receive FIS 0x27H (ie, the error detected command in Table 1), the disk enters the DFPDMAQ12 state (for host damage_clear busy state), indicating that an error exists.

[0078] Step 303, when the disk returns FIS 0x34H to the SATA controller, after receiving FIS 0x34H, the SATA controller confirms that the previous FIS 0x27H has been successfully transmitted (i.e., the FIS transmission in Table 1 is completed), so the SATA controller can enter the DI0 state (for device idle), so that additional commands corresponding to different tag values ​​can be sent (host can issue additional commands with different tags).

[0079] The above steps 301 to 303 are the process of the SATA controller sending a FIS to a disk. When the PM needs to send multiple FIS to multiple disks, the steps 301 to 303 are repeated in series. After all FIS are sent and all read FPDMA queued commands to be completed are added to the command queue, multiple disks execute the following steps 304 to 306 in parallel.

[0080] Step 304, when a disk is ready to service commands in the command queue and send data (service command and send data), the disk sends FIS 0x41H to the SATA controller to inform the SATA controller that it is ready to send data. The SATA controller selects the DMA engine context accordingly to facilitate subsequent data reception (receive data).

[0081] Step 305: The disk sends FIS 0x46H to the SATA controller to send data to the SATA controller, and the SATA controller receives the data accordingly.

[0082] Step 306, after a disk finishes a command, the disk updates task file registers and sends FIS 0xA1H to the SATA controller to inform the SATA controller that the command has been completed, and the SATA controller interrupts the host accordingly.

[0083] As disks are used continuously, they may fail, affecting the security and reliability of data storage. Therefore, it is necessary to determine the failure status of the disk so that the disk can be repaired or replaced when it fails. Related Technology When determining the fault condition of the disk, the processor (including but not limited to the central processing unit (CPU), etc.) sends an IO instruction to the small computer system interface (scsi) layer, and the time of sending the IO instruction is recorded as time A. The scsi layer forwards the IO instruction to the driver layer (also called SATA driver), and the driver layer adds FIS 0x27H to the sending queue of the SATA controller. The SATA controller and PM send FIS 0x27H in sequence. After the disk receives FIS 0x27H, the disk and PM send FIS 0x34H in sequence. The driver layer can perceive that the SATA controller receives FIS 0x34H (including steps 301 to 303 above), and then the disk transmits data according to steps 303 to 306 above. When the data transmission process is completed, the driver layer sends a response to the scsi that the IO instruction has been processed, and the scsi layer sends a response to the processor that the IO instruction has been processed. The time when the response sent by the scsi is received is recorded as time B, and the difference between time B and time A is used as the time length for the disk to process the IO instruction. If this time exceeds the threshold, the disk is considered to have failed.

[0084] It can be seen that the time taken by a disk to process an IO instruction includes the above steps 301 to 303, and according to the above description, it can be known that steps 301 to 303 need to be repeatedly executed in series. Therefore, if the SATA controller sends a FIS to a faulty disk X, and the execution time of steps 301 to 303 corresponding to the disk X is extended due to the fault of the disk X (for example, from a few milliseconds to 6-10 seconds), then not only will the time taken by the disk X to process the IO instruction increase, but the SATA controller will also send the FIS to the next disk later, because the SATA controller needs to wait until the steps 301 to 303 corresponding to the disk X are executed before sending the FIS to the next disk. Accordingly, the next disk also needs to wait for the FIS sent by the SATA controller. Therefore, regardless of whether the next disk fails or not, the time taken by the next disk to process the IO instruction will increase (because it is affected by the disk X). Similarly, the time taken by the subsequent disks to process the IO instruction will also increase due to the influence of the disk X. From the perspective of the upper-level software, these disks are slow and cannot meet the business needs of the upper-level software.

[0085] For example, see Figure 2If the first disk fails and the other three disks are not failed, not only will the time it takes for the first disk to process the IO command increase, but the other three disks will also be affected by the first disk, and the time it takes for the other three disks to process the IO command will also increase.

[0086] It is precisely because the time it takes for a disk to process an IO instruction will be affected by other disks. Therefore, if, according to the relevant technology, a disk is considered to have failed when the time it takes to process an IO instruction exceeds the threshold, there will be misjudgment, because based solely on the time it takes for the disk to process the IO instruction, it is impossible to distinguish whether the disk itself has a fault and the time it takes to process the IO instruction has increased, or whether the disk itself has no fault but is affected by other disks and the time it takes to process the IO instruction has increased. It is not difficult to see that the relevant technology is not accurate enough to determine the fault condition of the disk, and it may still need to rely on manual judgment of the fault condition of the disk.

[0087] In this regard, the embodiment of the present application provides a method for determining a fault condition to improve the accuracy of determining a fault condition of a disk. Before describing the method provided by the embodiment of the present application, the hardware architecture involved in the embodiment of the present application (see Figure 4 and Figure 5 ) and software architecture (see Figure 6 ) for explanation.

[0088] For example, the embodiment of the present application provides a processor, a SATA controller, a PM, and multiple disks in the hardware architecture. The disks include but are not limited to mechanical hard disk drives (HDDs), etc. The embodiment of the present application does not limit the type of disks. In some implementations, such as Figure 4 As shown, a system on chip (SOC) integrates a processor and a SATA controller, the SATA controller is connected to a PM, and the PM is connected to multiple disks. In other embodiments, Figure 5 As shown, the processor and the SATA controller are not integrated into one, the SATA controller is a separate controller, the processor can be connected to the SATA controller through a peripheral component interconnect express (PCIe) bus or the like, the SATA controller is connected to the PM, and the PM is connected to multiple disks.

[0089] In an exemplary embodiment, if Figure 6As shown, the embodiment of the present application provides a business layer and a driver layer located in the operating system in the software architecture, and the business layer and the driver layer determine the fault condition of the disk respectively, so that a more accurate fault condition can be determined by combining the fault conditions of the two disks. Among them, the operating system can be used in conjunction with the processor, and the operating system includes but is not limited to Linux.

[0090] The present application embodiment provides a method for determining a fault condition, which can be executed by the business layer or by other objects (such as software modules or hardware components), and the present application embodiment does not limit this. Take the business layer executing this method as an example, Figure 7 As shown, the method includes the following steps 701 to 703.

[0091] Step 701, obtaining a first fault condition of a plurality of disks, the first fault condition being determined by a driver layer and then reported, and the plurality of disks being connected to a PM.

[0092] In an embodiment of the present application, the driver layer determines (or detects) the first fault conditions of multiple disks and reports the first fault conditions of multiple disks. The business layer can obtain the first fault conditions of multiple disks by receiving the report from the driver layer.

[0093] For example, the embodiment of the present application can set a control bit (slow_control) and a result bit (slow_result), and obtain the first fault conditions of multiple disks through the control bit and the result bit. The control bit is used to indicate the determination and acquisition of the first fault condition, and the result bit is used to indicate the faulty disk when the first fault condition indicates that there is a faulty disk.

[0094] In the initial state, when the control bit is the first value (including but not limited to 0xFF), it is used to indicate that the first fault condition needs to be determined. When the driver layer detects that the control bit is the first value, it starts to determine the first fault condition. Among them, the initial state can be the state entered after the device where the driver layer is located is turned on or restarted, or it can be the state entered after the device performs other execution actions, which is not limited here. If the first fault condition determined by the driver layer indicates that there is no faulty disk, the control bit still maintains the first value, and the driver layer can continue to determine the first fault condition. Alternatively, if the first fault condition determined by the driver layer indicates that there is at least one faulty disk, the driver layer modifies the control bit from the first value to the second value (including but not limited to 0xAA) to indicate that the first fault condition has been determined and the first fault condition indicates that there is at least one faulty disk, and the driver layer also suspends the determination of the first fault condition and writes the disk identifier of the faulty disk into the result bit, so that the result bit is modified from the third value (including but not limited to 0x00) in the initial state to the disk identifier of the faulty disk, and the disk identifier is used to uniquely indicate the faulty disk.

[0095] When the business layer detects that the control bit is the first value, it means that the driver layer has not yet identified any faulty disks, and the first fault conditions of multiple disks obtained by the business layer indicate that there is no faulty disk. Accordingly, the business layer does not need to detect the result bit, nor does it need to modify the control bit and the result bit. When the business layer detects that the control bit is the second value, it means that the driver layer has identified at least one faulty disk, and the first fault conditions of multiple disks obtained by the business layer indicate that there is at least one faulty disk. Accordingly, the business layer detects the result bit, obtains the disk identifier of the faulty disk, and uses the disk uniquely indicated by the disk identifier as the faulty disk. The business layer also modifies the control bit from the second value to the fourth value (including but not limited to 0xBB) to indicate that the result bit has been detected and the disk identifier of the faulty disk has been obtained.

[0096] In addition, the business layer can also modify the control bit from the fourth value to the fifth value (including but not limited to 0x55) to indicate that the driver layer needs to restart determining the fault condition. According to the above description, the driver layer will suspend determining the first fault condition after modifying the control bit to the second value. When the driver layer detects that the control bit is the fifth value, it can restart determining the fault conditions of multiple disks.

[0097] It should be understood that the above first value, second value, third value, fourth value and fifth value are all examples. The embodiment of the present application can also set the control bit and the result bit to other values ​​according to actual needs, as long as the first value, second value, third value, fourth value and fifth value are different.

[0098] In some embodiments, the first fault condition may indicate that there is no faulty disk, that is, each disk among the multiple disks is not faulty, or the first fault condition may also indicate that there is a faulty disk, that is, one disk among the multiple disks is faulty. For any one of the multiple disks, if any one of the multiple disks is the first disk among the multiple disks whose time difference after the start determination time is greater than or equal to the first threshold, then the any one of the disks is a faulty disk. The time difference of any one of the disks is the difference between the first moment and the second moment, the first moment is the moment when the driver layer adds the first key frame to the controller's sending queue, and the controller is connected to the PM, and the second moment is the moment when the driver layer perceives that the controller receives the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

[0099] The controller is the SATA controller mentioned above. For the connection between the SATA controller and the PM, refer to Figure 4 and Figure 5 The corresponding description is sufficient and will not be repeated here. In addition, the starting determination time (or the detection starting point) is the time when the driver layer starts to determine the fault conditions of multiple disks. For example, according to the above description, the time when the initial state is entered can be used as the starting determination time, and the time when the business layer controls the driver layer to restart the determination of the fault condition can also be used as the starting determination time. In other words, there can be multiple starting determination times, so that the driver layer can determine the fault conditions of multiple disks multiple times.

[0100] In the embodiment of the present application, the first moment of a disk is the starting point of the disk processing an IO instruction, and the second moment of a disk is the end point of the disk processing an IO instruction. The first moment and the second moment can be reflected in the form of timestamps. The difference between the first moment and the second moment (that is, the result obtained by subtracting the first moment from the second moment) is used as the time difference of the disk, so that the time difference of a disk can reflect the response time of the disk to the IO instruction. The response time of the disk to the IO instruction is a key link in the process of the disk processing the IO instruction. Compared with the method in the related art that takes the moment when the processor sends the IO instruction to the SCSI layer as the starting point of the disk processing the IO instruction, and takes the moment when the response that the IO instruction sent by the SCSI layer has been processed is received as the end point of processing the IO instruction, the time difference of the disk determined by the embodiment of the present application, that is, the time difference for the disk to process the IO instruction, does not include the waiting time of the SCSI layer, the process time from the SCSI layer to the driver layer, the waiting time of the driver layer, etc., but starts from the time when the driver layer sends the IO instruction to the SATA controller logic (that is, the driver layer starts to add the first key frame to the sending queue of the SATA controller) to the time when the driver layer receives the response to the IO instruction (that is, the time when the driver layer perceives that the SATA controller has received the second key frame). Therefore, the time difference for the disk to process the IO instruction determined by the embodiment of the present application is more accurate, so that the first fault condition determined according to the time difference of the disk is also more accurate.

[0101] For multiple disks under a PM, the driver layer can determine the first moment and the second moment of each disk respectively, and calculate the difference between the first moment and the second moment of each disk to obtain the time difference of each disk. In addition, the driver layer can determine the order in which multiple IO instructions are sent, or the sending of multiple IO instructions is orderly, and each IO instruction corresponds to a disk for processing the IO instruction. Among the ordered multiple IO instructions, if the time difference of the disk is less than the first threshold when the first IO instruction is processed by the corresponding disk, then the disk corresponding to the first IO instruction does not have a fault. If the time difference of the disk is also less than the first threshold when the second IO instruction is processed by the corresponding disk, then the disk corresponding to the second IO instruction does not have a fault either, and so on, until when a certain IO instruction is processed by the corresponding disk, the time difference of the disk processing the IO instruction (that is, the time difference of the disk mentioned above) is greater than or equal to the first threshold for the first time, then the disk corresponding to the IO instruction is the first disk among the multiple disks whose time difference is greater than or equal to the first threshold, and the disk corresponding to the IO instruction has a fault, and the driver layer can regard the disk corresponding to the IO instruction as a faulty disk. In addition, if the time difference of an IO instruction is greater than or equal to the first threshold when being processed by the corresponding disk, but the disk is not the first disk under the PM whose time difference is greater than or equal to the first threshold, that is, there are other disks whose time difference is greater than or equal to the first threshold before this disk, then the driver layer will not treat this disk as a faulty disk.

[0102] Exemplarily, the embodiments of the present application can modify the Linux open source part of the SATA driver (including modules such as libata and libahci) (or modify the open source SATA driver), and the modifications are in compliance with the general public license (GPL), so that the driver layer adds detection of the first moment and the second moment during the process of each disk processing IO instructions, and adds the difference calculation between the first moment and the second moment to obtain the time difference of each disk.

[0103] For example, a first code is added at or near ahci_qc_issue in the Linux open source part, and the first code is used to increase the detection of the first moment by the driver layer. For another example, a second code is added at or near ata_tf_from_fis in the Linux open source part, and the second code is used to increase the detection of the second moment by the driver layer, and the driver layer calculates the difference between the first moment and the second moment to obtain the time difference.

[0104] For example, the first key frame is FIS 0x27H, and the second key frame is FIS 0x34H. Figure 8As shown, the first moment is the moment when the driver layer adds FIS 0x27H to the sending queue of the SATA controller. After that, the SATA controller sends FIS 0x27H to the PM according to the requirements of the SATA protocol, and the PM forwards it to the corresponding disk according to the requirements of the SATA protocol. After receiving FIS 0x27H, the disk sends FIS 0x34H to the PM (the principle is detailed in the description corresponding to steps 301 to 303 above, which will not be repeated here). After that, PM forwards the received FIS 0x34H sent by the disk to the SATA controller according to the requirements of the SATA protocol. The driver layer senses that the SATA controller receives the FIS 0x34H command, which is the second moment. Of course, the embodiment of the present application does not limit the types of the first key frame and the second key frame. The first key frame and the second key frame can also be other types of frames involved in the process of the disk processing the IO instruction.

[0105] It should be understood that in the embodiment of the present application, the first fault condition indicates that there is no faulty disk or there is one faulty disk, that is, the first fault condition indicates at most one faulty disk, and the faulty disk is the first disk among multiple disks whose time difference is greater than or equal to the first threshold. The reason is that in the PM expansion mode, among the multiple disks connected to the same PM, different disks will affect each other. If a disk fails and the time difference of the disk increases (the disk is the first disk whose time difference is greater than or equal to the first threshold), then the time difference of other disks under the same PM (that is, the time for other disks to process IO instructions) will also be affected and cause the time difference of the disk to increase (because different disks need to repeat the above steps 301 to 303 in serial), then for other disks, it is not easy to distinguish whether the time difference of other disks increases due to the failure of other disks themselves, or the time difference of other disks increases due to the influence of the failed disk.

[0106] For example, see Figure 8 , assuming that the driver layer adds IO instructions for multiple disks to the sending queue of the SATA controller at the same time, the time difference of disk 0 is T0, and the time difference of disk 1 is T1 (including T0, affected by disk 0), the time difference of disk 2 is T2 (including T1 and T0, affected by disk 1 and disk 0), and the time difference of disk 3 is T3 (including T2, T1 and T0, affected by disk 2, disk 1 and disk 0). If the driver layer determines that the time difference T0 of disk 0 is greater than or equal to the first threshold, disk 0 is regarded as a faulty disk indicated by the first fault condition, and disk 1, disk 2 and disk 3 are not regarded as faulty disks.

[0107] It is precisely because of the reasons described above that other disks are ignored in the embodiment of the present application, so that the first fault condition indicates at most one faulty disk, and the faulty disk is the first disk among multiple disks whose time difference is greater than or equal to the first threshold value. It can be seen that the first fault condition determined by the driver layer is based on the single processing process of the disk to the IO instruction, rather than the average statistics of the multiple processing processes of the disk to the IO instruction, but the performance of the disk in the process of processing the IO instruction is often more complicated, or in other words, there is a certain contingency in the single processing process of the disk to the IO instruction. Therefore, it may not be easy to set the above first threshold accurately, so that when the disk does not have a fault, the time difference of the disk is also greater than or equal to the first threshold value. In other words, even if the time difference of the disk is greater than or equal to the first threshold value, it cannot be confirmed that the disk must have a fault, and the first fault condition determined by the driver layer may still be inaccurate. In this regard, in the embodiment of the present application, the driver layer needs to report the first fault condition to the business layer. After obtaining the second fault conditions of multiple disks, the business layer combines the first fault condition with the second fault condition to improve the accuracy of determining the fault condition, as described in step 702 below.

[0108] In other embodiments, the first fault condition may indicate that there is no faulty disk, that is, each disk among the multiple disks is not faulty, or the first fault condition may also indicate that there is at least one faulty disk, that is, one or more disks among the multiple disks are faulty. For any one of the multiple disks, if any one of the multiple disks has a time difference greater than or equal to a second threshold value after the start determination time, then the any one of the multiple disks is a faulty disk. The time difference of any one of the disks is the difference between the third moment and the fourth moment, the third moment is the moment when the controller sends the first key frame added to the sending queue by the driver layer, and the controller is connected to the PM, and the fourth moment is the moment when the controller receives the second key frame, and the second key frame is sent by any one of the disks according to the received first key frame.

[0109] Among them, the controller, the first key frame and the second key frame can refer to the description in the previous implementation, and will not be repeated here. For multiple disks under a PM, the driver layer can determine the third moment and the fourth moment of each disk respectively, and calculate the difference between the third moment and the fourth moment of each disk (that is, the result obtained by subtracting the third moment from the fourth moment) to obtain the time difference of each disk. If the time difference of the disk is less than the second threshold when an IO instruction is processed by the corresponding disk, the disk corresponding to the IO instruction does not have a fault. If the time difference of the disk is greater than or equal to the second threshold when an IO instruction is processed by the corresponding disk, the disk corresponding to the IO instruction has a fault.

[0110] In this embodiment, since the time difference of the disk (i.e., the time difference of the disk processing the IO instruction) is no longer determined according to the first moment when the driver layer adds the first key frame to the sending queue of the SATA controller and the second moment when the driver layer senses that the SATA controller receives the second key frame as in the previous embodiment, but the time difference of the disk is determined according to the third moment when the SATA controller sends the first key frame and the fourth moment when the SATA controller receives the second key frame, the time difference of the disk determined in this embodiment is more accurate, and the time differences of different disks will not interfere with each other. Therefore, in this embodiment, it is not necessary to make the first fault condition indicate at most one faulty disk and the faulty disk is the first disk among multiple disks whose time difference is greater than or equal to the first threshold as a faulty disk, but the first fault condition can indicate at least one faulty disk. Of course, for this embodiment, the driver layer can also report the first fault condition to the business layer. After obtaining the second fault conditions of multiple disks, the business layer combines the first fault condition with the second fault condition to improve the accuracy of determining the fault condition, as described in step 702 below.

[0111] Step 702: obtaining a second fault condition of the plurality of disks, where the second fault condition is determined by the business layer according to an average processing time of each disk in the plurality of disks for an IO instruction.

[0112] In an embodiment of the present application, the business layer determines the second fault condition based on the average processing time of each disk among the multiple disks for the IO instruction. The second fault condition may indicate that there is no faulty disk, that is, each disk among the multiple disks is not faulty, or the second fault condition may also indicate that there is at least one faulty disk, that is, one or more disks among the multiple disks are faulty.

[0113] In an exemplary embodiment, for any one of the multiple disks, if the average processing time of any one of the disks for an IO instruction is greater than or equal to a third threshold, then the any one of the disks is a faulty disk. The average processing time of any one of the disks for an IO instruction is a ratio between a first value and a second value, the first value being the total processing time of any one of the disks for an IO instruction within a reference period, and the second value being the total number of IO instructions processed by any one of the disks within the reference period.

[0114] For each of the multiple disks, the business layer can determine the first value and the second value of each disk respectively, and calculate the ratio between the first value and the second value of each disk to obtain the average processing time of each disk for IO instructions during the reference period. Exemplarily, the business layer can determine the first value and the second value of each disk through the cmd_age or / proc / diskstats of the scsi layer, which is not limited here. If the average processing time of a disk for IO instructions is less than the third threshold, the disk is not faulty. If the average processing time of a disk for IO instructions is greater than or equal to the third threshold, the disk is faulty and the business layer can treat the disk as a faulty disk.

[0115] It can be seen that the second fault condition determined by the business layer is not based on the single processing process of the disk to the IO instruction, but based on the average statistics of the multiple processing processes of the disk to the IO instruction, thereby avoiding the randomness of the single processing process of the disk to the IO. Compared with the first fault condition determined by the driver layer, the second fault condition determined by the business layer is more accurate, or in other words, if the second fault condition determined by the business layer indicates at least one faulty disk, it is more likely that the indicated faulty disk actually has a fault. However, the at least one faulty disk indicated by the second fault condition determined by the business layer may include two types of disks, one is a disk that actually has a fault, and the other is a disk that does not have a fault but is affected by the faulty disk. It is not easy for the business layer to directly distinguish between these two types of disks. Therefore, the embodiment of the present application combines the first fault condition with the second fault condition to determine a more accurate third fault condition. See the description in step 703 below for details.

[0116] It should be understood that the embodiment of the present application does not limit the execution order of step 701 and step 702. That is, step 701 can be executed first and then step 702, or step 702 can be executed first and then step 701, or step 701 and step 802 can be executed synchronously. Exemplarily, step 702 can be executed first, and when the second fault condition of the multiple disks obtained in step 702 indicates that there is at least one faulty disk, step 701 is executed. Correspondingly, when the second fault condition of the multiple disks obtained in step 702 indicates that there is no faulty disk, the first fault condition reported by the driver layer in step 701 can be ignored, and the driver layer can be controlled to re-determine the fault conditions of the multiple disks. For example, the business layer can reset the starting determination time when the second fault condition indicates that there is no faulty disk, and when it does not determine the faulty disk again within a subsequent period of time, so as to control the driver layer to re-determine the fault conditions of the multiple disks.

[0117] Of course, the embodiment of the present application does not limit the timing of controlling the driver layer to redetermine the fault conditions of multiple disks. After the first fault condition of multiple disks has been obtained, the driver layer can be controlled to redetermine the fault conditions of multiple disks at any time according to actual needs. For example, according to the above description, by modifying the control bit to 0x55, the driver layer redetermines the fault conditions of multiple disks. Therefore, in an exemplary embodiment, after obtaining the first fault condition of multiple disks, the method provided by the embodiment of the present application may also include: controlling the driver layer to redetermine the fault conditions of multiple disks.

[0118] Still according to the above description, it can be known that the first fault condition determined by the driver layer is based on the single processing process of the disk to the IO instruction, and because of the influence between different disks, the first fault condition determined by the driver layer may indicate at most one faulty disk, and the faulty disk is the first disk among multiple disks whose time difference is greater than or equal to the first threshold. If the driver layer is not controlled to redetermine the fault condition, the driver layer may only have a chance to obtain one first fault condition during one operation of the device where the driver layer is located. Controlling the driver layer to redetermine the fault condition is conducive to the driver layer determining multiple fault conditions, and multiple fault conditions can cooperate with the fault conditions determined by the business layer, so that during one operation of the device where the driver layer is located, the accurate fault condition can be determined through the cooperation between the driver layer and the business layer for multiple times.

[0119] Step 703: Determine a third fault condition of the plurality of disks according to the first fault condition and the second fault condition.

[0120] Among them, the third fault condition of multiple disks may indicate that there is no faulty disk among the multiple disks. In this case, the above steps 701 to 703 can be repeatedly performed, or other methods for determining the fault condition can be performed to continue to determine the fault condition of the disk during the subsequent use of the disk. Alternatively, the third fault condition of multiple disks may also indicate that there is at least one faulty disk among the multiple disks. In this case, a prompt message can be issued to prompt the user to cancel the use (including but not limited to unplugging) of at least one faulty disk indicated by the third fault condition, or the at least one faulty disk indicated by the third fault condition can be automatically isolated to prevent the faulty disk from affecting the security and reliability of data storage. Exemplarily, the automatic isolation method includes but is not limited to deleting from the SCSI layer, such as deleting at least one faulty disk indicated by the third fault condition from the SCSI layer through a SCSI remove (remove) instruction. This isolation method is more thorough and complete than isolating the faulty disk only in the upper-layer software.

[0121] Next, a method for determining the third fault condition of multiple disks according to the first fault condition and the second fault condition is described, including but not limited to the following methods 1 to 3.

[0122] Mode 1: If the first fault condition indicates that there is no faulty disk, and the second fault condition indicates that there is no faulty disk (referred to as condition 1), it is determined that the third fault condition indicates that there is no faulty disk.

[0123] In the case that neither the driver layer nor the business layer detects a faulty disk, it is sufficient to determine that there is no faulty disk among the multiple disks.

[0124] Mode 2: If the failed disk indicated by the first fault condition and the failed disk indicated by the second fault condition include the same second disk, the second disk is used as the failed disk indicated by the third fault condition.

[0125] When the faulty disk detected by the driver layer overlaps with the faulty disk detected by the service layer (ie, the same second disk), the same second disk is used as the faulty disk, so that the third fault condition indicates that the second disk is the faulty disk.

[0126] Exemplarily, the failed disk indicated by the first fault condition and the failed disk indicated by the second fault condition are the same second disk, including: the first fault condition indicates that there are one or more failed disks, the second fault condition indicates that there are one or more failed disks, and the failed disk indicated by the first fault condition and the failed disk indicated by the second fault condition both include the same second disk (referred to as Case 2). For example, the first fault condition indicates that there is a failed disk, and the second fault condition indicates that there is a failed disk, and both of these failed disks are the second disk. For another example, the first fault condition indicates that there is a failed disk, and this failed disk is the second disk, and the second fault condition indicates that there are multiple failed disks, and the multiple failed disks include the second disk, which are not listed one by one here.

[0127] Mode 3: If the first fault condition is different from the second fault condition, the third fault condition is determined in any one of the following ways.

[0128] In method a of method 3, the second fault condition is used as the third fault condition. According to the description in step 702 above, the second fault condition determined by the business layer is more accurate than the first fault condition determined by the driver layer. Therefore, when the first fault condition is different from the second fault condition, the second fault condition can be used as the standard and directly used as the third fault condition.

[0129] In method b of method 3, a failed disk determined according to a certain specific rule from among multiple disks is used as a failed disk indicated by the third failure condition. For example, a set of failed disks indicated by the first failure condition and failed disks indicated by the second failure condition is determined from among multiple disks, and at least one failed disk is determined from the set according to a certain specific rule as a failed disk indicated by the third failure condition.

[0130] In method c of method 3, the third fault condition is determined according to the reference rule. For example, the third fault condition is determined by switching to other fault condition determination methods, the third fault condition is determined according to the historical fault conditions of each disk in the plurality of disks, the third fault condition is determined manually, etc. The third fault condition is not limited in the embodiment of the present application.

[0131] Method d in Method 3 determines the third fault condition according to at least one of the fourth fault condition and the fifth fault condition, the fourth fault condition is determined by controlling the driver layer, and the fifth fault condition is determined by controlling the business layer. The fourth fault condition may be one or more (i.e., the driver layer may perform one or more tests), and the fifth fault condition may also be one or more (i.e., the business layer may perform one or more tests continuously). Compared with the above methods a, b, and c, when the first fault condition is different from the second fault condition, the method d enables at least one of the driver layer and the business layer to re-detect the faulty disk and re-determine the fault condition, so as to determine the third fault condition according to the re-determined fault condition (i.e., the fourth fault condition and the fifth fault condition), thereby further eliminating contingency and improving the accuracy of the determined third fault condition.

[0132] Exemplarily, the second fault condition is determined by the business layer for a reference period. When the business layer needs to determine the fifth fault condition, the business layer can select an update period different from the reference period, and determine the fifth fault condition for the update period. The reference period and the update period may or may not overlap. In the embodiment of the present application, a time window can be set on the time axis, and an update period can be selected by sliding the time window based on the reference period. In addition, the way in which the business layer controls the driver layer to restart determining the fault condition has been described in step 701 above and will not be repeated here.

[0133] In the embodiment of the present application, the first fault condition is different from the second fault condition and can be further divided into multiple conditions. The implementation method of method d in different conditions may also be different, as described below for details.

[0134] In some implementations, the first fault condition is different from the second fault condition, including: the faulty disk indicated by the first fault condition is different from the faulty disk indicated by the second fault condition (referred to as condition 3). The first fault condition indicates the existence of one or more faulty disks, the second fault condition indicates the existence of one or more faulty disks, and the faulty disks indicated by the first fault condition and the faulty disks indicated by the second fault condition do not have any identical disks. For example, the first fault condition indicates the existence of one faulty disk, and the second fault condition indicates the existence of one faulty disk, but the two faulty disks are different. For another example, the first fault condition indicates the existence of one faulty disk, and the second fault condition indicates the existence of multiple faulty disks, but none of the multiple faulty disks is the same as the faulty disk indicated by the first fault condition, and they are not listed one by one here.

[0135] Method d corresponding to situation 3 includes: if the fifth fault condition is the same as the second fault condition, then the second fault condition is taken as the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is the same first disk as the faulty disk indicated by the first fault condition, then the first disk is taken as the faulty disk indicated by the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is different from the faulty disk indicated by the first fault condition, then the third fault condition is determined according to the first setting method.

[0136] That is to say, when both the driver layer and the business layer have detected a faulty disk, but the detected faulty disks do not overlap at all, the business layer re-determines the fault condition and obtains the fifth fault condition. If the fifth fault condition is the same as the second fault condition, it means that the second fault condition has eliminated contingency and has sufficient reliability, so the second fault condition (i.e., the fifth fault condition) can be directly used as the third fault condition. If the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition includes the same first disk as the faulty disk indicated by the first fault condition (equivalent to entering the above situation 2), it means that the same first disk has a certain reliability, and the same first disk can be used as the faulty disk indicated by the third fault condition. If the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is different from the faulty disk indicated by the first fault condition, this situation is relatively rare, and the third fault condition can be determined according to the first setting method. Exemplarily, the first setting method includes at least one of method a, method b, method c, and method d, and the embodiment of the present application does not limit the first setting method.

[0137] In some other implementations, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is at least one faulty disk, and the second fault condition indicates that there is no faulty disk (referred to as condition 4).

[0138] Method d corresponding to situation 4 includes: if the fifth fault condition indicates that there is no faulty disk, determining that the third fault condition indicates that there is no faulty disk; or, if the fifth fault condition indicates that there is at least one faulty disk, determining the third fault condition based on the fifth fault condition and the first fault condition.

[0139] That is to say, when the driver layer detects a faulty disk but the business layer does not detect a faulty disk, if the fifth fault condition indicates that there is no faulty disk, that is, the business layer still does not detect the faulty disk after re-detection, then it can be determined that the result that there is no faulty disk is reliable, and thus it is determined that the third fault condition indicates that there is no faulty disk. Alternatively, if the fifth fault condition indicates that there is a faulty disk, that is, the business layer detects the faulty disk after re-detection, then the fifth fault condition and the first fault condition may enter the above situation 2 or situation 3, and the third fault condition can be determined in a manner corresponding to the situation entered, which has been explained above and will not be repeated here.

[0140] In some further implementations, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is no faulty disk, and the second fault condition indicates that there is at least one faulty disk (referred to as condition 5).

[0141] Method d corresponding to situation 5 includes: if the reference condition is met, the third fault condition is determined according to the second setting method, and meeting the reference condition includes: the fourth fault condition indicates that there is no faulty disk, and the fifth fault condition indicates that there is at least one faulty disk; or, if the reference condition is not met, the third fault condition is determined based on the fourth fault condition and the fifth fault condition.

[0142] That is to say, when the driver layer does not detect the faulty disk, but the business layer detects the faulty disk, the fourth fault condition indicates that there is no faulty disk, and the fifth fault condition indicates that there is at least one faulty disk, that is, the driver layer still does not detect the faulty disk after re-detection, the fourth fault condition is the same as the first fault condition, and the business layer still detects at least one faulty disk after re-detection, and the fifth condition is the same as the second fault condition. If the reference condition is met, that is, the driver layer and the business layer still stay in condition 5 after re-detection, the third fault condition is determined according to the second setting method. Exemplarily, the second setting method includes at least one of method a, method b, method c and method d, and the second setting method can be the same or different from the first setting method above, and the embodiment of the present application does not limit this. Alternatively, if the reference condition is not met, that is, it does not stay in condition 5, but enters condition 1, condition 2, condition 3 or condition 4 above, the third fault condition can be determined in the manner corresponding to the entered condition, which has been explained above and will not be repeated here.

[0143] In addition, regardless of the above methods or situations, the embodiments of the present application can record logs, which can be used to subsequently improve the method of determining the fault situation and the method of handling various fault situations.

[0144] So far, the business layer determines the third fault condition of the multiple disks according to the first fault condition and the second fault condition. After determining the third fault condition, the business layer may continue to perform other steps.

[0145] In an exemplary embodiment, after determining the third fault condition of multiple disks according to the first fault condition and the second fault condition, the method further includes: controlling the driver layer to determine the fault condition of the reference disk. According to the above description, when the third fault condition indicates that there is at least one disk, the faulty disk indicated by the third fault condition will be deleted from the multiple disks. In this case, the reference disk includes the remaining disks after deleting the faulty disk indicated by the third fault condition from the multiple disks. Of course, if the third fault condition indicates that there is no faulty disk, there is no need to delete the disk, and the reference disk is the multiple disks. Exemplarily, the business layer can control the driver layer to determine the fault condition of the reference disk when the third fault condition is obtained and the faulty disk is not detected within a specified time period, so as to trigger repeated execution of steps 701 to 703.

[0146] In an exemplary embodiment, after determining the third fault condition of multiple disks based on the first fault condition and the second fault condition, the method also includes: if the third fault condition indicates that there is at least one failed disk, deleting the disk indicated by the third fault condition from the multiple disks to obtain an updated disk group; determining the fault condition of the updated disk group; if the fault condition of the updated disk group indicates that there is at least one failed disk, adding the failed disk indicated by the third fault condition to the updated disk group to obtain a recovered disk group, and determining the fault condition of the recovered disk group.

[0147] After obtaining the updated disk group, the business layer continues to detect and obtain the fault condition of the updated disk group. If the fault condition of the updated disk group indicates that there is still at least one faulty disk in the disk group, it may be that there is still a faulty disk in the updated disk group, indicating that the faulty disk indicated by the third fault condition previously determined may not be accurate enough, for example, there may not be a fault. Therefore, the business layer can retry once: add the deleted faulty disk indicated by the third fault condition back to the updated disk group, for example, at least one faulty disk indicated by the third fault condition can be re-added to the updated disk group through the scsi add command, so as to restore the updated disk group to the restored multiple disk groups, and then determine the fault condition of the restored multiple disk groups, thereby further improving the accuracy of determining the fault condition. Exemplarily, the business layer can perform the retry process multiple times, for example, the number of times the business layer repeats the process can be equal to the number of multiple disks connected to the PM.

[0148] According to the above description of the method, in the embodiment of the present application, the exemplary steps performed by the driver layer can be seen in Fig. 9 That is, when the first disk among multiple disks whose time difference is greater than or equal to the first threshold is detected, the disk is reported to the business layer as a faulty disk. Afterwards, if the driver layer receives an instruction from the business layer to restart the detection, it restarts the detection; otherwise, it ensures to wait, that is, suspend the detection. The steps executed by the business layer can be seen in Fig.10 When the business layer detects a faulty disk, it combines the faulty disk reported by the driver layer to jointly determine the exact faulty disk and isolate the exact faulty disk. The business layer can also ignore the faulty disk reported by the driver layer if no faulty disk is detected, and control the driver layer to restart detection if the business layer still fails to detect the faulty disk within a period of time.

[0149] The above is an overall description of the method for determining a fault condition provided in the embodiment of the present application. Below, the method for determining a fault condition is further described with examples.

[0150] 1. After the device is powered on or restarted, the default value of slow_control is 0xFF, which is a starting point. The driver layer starts to detect whether there is a faulty disk among multiple disks. If a disk is detected whose first time difference (the second moment when the driver layer perceives that the SATA controller receives the FIS 0x34H sent by the disk, minus the first moment when the driver layer adds the FIS 0x27H to the sending queue of the SATA controller) is greater than or equal to the first threshold, slow_control is set to 0xAA, subsequent detection is suspended, and the faulty disk detected by the driver layer is filled into slow_result.

[0151] 2. After the device is powered on or restarted, the business layer starts to run synchronously. The business layer calculates the average processing time of the IO instructions of each disk in the reference period. If there is a disk whose average processing time exceeds the third threshold (for example, 100 milliseconds), it is regarded as a faulty disk determined by the business layer. If slow_control has been set to 0xAA, the result of slow_result is read to obtain the faulty disk detected by the driver layer, and slow_control is set to 0xBB. Alternatively, even if the business layer has not determined the faulty disk, and if slow_control has been set to 0xAA, the result of slow_result can be cleared and slow_control can be set to 0xBB. On this basis, if the business layer continues to fail to determine the faulty disk (for example, the business layer has not detected a disk with an average processing time exceeding 100 milliseconds for 50 consecutive cycles), and slow_control has been set to 0xBB, slow_control can be modified from 0xBB to 0x55 to control the driver layer to restart determining the fault condition.

[0152] 3. The business layer determines the third failure condition of multiple disks according to the first failure condition and the second failure condition as follows.

[0153] 1. If neither the driver layer nor the business layer detects a faulty disk, the third fault condition indicates that there is no faulty disk.

[0154] 2. The driver layer detects a faulty disk, the business layer detects a faulty disk, and the two faulty disks are the same, then it is determined that the faulty disk is the same faulty disk.

[0155] 3. The driver layer detects a faulty disk, which is one of the multiple faulty disks detected by the business layer, and determines that the faulty disk is the same faulty disk.

[0156] 4. The driver layer detects a faulty disk, and the business layer detects a faulty disk, but there is no identical faulty disk. Then the faulty disk detected by the business layer later is determined again. If the faulty disk detected by the business layer N times in a row is the same, then the faulty disk reported by the driver layer can be ignored, and the faulty disk detected by the business layer is used as the faulty disk. If the faulty disks detected by the business layer N times in a row are different, but there is a disk that is the same as the faulty disk detected by the driver layer, then the same disk is used as the faulty disk. If the faulty disks detected by the business layer N times in a row are different, and there is no disk that is the same as the faulty disk detected by the driver layer, there can be multiple implementation plans. For example, the business layer can perform N detections again through a sliding window, the business layer can also arbitrarily determine a faulty disk, the business layer can also set the starting determination time of the driver layer, obtain the newly detected faulty disk of the driver layer, and so on. Not all of them are enumerated here. In addition, logs can be recorded to further improve the processing plan later.

[0157] 5. If the driver layer detects a faulty disk and the business layer detects multiple faulty disks, but there is no identical faulty disk, the faulty disk detected by the business layer will be re-determined. If the faulty disk detected by the business layer for N consecutive times does not contain any disk identical to the faulty disk detected by the driver layer, there are multiple implementation options. For example, the business layer can perform N detections again through a sliding window, the business layer can also arbitrarily determine a faulty disk, the business layer can also set the starting determination time of the driver layer, obtain the newly detected faulty disk by the driver layer, and so on. In addition, logs can be recorded to further improve the processing plan later.

[0158] 6. If the driver layer detects a faulty disk, but the business layer does not, then record the status and wait for the next round of faulty disk detection by the business layer to confirm again. If the business layer does not detect the faulty disk after N consecutive detections, it is considered that the driver layer misjudged and the faulty disk result reported by the driver layer is ignored. Set slow_control to 0x55 to notify the driver layer to start re-detection.

[0159] 7. If the driver layer does not detect a faulty disk, but the business layer detects one or more faulty disks, the detection result of the business layer can be used directly, or the judgment can be made again based on the faulty disks detected by the business layer later. If the situation falls into the first six categories, the corresponding situation will be handled accordingly. If the situation falls into none of the first six categories, certain rules can be used for judgment. In other words, this scenario can only record logs to facilitate further improvement of the handling plan.

[0160] 4. After the exact faulty disk is identified, the business layer will completely delete the faulty disk from the SCSI layer through the SCSI remove command, thereby preventing the faulty disk from affecting other disks under the PM where the faulty disk is located. In some embodiments, the business layer is the upper layer software of the driver layer. For example, the business layer is software for managing redundant arrays of independent disks (RAID), and if the RAID status is normal, the RAID can mark the identified faulty disk as faulty, and the RAID status changes from normal to degraded. In other embodiments, the business layer is not RAID, and there is no need to perform the marking and status modification steps.

[0161] 5. If the business layer does not detect a faulty disk within a period of time, it can be considered that the disks during this period are all non-faulty disks and the disk access is normal. It can be considered as a new starting time. The business layer sets slow_control to 0x55 to notify the driver layer to start re-detection, thereby starting the next round of repetition of one to five.

[0162] 6. If the business layer completely deletes the identified faulty disk from the SCSI layer, and then detects a new faulty disk under the PM where the faulty disk is located, then perhaps a new faulty disk has appeared under the PM, or perhaps the previously identified faulty disk is incorrect. Therefore, the business layer can retry, restore the previously identified faulty disk through the SCSI add command, re-detect the faulty disk, and perform the operations described above. Exemplarily, the business layer retries M times and then ends, where M is the number of disks connected to the PM. In some embodiments, the business layer is RAID, and after restoring the previously identified faulty disk through the SCSI add command, an additional RAID fast reconstruction is required.

[0163] In the above one to six, the interaction between the business layer and the driver layer is involved, and the interaction is exemplified by the following Table 3. It should be understood that the number of the interaction file slow control is one and the number of the interaction file slow result is four in the following Table 3 for example only, and the embodiment of the present application can also make the number of the interaction file slow control four and the number of the interaction file slow result one, and the number of interaction files can change with the number of PMs and the number of disks connected to the PMs.

[0164] Table 3

[0165]

[0166] To summarize, in the PM expansion scenario, the embodiment of the present application not only obtains the first fault condition determined and reported by the driver layer, but also obtains the second fault condition determined by the business layer based on the average processing time of each disk for IO instructions, so that a more accurate third fault condition can be determined by combining the first fault condition and the second fault condition, thereby improving the accuracy of determining the faulty disk and ensuring the security and reliability of data storage.

[0167] The above describes the method for determining a fault condition provided by the embodiment of the present application. Corresponding to the above method, the embodiment of the present application also provides a device for determining a fault condition. The device is used to Fig.11 The modules shown above perform the Figure 7 The method for determining fault conditions is shown in Fig.11 As shown, the device for determining a fault condition provided in an embodiment of the present application includes the following modules.

[0168] An acquisition module 1101 is used to acquire a first fault condition of a plurality of disks, the first fault condition is determined by the driver layer and then reported, and the plurality of disks are connected to the PM;

[0169] The acquisition module 1101 is further used to acquire a second fault condition of the plurality of disks, where the second fault condition is determined by the business layer according to an average processing time of each of the plurality of disks for an IO instruction;

[0170] The determination module 1102 is configured to determine a third fault condition of the plurality of disks according to the first fault condition and the second fault condition.

[0171] In one possible implementation, the first fault condition indicates that there is no faulty disk, or the first fault condition indicates that there is a faulty disk; for any one of the multiple disks, if any one of the disks is the first disk among the multiple disks whose time difference after the starting determination time is greater than or equal to the first threshold, then any one of the disks is a faulty disk; the time difference of any one of the disks is the difference between the first moment and the second moment, the first moment is the moment when the driver layer adds the first key frame to the controller's sending queue, and the controller is connected to the PM, and the second moment is the moment when the driver layer perceives that the controller has received the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

[0172] In one possible implementation, the first fault condition indicates that there is no faulty disk, or the first fault condition indicates that there is at least one faulty disk; for any one of the multiple disks, if any one of the disks is a disk whose time difference among the multiple disks after the starting determined time is greater than or equal to the second threshold, then any one of the disks is a faulty disk; the time difference of any one of the disks is the difference between a third moment and a fourth moment, the third moment is the moment when the controller sends the first key frame added to the sending queue by the driver layer, and the controller is connected to the PM, and the fourth moment is the moment when the controller receives the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

[0173] In one possible implementation, the second fault condition indicates that there is no faulty disk, or the second fault condition indicates that there is at least one faulty disk; for any one of the multiple disks, if the average processing time of any one of the disks for IO instructions is greater than or equal to a third threshold, then any one of the disks is a faulty disk; wherein the average processing time of any one of the disks for IO instructions is a ratio between a first value and a second value, the first value being the total time for any one of the disks to process IO instructions within a reference time period, and the second value being the total number of IO instructions processed by any one of the disks within the reference time period.

[0174] In a possible implementation, the device further includes: a control module, configured to control the driver layer to redetermine the failure conditions of the multiple disks.

[0175] In one possible implementation, the determination module 1102 is also used to, if the third fault condition indicates that there is at least one faulty disk, delete the disk indicated by the third fault condition from multiple disks to obtain an updated disk group; determine the fault condition of the updated disk group; if the fault condition of the updated disk group indicates that there is at least one faulty disk, add the faulty disk indicated by the third fault condition to the updated disk group to obtain a recovered disk group, and determine the fault condition of the recovered disk group.

[0176] In one possible implementation, the determination module 1102 is used to determine the third fault condition if the first fault condition is different from the second fault condition in any one of the following ways: determine the third fault condition based on at least one of the fourth fault condition and the fifth fault condition, the fourth fault condition being determined by controlling the driver layer, and the fifth fault condition being determined by controlling the business layer; taking the second fault condition as the third fault condition; taking a faulty disk randomly determined from multiple disks as the faulty disk indicated by the third fault condition; and determining the third fault condition according to a reference rule.

[0177] In one possible implementation, the first fault condition is different from the second fault condition, including: the faulty disk indicated by the first fault condition is different from the faulty disk indicated by the second fault condition; a determination module 1102 is used to, if the fifth fault condition is the same as the second fault condition, treat the second fault condition as the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is the same first disk as the faulty disk indicated by the first fault condition, treat the first disk as the faulty disk indicated by the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is different from the faulty disk indicated by the first fault condition, determine the third fault condition according to the first setting method.

[0178] In one possible implementation, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is at least one faulty disk, and the second fault condition indicates that there is no faulty disk; a determination module 1102 is used to determine that the third fault condition indicates that there is no faulty disk if the fifth fault condition indicates that there is no faulty disk; or, if the fifth fault condition indicates that there is at least one faulty disk, determine the third fault condition based on the fifth fault condition and the first fault condition.

[0179] In one possible implementation, the first fault condition is different from the second fault condition, including: the first fault condition indicates that there is no faulty disk, and the second fault condition indicates that there is at least one faulty disk; the determination module 1102 is used to determine the third fault condition according to the second setting method if the reference condition is met, and the reference condition is met. The fourth fault condition indicates that there is no faulty disk, and the fifth fault condition indicates that there is at least one faulty disk; or, if the reference condition is not met, the third fault condition is determined according to the fourth fault condition and the fifth fault condition.

[0180] In a possible implementation, the determination module 1102 is configured to use the second disk as the failed disk indicated by the third fault condition if the failed disk indicated by the first fault condition and the failed disk indicated by the second fault condition include the same second disk.

[0181] It should be understood that the above Fig.11 The device shown in the figure has the following beneficial effects when realizing its function: Figure 7 The method of determining a fault condition shown has the same beneficial effects. Fig.11When the device shown implements its functions, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0182] See also Fig.12 , Fig.12 A schematic structural diagram of an exemplary device 1200 for determining a fault condition of the present application is shown. The device 1200 for determining a fault condition includes at least one processor 1201 , a memory 1203 , and at least one network interface 1204 .

[0183] Processor 1201 is, for example, a general-purpose central processing unit (CPU), a digital signal processor (DSP), a network processor (NP), a graphics processing unit (GPU), a neural-network processing units (NPU), a data processing unit (DPU), a microprocessor or one or more integrated circuits or application-specific integrated circuits (ASICs) for implementing the solution of the present application, a programmable logic device (PLD), other general-purpose processors or other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components or any combination thereof. PLD is, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting an advanced RISC machines (ARM) architecture. It can implement or execute various logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0184] Optionally, the device 1200 for determining a fault condition further includes a bus 1202. The bus 1202 is used to transmit information between components of the device 1200 for determining a fault condition. The bus 1202 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus 1202 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 The fact that only one line is used in the diagram does not mean that there is only one bus or only one type of bus.

[0185] The memory 1203 is, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache.

[0186] By way of example but not limitation, many forms of ROM and RAM are available. For example, ROM is a compact disc read-only memory (CD-ROM). RAM includes, but is not limited to, static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0187] The memory 1203 may also be other types of storage devices that can store static information and instructions. Or it may be other types of dynamic storage devices that can store information and instructions. Or it may be other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, blue-ray disk, etc.), magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by the computer, but is not limited to this. The memory 1203, for example, exists independently and is connected to the processor 1201 through the bus 1202. The memory 1203 may also be integrated with the processor 1201.

[0188] The network interface 1204 uses any transceiver-like device to communicate with other devices or communication networks, and the communication network can be Ethernet, radio access network (RAN) or wireless local area network (WLAN), etc. The network interface 1204 may include a wired network interface and may also include a wireless network interface. Specifically, the network interface 1204 may be an Ethernet interface, such as a fast Ethernet (FE) interface, a gigabit Ethernet (GE) interface, an asynchronous transfer mode (ATM) interface, a WLAN interface, a cellular network interface or a combination thereof. The Ethernet interface may be an optical interface, an electrical interface or a combination thereof. In some embodiments of the present application, the network interface 1204 may be used to determine that the device 1200 in the fault condition communicates with other devices.

[0189] In a specific implementation, as some embodiments, the processor 1201 may include one or more CPUs, such as Fig.12 0 and CPU1 shown in FIG. Each of these processors may be a single-core processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0190] In a specific implementation, as some embodiments, the device 1200 for determining a fault condition may include multiple processors, such as Fig.12 1 and 1205. Each of these processors may be a single-core processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0191] In some implementations, the memory 1203 is used to store program instructions 1210 for executing the solution of the present application, and the processor 1201 can execute the program instructions 1210 stored in the memory 1203. That is, the device 1200 for determining a fault condition can implement the method provided in the method embodiment through the processor 1201 and the program instructions 1210 in the memory 1203, that is, Figure 7 The method shown. The program instruction 1210 may include one or more software modules. Optionally, the processor 1201 itself may also store program instructions for executing the solution of the present application.

[0192] In a specific implementation process, the device 1200 for determining a fault condition of the present application may correspond to a device where a module in the form of software for executing the above method is located, or a component in the form of hardware for executing the above method. The processor 1201 in the device 1200 for determining a fault condition reads the instructions in the memory 1203, so that Fig.12 The device 1200 for determining a fault condition shown can perform all or part of the steps in the method embodiment.

[0193] The device 1200 for determining a fault condition may also correspond to the above Fig.11 The device shown, Fig.11 Each functional module in the illustrated apparatus is implemented using software of the apparatus 1200 for determining a fault condition. In other words, Fig.11 The functional modules included in the illustrated apparatus are generated after the processor 1201 of the device 1200 for determining a fault condition reads the program instructions 1210 stored in the memory 1203 .

[0194] in, Figure 7 Each step of the method shown is completed by an integrated logic circuit of hardware or software instructions in the processor of the device 1200 that determines the fault condition. The steps of the method embodiment disclosed in the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method embodiment in combination with its hardware. To avoid repetition, it is not described in detail here.

[0195] In an exemplary embodiment, a computer-readable storage medium is provided, wherein at least one computer instruction is stored in the computer-readable storage medium, and the at least one computer instruction is loaded and executed by a processor to enable a computer to implement Figure 7 The method for determining fault conditions is shown.

[0196] Exemplarily, a computer program or a computer program product is provided, the computer program or the computer program product comprising: computer instructions, when the computer instructions are executed by a computer, causing the computer to execute Figure 7 The method for determining fault conditions is shown.

[0197] In an exemplary embodiment, a chip is provided, including a processor for calling and executing instructions stored in a memory, so that a computer equipped with the chip executes Figure 7 The method for determining fault conditions is shown.

[0198] Exemplarily, another chip is provided, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the computer equipped with the chip executes Figure 7 The method for determining fault conditions is shown.

[0199] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk), etc.

[0200] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items with substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another.

[0201] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0202] The term "at least one" in this application means one or more, and the term "multiple" in this application means two or more, for example, multiple disks means two or more disks. The terms "system" and "network" are often used interchangeably herein.

[0203] It should be understood that the terms used in the description of the various examples herein are only for describing specific examples and are not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0204] It should also be understood that the term "and / or" used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "and / or" is a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0205] It should also be understood that the terms "if" and "if" may be interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting." Similarly, the phrases "if it is determined that ..." or "if [a stated condition or event] is detected" may be interpreted to mean "upon determining that ..." or "in response to determining that ..." or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0206] The above description is only an embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining a fault condition, characterized in that: The method comprises: Acquire a first fault condition of a plurality of disks, the first fault condition being reported after being determined by a driver layer, the plurality of disks being connected to a port multiplexer PM; Obtain a second fault condition of the multiple disks, where the second fault condition is determined by the business layer according to an average processing time of each disk in the multiple disks for an input and output IO instruction; A third failure condition of the plurality of disks is determined based on the first failure condition and the second failure condition.

2. The method according to claim 1, characterized in that The first failure condition indicates that there is no failed disk, or the first failure condition indicates that there is a failed disk; For any one of the multiple disks, if the any one of the multiple disks is the first disk among the multiple disks whose time difference after the start determination time is greater than or equal to the first threshold, then the any one of the disks is a failed disk; The time difference of any one of the disks is the difference between the first moment and the second moment, the first moment being the moment when the driver layer adds the first key frame to the sending queue of the controller, and the controller is connected to the PM, and the second moment is the moment when the driver layer senses that the controller has received the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

3. The method according to claim 1, characterized in that The first failure condition indicates that there is no failed disk, or the first failure condition indicates that there is at least one failed disk; For any one of the multiple disks, if the time difference of the any one disk after the start determination time is greater than or equal to the second threshold among the multiple disks, then the any one disk is a failed disk; The time difference of any one of the disks is the difference between the third moment and the fourth moment, the third moment is the moment when the controller sends the first key frame added to the sending queue by the driver layer, the controller is connected to the PM, and the fourth moment is the moment when the controller receives the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

4. The method according to any one of claims 1 to 3, characterized in that: The second failure condition indicates that there is no failed disk, or the second failure condition indicates that there is at least one failed disk; For any one of the multiple disks, if the average processing time of the IO instruction by the any one of the disks is greater than or equal to a third threshold, the any one of the disks is a faulty disk; Among them, the average processing time of any one disk for IO instructions is the ratio between a first value and a second value, the first value is the total time for any one disk to process IO instructions within a reference time period, and the second value is the total number of IO instructions processed by any one disk within the reference time period.

5. The method according to any one of claims 1 to 4, characterized in that: After obtaining the first failure conditions of the plurality of disks, the method further includes: The driver layer is controlled to re-determine the failure status of the plurality of disks.

6. The method according to any one of claims 1 to 5, characterized in that: After determining a third fault condition of the plurality of disks according to the first fault condition and the second fault condition, the method further comprises: If the third fault condition indicates that there is at least one faulty disk, deleting the disk indicated by the third fault condition from the plurality of disks to obtain an updated disk group; Determining a failure condition of the updated disk group; If the fault condition of the updated disk group indicates that there is at least one faulty disk, the faulty disk indicated by the third fault condition is added to the updated disk group to obtain a restored disk group, and the fault condition of the restored disk group is determined.

7. The method according to any one of claims 1 to 6, characterized in that: Determining a third fault condition of the plurality of disks according to the first fault condition and the second fault condition comprises: If the first fault condition is different from the second fault condition, the third fault condition is determined in any one of the following ways: Determine the third fault condition according to at least one of a fourth fault condition and a fifth fault condition, wherein the fourth fault condition is determined by controlling the driver layer, and the fifth fault condition is determined by controlling the service layer; Taking the second fault condition as the third fault condition; using a faulty disk randomly determined from the plurality of disks as a faulty disk indicated by the third fault condition; The third fault case is determined according to a reference rule.

8. The method according to claim 7, characterized in that The first fault condition is different from the second fault condition, including: the faulty disk indicated by the first fault condition is different from the faulty disk indicated by the second fault condition; Determining the third fault condition according to at least one of the fourth fault condition and the fifth fault condition comprises: If the fifth fault condition is the same as the second fault condition, the second fault condition is used as the third fault condition; Alternatively, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition and the faulty disk indicated by the first fault condition are the same first disk, the first disk is used as the faulty disk indicated by the third fault condition; Alternatively, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is different from the faulty disk indicated by the first fault condition, the third fault condition is determined according to the first setting method.

9. The method according to claim 7, characterized in that: The first fault condition is different from the second fault condition, including: the first fault condition indicates that there is at least one failed disk, and the second fault condition indicates that there is no failed disk; Determining the third fault condition according to at least one of the fourth fault condition and the fifth fault condition comprises: If the fifth fault condition indicates that there is no failed disk, determining that the third fault condition indicates that there is no failed disk; Alternatively, if the fifth fault condition indicates that there is at least one failed disk, the third fault condition is determined based on the fifth fault condition and the first fault condition.

10. The method according to claim 7, characterized in that The first fault condition is different from the second fault condition, including: the first fault condition indicates that there is no failed disk, and the second fault condition indicates that there is at least one failed disk; Determining the third fault condition according to at least one of the fourth fault condition and the fifth fault condition comprises: If the reference condition is met, the third fault condition is determined according to the second setting mode, wherein the reference condition is met including: the fourth fault condition indicates that there is no faulty disk, and the fifth fault condition indicates that there is at least one faulty disk; Alternatively, if the reference condition is not met, the third fault condition is determined according to the fourth fault condition and the fifth fault condition.

11. The method according to any one of claims 1 to 6, characterized in that: Determining a third fault condition of the plurality of disks according to the first fault condition and the second fault condition comprises: If the failed disk indicated by the first fault condition and the failed disk indicated by the second fault condition include the same second disk, the second disk is used as the failed disk indicated by the third fault condition.

12. A device for determining a fault condition, characterized in that: The device comprises: An acquisition module, used for acquiring a first fault condition of a plurality of disks, the first fault condition being reported after being determined by the driver layer, the plurality of disks being connected to the port multiplexer PM; The acquisition module is further used to acquire a second fault condition of the multiple disks, where the second fault condition is determined by the business layer according to an average processing time of each disk in the multiple disks for an input and output IO instruction; A determination module is used to determine a third fault condition of the plurality of disks according to the first fault condition and the second fault condition.

13. The device according to claim 12, characterized in that The first failure condition indicates that there is no failed disk, or the first failure condition indicates that there is a failed disk; For any one of the multiple disks, if the any one of the multiple disks is the first disk among the multiple disks whose time difference after the start determination time is greater than or equal to the first threshold, then the any one of the disks is a failed disk; The time difference of any one of the disks is the difference between the first moment and the second moment, the first moment being the moment when the driver layer adds the first key frame to the sending queue of the controller, and the controller is connected to the PM, and the second moment is the moment when the driver layer senses that the controller has received the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

14. The device according to claim 12, characterized in that The first failure condition indicates that there is no failed disk, or the first failure condition indicates that there is at least one failed disk; For any one of the multiple disks, if the time difference of the any one disk after the start determination time is greater than or equal to the second threshold among the multiple disks, then the any one disk is a failed disk; The time difference of any one of the disks is the difference between the third moment and the fourth moment, the third moment is the moment when the controller sends the first key frame added to the sending queue by the driver layer, the controller is connected to the PM, and the fourth moment is the moment when the controller receives the second key frame, and the second key frame is sent by any one of the disks based on the received first key frame.

15. The device according to any one of claims 12 to 14, characterized in that: The second failure condition indicates that there is no failed disk, or the second failure condition indicates that there is at least one failed disk; For any one of the multiple disks, if the average processing time of the IO instruction by the any one of the disks is greater than or equal to a third threshold, the any one of the disks is a faulty disk; Among them, the average processing time of any one disk for IO instructions is the ratio between a first value and a second value, the first value is the total time for any one disk to process IO instructions within a reference time period, and the second value is the total number of IO instructions processed by any one disk within the reference time period.

16. The device according to any one of claims 12 to 15, characterized in that: The device also includes: The control module is used to control the driver layer to redetermine the failure conditions of the multiple disks.

17. The device according to any one of claims 12 to 16, characterized in that: The determination module is also used to, if the third fault condition indicates that there is at least one faulty disk, delete the disk indicated by the third fault condition from the multiple disks to obtain an updated disk group; determine the fault condition of the updated disk group; if the fault condition of the updated disk group indicates that there is at least one faulty disk, add the faulty disk indicated by the third fault condition to the updated disk group to obtain a recovered disk group, and determine the fault condition of the recovered disk group.

18. The device according to any one of claims 12 to 17, characterized in that: The determining module is configured to determine the third fault condition in any one of the following ways if the first fault condition is different from the second fault condition: Determine the third fault condition according to at least one of a fourth fault condition and a fifth fault condition, wherein the fourth fault condition is determined by controlling the driver layer, and the fifth fault condition is determined by controlling the service layer; Taking the second fault condition as the third fault condition; using a faulty disk randomly determined from the plurality of disks as a faulty disk indicated by the third fault condition; The third fault case is determined according to a reference rule.

19. The device according to claim 18, characterized in that The first fault condition is different from the second fault condition, including: the faulty disk indicated by the first fault condition is different from the faulty disk indicated by the second fault condition; The determination module is used to, if the fifth fault condition is the same as the second fault condition, use the second fault condition as the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is the same first disk as the faulty disk indicated by the first fault condition, use the first disk as the faulty disk indicated by the third fault condition; or, if the fifth fault condition is different from the second fault condition, and the faulty disk indicated by the fifth fault condition is different from the faulty disk indicated by the first fault condition, determine the third fault condition according to the first setting method.

20. The device according to claim 18, characterized in that The first fault condition is different from the second fault condition, including: the first fault condition indicates that there is at least one failed disk, and the second fault condition indicates that there is no failed disk; The determination module is used to determine that the third fault condition indicates that there is no faulty disk if the fifth fault condition indicates that there is no faulty disk; or, if the fifth fault condition indicates that there is at least one faulty disk, determine the third fault condition based on the fifth fault condition and the first fault condition.

21. The device according to claim 18, characterized in that The first fault condition is different from the second fault condition, including: the first fault condition indicates that there is no failed disk, and the second fault condition indicates that there is at least one failed disk; The determination module is used to determine the third fault condition according to the second setting method if a reference condition is met, and the reference condition is met including: the fourth fault condition indicates that there is no faulty disk, and the fifth fault condition indicates that there is at least one faulty disk; or, if the reference condition is not met, determine the third fault condition according to the fourth fault condition and the fifth fault condition.

22. The device according to any one of claims 12 to 17, characterized in that: The determination module is configured to use the second disk as the failed disk indicated by the third fault condition if the failed disk indicated by the first fault condition and the failed disk indicated by the second fault condition include the same second disk.

23. A device for determining a fault condition, characterized in that The device includes a memory and a processor; the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor, so that the device for determining the fault condition implements the method for determining the fault condition described in any one of claims 1-11.

24. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer instruction, and the at least one computer instruction is loaded and executed by a processor to enable a computer to implement the method for determining a fault condition as described in any one of claims 1-11.

25. A computer program product, characterized in that The computer program product comprises computer instructions, which are executed by a processor to enable a computer to implement the method for determining a fault condition as claimed in any one of claims 1 to 11.

Images