Resource pool fault locating method and device

Abstract

The application provides a resource pool fault positioning method and device, relates to the technical field of cloud computing, and comprises the following steps: determining that a current fault occurs in a resource pool based on the frequency of abnormal events of the resource pool in a current time slice; determining a fault topology corresponding to the current time slice based on the abnormal events in the current time slice and a device topology corresponding to the current time slice of the resource pool; merging the fault topologies corresponding to a plurality of time slices corresponding to the current fault to determine a fault topology corresponding to the current fault; and determining a root cause device corresponding to the current fault based on the influence degree of each device in the fault topology corresponding to the current fault on the remaining devices in the resource pool. The method and device provided by the application improve the efficiency of resource pool fault positioning and improve the timeliness and accuracy of resource pool fault positioning.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cloud computing, and in particular to a resource pool fault locating method and device. BACKGROUND

[0002] The development of cloud computing technology changes the deployment architecture of the original business system. By pooling physical resources such as computing, storage, and network, the on-demand acquisition and flexible migration of resources by the business are realized, thereby improving the utilization of physical resources and the high availability of the business. The resource pool corresponding to cloud computing includes multiple professional and multi-vendor physical devices and virtual devices such as computing, storage, network, security, and virtualization. The failure of a device usually triggers alarms or other abnormal events of the opposite device or cross-level physical devices and virtual devices, which causes difficulties in fault detection and localization.

[0003] The related art needs to rely on operation and maintenance engineers with operation and maintenance experience and professional skills to locate the fault of the resource pool corresponding to cloud computing. The efficiency of fault location is low, and the timeliness and accuracy of fault location cannot be achieved. SUMMARY

[0004] The present application provides a resource pool fault locating method and device to solve the technical problem of how to improve the efficiency of fault location and improve the timeliness and accuracy of fault location.

[0005] The present application provides a resource pool fault locating method, comprising:

[0006] determining that the resource pool has a current fault based on the abnormal event frequency of the resource pool in a current time slice;

[0007] determining a fault topology corresponding to the current time slice based on the abnormal events in the current time slice and the device topology of the resource pool corresponding to the current time slice;

[0008] combining the fault topologies corresponding to the multiple time slices of the current fault to determine a fault topology corresponding to the current fault;

[0009] determining a root cause device corresponding to the current fault based on the influence of each device in the fault topology corresponding to the current fault on the remaining devices in the resource pool.

[0010] In some embodiments, the determining that the resource pool has a current fault based on the abnormal event frequency of the resource pool in a current time slice comprises:

[0011] obtaining the abnormal event frequency of the resource pool in the current time slice;

[0012] If the frequency of abnormal events in the current time slice is greater than a preset frequency threshold, it is determined that the resource pool has experienced the current failure.

[0013] The preset frequency threshold is determined based on the mean and standard deviation of the frequency of abnormal events occurring in the resource pool during normal operation.

[0014] In some embodiments, determining the fault topology corresponding to the current time slice based on the abnormal events within the current time slice and the device topology of the resource pool corresponding to the current time slice includes:

[0015] Obtain the devices corresponding to the resource pool in the current time slice and the corresponding relationships between the devices;

[0016] Based on the devices and the correspondence between them, the device topology of the resource pool corresponding to the current time slice is constructed;

[0017] Based on the abnormal events within the current time slice, the devices in the device topology are labeled to obtain the fault topology corresponding to the current time slice.

[0018] In some embodiments, merging the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault includes:

[0019] Determine multiple time slices corresponding to the current fault;

[0020] The fault topology corresponding to the first time slice among the plurality of time slices is taken as the initial fault topology corresponding to the current fault.

[0021] The first device corresponding to the remaining time slices in the plurality of time slices is incorporated into the initial fault topology to obtain the fault topology corresponding to the current fault.

[0022] The first device is the device that experienced an abnormal event in the remaining time slice.

[0023] In some embodiments, incorporating the first device corresponding to the remaining time slices in the plurality of time slices into the initial fault topology to obtain the fault topology corresponding to the current fault includes:

[0024] Determine the current time slice to be merged;

[0025] Determine the first set of devices and the second set of devices corresponding to the current time slice to be merged;

[0026] If the first set of devices intersects with the initial fault topology, or if the first set of devices does not intersect with the initial fault topology but the second set of devices intersects with the initial fault topology, the devices in the first set of devices are incorporated into the initial fault topology to obtain an updated initial fault topology.

[0027] If the current time slice to be merged is the last remaining time slice, the updated initial fault topology will be used as the fault topology corresponding to the current fault.

[0028] The second device is a device adjacent to the first device that does not experience any abnormal events.

[0029] In some embodiments, determining the root cause device corresponding to the current fault based on the impact of each device in the fault topology corresponding to the current fault on the remaining devices in the resource pool includes:

[0030] Based on the fault topology corresponding to the current fault, determine the device to be analyzed.

[0031] Based on the degree of influence of the current device to be analyzed on the other devices in the resource pool, determine the set of devices that are affected and the set of devices that are not affected corresponding to the current device to be analyzed.

[0032] If the object device of the abnormal event belongs to the set of affected devices, increase the influence degree coefficient of the device currently being analyzed;

[0033] If the device involved in the abnormal event belongs to the set of unaffected devices, reduce the impact coefficient of the device currently being analyzed.

[0034] The device corresponding to the maximum value of the influence coefficient in the fault topology corresponding to the current fault is determined as the root cause device corresponding to the current fault.

[0035] In some embodiments, determining the set of affected devices and the set of non-affected devices corresponding to the current device to be analyzed based on the degree of influence of the current device to be analyzed on the other devices in the resource pool includes:

[0036] Based on the propagation relationship of abnormal events, the remaining devices in the resource pool are grouped to determine the set of affected devices and the set of non-affected devices corresponding to the device to be analyzed.

[0037] This application provides a resource pool fault location device, including:

[0038] The fault diagnosis module is used to determine the current fault of the resource pool based on the frequency of abnormal events in the current time slice.

[0039] The topology generation module is used to determine the fault topology corresponding to the current time slice based on the abnormal events in the current time slice and the device topology of the resource pool in the current time slice.

[0040] The slice merging module is used to merge the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault.

[0041] The fault location module is used to determine the root cause device corresponding to the current fault based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool.

[0042] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the resource pool fault location method.

[0043] This application provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the resource pool fault location method described above.

[0044] The resource pool fault location method and apparatus provided in this application determine the current fault in the resource pool based on the frequency of abnormal events in the current time slice; determine the fault topology corresponding to the current time slice based on the abnormal events within the current time slice and the device topology corresponding to the resource pool in the current time slice; merge the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault; and determine the root cause device corresponding to the current fault based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool. This method enables fault location of the resource pool without relying on the operation and maintenance experience and professional knowledge of operation and maintenance engineers, but rather based on the frequency of abnormal events, device topology, and the degree of influence of each device on the other devices in the resource pool. This reduces errors from human judgment, achieves rapid identification and accurate location of resource pool faults, improves the efficiency, timeliness, and accuracy of resource pool fault location. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0046] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is one of the flowcharts of the resource pool fault location method provided in this application.

[0048] Figure 2 This is a schematic diagram of the fault topology provided in this application.

[0049] Figure 3 This is one of the fault topology diagrams provided in this application after merging time slices.

[0050] Figure 4 This is the second schematic diagram of the fault topology after merging time slices provided in this application.

[0051] Figure 5 This is the third schematic diagram of the fault topology after merging time slices provided in this application.

[0052] Figure 6 This is a schematic diagram illustrating the selection of time slices provided in this application.

[0053] Figure 7 This is a schematic diagram of the root cause device cluster matching provided in this application.

[0054] Figure 8 This is a schematic diagram of the abnormal event propagation relationship provided in this application.

[0055] Figure 9 This is the second flowchart of the resource pool fault location method provided in this application.

[0056] Figure 10 This is a schematic diagram of the structure of the resource pool fault location device provided in this application.

[0057] Figure 11 This is a schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0058] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0059] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, units, or modules is not necessarily limited to those explicitly listed, but may include other steps, units, or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0060] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of personal information all comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and they do not violate public order and good morals.

[0061] During the operation of a cloud computing resource pool, there may be a continuous stream of known anomalies (such as confirmed version defects, planned network operations, etc.) or warning messages (such as account expiration, license capacity, etc.). These types of messages are often interspersed with anomalies caused by sudden failures, interfering with fault diagnosis. For example, if a host in a resource pool goes down, it may trigger 36 related alarms, including 32 alarms from 6 switches, 2 alarms from 1 host, and 2 alarms from storage. Simultaneously, there may be 21 alarms unrelated to this specific failure during the same period.

[0062] The relevant technologies for identifying and locating resource pool faults mainly rely on the human experience and professional skills of operation and maintenance engineers: 1) Monitoring resource pool alarms and abnormal events. When unconfirmed alarms or abnormal events occur, the occurrence of faults can be determined, and simple faults can be located; 2) When a large number of abnormal events and alarms occur across disciplines and devices, operation and maintenance engineers need to locate faulty devices based on their professional knowledge and experience, following the timeline of alarms or abnormal events.

[0063] Relying entirely on operations engineers with experience and expertise to identify and locate faults in the resource pool affects the efficiency of fault location and fails to meet the timeliness and accuracy requirements of fault identification and location.

[0064] In order to address the shortcomings of related technologies, Figure 1 This is one of the flowcharts illustrating the resource pool fault location method provided in this application, such as... Figure 1 As shown, the method includes steps 110, 120, 130 and 140.

[0065] Step 110: Based on the frequency of abnormal events in the resource pool during the current time slice, determine the current failure of the resource pool.

[0066] Specifically, the execution subject of the resource pool fault location method provided in this application embodiment is a resource pool fault location device. This device can be implemented by software, such as a resource pool fault location program; or it can be a device that executes the resource pool fault location method, such as a mobile terminal, tablet computer, desktop computer, or server.

[0067] A resource pool is a mechanism or entity in the field of computer and information technology that centrally manages and allocates available resources. A resource pool can include various types of resources, such as computing resources (e.g., processors, memory), storage resources (e.g., hard disk space), network bandwidth, software licenses, virtual machine instances, container instances, etc. These resources can also be divided into physical resources and virtual resources. Physical resources refer to actual existing hardware resources. Virtual resources refer to resources obtained through software virtualization based on physical resources.

[0068] Fault identification refers to determining whether a resource pool has experienced or is experiencing a fault. Fault localization refers to identifying the root cause device among a large number of device objects involved in the abnormal events that occur during the fault occurrence process, and which abnormal event is the root cause abnormal event. The root cause device refers to the device or system involved in the fundamental cause of the abnormal events in the resource pool.

[0069] Anomalies typically refer to unusual or abnormal events or situations that occur during system operation. These anomalies may originate from software, hardware, or environmental factors and can affect the stability, performance, or security of the system. Anomaly frequency refers to the number of times an anomaly occurs.

[0070] A time slice refers to a time segment used for analyzing and statistically analyzing anomalous events. For example, the duration of a time slice can be determined with a granularity of 1 minute.

[0071] The duration of each time slice can be determined, and the frequency of abnormal events in the resource pool across each time slice can be statistically analyzed. If the frequency of abnormal events in the current time slice exceeds a preset value or changes rapidly, the resource pool can be considered to have failed. Considering that the resource pool may fail multiple times, the failure occurring in the current time slice can be defined as the current failure (also known as the current fault).

[0072] Step 120: Based on the abnormal events in the current time slice and the device topology of the resource pool in the current time slice, determine the fault topology corresponding to the current time slice.

[0073] Specifically, in the field of computer networks and communications, topology refers to the structure and layout of the physical or logical connections between devices, describing how devices are connected and how data is transmitted between them.

[0074] Device topology can be constructed based on physical devices, virtual devices, and the relationships between devices (including dynamic mapping and connection relationships) in a resource pool. For example, in daily operations and maintenance, the device topology of a resource pool can be generated through the resource pool configuration management database (CMDB). The generated device topology can also be called the dynamic three-dimensional topology of the resource pool. "Dynamic" means that the device relationships within the resource pool need to be updated in real time, such as the relationship between hosts and virtual machines; therefore, the device topology is updated in real time. The device topology corresponding to the current time slice may be different from that of the previous time slice. "Three-dimensional" means that the device topology includes devices at various levels within the resource pool, such as physical devices, virtual devices, and business systems.

[0075] Fault topology refers to the device topology that includes device information corresponding to abnormal events. It can identify the target devices (devices that generated or propagated abnormal events, etc.) corresponding to each abnormal event within the current time slice, and mark them in the resource pool according to the device topology of the current time slice to obtain the fault topology for the current time slice.

[0076] Step 130: Merge the fault topologies corresponding to multiple time slices for the current fault to determine the fault topology corresponding to the current fault.

[0077] Specifically, abnormal events caused by resource pool failures usually last for a long time and are generally not fully output within a single time slice. Therefore, there may be multiple time slices, either consecutive or discontinuous, that belong to the same failure. In practice, it is necessary to merge multiple time slices that belong to the same failure.

[0078] Merging multiple time slices is essentially merging the fault topologies corresponding to those time slices. The devices experiencing anomalous events in the fault topologies of the various time slices corresponding to the current fault may be different. These devices could all be the root cause devices for the current fault. Therefore, the fault topologies of the multiple time slices corresponding to the current fault can be merged to obtain a final fault topology that includes the devices that generated the anomalous events in the current fault.

[0079] Step 140: Based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool, determine the root cause device corresponding to the current fault.

[0080] Specifically, taking any device in the fault topology corresponding to the current fault as an example, this device may affect the other devices in the resource pool, or it may not affect the other devices in the resource pool. The greater the impact, the more likely the device is to be the root cause device corresponding to the current fault; the smaller the impact, the less likely the device is to be the root cause device corresponding to the current fault.

[0081] The system can identify the target device corresponding to the abnormal event and determine whether that target device is an affected device (affected by any device in the fault topology corresponding to the current fault). The degree of impact of that device on the remaining devices in the resource pool is determined based on the number of affected devices. It is understood that the greater the degree of impact, the more likely that device is to be the root cause device for the current fault.

[0082] The resource pool fault location method provided in this application determines the current fault in the resource pool based on the frequency of abnormal events in the current time slice; it determines the fault topology corresponding to the current time slice based on the abnormal events within the current time slice and the device topology of the resource pool in the current time slice; it merges the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault; and it determines the root cause device corresponding to the current fault based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool. This method enables fault location of the resource pool without relying on the operation and maintenance experience and professional knowledge of operation and maintenance engineers, but rather based on the frequency of abnormal events, device topology, and the degree of influence of each device on the other devices in the resource pool. This reduces errors from human judgment, achieves rapid identification and accurate location of resource pool faults, improves the efficiency, timeliness, and accuracy of resource pool fault location.

[0083] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0084] In some embodiments, determining the current failure of a resource pool based on the frequency of abnormal events in the current time slice includes:

[0085] Get the frequency of abnormal events in the resource pool during the current time slice;

[0086] If the frequency of abnormal events in the current time slice exceeds a preset frequency threshold, it is determined that the current failure has occurred in the resource pool.

[0087] The preset frequency threshold is determined based on the mean and standard deviation of the frequency of abnormal events that occur during the normal operation of the resource pool.

[0088] Specifically, the frequency (number of abnormal events per unit time) of abnormal events (alarms, logs, etc.) during normal operation of the resource pool (no faults, no network disruptions) can be statistically analyzed to obtain statistical characteristics. These statistical characteristics are usually relatively stable. Methods such as the KS test and histograms can be used to determine if they conform to a normal distribution. Based on the frequency of the statistically analyzed abnormal events, the mean can be calculated. and standard deviation .

[0089] Get the resource pool in the current time slice Frequency of abnormal events The k-sigma algorithm can be used to determine a preset frequency threshold to identify the frequency of abnormal events within a time slice. For example, if the k value in the k-sigma algorithm is 3, then the preset frequency threshold can be obtained as follows: If the current time slice Frequency of abnormal events Greater than the preset frequency threshold, i.e. This indicates that the frequency of abnormal events in the current time slice is high, and the resource pool has experienced its current failure. The value of k can be adjusted based on actual operational conditions.

[0090] The resource pool fault location method provided in this application determines a preset frequency threshold based on the mean and standard deviation of the frequency of abnormal events occurring during normal operation of the resource pool. The frequency of abnormal events in the current time slice is compared with the preset frequency threshold to determine the current fault in the resource pool, thereby improving the timeliness and accuracy of resource pool fault location.

[0091] In some embodiments, based on abnormal events within the current time slice and the device topology of the resource pool corresponding to the current time slice, the fault topology corresponding to the current time slice is determined, including:

[0092] Obtain the devices corresponding to the resource pool in the current time slice and the corresponding relationships between the devices;

[0093] Based on the devices and the correspondence between them, construct the device topology of the resource pool corresponding to the current time slice;

[0094] Based on the abnormal events within the current time slice, the devices in the device topology are labeled to obtain the fault topology corresponding to the current time slice.

[0095] Specifically, Figure 2 This is a schematic diagram of the fault topology provided in this application, such as... Figure 2As shown, a resource pool includes devices such as virtual hard disks (vDisk0, vDisk1, and vDisk2), storage management devices (DSM0, DSM1, and DSM2), switches (MTOR, STTOR, and TTOR), hosts (Host0, Host1, and Host2), and virtual machines (vm0, vm1, and vm2). The solid lines in the diagram represent the correspondence between devices. These correspondences can include connection relationships, data transfer relationships, and dynamic mapping relationships. Solid boxes in the diagram represent devices marked with abnormal events (having abnormalities), while dashed boxes represent devices not marked with abnormal events (no abnormalities).

[0096] Based on the devices and their corresponding relationships, the device topology corresponding to the resource pool in the current time slice can be constructed. Abnormal events typically involve target devices. By marking the target devices involved in abnormal events within the current time slice in the constructed device topology, the fault topology corresponding to the current time slice can be obtained.

[0097] The fault topology not only represents the devices and their correspondences in the resource pool corresponding to the current time slice, but also the device information related to the abnormal event.

[0098] The resource pool fault location method provided in this application determines the fault topology corresponding to the current time slice based on the abnormal events in the current time slice and the device topology of the resource pool in the current time slice. This method can accurately represent the devices in the resource pool, the device correspondence, and the devices involved in the abnormal events, thereby improving the timeliness and accuracy of resource pool fault location.

[0099] In some embodiments, the fault topologies corresponding to multiple time slices corresponding to the current fault are merged to determine the fault topology corresponding to the current fault, including:

[0100] Determine the multiple time slices corresponding to the current fault;

[0101] The fault topology corresponding to the first time slice among multiple time slices is taken as the initial fault topology corresponding to the current fault.

[0102] The first device corresponding to the remaining time slices in multiple time slices is incorporated into the initial fault topology to obtain the fault topology corresponding to the current fault.

[0103] The first device is the device that experiences an abnormal event in the remaining time slices.

[0104] Specifically, the current fault usually persists across multiple time slices, and there may be situations where the current fault exists in multiple consecutive or discontinuous time slices.

[0105] The fault topology corresponding to the first time slice out of multiple time slices is taken as the initial fault topology corresponding to the current fault. The first device is defined as the device that has an abnormal event in the remaining time slices; the second device is the device that is adjacent to the first device (and has a corresponding relationship in the device topology) and does not have an abnormal event.

[0106] The first device corresponding to the other time slices in multiple time slices may be associated with the device in the initial fault topology corresponding to the current fault. In other words, it may be the root cause device that caused the abnormal event. Therefore, the first device can be incorporated into the initial fault topology to merge multiple time slices and obtain the fault topology corresponding to the current fault.

[0107] The resource pool fault location method provided in this application merges the fault topologies corresponding to multiple time slices for the current fault to determine the fault topology corresponding to the current fault. This method can comprehensively consider all possible root cause devices, thereby improving the timeliness and accuracy of resource pool fault location.

[0108] In some embodiments, the first devices corresponding to the remaining time slices in a plurality of time slices are incorporated into the initial fault topology to obtain the fault topology corresponding to the current fault, including:

[0109] Determine the current time slice to be merged;

[0110] Determine the first and second device sets corresponding to the current time slice to be merged;

[0111] If the first set of devices intersects with the initial fault topology, or if the first set of devices does not intersect with the initial fault topology but the second set of devices intersects with the initial fault topology, the devices in the first set of devices are merged into the initial fault topology to obtain the updated initial fault topology.

[0112] If the current time slice to be merged is the last remaining time slice, the updated initial fault topology will be used as the fault topology corresponding to the current fault.

[0113] The second device is the device adjacent to the first device that does not experience any abnormal events.

[0114] Specifically, the initial fault topology can be represented as G. It can be defined as follows: Slice the time. Representing time slices The corresponding set of first devices (devices that experienced abnormal events), Representing time slices The corresponding set of second devices (devices adjacent to the first device that do not experience abnormal events). The updated initial fault topology, which is the time slice and time slices The fault topology obtained after merging the previous time slices. This represents the set of the first devices corresponding to the current fault after the slice is merged (all devices that experienced the abnormal event). This represents the set of second devices corresponding to the current fault after the slice is merged (all devices adjacent to the first device and not experiencing any abnormal events).

[0115] Methods for merging time slices can include:

[0116] Step 1: Set the first time slice detected for the current fault (time slice) ), and according to The resulting fault topology Let G be the initial fault topology for the current fault.

[0117] Step 2: Slice by time For the current time slice to be merged, determine the first set of devices. Second equipment set .

[0118] If the first set of devices If there is an intersection with the initial fault topology G, then the devices in the first device set are incorporated into the initial fault topology to obtain the updated initial fault topology, which can be represented as follows: .

[0119] Or, if the first set of devices The second set of devices has no intersection with the initial fault topology G. If there is an intersection with the initial fault topology G, then the devices in the first device set are incorporated into the initial fault topology to obtain the updated initial fault topology, which can be represented as follows: .

[0120] Step 3: If none of the conditions in Step 2 are met, then there is no need to incorporate the devices in the first set of devices into the initial fault topology, which can be represented as follows: .

[0121] If the current time slice to be merged is the last remaining time slice, the updated initial fault topology will be used as the fault topology corresponding to the current fault.

[0122] The following example illustrates the fault topology in the above embodiment. A resource pool experiences a time slice... When an abnormal event occurs, we can obtain the first set of devices, the second set of devices, and the initial fault topology. This can be represented as:

[0123]

[0124]

[0125]

[0126] Figure 3 This is one of the fault topology diagrams provided in this application after merging time slices, such as... Figure 3 As shown, time slices were further detected. Anomalies also occurred, affecting the time slice. The faults were labeled, and time slices were obtained. The first set of devices and the second set of devices can be represented as:

[0127]

[0128]

[0129] judge and If there is an intersection, then:

[0130]

[0131]

[0132] Figure 4 This is the second schematic diagram of the fault topology after merging time slices provided in this application, as shown below. Figure 4 As shown, time slices were further detected. An abnormal event also occurred, affecting the time slice. The faults were labeled, and time slices were obtained. The first set of devices and the second set of devices can be represented as:

[0133]

[0134]

[0135] judge and There is no intersection, but and If there is no intersection, then:

[0136]

[0137]

[0138] Figure 5 This is the third diagram of the fault topology after merging time slices provided in this application. Figure 5Indicates the time slice Time slice and time slices The fault topology obtained after merging.

[0139] The resource pool fault location method provided in this application incorporates the first device corresponding to the remaining time slices into the initial fault topology to obtain the fault topology corresponding to the current fault. This method can comprehensively consider all possible root cause devices, thereby improving the timeliness and accuracy of resource pool fault location.

[0140] In some embodiments, this application provides a method for selecting multiple time slices corresponding to the current fault, the method steps of which are as follows:

[0141] Step 1: Determine the time slice in which the current failure occurs in the resource pool as the first time slice. Select 9 time slices by a preset time length (e.g., the preset time length can be 30 minutes) or a first preset number (e.g., the first preset number can be 9). The 9 time slices after the first time slice can be merged according to the method of the aforementioned embodiment.

[0142] In the aforementioned embodiments, merged time slices can be referred to as valid slices, and unmerged time slices can be referred to as invalid slices.

[0143] Step 2: In addition to the nine subsequent fault slices mentioned above, examine and merge the multiple consecutive time slices following the last valid slice (the merged fault slice). The number of consecutive time slices can be selected based on a second preset number (e.g., the second preset number can be 3).

[0144] If all time slices in the second preset number are invalid, then time slices after the last time slice in the second preset number will no longer be considered, and the time slices of the first preset number and the second preset number will be used as the selected time slices.

[0145] If there is a valid slice in the second preset number of time slices, then based on the last valid slice in the second preset number of time slices, the second preset number of time slices are determined again for examination. If there is a valid slice in the determined time slices, the time slices are selected and examined in the same way as above until all the determined second preset number of time slices are invalid slices.

[0146] Figure 6 This is a schematic diagram illustrating the selection of time slices provided in this application, such as... Figure 6As shown, based on time slice 1, the current failure in the resource pool is determined, the failure merging process is initiated, and subsequent time slices 2-10 are examined. Simultaneously, the three slices following time slice 10 (a valid slice) are examined. Slice 13 is a valid slice, so slices 14-16 are examined. Slices 14-16 do not contain any valid slices, so subsequent slices are no longer examined. In the diagram, solid lines represent valid slices, and dashed lines represent invalid slices.

[0147] In some embodiments, determining the root cause device for the current fault based on the impact of each device in the fault topology corresponding to the current fault on the remaining devices in the resource pool includes:

[0148] Based on the fault topology corresponding to the current fault, determine the device to be analyzed.

[0149] Based on the degree of impact of the current device to be analyzed on the other devices in the resource pool, determine the set of affected devices and the set of non-affected devices corresponding to the current device to be analyzed.

[0150] If the device involved in an abnormal event belongs to the set of affected devices, increase the impact coefficient of the device currently being analyzed.

[0151] When the device involved in an abnormal event belongs to the set of non-affected devices, reduce the impact coefficient of the device currently being analyzed.

[0152] The device with the maximum influence coefficient in the fault topology corresponding to the current fault is identified as the root cause device of the current fault.

[0153] Specifically, a stack-based matching algorithm can be used to identify the root cause device corresponding to the current fault by applying a stack-based matching algorithm to each of the object devices corresponding to the abnormal events on the fault topology.

[0154] In the fault topology corresponding to the current fault, the object device corresponding to the abnormal event can be selected one by one as the current device to be analyzed.

[0155] Based on the impact of the device being analyzed on the other devices in the resource pool, the remaining devices are grouped into two sets: the affected devices set and the unaffected devices set. Devices that can be affected by the device being analyzed are assigned to the affected devices set, while those that cannot be affected are assigned to the unaffected devices set.

[0156] Determine the root cause device for the current fault using the following steps:

[0157] Step 1: Set the degree of influence coefficient This is used to indicate the degree of influence of the device being analyzed on the other devices in the resource pool. The initial value can be 0.

[0158] For each anomalous event, the affected device is matched with the set of affected devices and the set of unaffected devices. If the affected device belongs to the set of affected devices, the influence coefficient of the device being analyzed is increased (e.g., by incrementing the coefficient). If the affected device belongs to the set of unaffected devices, the influence coefficient of the device being analyzed is decreased (e.g., by decrementing the coefficient). After analyzing all the affected devices for all anomalous events using the above method, the influence coefficient of the device being analyzed can be obtained.

[0159] Therefore, the degree of influence coefficient can also be called the matching degree.

[0160] Step 2: Analyze each device in the fault topology corresponding to the current fault to obtain the influence coefficient of each device. The maximum value of the influence coefficient is then selected. The corresponding device was identified as the root cause device for the current fault.

[0161] Figure 7 This is a schematic diagram of the root cause device cluster matching provided in this application, such as... Figure 7 As shown, Host0 and STTOR are the devices to be analyzed (hypothetical fault points), respectively. The influence coefficient (match degree) of Host0 is 6, and the influence coefficient (match degree) of STTOR is -2. The match degree of each "hypothetical fault point" is examined one by one, and the "hypothetical fault point" with the highest match degree is the root cause device of the current fault.

[0162] The resource pool fault location method provided in this application improves the timeliness and accuracy of resource pool fault location by using a stack matching algorithm to determine the root cause device corresponding to the current fault based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool.

[0163] In some embodiments, based on the degree of influence of the device to be analyzed on the other devices in the resource pool, the set of affected devices and the set of non-affected devices corresponding to the device to be analyzed are determined, including:

[0164] Based on the propagation relationship of abnormal events, the remaining devices in the resource pool are grouped to determine the set of affected devices and the set of unaffected devices corresponding to the device to be analyzed.

[0165] Specifically, if a physical or virtual device in the resource pool malfunctions, surrounding related devices will detect the anomaly within a short period of time and report it as an alarm or other abnormal event.

[0166] In other words, there is an abnormal event propagation relationship between the various devices in the resource pool. For example, the device that causes an abnormal event can be the abnormal event source object, and the device that detects the abnormal event can be the abnormal event propagation object. In this case, there is an abnormal event propagation relationship between the source object and the propagation object.

[0167] Figure 8 This is a schematic diagram of the abnormal event propagation relationship provided in this application, such as... Figure 8 As shown, in the device topology, the abnormal event propagation relationship between adjacent device objects may have various situations such as "many-to-many", "one-to-many", and "one-to-one".

[0168] An exception propagation model can be established based on the propagation relationships of exception events. The exception propagation model describes the source object of the exception event, the object from which the exception event propagates, and the direction of the exception event's propagation.

[0169] An exception propagation model can be defined for physical and virtual devices such as hosts, switches, storage nodes, and storage pools when they fail. It is understandable that different exceptions will correspond to different propagation models. Since the device experiencing an exception may continue to propagate the exception message to more devices, forming an exception propagation chain, the exception propagation model can include at least two devices.

[0170] The resource pool fault location method provided in this application improves the timeliness and accuracy of resource pool fault location by grouping the remaining devices in the resource pool according to the abnormal event propagation relationship and determining the set of affected and unaffected devices corresponding to the device to be analyzed.

[0171] Figure 9 This is the second flowchart of the resource pool fault location method provided in this application, as shown below. Figure 9 As shown, the method includes:

[0172] Step 910: Obtain statistical characteristics of abnormal events during the normal operation of the resource pool;

[0173] Step 920: Obtain the dynamic three-dimensional topology of the resource pool;

[0174] Step 930: Obtain the abnormal event propagation model for different devices;

[0175] Step 940: Examine the changes in the frequency of anomalous events by time slice;

[0176] Step 950: Generate the initial fault topology;

[0177] Step 960: Merge time slices;

[0178] Step 970: Determine the root cause device of the fault by matching the different clusters.

[0179] The resource pool fault location method provided in this application utilizes the statistical characteristics of abnormal events, the dynamic three-dimensional topology of the resource pool, and the abnormal event propagation model. It also achieves rapid identification and accurate location of resource pool faults through time slice merging algorithm and fault root cause device cluster matching algorithm. This reduces the technical threshold for operation and maintenance personnel, enhances the operation and maintenance capabilities of the resource pool, and improves user experience.

[0180] The apparatus provided in the embodiments of this application is described below. The apparatus described below can be referred to in correspondence with the method described above.

[0181] Figure 10 This is a schematic diagram of the structure of the resource pool fault location device provided in this application, as shown below. Figure 10 As shown, the device includes:

[0182] The fault judgment module 1010 is used to determine the current fault of the resource pool based on the frequency of abnormal events in the current time slice.

[0183] The topology generation module 1020 is used to determine the fault topology corresponding to the current time slice based on the abnormal events in the current time slice and the device topology of the resource pool in the current time slice.

[0184] The slice merging module 1030 is used to merge the fault topologies corresponding to multiple time slices corresponding to the current fault and determine the fault topology corresponding to the current fault.

[0185] The fault location module 1040 is used to determine the root cause device of the current fault based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool.

[0186] The resource pool fault location device provided in this application determines the current fault in the resource pool based on the frequency of abnormal events in the current time slice; it determines the fault topology corresponding to the current time slice based on the abnormal events in the current time slice and the device topology of the resource pool in the current time slice; it merges the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault; and it determines the root cause device corresponding to the current fault based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool. This achieves fault location of the resource pool without relying on the operation and maintenance experience and professional knowledge of operation and maintenance engineers, but rather based on the frequency of abnormal events, device topology, and the degree of influence of each device on the other devices in the resource pool. This reduces the error of human judgment, realizes rapid identification and accurate location of resource pool faults, improves the efficiency of resource pool fault location, and enhances the timeliness and accuracy of resource pool fault location.

[0187] In some embodiments, the fault diagnosis module is used for:

[0188] Get the frequency of abnormal events in the resource pool during the current time slice;

[0189] If the frequency of abnormal events in the current time slice exceeds a preset frequency threshold, it is determined that the current failure has occurred in the resource pool.

[0190] The preset frequency threshold is determined based on the mean and standard deviation of the frequency of abnormal events that occur during the normal operation of the resource pool.

[0191] In some embodiments, the topology generation module is used for:

[0192] Obtain the devices corresponding to the resource pool in the current time slice and the corresponding relationships between the devices;

[0193] Based on the devices and the correspondence between them, construct the device topology of the resource pool corresponding to the current time slice;

[0194] Based on the abnormal events within the current time slice, the devices in the device topology are labeled to obtain the fault topology corresponding to the current time slice.

[0195] In some embodiments, the slice merging module is used for:

[0196] Determine the multiple time slices corresponding to the current fault;

[0197] The fault topology corresponding to the first time slice among multiple time slices is taken as the initial fault topology corresponding to the current fault.

[0198] The first device corresponding to the remaining time slices in multiple time slices is incorporated into the initial fault topology to obtain the fault topology corresponding to the current fault.

[0199] The first device is the device that experiences an abnormal event in the remaining time slices.

[0200] In some embodiments, the slice merging module is used for:

[0201] Determine the current time slice to be merged;

[0202] Determine the first and second device sets corresponding to the current time slice to be merged;

[0203] If the first set of devices intersects with the initial fault topology, or if the first set of devices does not intersect with the initial fault topology but the second set of devices intersects with the initial fault topology, the devices in the first set of devices are merged into the initial fault topology to obtain the updated initial fault topology.

[0204] If the current time slice to be merged is the last remaining time slice, the updated initial fault topology will be used as the fault topology corresponding to the current fault.

[0205] The second device is the device adjacent to the first device that does not experience any abnormal events.

[0206] In some embodiments, the fault location module is used for:

[0207] Based on the fault topology corresponding to the current fault, determine the device to be analyzed.

[0208] Based on the degree of impact of the current device to be analyzed on the other devices in the resource pool, determine the set of affected devices and the set of non-affected devices corresponding to the current device to be analyzed.

[0209] If the device involved in an abnormal event belongs to the set of affected devices, increase the impact coefficient of the device currently being analyzed.

[0210] When the device involved in an abnormal event belongs to the set of non-affected devices, reduce the impact coefficient of the device currently being analyzed.

[0211] The device with the maximum influence coefficient in the fault topology corresponding to the current fault is identified as the root cause device of the current fault.

[0212] In some embodiments, the fault location module is used for:

[0213] Based on the propagation relationship of abnormal events, the remaining devices in the resource pool are grouped to determine the set of affected devices and the set of unaffected devices corresponding to the device to be analyzed.

[0214] Figure 11 This is a schematic diagram of the structure of the electronic device provided in this application, such as... Figure 11 As shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130, and a communications bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the communications bus 1140. The processor 1110 can call logical commands in the memory 1130 to execute the methods described in the above embodiments, for example:

[0215] Based on the frequency of abnormal events in the resource pool in the current time slice, determine the current fault in the resource pool; based on the abnormal events in the current time slice and the device topology of the resource pool in the current time slice, determine the fault topology corresponding to the current time slice; merge the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault; based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool, determine the root cause device corresponding to the current fault.

[0216] Furthermore, the logical commands in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0217] The processor in the electronic device provided in this application embodiment can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effect, which will not be repeated here.

[0218] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0219] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0220] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0221] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0222] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0223] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for locating resource pool faults, characterized in that, include: Based on the frequency of abnormal events in the resource pool in the current time slice, it is determined that the resource pool has experienced the current failure. Based on the abnormal events within the current time slice and the device topology of the resource pool corresponding to the current time slice, the fault topology corresponding to the current time slice is determined. Merge the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault. Based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool, the root cause device corresponding to the current fault is determined. The process of determining the root cause device for the current fault based on the impact of each device in the fault topology corresponding to the current fault on the remaining devices in the resource pool includes: Based on the fault topology corresponding to the current fault, determine the device to be analyzed. Based on the degree of influence of the current device to be analyzed on the other devices in the resource pool, determine the set of devices that are affected and the set of devices that are not affected corresponding to the current device to be analyzed. If the object device of the abnormal event belongs to the set of affected devices, increase the influence degree coefficient of the device currently being analyzed; If the device involved in the abnormal event belongs to the set of unaffected devices, reduce the impact coefficient of the device currently being analyzed. The device corresponding to the maximum value of the influence coefficient in the fault topology corresponding to the current fault is determined as the root cause device corresponding to the current fault.

2. The resource pool fault location method according to claim 1, characterized in that, The determination of the current failure of the resource pool based on the frequency of abnormal events in the current time slice includes: Obtain the frequency of abnormal events in the resource pool during the current time slice; If the frequency of abnormal events in the current time slice is greater than a preset frequency threshold, it is determined that the resource pool has experienced the current failure. The preset frequency threshold is determined based on the mean and standard deviation of the frequency of abnormal events occurring in the resource pool during normal operation.

3. The resource pool fault location method according to claim 1, characterized in that, The step of determining the fault topology corresponding to the current time slice based on the abnormal events within the current time slice and the device topology of the resource pool in the current time slice includes: Obtain the devices corresponding to the resource pool in the current time slice and the corresponding relationships between the devices; Based on the devices and the correspondence between them, the device topology of the resource pool corresponding to the current time slice is constructed; Based on the abnormal events within the current time slice, the devices in the device topology are labeled to obtain the fault topology corresponding to the current time slice.

4. The resource pool fault location method according to claim 1, characterized in that, The step of merging the fault topologies corresponding to multiple time slices for the current fault to determine the fault topology for the current fault includes: Determine multiple time slices corresponding to the current fault; The fault topology corresponding to the first time slice among the plurality of time slices is taken as the initial fault topology corresponding to the current fault. The first device corresponding to the remaining time slices in the plurality of time slices is incorporated into the initial fault topology to obtain the fault topology corresponding to the current fault. The first device is the device that experienced an abnormal event in the remaining time slice.

5. The resource pool fault location method according to claim 4, characterized in that, The step of incorporating the first device corresponding to the remaining time slices in the plurality of time slices into the initial fault topology to obtain the fault topology corresponding to the current fault includes: Determine the current time slice to be merged; Determine the first set of devices and the second set of devices corresponding to the current time slice to be merged; If the first set of devices intersects with the initial fault topology, or if the first set of devices does not intersect with the initial fault topology but the second set of devices intersects with the initial fault topology, the devices in the first set of devices are incorporated into the initial fault topology to obtain an updated initial fault topology. If the current time slice to be merged is the last remaining time slice, the updated initial fault topology will be used as the fault topology corresponding to the current fault. The second device is a device adjacent to the first device that does not experience any abnormal events.

6. The resource pool fault location method according to claim 1, characterized in that, The step of determining the set of affected devices and the set of non-affected devices corresponding to the current device under analysis based on the degree of influence of the current device under analysis on the other devices in the resource pool includes: Based on the propagation relationship of abnormal events, the remaining devices in the resource pool are grouped to determine the set of affected devices and the set of non-affected devices corresponding to the device to be analyzed.

7. A resource pool fault location device, characterized in that, include: The fault diagnosis module is used to determine the current fault of the resource pool based on the frequency of abnormal events in the current time slice. The topology generation module is used to determine the fault topology corresponding to the current time slice based on the abnormal events in the current time slice and the device topology of the resource pool in the current time slice. The slice merging module is used to merge the fault topologies corresponding to multiple time slices corresponding to the current fault to determine the fault topology corresponding to the current fault. The fault location module is used to determine the root cause device corresponding to the current fault based on the degree of influence of each device in the fault topology corresponding to the current fault on the other devices in the resource pool. The process of determining the root cause device for the current fault based on the impact of each device in the fault topology corresponding to the current fault on the remaining devices in the resource pool includes: Based on the fault topology corresponding to the current fault, determine the device to be analyzed. Based on the degree of influence of the current device to be analyzed on the other devices in the resource pool, determine the set of devices that are affected and the set of devices that are not affected corresponding to the current device to be analyzed. If the object device of the abnormal event belongs to the set of affected devices, increase the influence degree coefficient of the device currently being analyzed; If the device involved in the abnormal event belongs to the set of unaffected devices, reduce the impact coefficient of the device currently being analyzed. The device corresponding to the maximum value of the influence coefficient in the fault topology corresponding to the current fault is determined as the root cause device corresponding to the current fault.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the resource pool fault location method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the resource pool fault location method as described in any one of claims 1 to 6.

Images