Method and apparatus for anomaly detection in distributed storage systems
By sending requests to segment units in a distributed storage system and using the return information from other segment units to determine the status, the problem of difficulty in quickly and accurately detecting segment unit anomalies in existing technologies is solved, and fast and accurate anomaly detection and diagnosis are achieved.
Patent Information
- Application Number
- CN202211611973.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-15
- Publication Date
- 2025-10-31
- Estimated Expiration
- 2042-12-15
AI Technical Summary
In distributed storage systems, network anomalies between nodes and device anomalies within individual nodes make it difficult to accurately and quickly detect and diagnose segment unit anomalies. Existing technologies rely on manual analysis, which is time-consuming and not real-time.
By sending requests to each segment unit, receiving abnormal information, and determining the current status based on the return information from other segment units, abnormalities can be quickly detected using heartbeat signals or data operation requests. Combined with the status confirmation mechanism of master and slave segment units, misjudgments can be avoided.
It enables rapid and accurate detection of segment unit anomalies, reduces misdiagnosis, and improves the real-time performance and efficiency of the system.
Smart Images

Figure CN115686922B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of data storage, and in particular to an anomaly detection method, apparatus, storage medium, and electronic device for distributed storage systems. Background Technology
[0002] A large-scale distributed storage system typically consists of numerous segments, each composed of segment units from different machines. Each segment unit can be a different device on a different machine, such as a segment of storage space on a storage medium. These machines are connected via a network to form a distributed cluster. Multiple segment units can reside on the same node.
[0003] After a period of operation, distributed clusters of a certain size may experience network anomalies between nodes or device malfunctions (motherboard, network card, storage media, etc.) within individual nodes. When these anomalies occur, one or more segments may be unable to perform normal business operations. Correct and rapid anomaly detection and diagnosis for each segment is the foundation for ensuring the continuous operation of distributed storage systems under abnormal conditions.
[0004] However, under the combined influence of factors such as cluster network anomalies and internal device anomalies of individual nodes, it is relatively difficult to accurately diagnose anomalies in a specific segment unit within a segment, especially in distributed processing architectures.
[0005] In traditional distributed storage systems, node failures and network connectivity issues are typically handled by operations and maintenance personnel. When a system performance degradation or unavailability is detected, the process involves observing and analyzing various host and network-related operational alerts, summarizing and analyzing these alerts to identify the fault location and scope of impact, and then manually addressing the issue. This manual approach is labor-intensive and cannot achieve real-time or accurate results. Summary of the Invention
[0006] This application provides an anomaly detection method, apparatus, storage medium, electronic device, and computer program product for a distributed storage system.
[0007] In a first aspect, embodiments of this application provide an anomaly detection method for a distributed storage system, used in an electronic device, wherein the distributed storage system includes multiple segments, each segment including multiple segment units, and the method includes:
[0008] Send requests to the plurality of segment units in each segment;
[0009] Receive exception information, which is issued by one or more segment units in response to the request;
[0010] For each segment unit that issued the abnormal information, the current state of each segment unit is determined based on the return information from other segment units.
[0011] In one possible implementation of the first aspect above, the request is a heartbeat signal, wherein the heartbeat signal is sent to the plurality of segment units at one or more predetermined time intervals, and the abnormal information is received from the one or more segment units when the number of timeouts of the heartbeat signal is greater than or equal to a predetermined threshold.
[0012] In one possible implementation of the first aspect above, the request is a data operation request, wherein the data operation request is sent to the plurality of segment units, and the exception information is received directly from the one or more segment units.
[0013] In one possible implementation of the first aspect above, one or more of the predetermined time intervals may be the same or different.
[0014] In one possible implementation of the first aspect described above, the heartbeat signal is transmitted each time at a different predetermined time interval or at the same predetermined time interval.
[0015] In one possible implementation of the first aspect described above, the multiple segment units include a master segment unit and multiple slave segment units.
[0016] When the received abnormal information comes from the master segment unit, the current state of the master segment unit is determined based on the return information from the multiple slave segment units.
[0017] In one possible implementation of the first aspect above, it further includes:
[0018] Each of the slave segment units sends a detection request to the master segment unit;
[0019] The number of slave units that receive the return information from the master unit is counted, wherein the return information is generated by the master unit in response to the detection request;
[0020] Based on the quantity, the current state of the main segment unit is determined.
[0021] In one possible implementation of the first aspect above, when the quantity is greater than or equal to a predetermined threshold, the current state of the main segment unit is determined to be a normal state; otherwise, it is determined to be an abnormal state.
[0022] In one possible implementation of the first aspect described above, the multiple segment units include a master segment unit and multiple slave segment units.
[0023] When the received abnormal information comes from the slave segment unit, the current state of the slave segment unit is determined according to the return information from the master segment unit.
[0024] In one possible implementation of the first aspect above, it further includes:
[0025] The master segment unit sends a detection request to the slave segment unit;
[0026] The current state of the slave segment unit is determined based on whether the master segment unit receives the return information from the slave segment unit, wherein the return information is generated by the slave segment unit in response to the detection request.
[0027] In one possible implementation of the first aspect above, if the master segment unit receives the return information from the slave segment unit, then the current state of the slave segment unit is determined to be a normal state; otherwise, it is determined to be an abnormal state.
[0028] Secondly, embodiments of this application provide an anomaly detection device for a distributed storage system, the distributed storage system comprising multiple segments, each segment comprising multiple segment units, the device comprising:
[0029] A sending unit is used to send requests to the plurality of segment units;
[0030] A receiving unit is configured to receive abnormal information, which is sent by one or more segment units in response to the request;
[0031] The determining unit is used to determine the current state of each segment unit that issued the abnormal information, based on the return information from other segment units.
[0032] Thirdly, embodiments of this application provide a computer program product including computer-executable instructions, which are executed by a processor to implement the anomaly detection method for a distributed storage system described in the first aspect.
[0033] Fourthly, embodiments of this application provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the anomaly detection method for a distributed storage system described in the first aspect.
[0034] Fifthly, embodiments of this application provide an electronic device, including: one or more processors; one or more memories; wherein the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device performs the anomaly detection method for a distributed storage system described in the first aspect.
[0035] In this invention, it is possible to quickly detect whether a segment unit is abnormal, and in the event of an abnormality, by requesting other segment units within the segment to further confirm the state of the segment unit, the state of the segment unit can be quickly and accurately determined (diagnosed), thus avoiding misdiagnosis. Attached Figure Description
[0036] Figure 1 A block diagram of an electronic device is shown according to some embodiments of this application;
[0037] Figure 2 A schematic diagram of a distributed storage system is shown according to some embodiments of this application;
[0038] Figure 3 According to some embodiments of this application, a flowchart of an anomaly detection method for a distributed storage system is shown;
[0039] Figure 4 According to some embodiments of this application, another flowchart of an anomaly detection method for a distributed storage system is shown;
[0040] Figure 5 According to some embodiments of this application, another flowchart of an anomaly detection method for a distributed storage system is shown;
[0041] Figure 6 According to some embodiments of this application, a structural diagram of an anomaly detection device for a distributed storage system is shown. Detailed Implementation
[0042] The illustrative embodiments of this application include, but are not limited to, anomaly detection methods, apparatuses, media, electronic devices, and computer program products for distributed storage systems.
[0043] The embodiments of this application will now be described in further detail with reference to the accompanying drawings.
[0044] Figure 1 A block diagram of an electronic device is shown according to some embodiments of this application.
[0045] like Figure 1 As shown, the electronic device 100 may include one or more processors 102, a system motherboard 108 connected to at least one of the processors 102, system memory 104 connected to the system motherboard 108, non-volatile memory (NVM) 106 connected to the system motherboard 108, and a network interface 110 connected to the system motherboard 108.
[0046] Processor 102 may include one or more single-core or multi-core processors. Processor 102 may include any combination of a general-purpose processor (CPU) and a dedicated processor (e.g., a graphics processing unit, application processor, baseband processor, etc.). A graphics processing unit (GPU) is a dedicated processor that, compared to a general-purpose processor, has an order of magnitude higher number of cores and powerful parallel computing capabilities, and is widely used in computer graphics processing. In embodiments of the present invention, processor 102 may be configured to execute according to... Figure 3 One or more embodiments of the various embodiments shown.
[0047] In some embodiments, the system motherboard 108 may include any suitable interface controller (not included in...). Figure 1 (as shown in the figure) to provide any suitable interface to at least one of the processors 102 and / or any suitable device or component communicating with the system motherboard 108.
[0048] In some embodiments, system motherboard 108 may include one or more memory controllers to provide an interface to system memory 104. System memory 104 may be used to load and store data and / or instructions 120. In some embodiments, system memory 104 of electronic device 100 may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).
[0049] The non-volatile memory 106 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions 120. In some embodiments, the non-volatile memory 106 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of HDD (Hard Disk Drive), CD (Compact Disc) drive, and DVD (Digital Versatile Disc) drive.
[0050] The non-volatile memory 106 may include a portion of the storage resources installed on the device of the electronic device 100, or it may be accessible by an external device, but is not necessarily part of the external device. For example, the non-volatile memory 106 may be accessed over a network via the network interface 110.
[0051] Specifically, system memory 104 and non-volatile memory 106 may each include a temporary copy and a permanent copy of instruction 120. Instruction 120 may include, when executed by at least one of processors 102, causing electronic device 100 to perform, as Figure 3The instructions for the method shown. In some embodiments, the instructions 120, hardware, firmware and / or their software components may additionally / alternatively be located in the system motherboard 108, network interface 110 and / or processor 102.
[0052] Network interface 110 may include a transceiver for providing a radio interface to electronic device 100, thereby enabling communication with any other suitable device (e.g., front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 110 may be integrated into other components of electronic device 100. For example, network interface 110 may be integrated into at least one of processor 102, system memory 104, non-volatile memory 106, and firmware device (not shown) with instructions, which, when at least one of processor 102 executes the instructions, enable electronic device 100 to perform... Figure 3 One or more embodiments of the various embodiments shown.
[0053] Network interface 110 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, network interface 110 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.
[0054] In one embodiment, at least one of the processors 102 may be packaged together with one or more controllers for the system motherboard 108 to form a system-in-package (SiP). In another embodiment, at least one of the processors 102 may be integrated on the same die with one or more controllers for the system motherboard 108 to form a system-on-a-chip (SoC).
[0055] The electronic device 100 may further include an input / output (I / O) device 112 connected to the system motherboard 108. The I / O device 112 may include a user interface enabling a user to interact with the electronic device 100; the peripheral component interface is designed to allow peripheral components to also interact with the electronic device 100. In some embodiments, the electronic device 100 may also include sensors for determining at least one of environmental conditions and location information related to the electronic device 100.
[0056] In some embodiments, the I / O device 112 may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., an LED flash), a keyboard, and a graphics card. The graphics card consists of a graphics processor that integrates an I / O interface conforming to a data transfer protocol specification (such as a PCIe interface).
[0057] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.
[0058] In some embodiments, the sensors may include, but are not limited to, gyroscope sensors, accelerometers, proximity sensors, ambient light sensors, and positioning units. The positioning unit may also be part of or interact with network interface 110 to communicate with components of the positioning network, such as Global Positioning System (GPS) satellites.
[0059] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
[0060] Program code can be applied to input instructions to perform the functions described in this invention and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a system for processing instructions including processor 102 includes any system having a processor such as a digital signal processor (DSP), microcontroller, application-specific integrated circuit (ASIC), or microprocessor.
[0061] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this invention are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.
[0062] One or more aspects of at least one embodiment can be implemented by instructions stored on a computer-readable storage medium, which, when read and executed by a processor, enable an electronic device to implement the methods of the embodiments described in this invention.
[0063] To facilitate understanding, the relevant terms and concepts that may be involved in the embodiments of this application will be introduced below.
[0064] Volume: A storage area in a distributed storage system.
[0065] Segment: The basic logical unit that makes up a volume. A volume has n segments (n≥1). The number of segments in a volume = the size of the volume / the size of the segment.
[0066] Segment unit: The basic or physical unit that makes up a segment. There are various segment units with different functions, including master segment units and slave segment units. A segment unit corresponds to a contiguous space on the physical disk and can store user data.
[0067] Storage medium: A storage medium used to store data, such as a physical disk.
[0068] Storage node: A storage node can contain multiple storage media, and a storage media can only belong to one storage node.
[0069] It's understandable that storage nodes and physical disks are both physical concepts. A storage node manages several physical disks (i.e., data disks), and at any given time, the same physical disk is managed by only one storage node. A volume is a logical concept; different storage block spaces can be divided on a single physical disk to form different volumes. That is, segment units of the same volume can be distributed across different physical disks.
[0070] The anomaly detection method for distributed storage systems provided in this application can be applied to... Figure 1 The electronic device 100 shown may be, for example, a data drive, a computer, a mobile device, a platform device, etc.
[0071] Figure 2 This is a schematic diagram of a distributed storage system 20 according to an embodiment of the present invention. Figure 2 As shown, the distributed storage system 20 includes a data driver 21 and a segment 22. The segment 22 includes, for example, a master segment unit P1 and two slave segment units S1 and S2.
[0072] It is understood that the distributed storage system 20 may include multiple data drives and multiple volumes (not shown in the figure), each volume includes multiple segments, and each segment includes multiple segment units. Figure 2 Section 22 is located in one of the volumes.
[0073] Figure 3 This is a flowchart of an anomaly detection method for a distributed storage system according to the present invention. In this embodiment, data driver 21 is used as an example for explanation. It is understood that the following method is executed by data driver 21.
[0074] See Figure 3 In step S31, requests are sent to multiple segment units within each segment. It is understood that the multiple segment units within each segment can reside in different storage nodes; by sending the request to these storage nodes, requests can be sent to multiple segment units within each segment.
[0075] In step S32, an exception message is received, which is issued by one or more segment units in response to the request.
[0076] It is understandable that these segment units that issue abnormal information may belong to different segments and reside in different storage nodes.
[0077] In step S33, for each segment unit that issued an abnormal message, the current state of each segment unit is determined based on the return information from other segment units.
[0078] For example, for a segment unit that issued an exception message, the current state of the segment unit is determined based on the return information of other segment units belonging to the same segment.
[0079] For example, the above request is, for instance, a heartbeat signal, which is sent to multiple segment units in each segment at one or more predetermined time intervals, and when the number of timeouts of the heartbeat signal is greater than or equal to a predetermined threshold, abnormal information is received from one or more segment units.
[0080] One or more predetermined time intervals can be different. For example, there are three predetermined time intervals T1, T2, and T3, where T1 = 100ms, T2 = 200ms, and T3 = 400ms. Alternatively, one or more predetermined time intervals can be the same, for example, the three predetermined time intervals T1 = T2 = T3 = 100ms.
[0081] It is understood that the number of predetermined time intervals and the specific time intervals in the above examples are for illustrative purposes only and are not subject to any limitation.
[0082] The predetermined threshold can be adjusted according to the network environment and can be any value without restriction.
[0083] For example, the predetermined threshold is, for instance, 3. For example, a first predetermined time interval T1 = 100 ms is used to send heartbeat signal 1 to multiple segment units. If heartbeat signal 1 fails to return normally, causing a timeout, the interval for subsequent heartbeat signal transmissions can be set, for example, a second predetermined time interval T2 = 200 ms. Thus, the next heartbeat detection 2 is sent after 200 ms.
[0084] If heartbeat detection 2 returns normally, the time interval for sending heartbeat signals thereafter returns to T1 = 100ms. If heartbeat signal 2 fails to return normally and causes a timeout, the interval for sending subsequent heartbeat signals is set to, for example, a third predetermined time interval T3 = 400ms. Thus, the next heartbeat detection 3 will be sent after 400ms.
[0085] If heartbeat detection 3 returns normally, the time interval for sending heartbeat signals thereafter will be restored to T1=100ms. If heartbeat signal 3 fails to return normally and causes a timeout, and the number of timeouts for the heartbeat signal is three, which is greater than or equal to the aforementioned predetermined threshold 3, then it can be determined that an abnormality has occurred in the network environment.
[0086] Understandably, the network anomaly was identified after a detection time of T1+T2+T3=700ms. This millisecond-level network detection mechanism ensures rapid awareness of network anomalies, facilitating subsequent processing.
[0087] It is understandable that network environment anomalies can be caused by storage media failure, storage node failure, network failure, and so on.
[0088] In the example above, heartbeat signals are sent at different predetermined time intervals each time. It is understandable that heartbeat signals can be sent at the same predetermined time interval each time.
[0089] For example, the above request is a data operation request, in which the data operation request is sent to multiple segment units and exception information is received directly from one or more of the segment units.
[0090] It is understandable that data operation requests can be, for example, requests to read or write data to each segment unit, or requests to perform other operations on the data.
[0091] For example, when a storage node fails, all segment units located on that node will respond to the data operation request and directly return an exception message. Understandably, in this case, the exception message can be returned much faster than the detection mechanism using heartbeat signals described above.
[0092] As mentioned above, network anomalies can be quickly detected, and abnormal information can be received from some segment units. However, if a segment unit sends out abnormal information due to, for example, a brief network jitter, and it is immediately determined that the segment unit is in an abnormal state, it will lead to misjudgment and thus cause a large amount of unnecessary data migration.
[0093] The following details the process of further determining the state of these segment units that have issued abnormal information.
[0094] Each segment comprises multiple segment units, which in turn include one master segment unit and multiple slave segment units. That is, each segment includes one master segment unit and multiple slave segment units. For example, such as... Figure 2 As shown, segment 22 includes a main segment unit P1 and two slave segment units S1 and S2.
[0095] When an error message is received from the master segment unit P1, the current state of the master segment unit P1 is determined based on the return information from the two slave segment units S1 and S2. In other words, the current state of the master segment unit P1 is determined based on the return information from the two slave segment units S1 and S2, which are located in the same segment as the master segment unit P1.
[0096] See Figure 4 In step S3311, each slave segment unit S1 and S2 sends a detection request to the master segment unit P1. It can be understood that by sending a detection instruction to each slave segment unit S1 and S2, each slave segment unit S1 and S2 sends a detection request to the master segment unit P1.
[0097] In step S3312, the number of slave units that receive return information from master unit P1 is counted, wherein the return information is generated by master unit P1 in response to the detection request.
[0098] Understandably, when the master segment unit P1 is functioning normally, it will respond to the detection request by generating return information and sending it back to the slave segment units S1 and S2. Conversely, when the master segment unit P1 malfunctions, it will not send any return information.
[0099] If both slave units S1 and S2 receive return information, then the count is 2.
[0100] In step S3313, the current state of the main segment unit P1 is determined based on the quantity. For example, when the quantity is greater than or equal to a predetermined threshold, the current state of the main segment unit P1 is determined to be a normal state; otherwise, it is determined to be an abnormal state.
[0101] The predetermined threshold is determined based on a majority rule, for example, predetermined threshold = [N / 2] + 1, where N represents the number of segment units in the segment, and [N / 2] represents the integer part of the number. For example, if the number of segment units N in the segment is 3, then predetermined threshold = [3 / 2] + 1 = 2.
[0102] In this embodiment, if the number obtained from the above statistics is 2, which is greater than or equal to the predetermined threshold of 2, then it can be determined that the current state of the main segment unit P1 is a normal state.
[0103] It is understandable that if neither segment unit S1 nor S2 receives any return information, the count is 0, which is less than the predetermined threshold of 2. In this case, the current state of the main segment unit P1 is determined to be an abnormal state.
[0104] Understandably, if either segment unit S1 or S2 experiences an anomaly, it will not receive any return information. For example, if segment unit S1 experiences an anomaly and does not receive any return information, while segment unit S2 is normal and receives return information, the count is 1, which is less than the predetermined threshold 2. In this case, the current state of the main segment unit P1 is determined to be an anomaly.
[0105] Understandably, when the main segment unit P1 sends out an abnormal message, it further judges based on the return information received by the slave segment units S1 and S2 from the main segment unit P1, so as to accurately determine whether the state of the main segment unit P1 is normal or abnormal, thus avoiding misjudgment.
[0106] On the other hand, when the received abnormal information comes from the slave segment unit, the current state of the slave segment unit is determined based on the return information from the master segment unit.
[0107] For example, if an exception message is sent from segment unit S1, the current state of segment unit S1 can be determined based on the return information from master segment unit P1.
[0108] See Figure 5 In step S3321, the master segment unit P1 sends a detection request to the slave segment unit S1. It can be understood that by sending a detection command to the master segment unit P1, the master segment unit P1 sends a detection request to the slave segment unit S1.
[0109] In step S3322, the current state of the slave segment unit S1 is determined based on whether the master segment unit P1 receives return information from the slave segment unit S1, wherein the return information is generated by the slave segment unit S1 in response to the detection request.
[0110] Understandably, when the slave segment unit S1 is functioning normally, it will generate return information in response to the detection request and send that return information back to the master segment unit P1. Conversely, when the slave segment unit S1 malfunctions, it will not send any return information.
[0111] If the master segment unit P1 receives a return message from the slave segment unit S1, it determines that the current state of the slave segment unit S1 is normal; otherwise, it determines that it is abnormal.
[0112] In this invention, it is possible to quickly detect whether a segment unit is abnormal, and in the event of an abnormality, by requesting other segment units within the segment to further confirm the state of the segment unit, the state of the segment unit can be quickly and accurately determined (diagnosed), thus avoiding misdiagnosis.
[0113] The present invention also provides an anomaly detection device 60 for a distributed storage system, the distributed storage system comprising multiple segments, each segment comprising multiple segment units. For example... Figure 6 As shown, the device 60 includes: a sending unit 61, configured to send a request to the plurality of segment units; a receiving unit 62, configured to receive abnormal information, the abnormal information being sent by one or more segment units in response to the request; and a determining unit 63, configured to determine the current state of each segment unit for each segment unit that sent the abnormal information, based on the return information from other segment units.
[0114] It is understood that the transmitting unit 61, the receiving unit 62, and the determining unit 63 can be implemented by a processor 102 in the electronic device 100 that has the functions of these modules or units. The embodiments disclosed above are method embodiments corresponding to this embodiment, and this embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.
[0115] The present invention also provides a computer program product, including computer-executable instructions, which are executed by processor 102 to implement the anomaly detection method for a distributed storage system of the present invention.
[0116] The present invention also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the anomaly detection method for a distributed storage system of the present invention.
[0117] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0118] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art should understand that various changes in form and detail may be made thereto without departing from the spirit and scope of this application.
[0119] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0120] It should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of the single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.
[0121] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose.
[0122] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.
Claims
1. An anomaly detection method for a distributed storage system, used in electronic devices, characterized in that, The distributed storage system includes multiple segments, each segment includes multiple segment units, and the multiple segment units include a master segment unit and multiple slave segment units. The method includes: Send requests to the plurality of segment units in each segment; Receive exception information, which is issued by one or more segment units in response to the request; For each segment unit that issued the aforementioned abnormal information, the current state of each segment unit is determined based on the return information from other segment units. When the received abnormal information originates from the master segment unit, each slave segment unit sends a detection request to the master segment unit; the number of slave segment units that received the return information from the master segment unit is counted, wherein the return information is generated by the master segment unit in response to the detection request; based on the count, the current state of the master segment unit is determined. When the received abnormal information comes from the slave segment unit, the master segment unit sends a detection request to the slave segment unit; the current state of the slave segment unit is determined based on whether the master segment unit receives the return information from the slave segment unit, wherein the return information is generated by the slave segment unit in response to the detection request.
2. The anomaly detection method according to claim 1, characterized in that, The request is a heartbeat signal, wherein the heartbeat signal is sent to the plurality of segment units at one or more predetermined time intervals, and when the number of timeouts of the heartbeat signal is greater than or equal to a predetermined threshold, the abnormal information is received from the one or more segment units.
3. The anomaly detection method according to claim 1, characterized in that, The request is a data operation request, wherein the data operation request is sent to the plurality of segment units, and the abnormal information is received directly from the one or more segment units.
4. The anomaly detection method according to claim 2, characterized in that, One or more of the predetermined time intervals may be the same or different.
5. The anomaly detection method according to claim 4, characterized in that, The heartbeat signal is sent at different or the same predetermined time interval each time.
6. The anomaly detection method according to claim 1, characterized in that, When the quantity is greater than or equal to a predetermined threshold, the current state of the main segment unit is determined to be a normal state; otherwise, it is determined to be an abnormal state.
7. The anomaly detection method according to claim 1, characterized in that, If the master segment unit receives the return information from the slave segment unit, it determines that the current state of the slave segment unit is normal; otherwise, it determines that it is abnormal.
8. An anomaly detection device for a distributed storage system, characterized in that, The distributed storage system includes multiple segments, each segment includes multiple segment units, and the multiple segment units include a master segment unit and multiple slave segment units. The device includes: A sending unit is used to send requests to the plurality of segment units; A receiving unit is configured to receive abnormal information, which is sent by one or more segment units in response to the request; The determining unit is used to determine the current state of each segment unit that issued the abnormal information, based on the return information from other segment units. Specifically, when the received abnormal information originates from the master segment unit, each slave segment unit sends a detection request to the master segment unit; the number of slave segment units that receive the return information from the master segment unit is counted, wherein the return information is generated by the master segment unit in response to the detection request; based on the count, the current state of the master segment unit is determined. When the received abnormal information comes from the slave segment unit, the master segment unit sends a detection request to the slave segment unit; the current state of the slave segment unit is determined based on whether the master segment unit receives the return information from the slave segment unit, wherein the return information is generated by the slave segment unit in response to the detection request.
9. A computer program product comprising computer-executable instructions, characterized in that, The instructions are executed by the processor to implement the anomaly detection method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on a computer, cause the computer to perform the anomaly detection method according to any one of claims 1-7.
11. An electronic device, characterized in that, include: One or more processors; One or more memory units; The one or more memories store one or more programs, which, when executed by the one or more processors, cause the electronic device to perform the anomaly detection method according to any one of claims 1-7.
Citation Information
Patent Citations
Election control method and device
CN108494585A
Distributed storage system and method for deleting volumes in distributed storage system
CN112256202A
System fault processing method and device, computer equipment and storage medium
CN112328421A