Object storage device isolation method and apparatus, computer device, and storage medium

Abstract

The application relates to the technical field of computers and discloses an object storage device isolation method and device, a computer device and a storage medium, the method is executed by an object storage device and comprises the following steps: when it is found that the working efficiency of the object storage device itself is lower than a preset efficiency threshold, a notification message is generated; the notification message is sent to a monitoring end through a preset message transmission channel, so that the monitoring end completes a slow disk control operation according to the notification message and sends an isolation control instruction to the object storage device, wherein the preset message transmission channel is an independently running message transmission channel; an isolation control instruction fed back by the monitoring end is received, the isolation control instruction comprises an isolation signal; and the object storage device itself is controlled to enter an isolation state according to the isolation signal. The application can timely complete the isolation of the object storage device with a slow disk, ensure that messages can be timely distributed to other normal object storage devices, and improve system availability.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to object storage device isolation methods, apparatus, computer equipment, and storage media. Background Technology

[0002] Distributed storage hard drives age over time, leading to issues like slow disks. When slow disks exist in a distributed system, their inefficient message processing severely impacts the overall message processing speed. Current slow disk detection mechanisms rely heavily on network communication between the monitoring terminal and the object storage device. When slow disk isolation is required, the slow disk's slow message processing speed can prevent timely isolation, hindering the effective distribution of messages to other healthy nodes. This results in front-end service lag and disrupts normal operations. Summary of the Invention

[0003] In view of this, the present invention provides an object storage device isolation method, apparatus, computer device, and storage medium to solve the problem of untimely isolation of slow disks.

[0004] In a first aspect, the present invention provides an object storage device isolation method, the method being executed by the object storage device, comprising:

[0005] When the working efficiency of the object storage device itself is detected to be lower than a preset efficiency threshold, a notification message is generated.

[0006] The notification message is sent to the monitoring terminal through the preset message transmission channel so that the monitoring terminal can complete the slow disk control operation according to the notification message and send the isolation control command to the object storage device. The preset message transmission channel is an independently operating message transmission channel.

[0007] Receive isolation control commands from the monitoring terminal, which include isolation signals;

[0008] Based on the isolation signal, the control object storage device itself enters an isolation state.

[0009] The above method generates a notification message when the efficiency of the object storage device falls below a preset efficiency threshold. This message is then sent to the monitoring terminal via a preset message transmission channel. The monitoring terminal performs slow disk control operations based on the notification and sends isolation control commands to the object storage device. This preset message transmission channel is an independently operating channel. The monitoring terminal receives the isolation control commands, which include an isolation signal. Based on the isolation signal, the object storage device enters an isolation state. When the object storage device detects that its efficiency is below the preset efficiency threshold, it generates a slow disk message upon identifying itself as a slow disk. This message is then transmitted to the monitoring terminal via an independent message transmission channel, such as a separate thread. This ensures timely transmission of slow disk messages to the monitoring terminal for processing, eliminating the need for them to queue with other pending messages. The monitoring terminal is promptly notified to perform slow disk control operations, and the control terminal, after completing the slow disk control operations, promptly notifies the object storage device to complete its isolation settings. This significantly reduces message backlog and front-end business delays caused by slow disk isolation delays.

[0010] In one optional implementation, when the working efficiency of the object storage device itself is detected to be lower than a preset efficiency threshold, a notification message is generated, specifically including:

[0011] Obtain the pre-created isolation signal corresponding to the object storage device, the network address of the node where the object storage device is located, the device identifier of the object storage device, the slot information of the object storage device, and the drive letter information of the object storage device;

[0012] A notification message is generated based on the isolation signal, network address, device identifier, slot information, and drive letter information.

[0013] By using the above method, when the working efficiency of the object storage device is detected to be lower than the preset efficiency threshold, all the information required by the monitoring end to perform slow disk control operation is obtained, and a notification message is generated based on the above information. The information required by the monitoring end is sent at once, which can not only improve the sending efficiency, but also avoid the problem of incomplete message reception due to abnormalities or other reasons when sending multiple times, thus preventing the slow disk control operation from being completed.

[0014] In one alternative implementation, the method further includes:

[0015] Use a pre-created, independent listening thread to listen for isolation control commands.

[0016] By using the above method and employing an independent listening thread, interference from other information can be avoided, and it can also be ensured that isolation control commands sent from the monitoring end to the object storage device can be successfully received.

[0017] Secondly, the present invention provides an object storage device isolation method, which is executed by a monitoring terminal and includes:

[0018] When a notification message is received through the message transmission channel between the monitoring terminal and the object storage device, the slow disk control operation is completed according to the notification message. The message transmission channel is an independently running thread channel.

[0019] Send isolation control commands to the object storage device so that the object storage device can isolate itself according to the isolation control commands.

[0020] In this manner, when a notification message is received through the message transmission channel between the monitoring terminal and the object storage device, a slow disk control operation is completed based on the notification message. This message transmission channel is an independently running thread channel. An isolation control command is sent to the object storage device, enabling it to isolate itself according to the command. Receiving the notification message through an independently running thread channel ensures efficient message transmission, timely completion of slow disk control operations, and prompt sending of isolation control commands to the object storage device. This ensures the slow disk can complete its isolation operation promptly, allowing front-end services to be promptly allocated to normal object storage devices for processing, thus guaranteeing the smooth operation of front-end services.

[0021] In one optional implementation, the notification message includes: an isolation signal corresponding to the object storage device, and the network address of the node where the object storage device is located; sending an isolation control command to the object storage device specifically includes:

[0022] The node where the object storage device is located is determined by its network address;

[0023] Establish a remote connection between the monitoring terminal and the node;

[0024] Use nodes to send isolation signals to object storage devices.

[0025] By using the above method, the node where the object storage device is located is determined by the network address, and a remote connection is established between the monitoring terminal and the node. The isolation signal is then sent to the object storage device through the node, ensuring that the monitoring terminal can successfully send the isolation signal to the object storage device from any node.

[0026] In an optional implementation, when the notification message also includes the device identifier of the object storage device, upon receiving the notification message through the message transmission channel between the monitoring terminal and the object storage device, the slow disk control operation is completed according to the notification message, including:

[0027] The device identification controls the storage device to stop read and write operations.

[0028] Using the above method, the object storage device can be controlled to stop read and write operations based on the device identifier, and the front-end messages can be distributed to other normal object storage devices to ensure that the front-end messages can be processed in a timely manner.

[0029] In one optional implementation, stopping read / write operations on the object storage device based on the device identifier includes:

[0030] The status of the object storage device is found in the preset status table based on the device identifier.

[0031] Mark the object storage device as offline. The offline status is used to indicate that the object storage device has stopped reading and writing operations.

[0032] In an optional implementation, when the notification message also includes slot information and disk letter information of the object storage device, when the notification message is received through the message transmission channel between the monitoring terminal and the object storage device, the slow disk control operation is completed according to the notification message, including:

[0033] Generate slow disk alarm information based on slot information and disk letter information;

[0034] The alarm information is sent to the terminal device, and the alarm information is used to indicate the replacement of the object storage device.

[0035] Using the above method, slow disk alarm information is generated based on slot information and disk letter information, and sent to terminal devices. This allows users to quickly locate the slow disk based on the alarm information and replace it in a timely manner.

[0036] Thirdly, the present invention provides an object storage device that performs object storage device isolation, comprising:

[0037] The generation module is used to generate a notification message when the working efficiency of the object storage device itself is lower than a preset efficiency threshold.

[0038] The first sending module is used to send notification messages to the monitoring terminal through a preset message transmission channel, so that the monitoring terminal can complete the slow disk control operation according to the notification message and send isolation control instructions to the object storage device. The preset message transmission channel is an independently operating message transmission channel.

[0039] The receiving module is used to receive isolation control commands fed back from the monitoring terminal. The isolation control commands include isolation signals.

[0040] The control module is used to control the object storage device to enter the isolation state based on the isolation signal.

[0041] Fourthly, the present invention provides a monitoring terminal for performing object storage device isolation, comprising:

[0042] The slow disk control module is used to complete the slow disk control operation according to the notification message received through the message transmission channel between the monitoring terminal and the object storage device. The message transmission channel is an independently running thread channel.

[0043] The second sending module is used to send isolation control commands to the object storage device so that the object storage device can perform isolation operations according to the isolation control commands.

[0044] Fifthly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the object storage device isolation method of the first aspect or any corresponding embodiment described above, or to perform the object storage device isolation method of the second aspect or any corresponding embodiment described above.

[0045] In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the object storage device isolation method of the first aspect or any corresponding embodiment described above, or to cause a computer to perform the object storage device isolation method of the second aspect or any corresponding embodiment described above. Attached Figure Description

[0046] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating an object storage device isolation method according to an embodiment of the present invention;

[0048] Figure 2 This is a flowchart illustrating another object storage device isolation method according to an embodiment of the present invention;

[0049] Figure 3 This is a flowchart illustrating another object storage device isolation method according to an embodiment of the present invention;

[0050] Figure 4 This is a flowchart illustrating another object storage device isolation method according to an embodiment of the present invention;

[0051] Figure 5 This is a flowchart illustrating another object storage device isolation method according to an embodiment of the present invention;

[0052] Figure 6 This is a block diagram of an object storage device structure according to an embodiment of the present invention, which describes an object storage device isolation method.

[0053] Figure 7 This is a block diagram of a monitoring terminal structure for an object storage device isolation method according to an embodiment of the present invention;

[0054] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0056] With the development of the internet industry, the demand for storage is increasing, leading to the emergence of distributed storage. Distributed storage distributes data across multiple storage servers, creating a virtual storage device from these dispersed resources. This improves system reliability, availability, and access efficiency, and also facilitates scalability. However, the hard drives in distributed storage will age over time, leading to problems such as slow disks. In distributed file systems, slow disks refer to low-performance hard drives in the storage system. The presence of slow disks can affect the performance of the associated group and even the entire business system.

[0057] The OSD service on an object-based storage device (OSD) node can self-monitor the read / write latency and other operational status of the hard drive's input / output (IO). If the OSD's IO read / write latency exceeds the slow disk warning threshold and this state persists for a period of time, the OSD will automatically identify itself as a slow disk. Upon detecting a slow disk, the OSD will send a slow disk message to the monitoring terminal (MON) via a network connection. After receiving the slow disk message, the MON will isolate the OSD as a slow disk and report the slow disk alarm to the client. After the slow OSD is isolated, automatic balancing will be triggered, migrating the data on the slow OSD to the primary node, backup node, or other available high-performance node OSDs, thus avoiding the impact of low-performance or slow-responding OSD nodes on data read / write operations in the storage cluster.

[0058] In one related technology, the current slow disk detection mechanism relies too heavily on network communication between the MON and OSD. After the OSD detects a slow disk, it sends a slow disk message to the MON. Then, the MON reports an alarm and returns an acknowledgment character (ACK) message to the OSD. The OSD process exits after receiving the ACK message.

[0059] The OSD has a separate thread that handles messages between the MON and the OSD. These messages include not only slow disk messages but also others such as network connection authentication messages and Placement Group (PG) status messages. These messages are queued and await scheduling by the processing thread. When processing PG status messages, the OSD lock is held. If the OSD experiences a slow disk, the slow disk causes the IO processing thread to release the OSD lock slowly, resulting in a large backlog of messages that cannot be processed in time. This prevents the exchange of slow disk messages from being completed, hindering timely slow disk fault isolation, preventing slow disk alarms from being reported correctly, and preventing the slow disk process from exiting for fault isolation, ultimately causing front-end business processes to freeze.

[0060] Based on the above problems, according to an embodiment of the present invention, an embodiment of an object storage device isolation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0061] This embodiment provides an object storage device isolation method, which can be used for the aforementioned object storage devices, such as OSDs. Figure 1 This is a flowchart of an object storage device isolation method according to an embodiment of the present invention, such as... Figure 1 As shown, Figure 1 The process shown includes the following steps:

[0062] Step S101: When the working efficiency of the object storage device itself is detected to be lower than a preset efficiency threshold, a notification message is generated.

[0063] Specifically, object storage devices, such as OSDs, can monitor their own efficiency during read and write operations, including IO read / write latency and message processing speed. If the OSD's IO read / write latency exceeds the slow disk warning threshold, and this state lasts longer than a preset slow disk time threshold, the OSD will automatically identify and determine that it is a slow disk. When the OSD determines that it is a slow disk, it will obtain parameter information related to slow disk processing, such as the object storage device's attribute information, location information, and slow disk identifier, and generate a notification message.

[0064] Step S102: The notification message is sent to the monitoring terminal through a preset message transmission channel so that the monitoring terminal can complete the slow disk control operation according to the notification message and send the isolation control command to the object storage device.

[0065] Specifically, the default message transmission channel is an independently operating one. Notification messages are used to indicate a slow disk event on the object storage device. For example, the monitoring end can start a dedicated thread as a message transmission channel to send notification messages for slow disk events. This way, slow disk notification messages don't need to be queued for processing; the monitoring end can receive the notification messages promptly, complete the slow disk control operation, and send isolation control commands to the object storage device. This allows the slow disk to be isolated promptly, and front-end business messages can be promptly distributed to other normally operating object storage devices, ensuring timely processing.

[0066] In an optional example, the message transmission channel for transmitting slow disk notification messages can be used only for transmitting slow disk notification messages, or it can be used to transmit a small number of other important messages, such as high-priority alarm messages and fault messages. This can improve the processing rate of high-priority messages, ensure the stable operation of the system, and also improve the utilization of the message transmission channel. It does not require starting a separate thread for each high-priority message, thus not consuming too many resources.

[0067] Step S103: Receive isolation control instructions from the monitoring terminal, which include isolation signals.

[0068] Specifically, after receiving a notification message from the object storage device that it has identified itself as a slow disk, the monitoring terminal needs to process the slow disk by sending an isolation control command to the object storage device where the slow disk occurred. The isolation control command may include an isolation signal, which can be generated by the object storage device. For example, the object storage device may register an isolation signal ID during startup and listen for this isolation signal. The monitoring terminal can include this isolation signal when generating the isolation control command. When the object storage device receives this isolation signal, it can activate its own isolation mechanism. In an optional example, the isolation signal can also be generated by the monitoring terminal. The specific method depends on the actual situation and will not be limited here.

[0069] Step S104: Based on the isolation signal, control the object storage device itself to enter the isolation state.

[0070] Specifically, when an object storage device receives an isolation signal, it activates its own isolation mechanism, exits its own process, and enters an isolated state, which facilitates subsequent fault detection or disk replacement.

[0071] The object storage device isolation method provided in this embodiment generates a notification message when the working efficiency of the object storage device is detected to be lower than a preset efficiency threshold. This notification message is then sent to the monitoring end via a preset message transmission channel, allowing the monitoring end to perform slow disk control operations based on the notification message and send isolation control commands to the object storage device. The preset message transmission channel is an independently operating message transmission channel. The method also receives the isolation control commands from the monitoring end, which include an isolation signal. Based on the isolation signal, the object storage device is controlled to enter an isolation state. When the object storage device detects that its working efficiency is lower than the preset efficiency threshold, it identifies itself as a slow disk and generates a slow disk message, which is transmitted to the monitoring end via an independent message transmission channel, such as a separate thread. This allows for timely transmission of slow disk messages to the monitoring end for processing, eliminating the need for them to wait in a queue with other pending messages. This enables timely notification to the monitoring end for slow disk control operations. After completing the slow disk control operations, the control end can also promptly notify the object storage device to complete its isolation settings. This significantly reduces the problem of message backlog and front-end business lag caused by slow disk isolation delays.

[0072] This embodiment provides an object storage device isolation method, which can be used for the aforementioned object storage devices, such as OSDs. Figure 2 This is a flowchart of an object storage device isolation method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0073] Step S201: When the working efficiency of the object storage device itself is detected to be lower than a preset efficiency threshold, a notification message is generated.

[0074] Specifically, step S201 above includes:

[0075] Step S2011: Obtain the pre-created isolation signal corresponding to the object storage device, the network address of the node where the object storage device is located, the device identifier of the object storage device, the slot information of the object storage device, and the drive letter information of the object storage device.

[0076] Specifically, the isolation signal corresponding to the pre-created object storage device can be a pre-registered isolation signal. When the object storage device is identified as a slow disk, the object storage device obtains the pre-registered isolation information ID, the network address of the node where the object storage is located (i.e., the IP address of the node), the device identifier of the object storage device (e.g., device ID), and the slot information of the object storage device, and assembles this information.

[0077] Step S2012: Generate a notification message based on the isolation signal, network address, device identifier, slot information, and drive letter information.

[0078] Specifically, isolation signals, network addresses, device identifiers, slot information, and drive letter information can be assembled to generate a notification message indicating that the object storage device itself is experiencing a slow disk phenomenon, and the information required by the monitoring end can be sent to the monitoring end all at once.

[0079] In an optional example, in addition to the information mentioned above, other information may be included, such as the attribute information of the object storage device, which is needed when handling slow disk isolation. This information is then assembled to generate a notification message and sent to the monitoring end so that the monitoring end can perform the corresponding processing.

[0080] In step S202, the notification message is sent to the monitoring terminal through a preset message transmission channel so that the monitoring terminal can complete the slow disk control operation according to the notification message and send the isolation control command to the object storage device.

[0081] Step S203: Receive the isolation control command fed back from the monitoring terminal. The isolation control command includes an isolation signal.

[0082] Step S204: Based on the isolation signal, the control object storage device itself enters the isolation state. For details of steps S202, S203, and S204, please refer to [link to relevant documentation]. Figure 1 Steps S102, S103, and S104 in the illustrated embodiment will not be described again here.

[0083] By using the above method, when the working efficiency of the object storage device is detected to be lower than the preset efficiency threshold, all the information required by the monitoring end to perform slow disk control operation is obtained, and a notification message is generated based on the above information. The information required by the monitoring end is sent at once, which can not only improve the sending efficiency, but also avoid the problem of incomplete message reception due to abnormalities or other reasons when sending multiple times, thus preventing the slow disk control operation from being completed.

[0084] In one optional implementation, the object storage device isolation method further includes:

[0085] Use a pre-created, independent listening thread to listen for isolation control commands.

[0086] Specifically, each object storage device in the distributed cluster can register a dedicated isolation semaphore at startup and start a separate thread to listen for it. The advantage of this is that the thread runs independently and is only responsible for listening to the isolation semaphore. The monitoring end can broadcast the isolation signal by message subscription. Because the object storage device has set up a listening thread and runs independently, it can quickly receive its own isolation signal, start its own isolation mechanism, and complete the isolation operation in a timely manner.

[0087] This embodiment provides a method for isolating object storage devices, which can be used on a monitoring end, such as in a distributed storage system like a Monolithic Object (MON). Figure 3 This is a flowchart of an object storage device isolation method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0088] Step S301: When a notification message is received through the message transmission channel between the monitoring terminal and the object storage device, the slow disk control operation is completed according to the notification message.

[0089] Specifically, the monitoring end can develop a dedicated message transmission channel for transmitting slow disk isolation messages. For example, a dedicated interface can be developed to listen for notification messages from object storage devices, which refer to slow disk messages. When a notification message is received, it is parsed, and the parsed message is used to perform slow disk control operations on the object storage device where the slow disk phenomenon has occurred.

[0090] In an optional example, slow disk control operations may include limiting the number or type of messages received by the object storage device experiencing slow disk issues, or reallocating messages belonging to that object storage device, etc.

[0091] Step S302: Send an isolation control command to the object storage device so that the object storage device can perform isolation operations according to the isolation control command.

[0092] Specifically, after the monitoring terminal completes the parsing of the notification message, it sends an isolation control command to the object storage device. For example, it can broadcast the isolation identifier of the parsed object storage device by means of message subscription. The isolation identifier can be an isolation signal ID. After receiving the isolation signal, the corresponding object storage device can start its own isolation operation in a timely manner.

[0093] In one alternative example, slow disk control operations and the sending of isolation control commands can be performed simultaneously, using a parallel processing approach. This results in higher isolation efficiency and faster completion of slow disk isolation. Alternatively, the slow disk control operations can be completed first, followed by the sending of isolation control commands. This ensures that all other necessary processing tasks are completed before slow disk isolation, thus guaranteeing a higher degree of completion of the slow disk isolation process.

[0094] In this manner, when a notification message is received through the message transmission channel between the monitoring terminal and the object storage device, a slow disk control operation is performed based on the notification message. This message transmission channel is an independently running thread channel. An isolation control command is sent to the object storage device, enabling it to isolate itself according to the command. Receiving notification messages through an independently running thread channel ensures efficient message transmission, timely completion of slow disk control operations, and prompt sending of isolation control commands to the object storage device. This ensures timely isolation of the slow disk, allowing front-end services to be promptly allocated to normal object storage devices for processing, guaranteeing smooth operation of front-end services and ensuring system stability and high availability.

[0095] In an optional implementation, when the notification message includes: an isolation signal corresponding to the object storage device and the network address of the node where the object storage device is located; step S302 specifically includes the following method steps:

[0096] Step a1: Determine the node where the object storage device is located by using its network address.

[0097] Specifically, the network address is used to locate the node where the object storage device is located, such as the IP address of the node where the object storage device is located.

[0098] Step a2: Establish a remote connection between the monitoring terminal and the node.

[0099] Specifically, since the monitoring terminal is located on a different node than the object storage device experiencing the slow disk issue, a remote connection can be established between the monitoring terminal and the node where the object storage device experiencing the slow disk issue is located. Remote connection commands, such as SSH commands, can be used to jump to the node where the object storage device is located.

[0100] Step a3: Use the node to send an isolation signal to the object storage device.

[0101] Specifically, the node where the slow disk phenomenon occurs sends an isolation signal to the object storage device where the slow disk phenomenon occurs.

[0102] By using the above method, the node where the object storage device is located is determined by the network address, and a remote connection is established between the monitoring terminal and the node. The isolation signal is then sent to the object storage device through the node, ensuring that the monitoring terminal can successfully send the isolation signal to the object storage device from any node.

[0103] In an optional implementation, the notification message further includes the device identifier of the object storage device, and step S301 above includes:

[0104] Step b1: Control the storage device to stop read / write operations based on the device identifier.

[0105] Specifically, the object storage device experiencing slow disk behavior is identified based on the device identifier, and the device is controlled to stop read and write operations.

[0106] In one optional implementation, step b1 specifically includes the following method steps:

[0107] Step b11: Search for the status of the object storage device in the preset status table based on the device identifier.

[0108] Specifically, the preset status table can be a read / write operation status table, such as the OSDMap table in a distributed object storage system. In this status table, different status values ​​represent the working status of the object storage device. For example, the status of an object storage device OSD that has a heartbeat and can read and write normally is UP (online), and the status of an object storage device that has stopped reading and writing is DOWN (offline). The status of the corresponding object storage device can be queried from this table based on the device identifier of the object storage device.

[0109] Step b12: Mark the object storage device as offline. The offline status is used to indicate that the object storage device has stopped reading and writing operations.

[0110] Specifically, the status of the identified object storage devices, i.e., those experiencing slow disk issues, is changed to DOWN. When an object storage device is marked as offline, the distributed storage system will stop read and write operations on that device. This allows front-end messages to be promptly distributed to other working object storage devices, ensuring timely processing.

[0111] In an optional implementation, when the notification message also includes slot information and drive letter information of the object storage device, step S301 above includes:

[0112] Step c1: Generate slow disk alarm information based on slot information and disk letter information.

[0113] Specifically, slow disk alarm information is generated based on slot information and drive letter information. Slot information indicates the location of the object storage device in the distributed cluster (e.g., node or partition), and drive letter information is the identifier of the object storage device. These two pieces of information can be used to accurately locate the object storage device.

[0114] Step c2: Send alarm information to the terminal device. The alarm information is used to indicate the replacement of the object storage device.

[0115] Specifically, alarm information is sent to the user's terminal device. The alarm method can be email, SMS, or highlighted on the device's visualization page, etc. Users can easily locate and replace the object storage device experiencing slow disk behavior based on the alarm information.

[0116] To make the object storage device isolation method provided by this invention easier to understand, this invention also provides an embodiment of a specific application scenario, such as the distributed file storage system Ceph, which includes multiple object storage devices (OSDs) and a monitoring endpoint (MON). OSDs are storage nodes responsible for storing and retrieving data. Each OSD instance manages a local file system that stores a portion of the data stored in the cluster. Multiple OSDs can form a logical group called a CRUSH rule, allowing the selection of a target OSD based on data type and replication level. This achieves load balancing and failover in Ceph, ensuring data reliability and high availability. The main task of the MON is to maintain the consistency of the cluster view. In maintaining consistency, the Paxos algorithm (protocol) is used and instantiated in the database for easy subsequent access. In the storage architecture, the MON is one of the key management nodes. They are core components of the cluster, monitoring the health status of all other nodes, monitoring metadata and configuration, and ensuring the consistency of all parts of the cluster.

[0117] Object storage device isolation methods based on the distributed file storage system Ceph, such as... Figure 4 As shown below, for ease of explanation, the slow disks listed are the OSDs where the slow disk phenomenon occurs. Figure 4The object storage device isolation method shown is divided into MON and OSD sides. The OSD side includes a slow disk message sending module and a slow disk isolation information listening module. The MON side can be divided into a slow disk message receiving module, a slow disk message processing module, and a slow disk isolation signal sending module. The way OSD notifies MON of slow disks no longer uses the existing network message mechanism. The MON side develops an interface to listen for slow disk messages from OSDs. After detecting a slow disk, the OSD directly calls this interface to notify the MON service. The slow disk message sending module is used to identify itself as a slow disk when the OSD performs its own read and write operations. It assembles information such as its registered slow disk isolation signal ID, the disk location information of the OSD, the OSD ID, and the network IP address of the node where the OSD is located into a slow disk message, and calls the slow disk message receiving interface provided by MON to inform MON of the slow disk message. The slow disk message receiving module can implement an interface to receive slow disk messages sent by OSDs, and the message receiving thread runs independently. Upon receiving a slow disk message, the slow disk message processing module parses the message. Based on the parsed OSD slow disk message, including the slow disk isolation signal ID, the disk location information of the OSD, the OSD ID, and the network IP of the node where the OSD resides, the module assembles the necessary information into a slow disk alarm message and reports it to the user in the form of an alarm, informing the user that a specific disk on a specific node has become slow and requesting timely replacement. The module also marks the OSD as DOWN in the OSDMap status table and calls the slow disk isolation signal sending module to isolate the slow OSD. The slow disk isolation signal sending module can jump to the node where the slow disk resides via SSH commands and send a signal based on the slow disk isolation signal ID. Since the slow disk has registered to listen for this signal during startup, it will receive the slow disk isolation signal. When the OSD's slow disk isolation signal listening module receives the slow disk isolation signal, it exits its own process, completing the fault isolation.

[0118] Specific slow disk isolation methods are as follows Figure 5 As shown: When the OSD starts, it initializes the slow disk fault signal listening module, which registers the slow disk isolation signal and starts a separate thread for listening. When the MON starts, it initializes the slow disk message listening module, the slow disk message processing module, and the slow disk fault signal sending module. During the OSD's business read / write operations, if a slow disk is detected, it sends a slow disk message to the MON through the slow disk message sending module. The MON parses the slow disk message and performs complete slow disk handling operations, such as issuing alarm messages, marking the slow disk's status as DOWN in the OSDMap table, etc. It also remotely connects to the node where the slow disk is located via SSH and sends a slow disk isolation signal to the corresponding slow disk based on the slow disk isolation signal ID. After the slow disk receives the slow disk isolation signal, it exits its own process, completing the slow disk isolation.

[0119] This embodiment also provides an object storage device that performs object storage device isolation to implement the above. Figure 1 or Figure 2 The embodiments and preferred embodiments described herein will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0120] This embodiment provides an object storage device that performs object storage device isolation, such as... Figure 6 As shown, it includes: a generation module 601, a first sending module 602, a receiving module 603, and a control module 604;

[0121] The generation module 601 is used to generate a notification message when it is detected that the working efficiency of the object storage device itself is lower than a preset efficiency threshold.

[0122] The first sending module 602 is used to send notification messages to the monitoring terminal through a preset message transmission channel, so that the monitoring terminal can complete the slow disk control operation according to the notification message and send isolation control instructions to the object storage device. The preset message transmission channel is an independently operating message transmission channel.

[0123] The receiving module 603 is used to receive isolation control commands fed back from the monitoring terminal, and the isolation control commands include isolation signals;

[0124] The control module 604 is used to control the object storage device to enter the isolation state based on the isolation signal.

[0125] In some alternative implementations, the generation module 601 is specifically used for:

[0126] Obtain the pre-created isolation signal corresponding to the object storage device, the network address of the node where the object storage device is located, the device identifier of the object storage device, the slot information of the object storage device, and the drive letter information of the object storage device; generate a notification message based on the isolation signal, network address, device identifier, slot information, and drive letter information.

[0127] In some alternative implementations, it also includes: a listening module 605;

[0128] The listening module 605 is used to listen for isolation control commands using a pre-created independent listening thread.

[0129] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0130] This embodiment also provides a monitoring terminal for implementing object storage device isolation, used to achieve the above. Figure 3 The embodiments and preferred embodiments, which have already been described, will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0131] This embodiment provides a monitoring terminal for performing object storage device isolation, such as... Figure 7 As shown, it includes: a slow disk control module 701 and a second sending module 702;

[0132] The slow disk control module 701 is used to complete the slow disk control operation according to the notification message when a notification message is received through the message transmission channel between the monitoring terminal and the object storage device. The message transmission channel is an independently running thread channel.

[0133] The second sending module 702 is used to send isolation control commands to the object storage device so that the object storage device can perform isolation operations on itself according to the isolation control commands.

[0134] In some optional implementations, when the notification message in the slow disk control module 701 includes the isolation signal corresponding to the object storage device and the network address of the node where the object storage device is located, the second sending module 702 includes:

[0135] The determination unit is used to determine the node where the object storage device is located by using the network address;

[0136] The remote connection unit is used to establish a remote connection between the monitoring terminal and the node;

[0137] The first transmitting unit is used to send isolation signals to the object storage device using the node.

[0138] In some optional implementations, when the notification message in the slow disk control module 701 includes the device identifier of the object storage device, the second sending module 702 includes:

[0139] The stop unit is used to control the storage device to stop read and write operations based on the device identifier.

[0140] In some alternative implementations, the stopping unit specifically includes:

[0141] The lookup sub-unit is used to search for the status of the object storage device in a preset status table based on the device identifier.

[0142] The tag subunit is used to mark the state of the object storage device as offline, which indicates that the object storage device should stop reading and writing operations.

[0143] In some optional implementations, when the notification message in the slow disk control module 701 includes slot information and disk letter information of the object storage device, the second sending module 702 includes:

[0144] The generation unit is used to generate slow disk alarm information based on slot information and disk letter information;

[0145] The second sending unit is used to send alarm information to the terminal device, and the alarm information is used to indicate the replacement of the object storage device.

[0146] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0147] In this embodiment, both the object storage device and the monitoring terminal for object storage device isolation are presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit), a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0148] This invention also provides a computer device having the above-described features. Figure 6 The object storage device shown is an object storage device isolated from the execution object storage device, or has the above-mentioned object storage device isolation. Figure 7 The monitoring terminal shown is isolated from the execution object storage device.

[0149] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0150] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0151] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0152] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0153] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0154] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0155] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0156] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for isolating object storage devices, characterized in that, The method is executed by an object storage device, and the method includes: When the working efficiency of the object storage device itself is detected to be lower than a preset efficiency threshold, a notification message is generated. The notification message is sent to the monitoring terminal through a preset message transmission channel, so that the monitoring terminal can complete the slow disk control operation according to the notification message and send the isolation control command to the object storage device. The preset message transmission channel is an independently operating message transmission channel. Receive isolation control commands fed back from the monitoring terminal, wherein the isolation control commands include isolation signals; Based on the isolation signal, the object storage device itself is controlled to enter an isolation state.

2. The method according to claim 1, characterized in that, When the working efficiency of the object storage device is detected to be lower than a preset efficiency threshold, a notification message is generated, specifically including: Obtain the pre-created isolation signal corresponding to the object storage device, the network address of the node where the object storage device is located, the device identifier of the object storage device, the slot information of the object storage device, and the drive letter information of the object storage device; The notification message is generated based on the isolation signal, the network address, the device identifier, the slot information, and the drive letter information.

3. The method according to claim 1 or 2, characterized in that, The method further includes: The isolation control commands are monitored using a pre-created independent listening thread.

4. A method for isolating object storage devices, characterized in that, The method is executed by the monitoring terminal, and the method includes: When a notification message is received through the message transmission channel between the monitoring terminal and the object storage device, the slow disk control operation is completed according to the notification message. The message transmission channel is an independently running thread channel. An isolation control command is sent to the object storage device so that the object storage device can isolate itself according to the isolation control command.

5. The method according to claim 4, characterized in that, When the notification message includes an isolation signal corresponding to the object storage device and the network address of the node where the object storage device is located, sending the isolation control command to the object storage device includes: The node where the object storage device is located is determined by the network address; Establish a remote connection between the monitoring terminal and the node; The isolation signal is sent to the object storage device using the node; or, When the notification message also includes the slot information and drive letter information of the object storage device, the notification message is received through the message transmission channel between the monitoring terminal and the object storage device. Based on the notification message, the slow disk control operation is completed, specifically including: A slow disk alarm message is generated based on the slot information and the disk letter information; The alarm information is sent to the terminal device, and the alarm information is used to instruct the replacement of the object storage device; or, When the notification message also includes the device identifier of the object storage device, the notification message is received through the message transmission channel between the monitoring terminal and the object storage device. Based on the notification message, the slow disk control operation is completed, including: The object storage device is controlled to stop read and write operations based on the device identifier.

6. The method according to claim 5, characterized in that, The step of stopping the read / write operation of the object storage device based on the device identifier includes: The status of the object storage device is searched in a preset status table based on the device identifier. The object storage device is marked as offline, which indicates that the object storage device should stop reading and writing operations.

7. An object storage device that performs object storage device isolation, characterized in that, The object storage device includes: The generation module is used to generate a notification message when it detects that the working efficiency of the object storage device itself is lower than a preset efficiency threshold. The first sending module is used to send the notification message to the monitoring terminal through a preset message transmission channel, so that the monitoring terminal can complete the slow disk control operation according to the notification message and send the isolation control command to the object storage device. The preset message transmission channel is an independently operating message transmission channel. The receiving module is used to receive the isolation control command fed back by the monitoring terminal, wherein the isolation control command includes an isolation signal; The control module is used to control the object storage device to enter an isolation state based on the isolation signal.

8. A monitoring terminal for performing object storage device isolation, characterized in that, The monitoring terminal includes: The slow disk control module is used to complete the slow disk control operation according to the notification message when a notification message is received through the message transmission channel between the monitoring terminal and the object storage device. The message transmission channel is an independently running thread channel. The second sending module is used to send an isolation control command to the object storage device so that the object storage device can perform isolation operations on itself according to the isolation control command.

9. A computer device, characterized in that, include: A memory and a processor are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the object storage device isolation method of any one of claims 1 to 3, or to perform the object storage device isolation method of any one of claims 4 to 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the object storage device isolation method of any one of claims 1 to 3, or to perform the object storage device isolation method of any one of claims 4 to 6.

Images