Distributed storage system hard disk management method, device, equipment and storage medium

Abstract

The embodiment of the present invention provides a hard disk management method, device, equipment and storage medium of a distributed storage system, including: a management end receives an offline request for a first target hard disk generated by an operation and maintenance service end in response to a second client, and sends an offline instruction to a first target storage node where the first target hard disk is located. The first target storage node responds to the offline instruction through the running storage service, deletes the first target hard disk from the online hard disk cache set, and sets the read and write handles of the first target hard disk to a switchable state, so that the first client disconnects the first session. The management end determines that all first session connections are disconnected, and forwards the first target hard disk offline completion information to the second client through the operation and maintenance service end. This can prevent the hard disk from being damaged during the hardware removal process without interrupting the business, thereby ensuring the safety of the hard disk.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a hard disk management method, device, equipment and storage medium of a distributed storage system. Background Art

[0002] With the advancement of science and technology, the amount of data that business systems need to store is also growing rapidly. Compared with traditional storage methods, distributed storage technology can better meet the needs of storage management, storage scalability, and fault tolerance of massive data. Distributed storage systems usually contain multiple storage nodes, and each storage node contains a large number of physical hard disks as storage media. In order to ensure the reliability of the storage system, the system usually stores data in multiple hard disks of multiple nodes.

[0003] As the distributed storage system is used for a long time and the amount of data continues to increase, it is also a very common situation in the distributed storage system that the capacity is insufficient and there is a need for expansion or there is a physical hard disk failure that needs to be replaced. When adding a new storage node in the storage system expansion scenario, in order to avoid data imbalance, it is usually necessary to replace the hard disk by pulling out the disk and floating the disk to achieve rapid balance; in addition, when the physical hard disk fails, the physical hard disk is replaced by hot-plugging the hard disk. If the disk is forcibly unplugged when the business is running normally, there is a certain probability that the original data information of the hard disk will be damaged. Summary of the invention

[0004] The embodiments of the present invention provide a hard disk management method, device, equipment and storage medium of a distributed storage system, which are used to solve the problem in the prior art that the hard disk may be damaged when the distributed storage system is forced to remove the hard disk when the business is running normally.

[0005] An embodiment of the present invention provides a hard disk management method for a distributed storage system, which is applied to a management end in the distributed storage system, and includes:

[0006] Receive an offline request for at least one first target hard disk generated by a second client from an operation and maintenance service end, and send an offline instruction to the first target storage node where each of the first target hard disks is located; so that the first target storage node responds to the offline instruction through the running storage service, deletes the first target hard disk from the online hard disk cache set, and sets the read and write handles of the first target hard disk to a switchable state, so that the first client disconnects the first session; wherein the first target hard disk is a hard disk in the target hard disk; the target hard disk is at least one hard disk in the online hard disk cache set of at least one target storage node determined in response to the operation request sent by the first client; for any of the target storage nodes , the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; the first session is a session corresponding to the first target hard disk; for any session, the session is a communication connection between one of the target storage nodes and one of the first clients, and the session corresponds to one of the target hard disks of the target storage node, the session is used to transmit read and write data of the corresponding target hard disk and send a read and write handle of the target hard disk to the corresponding first client, and the first client maintains the connection of the first session according to the read and write handle of the operation status sent by the first target node through the first session;

[0007] It is determined that all the first session connections are disconnected, and the first target hard disk offline completion information is sent to the second client through the operation and maintenance server.

[0008] Optionally, after determining that all the first session connections are disconnected, the method further includes:

[0009] For any of the first target hard disks, in response to a write request to the first target hard disk sent by the first client, determine the second target hard disk and notify the second target hard disk to update the read and write handles of the second target hard disk to an operating state at the second target storage node where the second target hard disk is located, and notify the second target storage node to establish a second session connection with the first client, so that the first client can write data to the second target hard disk through the second session; wherein, the second target hard disk is a hard disk in the online hard disk cache set of the second target storage node, and the second session corresponds to the second target hard disk.

[0010] Optionally, the target hard disk is determined by:

[0011] Determine at least one target storage node, and send first target hard disk query information indicating a target hard disk requirement to each of the target storage nodes;

[0012] Receive first feedback information indicating at least one target hard disk sent by each of the target storage nodes, and determine the target hard disk according to the first feedback information.

[0013] Optionally, determining the second target hard disk includes:

[0014] Sending second target hard disk query information indicating a second target hard disk requirement to the first target storage node;

[0015] Receive second feedback information indicating a second target hard disk sent by the first target storage node, and determine the second target hard disk according to the second feedback information.

[0016] Based on the same inventive concept, an embodiment of the present invention further provides a hard disk management method for a distributed storage system, which is applied to a target storage node in the distributed storage system, and includes:

[0017] In response to the offline instruction sent by the management end, the storage service deletes any first target hard disk from the online hard disk cache set, and updates all read and write handles of the first target hard disk to a switchable state; so that each of the first clients can disconnect the corresponding first session connection after failing to determine that the read and write handles of the first target hard disk are in the operation state; wherein the offline instruction is used to instruct at least one of the first target hard disks to go offline;

[0018] Among them, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; for any first session, the first session is a communication connection established with a first client after the read and write handle of the first target hard disk is updated to an operating state according to a notification from the management end, the first session is used to transmit read and write data of the corresponding first target hard disk and send the read and write handle of the first target hard disk to the corresponding first client, and the read and write handle in the operating state is used to instruct the first client to maintain the first session connection.

[0019] Optionally, after sending the switchable state read / write handle of the first target hard disk to the corresponding first client through each of the first sessions corresponding to the first target hard disk, the method further includes:

[0020] According to the notification of the management end, the read / write handle of at least one second target hard disk is updated to an operation state through the storage service, and for any second target hard disk, a second session connection with at least one first client is established according to the notification of the management end; wherein the second target hard disk is a hard disk in the online hard disk cache set, and the second session corresponds to the second target hard disk;

[0021] For any second session, the data uploaded by the first client is received through the storage service and saved to the second target hard disk; and the read and write handle of the second target hard disk is sent to the first client through the storage service.

[0022] Optionally, the first target hard disk is determined by:

[0023] Receiving, through the storage service, first target hard disk query information sent by the management end;

[0024] The storage service determines a first target hard disk that meets the target hard disk requirements indicated by the first target hard disk query information from an online hard disk cache set, generates first target hard disk feedback information indicating the first target hard disk, and sends it to the management end.

[0025] Optionally, before updating the read / write handle of the second target hard disk to an operation state through the storage service according to the notification of the management end, the method further includes:

[0026] Receiving, through the storage service, the second target hard disk inquiry information sent by the management end;

[0027] The storage service determines a second target hard disk that meets the second target hard disk requirement indicated by the second target hard disk query information from the online hard disk cache set, generates second target hard disk feedback information and sends it to the management end.

[0028] Optionally, the online hard disk cache set includes an available hard disk cache list and a busy hard disk cache list;

[0029] The method further comprises:

[0030] Updating the status information of each installed hard disk in real time through the storage service;

[0031] Whenever the status information of all hard disks installed in the target storage node is updated once, any hard disk in the online hard disk cache set is put into the available hard disk cache list or the busy hard disk cache list according to the status information of the hard disk.

[0032] Optionally, for any hard disk installed in the storage node, according to the status information of the hard disk, the hard disk is put into the available hard disk cache list or the busy hard disk cache list, specifically including at least one of the following:

[0033] Move a first target hard disk from the available hard disk cache list to the busy hard disk cache list; wherein the first target hard disk is a hard disk in the available hard disk cache list and has no available storage space;

[0034] Move the second hard disk from the busy hard disk cache list to the available hard disk cache list; wherein the second hard disk is located in the busy hard disk cache list and has the largest available storage space in the busy hard disk cache list;

[0035] Move the third hard disk from the available hard disk cache list to the busy hard disk cache list; wherein the third hard disk is located in the available hard disk cache list, and the hard disk utilization is the largest in the available hard disk cache list, and the hard disk utilization is greater than a preset hard disk utilization threshold;

[0036] The fourth hard disk and the fifth hard disk are exchanged in the list; wherein the fourth hard disk is located in the available hard disk cache list, and the hard disk utilization is the largest in the available hard disk cache list, and the hard disk utilization is less than a first preset hard disk utilization threshold; the fifth hard disk is located in the busy hard disk cache list, and the hard disk utilization is the smallest in the busy hard disk cache list, and the hard disk utilization is greater than a second preset hard disk utilization threshold;

[0037] The sixth hard disk and the seventh hard disk are exchanged in the list; wherein the sixth hard disk is located in the available hard disk cache list, and its available storage space is the smallest in the available hard disk cache list, and its available storage space is less than a first preset available storage space threshold; the seventh hard disk is located in the busy hard disk cache list, and its available storage space is the largest in the available hard disk cache list, and its available storage space is greater than a second preset available storage space threshold.

[0038] Optionally, the method further comprises:

[0039] When it is determined through the device management module that a new hard disk is installed on the target storage node, the new hard disk is added to the hard disk operation list through the storage service;

[0040] The hard disks in the hard disk operation list are initialized by the storage service, and the hard disks that have completed the initialization operation are added to the online hard disk cache set.

[0041] Based on the same inventive concept, an embodiment of the present invention further provides a management device for a distributed storage system, including:

[0042] An offline module is used to receive an offline request for at least one first target hard disk generated by the operation and maintenance server in response to the second client, and send an offline instruction to the first target storage node where each of the first target hard disks is located; so that the first target storage node responds to the offline instruction through the running storage service, deletes the first target hard disk from the online hard disk cache set, and sets the read and write handles of the first target hard disk to a switchable state, so that the first client disconnects the first session; wherein the first target hard disk is a hard disk in the target hard disk; the target hard disk is at least one hard disk in the online hard disk cache set of at least one target storage node determined in response to the operation request sent by the first client; for any of the target Storage node, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; the first session is a session corresponding to the first target hard disk; for any session, the session is a communication connection between one of the target storage node and one of the first clients, and the session corresponds to one of the target hard disks of the target storage node, the session is used to transmit read and write data of the corresponding target hard disk and send a read and write handle of the target hard disk to the corresponding first client, and the first client maintains the connection of the first session according to the read and write handle of the operation status sent by the first target node through the first session;

[0043] The offline completion module is used to determine that all the first session connections are disconnected, and forward the first target hard disk offline completion information to the second client through the operation and maintenance server.

[0044] Based on the same inventive concept, an embodiment of the present invention further provides a storage device of a distributed storage system, including:

[0045] The offline module is used to respond to the offline instruction sent by the management end through the storage service, delete any first target hard disk from the online hard disk cache set, and update all the read and write handles of the first target hard disk to a switchable state; so that each first client can disconnect the corresponding first session connection after failing to determine that the read and write handles of the first target hard disk are in the operation state; wherein the offline instruction is used to instruct at least one of the first target hard disks to go offline; wherein the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; for any first session, the first session is a communication connection established with one of the first clients after the read and write handles of the first target hard disk are updated to the operation state according to the notification of the management end, the first session is used to transmit the read and write data of the corresponding first target hard disk and send the read and write handles of the first target hard disk to the corresponding first client, and the read and write handles in the operation state are used to instruct the first client to maintain the first session connection.

[0046] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, comprising: a processor and a memory for storing instructions executable by the processor;

[0047] The processor is configured to execute the instructions to implement the hard disk management method of the distributed storage system applied to the management end or the target storage node.

[0048] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is used to implement the hard disk management method of the distributed storage system applied to the management end or the target storage node.

[0049] The beneficial effects of the present invention are as follows:

[0050] The hard disk management method, apparatus, device and storage medium of the distributed storage system provided by the embodiments of the present invention first delete the first target hard disk from the online hard disk cache set to avoid responding to the read and write IO of the first client again during the offline process of the first target hard disk; and also notify the target storage node to modify the read and write handle of the first target hard disk to disconnect the session corresponding to the first target hard disk, and then go offline and remove it when there is no data read and write operation on the first target hard disk, thereby avoiding damage to the hard disk during the hardware removal process without interrupting the business, thereby ensuring the safety of the hard disk. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of the structure of a distributed storage system in an embodiment of the present invention;

[0052] Figure 2 A schematic flow chart of a hard disk management method for a distributed storage system provided by an embodiment of the present invention and applied to a management end in a distributed storage system;

[0053] Figure 3 One of the flow diagrams of the hard disk management method of the distributed storage system provided by the embodiment of the present invention and applied to any storage node in the distributed storage system;

[0054] Figure 4 A second flow chart of a hard disk management method of a distributed storage system applied to any storage node in the distributed storage system provided by an embodiment of the present invention;

[0055] Figure 5 A schematic flow chart of a hard disk management method of a distributed storage system applied to a target storage node in a distributed storage system provided by an embodiment of the present invention;

[0056] Figure 6 A schematic flow chart of a hard disk management method of a distributed storage system applied to a first target storage node in a distributed storage system provided by an embodiment of the present invention;

[0057] Figure 7 A schematic flow chart of a hard disk management method of a distributed storage system applied to a second target storage node in a distributed storage system provided by an embodiment of the present invention;

[0058] Figure 8 A schematic diagram of a process of searching for a second target hard disk of a target storage node in a distributed storage system provided by an embodiment of the present invention;

[0059] Fig. 9 A schematic diagram of the structure of a management device for a distributed storage system provided by an embodiment of the present invention;

[0060] Fig.10 A schematic diagram of the structure of a storage device of a distributed storage system provided in an embodiment of the present invention;

[0061] Fig.11 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described below with reference to the accompanying drawings and examples. However, the exemplary embodiments can be implemented in various forms and should not be understood as being limited to the embodiments described herein; on the contrary, these embodiments are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the exemplary embodiments to those skilled in the art. The same figure marks in the figures represent the same or similar structures, and thus their repeated descriptions will be omitted. The words expressing position and direction described in the present invention are all explained using the accompanying drawings as examples, but changes can be made as needed, and the changes made are all included in the scope of protection of the present invention. The drawings of the present invention are only used to illustrate the relative position relationship and do not represent the true proportions.

[0063] It should be noted that specific details are described in the following description to facilitate a full understanding of the present invention. However, the present invention can be implemented in a variety of other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The subsequent description of the specification is a preferred embodiment of the present application, but the description is for the purpose of illustrating the general principles of the present application and is not intended to limit the scope of the present application. The scope of protection of the present application shall be determined by the definition of the attached claims.

[0064] First, the architecture of the distributed storage system involved in the embodiment of the present invention is introduced. Figure 1 As shown, the distributed storage system 10 includes: at least one storage node 11, a management terminal 12, and an operation and maintenance service terminal 13. Among them, for any storage node 11, at least one hard disk 11-1 is installed on the storage node. In the specific implementation process, the management terminal 12 and the operation and maintenance service terminal 13 can be hardware-discrete devices, or two virtual software modules implemented by the same hardware device; if the management terminal 12 and the operation and maintenance service terminal 13 are hardware-discrete devices, the management terminal 12 and the operation and maintenance service terminal can be a single hardware device, or a device cluster composed of multiple devices. Each storage node 11 in the distributed storage system is a hardware-discrete device, and the physical location of each storage node 11 is different during the implementation process; for any storage node 11, it can be a single hardware device or a device cluster composed of multiple devices. During the implementation process, the storage node 11 is connected to the first client 21 to perform operations such as data reading and writing with the first client 21. During the implementation process, the operation and maintenance server 13 is connected to the second client 22 to perform related offline operations of the hard disk 11 - 1 , which will be described in detail below.

[0065] The following specifically describes the hard disk management method, device, equipment and storage medium of the distributed storage system provided by the embodiments of the present invention in conjunction with the accompanying drawings.

[0066] The embodiment of the present invention provides a hard disk management method for a distributed storage system, which is applied to a management end in the distributed storage system, such as Figure 2 As shown, including:

[0067] S110. In response to the operation request sent by the first client, determine at least one target hard disk of at least one target storage node. For any target storage node, the target hard disk on the target storage node is a hard disk in the online hard disk cache set of the target storage node. For any target storage node, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection.

[0068] In the specific implementation process, the operation request may include a data read request, a data write request, etc. The operation request sent by the first client may correspond to one target hard disk or multiple target hard disks. If the operation request sent by the first client corresponds to multiple target hard disks, these target hard disks may be installed in the same target storage node or in different target storage nodes.

[0069] It should be noted that the term "first client" here does not refer to a specific client, but generally refers to a client that performs operations on the target hard disk request.

[0070] S120, notifying each of the target hard disks to update the read / write handle of the corresponding target hard disk to an operation state at the target storage node where it is located, and notifying each of the target storage nodes to establish a session connection with the first client. The read / write handle in the operation state is the read / write handle corresponding to the current operation of the target hard disk. For example, if the current operation of the target hard disk is to read data, then the read / write handle is a read handle; if the current operation of the target hard disk is to write data, then the read / write handle is a write handle.

[0071] For any session, the session is a communication connection between one of the target storage nodes and one of the first clients, and the session corresponds to one of the target hard disks of the target storage node. The session is used to transmit read and write data of the corresponding target hard disk and send the read and write handles of the target hard disk to the corresponding first client, and the first client maintains the connection of the first session based on the read and write handles of the operation status received from the first target node through the first session. In the specific implementation process, the target storage node can periodically send the heartbeat packets of the read and write handles of the target hard disk corresponding to the session to the first client through the session to maintain the connection of the session. For example, the operation request of the first client corresponds to three target hard disks A, B, and C, then the target storage node where the three target hard disks are located will establish three sessions corresponding to A, B, and C with the first client.

[0072] S130, receiving a response from the operation and maintenance server to a request for the offline operation of at least one of the first target hard disks generated by the second client, and sending an offline instruction to the first target storage node where each of the first target hard disks is located. The first target storage node responds to the offline instruction through the running storage service, deletes the first target hard disk from the online hard disk cache set, and sets the read and write handles of the first target hard disk to a switchable state, so that the first client disconnects the first session. The first target hard disk is a hard disk in the target hard disk; the first session is a session corresponding to the first target hard disk.

[0073] In the specific implementation process, an interactive page (such as a web page, etc.) can be deployed on the operation and maintenance server, and the user can use the second client to connect to the operation and maintenance server, and trigger an offline request for the first target hard disk by operating on the interactive page. The offline request includes the attribute information of the first target storage node to which the first target hard disk belongs (such as the Internet Protocol (IP) address, etc.), the attribute information of the first target hard disk (such as the number and port number of the first target hard disk in the first target storage node), and other information. In addition to triggering the offline request for the first target hard disk, the operation and maintenance server can also realize functions such as management and display of the operating status of each storage node and each hard disk in the distributed storage system.

[0074] In the specific implementation process, the user can trigger the offline request for the first target hard disk when the first target hard disk fails and needs to be removed for repair, or when the distributed storage system is expanded to achieve rapid data balancing. For the client, the data stored in the first target hard disk will be provided to the client through the backup technology adopted by the distributed system after the first target hard disk goes offline. Since this part is not the focus of this solution, it will not be elaborated.

[0075] S140: Determine that all the first session connections are disconnected, and forward the first target hard disk offline completion information to the second client through the operation and maintenance server.

[0076] Afterwards, the user can remove the first target hard disk after receiving the offline completion information of the first target hard disk through the second client.

[0077] In the specific implementation process, the above-mentioned various requests and information can be transmitted by HyperText Transfer Protocol (HTTP), HyperText Transfer Protocol Secure (HTTPS), etc., which are not limited here. The management end can determine that the first session connection is disconnected through feedback from the first storage node; or the target storage node can use a heartbeat mechanism to periodically send the read and write handle of the target hard disk to the first client, and determine that all first session connections are disconnected after a preset time after executing step S130, which is not limited here.

[0078] In this way, when the user needs to remove the first target hard disk, the embodiment of the present invention first deletes the first target hard disk from the online hard disk cache set to avoid responding to the read and write IO of the first client again during the offline process of the first target hard disk; and also notifies the target storage node to modify the read and write handle of the first target hard disk to disconnect the session corresponding to the first target hard disk, and then performs offline removal when there is no data read and write operation on the first target hard disk, thereby avoiding damage to the hard disk during the hardware removal process without interrupting the business, thereby ensuring the safety of the hard disk.

[0079] If the first client still needs to continue writing data to the distributed storage system after the first target hard disk is offline and removed during the process of writing data to the first target hard disk, then further, after determining that all the first session connections are disconnected, such as Figure 2 As shown, the method also includes:

[0080] S160. For any of the first target hard disks, in response to a write request to the first target hard disk sent by the first client, determine the second target hard disk and notify the second target hard disk to update the read and write handle of the second target hard disk to an operating state at the second target storage node where the second target hard disk is located, and notify the second target storage node to establish a second session connection with the first client, so that the first client can write data to the second target hard disk through the second session.

[0081] The second target hard disk is a hard disk in an online hard disk cache set of the second target storage node, and the second session corresponds to the second target hard disk.

[0082] In the specific implementation process, the first target storage node can be preferentially selected as the second target storage node, or another target storage node can be reselected as the second target storage node. As an optional implementation, if there are hard disks with storage space greater than or equal to the amount of data requested to be written by the first client in the online hard disk cache set of the first target storage node, the first target storage node is selected as the second target storage node, otherwise another target storage node is reselected as the second target storage node.

[0083] In this way, the first client can complete the writing of data and avoid the failure of the writing service caused by the removal of the first target hard disk.

[0084] If the operation request of the first client is a request that does not specify a target hard disk (for example, a write request indicating that data can be written to a non-specific target hard disk), then Figure 2 As shown, the target hard disk is determined by:

[0085] S101. Determine at least one target storage node, and send first target hard disk query information indicating target hard disk requirements to each of the target storage nodes.

[0086] In the specific implementation process, the target storage node can be determined by a preset selection rule. For example, when it is determined that the number of current storage nodes and the number of hard disks meet the preset numerical requirements, the target storage node is selected according to the preset priority of each storage node; otherwise, each target storage node is selected in turn.

[0087] S102: Receive first feedback information indicating at least one target hard disk sent by each of the target storage nodes, and determine the target hard disk according to the first feedback information.

[0088] For any target storage node, after receiving the first target hard disk query information, the target storage node will determine whether there is a target hard disk requirement that meets the first target hard disk query information (including whether the available storage space meets the requirements of the operation request of the first client, whether the number of concurrent connections of a single disk exceeds the set threshold, etc.) among the installed hard disks, and feedback the target hard disk found or the failure information that it is not found to the management end. If the number of target hard disks determined by the first feedback information is insufficient, the management end will reselect the target storage node and repeat the above steps to find the target hard disk again. The process of how to select the target hard disk from the hard disks installed in this target storage node will be explained in detail later, and will not be discussed here for the time being.

[0089] In order to facilitate data management, as an optional implementation, the second target hard disk can still be selected from the first target storage node. Figure 2 As shown, the second target hard disk is determined by:

[0090] S151. Send second target hard disk query information indicating a second target hard disk requirement to the first target storage node.

[0091] S152. Receive second feedback information indicating a second target hard disk sent by the first target storage node, and determine the second target hard disk according to the second feedback information.

[0092] In addition, if the first storage node fails to find the second target hard disk, it will feedback failure information to the management end, and the management end will reselect other storage nodes and send a second target hard disk query information. The selected storage node will then search the corresponding hard disk for a second target hard disk that meets the requirements indicated by the second target hard disk query information, and feedback the found second target hard disk or failure information that it was not found to the management end, so that the management end can obtain the result of the second target hard disk or repeat the above steps to find the second target hard disk again.

[0093] As another optional implementation, the management end may send second target hard disk query information to the storage nodes in sequence according to the preset storage node priority, and determine the second target hard disk based on the storage node feedback information; if the feedback is the second feedback information, the management end finally determines the second target hard disk according to the indication of the second feedback information; if the feedback is failure information, the above process is repeated until the second target storage node, i.e., the second target hard disk, is found.

[0094] For any target storage node, the process of how to select a target hard disk from the hard disks installed in the target storage node will be described in detail later and will not be discussed here for the time being.

[0095] Accordingly, an embodiment of the present invention also provides a hard disk management method for a distributed storage system. The following describes the implementation steps for different types of storage nodes in the distributed storage system. Since the ranges of the different types of storage nodes described below may overlap, if a storage node meets multiple types described below at the same time, the storage node will implement the corresponding steps of each type involved.

[0096] (I) Any storage node. Figure 3 As shown, the corresponding steps specifically include:

[0097] S201. When it is determined through the device management module that a new hard disk is installed on the target storage node, the new hard disk is added to a hard disk operation list through the storage service.

[0098] In the specific implementation process, if the operating system of the storage node adopts the Linux system, the device management tool udev can be used as the device management module. The new hard disk will interact with the system kernel through the Small Computer System Interface (SCSI) protocol. udev listens to the events sent by the kernel. If the occurrence of the device addition event is detected, it is determined that a new hard disk is installed on the target storage node. At this time, the shell script specified in the corresponding matching rule is called to send notification information to the storage service, where the notification information includes the hard disk name and the operation type information used to indicate that the hard disk is online. After receiving the information, the storage service adds the new hard disk to the storage service hard disk operation list.

[0099] S202. Initialize the hard disks in the hard disk operation list through the storage service, and add the hard disks that have completed the initialization operation to the online hard disk cache set. The online hard disk cache set is a set of hard disks in the storage node that allow the first client to perform read and write data operations through a session connection. In the specific implementation process, a separate thread started by the storage service processes the hard disk operations in the hard disk operation list in real time. For a new hard disk, when the storage server determines that the operation is a hard disk online based on the operation type information of the hard disk, the new hard disk is formatted and mounted, and other initialization operations are performed. After the initialization of the new hard disk is completed, the hard disk is added to the online hard disk cache set of the storage service so that the storage service can subsequently select a target hard disk to respond to the read and write IO of the first client.

[0100] Optionally, the online hard disk cache set includes an available hard disk cache list and a busy hard disk cache list. Figure 4 As shown, the method also includes:

[0101] S203: Update the status information of each installed hard disk in real time through the storage service.

[0102] During the specific implementation process, when the storage service is started and initialized, a separate thread can be created to cache and update the status information of all hard disks on this storage node in real time. The specific status information may include: disk slot number, disk utilization, total disk capacity, and disk available space, etc.

[0103] S204. Whenever the status information of all hard disks installed in the target storage node is updated once, any hard disk in the online hard disk cache set is placed in the available hard disk cache list or the busy hard disk cache list according to the status information of the hard disk.

[0104] In the specific implementation process, when the storage service is initialized, the number of hard disks installed online can be proportionally allocated and put into the available hard disk cache list or the busy hard disk cache list. Afterwards, during the operation of the storage node, every time the status information of all hard disks installed in the online hard disk cache set of the target storage node is updated, the available hard disk cache list and the busy hard disk cache list are adjusted in real time according to the updated hard disk status information.

[0105] Then further, for any hard disk in the online hard disk cache set, the hard disk is put into the available hard disk cache list or the busy hard disk cache list according to the status information of the hard disk, which specifically includes at least one of the following:

[0106] (1) Moving a first target hard disk from the available hard disk cache list to the busy hard disk cache list, wherein the first target hard disk is a hard disk in the available hard disk cache list that has no available storage space.

[0107] (2) Moving a second hard disk from the busy hard disk cache list to the available hard disk cache list, wherein the second hard disk is located in the busy hard disk cache list and has the largest available storage space in the busy hard disk cache list.

[0108] (3) Moving the third hard disk from the available hard disk cache list to the busy hard disk cache list, wherein the third hard disk is located in the available hard disk cache list, has the largest hard disk utilization in the available hard disk cache list, and has a hard disk utilization greater than a preset hard disk utilization threshold.

[0109] (4) Swap the positions of the fourth hard disk and the fifth hard disk in the list. The fourth hard disk is located in the available hard disk cache list, and its hard disk utilization is the largest in the available hard disk cache list, and its hard disk utilization is less than a first preset hard disk utilization threshold. The fifth hard disk is located in the busy hard disk cache list, and its hard disk utilization is the smallest in the busy hard disk cache list, and its hard disk utilization is greater than a second preset hard disk utilization threshold.

[0110] (5) Swap the positions of the sixth hard disk and the seventh hard disk in the list. The sixth hard disk is located in the available hard disk cache list, and its available storage space is the smallest in the available hard disk cache list, and its available storage space is smaller than a first preset available storage space threshold. The seventh hard disk is located in the busy hard disk cache list, and its available storage space is the largest in the available hard disk cache list, and its available storage space is larger than a second preset available storage space threshold.

[0111] In this way, the storage node manages each hard disk to accept the read and write operations of the first client by maintaining the available hard disk cache list and the busy hard disk cache list.

[0112] (ii) Target storage node, i.e., a storage node in the target storage node distributed storage system that has a corresponding target hard disk and performs data read and write operations with the first client. The target storage node includes the first target storage node. Figure 5 and Figure 6 As shown, the corresponding steps specifically include:

[0113] S210: updating the read / write handle of the target hard disk to an operation state according to the notification of the management end, and establishing a session connection with the first client.

[0114] In the specific implementation process, the read / write handle of the operation state is the read / write handle corresponding to the current operation of the target hard disk. For example, if the current operation of the target hard disk is to read data, then the read / write handle is a read handle; if the current operation of the target hard disk is to write data, then the read / write handle is a write handle.

[0115] In the specific implementation process, for any target hard disk, the target hard disk may establish session connections with multiple first clients respectively. For example, for target hard disk A, session connections may be established with first clients A, B, and C respectively at the same time.

[0116] S220. During the session maintenance process, the data uploaded by the first client through the session is received through the storage service and saved to the target hard disk, and / or data is read from the target hard disk and sent to the first client through the session; and the read and write handle of the target hard disk is sent to the first client. The read and write handle in the operating state is used to instruct the first client to maintain the first session connection. That is, the session is used to transmit the read and write data of the corresponding first target hard disk and send the read and write handle of the target hard disk to the corresponding first client. Accordingly, for the first session corresponding to the first target hard disk, the first session is used to transmit the read and write data of the corresponding first target hard disk and send the read and write handle of the first target hard disk to the corresponding first client.

[0117] In a specific implementation process, the target storage node may periodically send a heartbeat packet of the read / write handle of the target hard disk corresponding to the session to the first client through the session to maintain the connection of the session.

[0118] Alternatively, if Figure 5 and Figure 6As shown, if the operation request of the first client is a request that does not specify a target hard disk (for example, a write request indicating that data can be written to a non-specific target hard disk), the target hard disk (including the first target hard disk) is determined in the following manner:

[0119] S205: Receive, through the storage service, first target hard disk query information sent by the management end.

[0120] S206: Determine, through the storage service, from the online hard disk cache set a target hard disk that meets the target hard disk requirement indicated by the first target hard disk query information.

[0121] If the target hard disk is successfully determined, step S207 is executed. If the target hard disk is not successfully determined, step S208 is executed.

[0122] S207: Generate first target hard disk feedback information indicating the target hard disk and send it to the management end.

[0123] S208: Return failure information indicating that there is no target hard disk to the management terminal, so that the management terminal can select other storage nodes to find the target hard disk.

[0124] In addition, if the target hard disk is not successfully determined, a failure message indicating that there is no first target hard disk is returned to the management end, so that the management end selects other storage nodes to find the first target hard disk.

[0125] In the specific implementation process, if the online hard disk cache set includes an available hard disk cache list and a busy hard disk cache list, then an optimal hard disk can be selected to respond to IO write by polling the available hard disk cache list and the busy hard disk cache list respectively. Since the operating status of the hard disks in the available hard disk cache list is better than that in the busy hard disk cache list, it can be first determined whether a hard disk with non-zero storage space is cached in the available hard disk cache list. If so, the available hard disk cache list is traversed from the position of the last selected target hard disk recorded in the storage service thread, and it is determined in turn whether the hard disk meets the target hard disk requirements (including whether the available storage space meets the requirements of the operation request of the first client indicated by the first target hard disk query information, whether the number of concurrent single disks exceeds the set threshold, etc.), and the hard disk that meets the requirements is determined as the target hard disk to respond to IO write; if there is no target hard disk that meets the requirements in the available hard disk cache list, then try to traverse the busy hard disk cache list to find a hard disk that meets the requirements as the target hard disk to respond to IO write; if no target hard disk that meets the requirements is found in the available hard disk cache list and the busy hard disk cache list, a failure message indicating that there is no target hard disk can be returned to the management end, so that the management end selects other storage nodes to find the target hard disk.

[0126] (III) The first target storage node, that is, the target storage node where the first target hard disk that is instructed to go offline in the distributed storage system is located. The first target storage node belongs to the target storage node. Figure 6 As shown, after step S220, the following steps are specifically included:

[0127] S230. In response to the offline instruction sent by the management terminal, the storage service deletes any first target hard disk from the online hard disk cache set, and updates all read and write handles of the first target hard disk to a switchable state, so that each first client disconnects the corresponding first session connection. For any of the first sessions, the first session is the session corresponding to the first target hard disk. The offline instruction is used to instruct at least one of the first target hard disks to go offline.

[0128] As an optional implementation, after all read / write handles of the first target hard disk are updated to a switchable state, the read / write handles of the operation state are no longer sent to the corresponding first client through the first session. Afterwards, according to the heartbeat mechanism, when the first client still does not receive the read / write handles of the operation state of the first target hard disk sent by the first target storage node through the first session within a preset time, the first session connection with the first target storage node is disconnected.

[0129] As another optional implementation, after all read / write handles of the first target hard disk are updated to the switchable state, the switchable read / write handles of the first target hard disk are respectively sent to the corresponding first clients through each of the first sessions corresponding to the first target hard disk, so as to notify the first client to disconnect the first session.

[0130] (iv) A second target storage node, i.e., a storage node where a second target hard disk that takes over data writing from the first target hard disk in the distributed storage system is located. The second target storage node may be any storage node that meets the requirements in the distributed storage system, or further may be a target storage node, or further may be a first target storage node. Figure 7 As shown, the corresponding steps specifically include:

[0131] S310: updating the read / write handle of at least one second target hard disk to an operation state through the storage service according to the notification of the management end, and establishing a second session connection with at least one first client according to the notification of the management end for any second target hard disk. The second target hard disk is a hard disk in the online hard disk cache set, and the second session corresponds to the second target hard disk.

[0132] S320: For any of the second sessions, receiving the data uploaded by the first client through the storage service and saving it to the second target hard disk; and sending the read and write handle of the second target hard disk to the first client through the storage service.

[0133] In the specific implementation process, if the second target storage node is the same as the first target storage node, then step S310 is implemented after step S230. The implementation process of the second session is similar to that of the first session, and the relevant content of the first session can be referred to, which will not be elaborated here.

[0134] Furthermore, if Figure 7 As shown, before step S310, the second target storage node determines the second target hard disk in the following manner:

[0135] S301. Receive, through the storage service, second target hard disk query information sent by the management end.

[0136] S302, determining, through the storage service, from the online hard disk cache set a second target hard disk that meets the second target hard disk requirements indicated by the second target hard disk query information, generating second target hard disk feedback information indicating the second target hard disk and sending it to the management end.

[0137] In the specific implementation process, if the online hard disk cache set includes an available hard disk cache list and a busy hard disk cache list, and each time the second target hard disk is selected, a sequential selection is adopted, then the available hard disk cache list and the busy hard disk cache list can be polled to select an optimal hard disk to respond to IO write. Since the operating status of the hard disks in the available hard disk cache list is better than that of the hard disks in the busy hard disk cache list, if Figure 8 As shown, the step S302 can be specifically implemented in the following manner:

[0138] S401, determining whether there is a hard disk in the available hard disk cache list.

[0139] If the result of step S401 is yes, execute step S402; if the result of step S401 is no, execute step S413.

[0140] S402: Determine the location index of the last time a target hard disk was selected from the available hard disk cache list, which is recorded in the storage service thread.

[0141] S403, index=index+1.

[0142] S404: Determine whether the available storage space of the current hard disk is greater than the requirement of the operation request of the first client indicated by the query information of the second target hard disk.

[0143] If the result of step S404 is yes, execute step S405; if the result of step S404 is no, execute step S406.

[0144] S405: Determine whether the current hard disk concurrent number is less than a set threshold.

[0145] If the result of step S405 is yes, execute step S412; if the result of step S405 is no, execute step S406.

[0146] S406: Determine whether the current hard disk is the last hard disk in the available hard disk cache list.

[0147] If the result of step S406 is yes, execute step S407; if the result of step S406 is no, return to step S403.

[0148] S407: Determine the position index of the first hard disk in the busy hard disk cache list in the storage service thread, and execute step S409.

[0149] S409: Determine whether the available storage space of the current hard disk is greater than the requirement of the operation request of the first client indicated by the query information of the second target hard disk.

[0150] If the result of step S409 is yes, execute step S410; if the result of step S409 is no, execute step S411.

[0151] S410, determining whether the current hard disk concurrent number is less than a set threshold.

[0152] If the result of step S410 is yes, execute step S412; if the result of step S410 is no, execute step S411.

[0153] S411, determining whether the current hard disk is the last hard disk in the busy hard disk cache list.

[0154] If the result of step S411 is yes, execute step S413; if the result of step S411 is no, return to step S408.

[0155] S408, index=index+1.

[0156] S412: Determine the current hard disk as the second target hard disk, generate second target hard disk feedback information indicating the second target hard disk and send it to the management end, so as to replace the offline first target hard disk to respond to IO writing.

[0157] S413: Return failure information indicating that there is no second target hard disk to the management terminal, so that the management terminal can select other storage nodes to find the second target hard disk.

[0158] Based on the same inventive concept, an embodiment of the present invention further provides a management device for a distributed storage system, such as Fig. 9 As shown, including:

[0159] The offline module M101 is used to receive an offline request for at least one first target hard disk generated by the operation and maintenance server in response to the second client, and send an offline instruction to the first target storage node where each of the first target hard disks is located; so that the first target storage node responds to the offline instruction through the running storage service, deletes the first target hard disk from the online hard disk cache set, and sets the read and write handles of the first target hard disk to a switchable state, so that the first client disconnects the first session; wherein, the first target hard disk is a hard disk in the target hard disk; the target hard disk is at least one hard disk in the online hard disk cache set of at least one target storage node determined in response to the operation request sent by the first client; for any of the The target storage node, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; the first session is a session corresponding to the first target hard disk; for any session, the session is a communication connection between one of the target storage nodes and one of the first clients, and the session corresponds to one of the target hard disks of the target storage node, the session is used to transmit read and write data of the corresponding target hard disk and send a read and write handle of the target hard disk to the corresponding first client, and the first client maintains the connection of the first session according to the read and write handle of the operation status sent by the first target node through the first session;

[0160] The offline completion module M102 is used to determine that all the first session connections are disconnected, and forward the first target hard disk offline completion information to the second client through the operation and maintenance server.

[0161] Optionally, the device further comprises:

[0162] The replacement module M103 is used to determine the second target hard disk for any of the first target hard disks, in response to a write request to the first target hard disk sent by the first client, and to notify the second target hard disk to update the read and write handle of the second target hard disk to an operating state at the second target storage node where the second target hard disk is located, and to notify the second target storage node to establish a second session connection with the first client, so that the first client can write data to the second target hard disk through the second session; wherein, the second target hard disk is a hard disk in the online hard disk cache set of the second target storage node, and the second session corresponds to the second target hard disk.

[0163] Optionally, the target hard disk is determined by:

[0164] Determine at least one target storage node, and send first target hard disk query information indicating a target hard disk requirement to each of the target storage nodes;

[0165] Receive first feedback information indicating at least one target hard disk sent by each of the target storage nodes, and determine the target hard disk according to the first feedback information.

[0166] Optionally, determining the second target hard disk includes:

[0167] Sending second target hard disk query information indicating a second target hard disk requirement to the first target storage node;

[0168] Receive second feedback information indicating a second target hard disk sent by the first target storage node, and determine the second target hard disk according to the second feedback information.

[0169] Based on the same inventive concept, an embodiment of the present invention further provides a storage device of a distributed storage system, such as Fig.10 As shown, including:

[0170] The offline module M201 is used to respond to the offline instruction sent by the management end through the storage service, delete any first target hard disk from the online hard disk cache set, and update all the read and write handles of the first target hard disk to the switchable state; so that each of the first clients can disconnect the corresponding first session connection after failing to determine that the read and write handles of the first target hard disk are in the operation state; wherein the offline instruction is used to instruct at least one of the first target hard disks to go offline;

[0171] Among them, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; for any first session, the first session is a communication connection established with a first client after the read and write handle of the first target hard disk is updated to an operating state according to a notification from the management end, the first session is used to transmit read and write data of the corresponding first target hard disk and send the read and write handle of the first target hard disk to the corresponding first client, and the read and write handle in the operating state is used to instruct the first client to maintain the first session connection.

[0172] Optionally, the device further comprises:

[0173] The replacement module M202 is used to update the read and write handle of at least one second target hard disk to an operating state through the storage service according to the notification of the management end, and for any second target hard disk, establish a second session connection with at least one first client according to the notification of the management end; wherein the second target hard disk is a hard disk in an online hard disk cache set, and the second session corresponds to the second target hard disk; for any second session, receive the data uploaded by the first client through the storage service and save it to the second target hard disk; and send the read and write handle of the second target hard disk to the first client through the storage service.

[0174] Optionally, the first target hard disk is determined by:

[0175] Receiving, through the storage service, first target hard disk query information sent by the management end;

[0176] The storage service determines a first target hard disk that meets the target hard disk requirements indicated by the first target hard disk query information from an online hard disk cache set, generates first target hard disk feedback information indicating the first target hard disk, and sends it to the management end.

[0177] Optionally, the replacement module M202 is further used for:

[0178] Receiving, through the storage service, the second target hard disk inquiry information sent by the management end;

[0179] The storage service determines a second target hard disk that meets the second target hard disk requirement indicated by the second target hard disk query information from the online hard disk cache set, generates second target hard disk feedback information and sends it to the management end.

[0180] Optionally, the online hard disk cache set includes an available hard disk cache list and a busy hard disk cache list;

[0181] The device also includes:

[0182] A management module M203, used to update the status information of each installed hard disk in real time through the storage service;

[0183] Whenever the status information of all hard disks installed in the target storage node is updated once, any hard disk in the online hard disk cache set is put into the available hard disk cache list or the busy hard disk cache list according to the status information of the hard disk.

[0184] Optionally, for any hard disk installed in the storage node, according to the status information of the hard disk, the hard disk is put into the available hard disk cache list or the busy hard disk cache list, specifically including at least one of the following:

[0185] Move a first target hard disk from the available hard disk cache list to the busy hard disk cache list; wherein the first target hard disk is a hard disk in the available hard disk cache list and has no available storage space;

[0186] Move the second hard disk from the busy hard disk cache list to the available hard disk cache list; wherein the second hard disk is located in the busy hard disk cache list and has the largest available storage space in the busy hard disk cache list;

[0187] Move the third hard disk from the available hard disk cache list to the busy hard disk cache list; wherein the third hard disk is located in the available hard disk cache list, and the hard disk utilization is the largest in the available hard disk cache list, and the hard disk utilization is greater than a preset hard disk utilization threshold;

[0188] The fourth hard disk and the fifth hard disk are exchanged in the list; wherein the fourth hard disk is located in the available hard disk cache list, and the hard disk utilization is the largest in the available hard disk cache list, and the hard disk utilization is less than a first preset hard disk utilization threshold; the fifth hard disk is located in the busy hard disk cache list, and the hard disk utilization is the smallest in the busy hard disk cache list, and the hard disk utilization is greater than a second preset hard disk utilization threshold;

[0189] The sixth hard disk and the seventh hard disk are exchanged in the list; wherein the sixth hard disk is located in the available hard disk cache list, and its available storage space is the smallest in the available hard disk cache list, and its available storage space is less than a first preset available storage space threshold; the seventh hard disk is located in the busy hard disk cache list, and its available storage space is the largest in the available hard disk cache list, and its available storage space is greater than a second preset available storage space threshold.

[0190] Optionally, the device further comprises:

[0191] The new hard disk loading module M204 is used to add the new hard disk to the hard disk operation list through the storage service when it is determined through the device management module that a new hard disk is installed on the target storage node; initialize the hard disks in the hard disk operation list through the storage service, and add the hard disks that have completed the initialization operation to the online hard disk cache collection.

[0192] In the several embodiments provided in the present application, it should be understood that the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0193] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0195] Since the specific manner in which each module of the device performs operations has been described in detail in the embodiment of the hard disk management method of the distributed storage system, it will not be repeated here.

[0196] In a fifth aspect, based on the same inventive concept, an embodiment of the present invention further provides an electronic device, such as Fig.11 As shown, it includes: a processor 110 and a memory 120 for storing executable instructions of the processor 110;

[0197] The processor 110 is configured to execute the instructions to implement the image compression method as described in the first aspect, or to implement the image decompression method as described in the second aspect.

[0198] In the specific implementation process, the device may have relatively large differences due to different configurations or performances, and may include one or more processors 110, memory 120, and computer-readable storage medium 130. The memory 120 and / or computer-readable storage medium 130 include one or more applications 131 or data 132. The memory 120 and / or computer-readable storage medium 130 may also include one or more operating systems 133, such as Windows, Mac OS, Linux, IOS, Android, Unix, FreeBSD, etc. Among them, the memory 120 and the computer-readable storage medium 130 may be short-term storage or persistent storage. The application 131 may include one or more modules ( Fig.11 ), each module may include a series of instruction operations. Further, the processor 110 may be configured to communicate with the computer-readable storage medium 130, and execute a series of instruction operations in the computer-readable storage medium 130 on the device. The device may also include one or more power supplies ( Fig.11 one or more network interfaces 140, the network interface 140 including a wired network interface 141 and / or a wireless network interface 142; one or more input / output interfaces 143.

[0199] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is used to implement the hard disk management method of the distributed storage system.

[0200] The hard disk management method, apparatus, device and storage medium of the distributed storage system provided by the embodiments of the present invention first delete the first target hard disk from the online hard disk cache set to avoid responding to the read and write IO of the first client again during the offline process of the first target hard disk; and also notify the target storage node to modify the read and write handle of the first target hard disk to disconnect the session corresponding to the first target hard disk, and then go offline and remove it when there is no data read and write operation on the first target hard disk, thereby avoiding damage to the hard disk during the hardware removal process without interrupting the business, thereby ensuring the safety of the hard disk.

[0201] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0202] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0203] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0204] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0205] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A hard disk management method for a distributed storage system, characterized in that: The management end used in the distributed storage system includes: Receive an offline request for at least one first target hard disk generated by a second client from an operation and maintenance service end, and send an offline instruction to the first target storage node where each of the first target hard disks is located; so that the first target storage node responds to the offline instruction through the running storage service, deletes the first target hard disk from the online hard disk cache set, and sets the read and write handles of the first target hard disk to a switchable state, so that the first client disconnects the first session; wherein the first target hard disk is a hard disk in the target hard disk; the target hard disk is at least one hard disk in the online hard disk cache set of at least one target storage node determined in response to the operation request sent by the first client; for any of the target storage nodes, The online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; the first session is a session corresponding to the first target hard disk; for any session, the session is a communication connection between one of the target storage node and one of the first clients, and the session corresponds to one of the target hard disks of the target storage node, the session is used to transmit the read and write data of the corresponding target hard disk and send the read and write handle of the target hard disk to the corresponding first client, and the first client maintains the connection of the first session based on the read and write handle of the operation status sent by the first target storage node through the first session; It is determined that all the first session connections are disconnected, and first target hard disk offline completion information is sent to the second client through the operation and maintenance server.

2. The method according to claim 1, characterized in that After determining that all the first session connections are disconnected, the method further includes: For any of the first target hard disks, in response to a write request to the first target hard disk sent by the first client, determine the second target hard disk and notify the second target hard disk to update the read and write handles of the second target hard disk to an operating state at the second target storage node where the second target hard disk is located, and notify the second target storage node to establish a second session connection with the first client, so that the first client can write data to the second target hard disk through the second session; wherein, the second target hard disk is a hard disk in the online hard disk cache set of the second target storage node, and the second session corresponds to the second target hard disk.

3. The method according to claim 1, characterized in that The target hard disk is determined by: Determine at least one target storage node, and send first target hard disk query information indicating a target hard disk requirement to each of the target storage nodes; Receive first feedback information indicating at least one target hard disk sent by each of the target storage nodes, and determine the target hard disk according to the first feedback information.

4. The method according to claim 2, characterized in that Determine the second target hard disk, including: Sending second target hard disk query information indicating a second target hard disk requirement to the first target storage node; Receive second feedback information indicating a second target hard disk sent by the first target storage node, and determine the second target hard disk according to the second feedback information.

5. A hard disk management method for a distributed storage system, characterized in that: The target storage nodes used in the distributed storage system include: The storage service responds to the offline instruction sent by the management end, for any first target hard disk, the first target hard disk is deleted from the online hard disk cache set, and all the read and write handles of the first target hard disk are set and updated to a switchable state; so that each first client can disconnect the corresponding first session connection after failing to determine that the read and write handles of the first target hard disk are in an operating state; wherein the offline instruction is used to instruct at least one of the first target hard disks to go offline; Among them, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; for any first session, the first session is a communication connection established with a first client after the read and write handle of the first target hard disk is updated to an operating state according to a notification from the management end, the first session is used to transmit the corresponding read and write data of the first target hard disk and send the read and write handle of the first target hard disk to the corresponding first client, and the read and write handle in the operating state is used to instruct the first client to maintain the first session connection.

6. The method according to claim 5, characterized in that After sending the switchable state read / write handle of the first target hard disk to the corresponding first client through each of the first sessions corresponding to the first target hard disk, the method further includes: According to the notification of the management end, the read / write handle of at least one second target hard disk is updated to an operation state through the storage service, and for any second target hard disk, a second session connection with at least one first client is established according to the notification of the management end; wherein the second target hard disk is a hard disk in the online hard disk cache set, and the second session corresponds to the second target hard disk; For any second session, the data uploaded by the first client is received through the storage service and saved to the second target hard disk; and the read and write handle of the second target hard disk is sent to the first client through the storage service.

7. The method according to claim 5, characterized in that The first target hard disk is determined by: Receiving, through the storage service, first target hard disk query information sent by the management end; The storage service determines a first target hard disk that meets the target hard disk requirements indicated by the first target hard disk query information from an online hard disk cache set, generates first target hard disk feedback information indicating the first target hard disk, and sends it to the management end.

8. The method according to claim 6, characterized in that Before updating the read / write handle of the second target hard disk to an operation state through the storage service according to the notification of the management end, the method further includes: Receiving, through the storage service, the second target hard disk inquiry information sent by the management end; The storage service determines a second target hard disk that meets the second target hard disk requirement indicated by the second target hard disk query information from the online hard disk cache set, generates second target hard disk feedback information and sends it to the management end.

9. The method according to claim 5, characterized in that The online hard disk cache set includes an available hard disk cache list and a busy hard disk cache list; The method further comprises: Updating the status information of each installed hard disk in real time through the storage service; Whenever the status information of all hard disks installed in the target storage node is updated once, any hard disk in the online hard disk cache set is put into the available hard disk cache list or the busy hard disk cache list according to the status information of the hard disk.

10. The method according to claim 9, characterized in that For any hard disk installed in the target storage node, according to the status information of the hard disk, the hard disk is put into the available hard disk cache list or the busy hard disk cache list, specifically including at least one of the following: Move a first target hard disk from the available hard disk cache list to the busy hard disk cache list; wherein the first target hard disk is a hard disk in the available hard disk cache list and has no available storage space; Move the second hard disk from the busy hard disk cache list to the available hard disk cache list; wherein the second hard disk is located in the busy hard disk cache list and has the largest available storage space in the busy hard disk cache list; Move the third hard disk from the available hard disk cache list to the busy hard disk cache list; wherein the third hard disk is located in the available hard disk cache list, and the hard disk utilization is the largest in the available hard disk cache list, and the hard disk utilization is greater than a preset hard disk utilization threshold; The fourth hard disk and the fifth hard disk are exchanged in the list; wherein the fourth hard disk is located in the available hard disk cache list, and the hard disk utilization is the largest in the available hard disk cache list, and the hard disk utilization is less than a first preset hard disk utilization threshold; the fifth hard disk is located in the busy hard disk cache list, and the hard disk utilization is the smallest in the busy hard disk cache list, and the hard disk utilization is greater than a second preset hard disk utilization threshold; The sixth hard disk and the seventh hard disk are exchanged in the list; wherein the sixth hard disk is located in the available hard disk cache list, and the available storage space is the smallest in the available hard disk cache list, and the available storage space is less than the first preset available storage space threshold; The seventh hard disk is located in the busy hard disk cache list, and has the largest available storage space in the available hard disk cache list, and the available storage space is greater than the second preset available storage space threshold.

11. The method according to claim 5, characterized in that The method further comprises: When it is determined through the device management module that a new hard disk is installed on the target storage node, the new hard disk is added to the hard disk operation list through the storage service; The hard disks in the hard disk operation list are initialized by the storage service, and the hard disks that have completed the initialization operation are added to the online hard disk cache set.

12. A management device for a distributed storage system, characterized in that: include: The offline module is used to receive an offline request for at least one first target hard disk generated by the operation and maintenance server in response to the second client, and send an offline instruction to the first target storage node where each of the first target hard disks is located; so that the first target storage node responds to the offline instruction through the running storage service, deletes the first target hard disk from the online hard disk cache set, and sets the read and write handles of the first target hard disk to a switchable state, so that the first client disconnects the first session; wherein the first target hard disk is a hard disk in the target hard disk; the target hard disk is at least one hard disk in the online hard disk cache set of at least one target storage node determined in response to the operation request sent by the first client; for any of the target storage Node, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; the first session is a session corresponding to the first target hard disk; for any session, the session is a communication connection between one of the target storage nodes and one of the first clients, and the session corresponds to one of the target hard disks of the target storage node, the session is used to transmit the read and write data of the corresponding target hard disk and send the read and write handle of the target hard disk to the corresponding first client, and the first client maintains the connection of the first session according to the read and write handle of the operation status sent by the first target storage node through the first session; The offline completion module is used to determine that all the first session connections are disconnected, and forward the first target hard disk offline completion information to the second client through the operation and maintenance server.

13. A storage device of a distributed storage system, characterized in that: include: The offline module is used to respond to the offline instruction sent by the management end through the storage service, delete any first target hard disk from the online hard disk cache set, and update all read and write handles of the first target hard disk to a switchable state; so that each first client can disconnect the corresponding first session connection after failing to determine that the read and write handles of the first target hard disk are in an operating state; wherein the offline instruction is used to instruct at least one of the first target hard disks to go offline; Among them, the online hard disk cache set is a set of hard disks in the target storage node that allow the first client to perform read and write data operations through a session connection; for any first session, the first session is a communication connection established with a first client after the read and write handle of the first target hard disk is updated to an operating state according to a notification from the management end, the first session is used to transmit the corresponding read and write data of the first target hard disk and send the read and write handle of the first target hard disk to the corresponding first client, and the read and write handle in the operating state is used to instruct the first client to maintain the first session connection.

14. An electronic device, characterized in that: include: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the hard disk management method of the distributed storage system as described in any one of claims 1-4, or to implement the hard disk management method of the distributed storage system as described in any one of claims 5-11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is used to implement the hard disk management method of the distributed storage system as described in any one of claims 1-4, or to implement the hard disk management method of the distributed storage system as described in any one of claims 5-11.

Images