Ceph distributed storage method, device, electronic device and storage medium

Abstract

The present invention relates to a Ceph distributed storage method, device, electronic device and storage medium. The method comprises: creating a cluster process and reading a configuration file, sending a connection request to a monitor, and the monitor establishing a connection channel with a client in response to the connection request. An IO read / write request is sent to a first object storage device of a hot standby non-volatile memory standard, and the first object storage device establishes a read / write channel with the client in response to the IO read / write request. The output byte stream is written to a replica node through the read / write channel, and the input byte stream is read from a master node or a slave node through the read / write channel according to a hot standby time point. The hot standby time point is the time point for switching the first object storage device as a temporary master node. When the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a Ceph distributed storage method, device, electronic device and storage medium. Background Art

[0002] Distributed storage technology is mainly divided into two categories, one is the distributed file system architecture with a centralized architecture, and the other is the completely decentralized distributed storage architecture. Among them, the distributed file system architecture with a centralized architecture is represented by GFS (distributed data system) and HDFS (Hadoop Distributed File System, distributed file management system), while the completely decentralized distributed storage architecture is represented by Ceph (open source distributed storage system), Swift (open source programming language), and GlusterFS (distributed file system).

[0003] When storing each piece of data, Ceph will calculate the location of the data storage and try to distribute the data evenly. At the same time, due to the use of CRUSH (pseudo-random data distribution function) and HASH (hash) algorithms, it does not have traditional single point failures, and the performance will not be affected as the scale expands. When a storage disk has a serious fault such as a bad sector, the storage capacity and performance of the storage system will decrease accordingly. After the faulty disk is removed from the cluster, the PG MAP (PG cluster index) in the cluster will be recalculated, and the cluster data will be balanced again, involving data redistribution. After the newly purchased disk device is added to the cluster, the PG MAP needs to be updated again to rebalance the recovery data. The cluster's data balance IO (input and output flow) will affect the Client's IO, thereby affecting the cluster's business IO, which will have a significant impact on the total time and performance of distributed storage.

[0004] Therefore, the existing distributed storage method needs to rebalance data when a new storage device is added, which affects the IO of the storage cluster, resulting in a longer distributed storage time, lower storage efficiency, and to some extent reduced performance of distributed storage. Summary of the invention

[0005] Based on this, it is necessary to provide a Ceph distributed storage method, device, electronic device and storage medium that can ensure distributed storage efficiency and storage performance when adding new storage devices to address the above technical problems.

[0006] The present invention provides a Ceph distributed storage method, the method comprising:

[0007] Create a cluster process and read a configuration file, send a connection request to a monitor, and the monitor establishes a connection channel with a client in response to the connection request;

[0008] Sending an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, wherein the first object storage device establishes a read / write channel with the client in response to the IO read / write request;

[0009] Writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, wherein the hot standby time point is the time point of switching the first object storage device as a temporary master node;

[0010] When the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

[0011] In one embodiment, the sending of an IO read / write request to a first object storage device of a hot standby non-volatile memory standard, wherein the first object storage device establishes a read / write channel with the client in response to the IO read / write request, comprises:

[0012] When the original master node fails, the cluster PG data in the original master node is called out, and a temporary takeover request is sent to the monitor, and the monitor synchronizes the cluster PG data to the first object storage device in response to the temporary takeover request;

[0013] After receiving the cluster PG data, the first object storage device acts as the temporary master node to replace the original master node and receives data synchronization from the slave node.

[0014] In one embodiment, the IO read / write request includes an input stream request and an output stream request; the IO read / write request is sent to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishes a read / write channel with the client in response to the IO read / write request, and then includes:

[0015] Sending the output stream request to the first object storage device, and after receiving the output stream request, the first object storage device writes the byte stream carried in the output stream request to the first replica node;

[0016] Sending the input stream request to the first object storage device, and after receiving the input stream request, the first object storage device reads data from the master node or the slave node according to the hot standby time point;

[0017] Among them, the replica node, the master node and the slave node are all object storage devices.

[0018] In one embodiment, writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, includes:

[0019] Determine, according to the hot standby time point, whether the first object storage device reads the first target data from the master node in response to the input stream request; if so,

[0020] The first target data is read from the master node first, and when the first target data does not exist in the master node, the first target data is read from the slave node; if not, then

[0021] Read the first target data from the slave node;

[0022] The master node includes a temporary master node, an original master node before a failure, and the second object storage device after becoming the master node.

[0023] In one embodiment, the method further comprises:

[0024] Determine, according to the hot standby time point, whether the first object storage device reads the second target data from the slave node in response to the input stream request; if so,

[0025] The second target data is read from the slave node first, and when the second target data does not exist in the slave node, the second target data is read from the master node; if not, then

[0026] Reading the second target data from the master node;

[0027] The master node includes a temporary master node, an original master node before a failure, and the second object storage device after becoming the master node.

[0028] In one embodiment, when the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node, including:

[0029] After the second object storage device joins the cluster, synchronizing the data of the temporary master node to the second object storage device, and synchronizing the data in the slave node to the second object storage device;

[0030] After the data of the temporary master node is synchronized to the second object storage device, the service of the temporary master node is exited and the storage data in the temporary master node is cleared, and the temporary master node is replaced with the second object storage device as the master node.

[0031] In one embodiment, the method further comprises:

[0032] After the second object storage device replaces the temporary master node as the master node, it receives an IO read / write request from the client, and establishes a read / write channel with the client in response to the IO read / write request;

[0033] The output byte stream carried in the IO read / write request is written to the replica node through the read / write channel, and the input byte stream marked in the IO read / write request is read from the master node or the slave node through the read / write channel according to the hot standby time point, and the input byte stream is stored in the second object storage device.

[0034] The present invention also provides a Ceph distributed storage device, the device comprising:

[0035] A connection request module, used to create a cluster process and read a configuration file, send a connection request to a monitor, and the monitor establishes a connection channel with a client in response to the connection request;

[0036] A read / write request module, configured to send an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, wherein the first object storage device establishes a read / write channel with the client in response to the IO read / write request;

[0037] A data read-write module, configured to write the output byte stream to the replica node through the read-write channel, and read the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, wherein the hot standby time point is the time point of switching the first object storage device as a temporary master node;

[0038] The master node switching module is used to synchronize the stored data from the first object storage device to the second object storage device after the second object storage device joins the cluster, and to format and clear the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

[0039] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the Ceph distributed storage method as described in any one of the above is implemented.

[0040] The present invention also provides a computer storage medium storing a computer program, wherein when the computer program is executed by a processor, the Ceph distributed storage method as described above is implemented.

[0041] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the Ceph distributed storage methods described above.

[0042] The above Ceph distributed storage method, device, electronic device and storage medium, the client creates a cluster process and reads the configuration file, sends a connection request to the monitor, the monitor responds to the connection request and establishes a connection channel with the client, and then sends an IO read and write request to the object storage device of the hot standby non-volatile memory standard, and the object storage device responds to the IO read and write request and establishes a read and write channel with the client. Then, the output byte stream is written to the replica node through the read and write channel, and the input byte stream is read from the master node or the slave node through the read and write channel according to the hot standby time point, and the hot standby time point is the time point when the object storage device of the hot standby non-volatile memory standard is switched as the temporary master node. In addition, when a new object storage device joins the cluster, the object storage device of the hot standby non-volatile memory standard will reversely synchronize the stored data to the new object storage device, and format and clear the stored data after the data synchronization is completed, and the new object storage device is used as the master node. The method can complete the failure service takeover in a short time through the Ceph hot standby service, ensure the stability of the cluster state without loss and reduction of the cluster distributed storage performance, and can also ensure the distributed storage efficiency when adding new storage devices. In addition, by creating device services, the scale of hot standby services can be effectively managed, thereby controlling the waste of spare storage devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 One of the flowcharts of the Ceph distributed storage method provided by the present invention;

[0045] Figure 2 A schematic diagram of a distributed storage architecture of a Ceph distributed storage method in a specific embodiment provided by the present invention;

[0046] Figure 3A schematic diagram of a distributed storage architecture after a new storage device of the Ceph distributed storage method in a specific embodiment of the present invention joins a cluster;

[0047] Figure 4 The second flowchart of the Ceph distributed storage method provided by the present invention;

[0048] Figure 5 The third flowchart of the Ceph distributed storage method provided by the present invention;

[0049] Figure 6 The fourth flowchart of the Ceph distributed storage method provided by the present invention;

[0050] Figure 7 The fifth flowchart of the Ceph distributed storage method provided by the present invention;

[0051] Figure 8 The sixth flowchart of the Ceph distributed storage method provided by the present invention;

[0052] Fig. 9 The seventh flowchart of the Ceph distributed storage method provided by the present invention;

[0053] Fig.10 A schematic diagram of the structure of the Ceph distributed storage device provided by the present invention;

[0054] Fig.11 This is a diagram of the internal structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] Combine the following Figures 1 to 11 The Ceph distributed storage method, device, electronic device and storage medium of the present invention are described.

[0057] like Figure 1 As shown, in one embodiment, a Ceph distributed storage method includes the following steps:

[0058] Step S110, creating a cluster process and reading a configuration file, sending a connection request to the monitor, and the monitor establishing a connection channel with the client in response to the connection request.

[0059] Specifically, the client first creates a cluster process and reads a configuration file, and sends a connection request to the monitor, and the monitor establishes a connection channel with the client in response to the connection request.

[0060] Step S120: Send an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishes a read / write channel with the client in response to the IO read / write request.

[0061] Specifically, the client sends an IO read / write request to the first object storage device of the hot standby non-volatile memory standard. After receiving the IO read / write request, the first object storage device establishes a read / write channel with the client in response to the IO read / write request.

[0062] Step S130, writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, where the hot standby time point is the time point for switching the first object storage device as the temporary master node.

[0063] Specifically, the client writes the output byte stream to the replica node through the read-write channel, and reads the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, where the hot standby time point is the time point for switching the first object storage device as the temporary master node.

[0064] Step S140: After the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

[0065] Specifically, when a new object storage device (i.e., the second object storage device) joins the cluster, the temporary master node (i.e., the first object storage device) reversely synchronizes its stored data to the new object storage device, and formats and clears the stored data after the data synchronization is completed, giving up its master node identity, and the new object storage device replaces the temporary master node as the new master node.

[0066] In the above Ceph distributed storage method, the client creates a cluster process and reads the configuration file, sends a connection request to the monitor, and the monitor establishes a connection channel with the client in response to the connection request, and then sends an IO read and write request to the object storage device of the hot standby non-volatile memory standard, and the object storage device establishes a read and write channel with the client in response to the IO read and write request. Then, the output byte stream is written to the replica node through the read and write channel, and the input byte stream is read from the master node or the slave node through the read and write channel according to the hot standby time point. The hot standby time point is the time point when the object storage device of the hot standby non-volatile memory standard is switched as the temporary master node. In addition, when a new object storage device joins the cluster, the object storage device of the hot standby non-volatile memory standard will reversely synchronize the stored data to the new object storage device, and format and clear the stored data after the data synchronization is completed, and the new object storage device is used as the master node. The method can complete the failure service takeover in a short time through the Ceph hot standby service, ensure the stability of the cluster state without loss and reduction of the cluster distributed storage performance, and can also ensure the distributed storage efficiency when adding new storage devices. In addition, by creating device services, the scale of hot standby services can be effectively managed, thereby controlling the waste of spare storage devices.

[0067] See also Figure 2 As shown, in a specific embodiment, the Ceph distributed storage method provided by the present invention, the client first creates a cluster process and reads the configuration file, and sends a connection request to the monitor Monitor, and the monitor Monitor responds to the connection request to establish a connection channel with the client. The original master node, that is, the original object storage device fails and calls out the cluster PG data, which will actively report to the Monitor (monitor or manager) to inform the hot standby non-volatile memory standard object storage device (i.e. OSD (NVMe)) to temporarily take over the service of the original master node and serve as a temporary master node. At the same time, the slave node (OSD (Replica)) will synchronize the data to the temporary master node OSD (NVMe).

[0068] Subsequently, the IO read and write from the client is directly connected to the temporary master node OSD (NVMe) for reading and writing. When OSD (NVMe) receives the byte output stream in the IO read and write, it will write the corresponding byte stream to other replica nodes. When OSD (NVMe) receives the byte input stream in the IO read and write, it will determine whether to read data from the master node or from the specified slave node based on the hot standby time point. If it is reading data from the master node, it will read data from the master node first, and read the corresponding data from the slave node when there is no data to be read in the master node.

[0069] In this embodiment, combined with Figure 3As shown in the figure, after the new object storage device enters the cluster, the temporary master node OSD (NVMe) will reverse synchronize its stored data to the new object storage device, and hand over the role of the master node after the data synchronization is completed. The new object storage device will serve as the new master node. At the same time, OSD (NVMe) will exit the service and format and clear its internal storage data.

[0070] like Figure 4 As shown, in one embodiment, the Ceph distributed storage method provided by the present invention sends an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishes a read / write channel with the client in response to the IO read / write request, and the method includes the following steps:

[0071] Step S410, when the original master node fails, the cluster PG data in the original master node is called out, and a temporary takeover request is sent to the monitor. The monitor synchronizes the cluster PG data to the first object storage device in response to the temporary takeover request.

[0072] Step S420, after receiving the cluster PG data, the first object storage device acts as a temporary master node to replace the original master node and receives data synchronization from the slave node.

[0073] like Figure 5 As shown, in one embodiment, the Ceph distributed storage method provided by the present invention sends an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishes a read / write channel with the client in response to the IO read / write request, and then includes the following steps:

[0074] Step S510: sending an output stream request to the first object storage device. After receiving the output stream request, the first object storage device writes the byte stream carried in the output stream request to the first replica node.

[0075] Step S520: Send an input stream request to the first object storage device. After receiving the input stream request, the first object storage device reads data from the master node or the slave node according to the hot standby time point.

[0076] Among them, the replica node, the master node and the slave node are all object storage devices.

[0077] like Figure 6 As shown, in one embodiment, the Ceph distributed storage method provided by the present invention writes the output byte stream to the replica node through the read-write channel, and reads the input byte stream from the master node or the slave node according to the hot standby time point through the read-write channel, specifically including the following steps:

[0078] Step S132: determining, according to the hot standby time point, whether the first object storage device reads the first target data from the primary node in response to the input stream request.

[0079] Step S134, reading the first target data from the master node first, and reading the first target data from the slave node when the first target data does not exist in the master node.

[0080] Specifically, when the judgment result in step S132 is that the first object storage device reads the first target data from the master node in response to the input stream request based on the hot standby time point, the first target data is read from the master node first, and when the first target data does not exist in the master node, the first target data is read from other slave nodes.

[0081] Step S136, reading the first target data from the slave node.

[0082] Specifically, when the judgment result in step S132 is that the first object storage device does not read the first target data from the master node in response to the input stream request based on the hot standby time point, the first target data is read from the corresponding slave node where the first target data exists.

[0083] The master node includes a temporary master node, an original master node before a failure, and a second object storage device after becoming a master node.

[0084] like Figure 7 As shown, in one embodiment, the Ceph distributed storage method provided by the present invention further includes the following steps:

[0085] Step S710 , determining, according to the hot standby time point, whether the first object storage device reads the second target data from the slave node in response to the input stream request.

[0086] Step S720: Read the second target data from the slave node preferentially, and when the second target data does not exist in the slave node, read the second target data from the master node.

[0087] Specifically, when the judgment result in step S710 is that the first object storage device reads the second target data from the designated slave node in response to the input stream request based on the hot standby time point, the second target data is read from the corresponding slave node preferentially, and when the second target data does not exist in the slave node, the second target data is read from the master node or other slave nodes.

[0088] Step S730, read the second target data from the master node.

[0089] Specifically, when the judgment result in step S710 is that the first object storage device does not read the second target data from the designated slave node in response to the input stream request according to the hot standby time point, the second target data is first read from the master node.

[0090] The master node includes a temporary master node, an original master node before a failure, and a second object storage device after becoming a master node.

[0091] like Figure 8 As shown, in one embodiment, the Ceph distributed storage method provided by the present invention, when the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, and formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node, specifically including the following steps:

[0092] Step S142: after the second object storage device joins the cluster, the data of the temporary master node is synchronized to the second object storage device, and the data in the slave node is synchronized to the second object storage device.

[0093] Step S144, after the data of the temporary master node is synchronized to the second object storage device, exit the service of the temporary master node and clear the stored data in the temporary master node, and replace the temporary master node with the second object storage device as the master node.

[0094] like Fig. 9 As shown, in one embodiment, the Ceph distributed storage method provided by the present invention further includes the following steps:

[0095] Step S910: After the second object storage device replaces the temporary master node as the master node, it receives an IO read / write request from the client, and establishes a read / write channel with the client in response to the IO read / write request.

[0096] Step S920, write the output byte stream carried in the IO read / write request to the replica node through the read / write channel, and read the input byte stream marked in the IO read / write request from the master node or the slave node through the read / write channel according to the hot standby time point, and store the input byte stream to the second object storage device.

[0097] The Ceph distributed storage device provided by the present invention is described below. The Ceph distributed storage device described below and the Ceph distributed storage method described above can refer to each other.

[0098] like Fig.10 As shown, in one embodiment, a Ceph distributed storage device includes a connection request module 1010, a read / write request module 1020, a data read / write module 1030, and a master node switching module 1040.

[0099] The connection request module 1010 is used to create a cluster process and read a configuration file, send a connection request to the monitor, and the monitor establishes a connection channel with the client in response to the connection request.

[0100] The read / write request module 1020 is used to send an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishes a read / write channel with the client in response to the IO read / write request.

[0101] The data read and write module 1030 is used to write the output byte stream to the replica node through the read and write channel, and read the input byte stream from the master node or the slave node through the read and write channel according to the hot standby time point. The hot standby time point is the time point for switching the first object storage device as the temporary master node.

[0102] The master node switching module 1040 is used to synchronize the stored data from the first object storage device to the second object storage device after the second object storage device joins the cluster, and format and clear the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

[0103] In this embodiment, the Ceph distributed storage device provided by the present invention further includes a temporary master node switching module, which is used to:

[0104] When the original master node fails, the cluster PG data in the original master node is called out, and a temporary takeover request is sent to the monitor. The monitor synchronizes the cluster PG data to the first object storage device in response to the temporary takeover request.

[0105] After receiving the cluster PG data, the first object storage device acts as a temporary master node to replace the original master node and receive data synchronization from the slave node.

[0106] In this embodiment, the Ceph distributed storage device provided by the present invention further includes a read and write request submodule, which is used to:

[0107] An output stream request is sent to the first object storage device. After receiving the output stream request, the first object storage device writes the byte stream carried in the output stream request to the second replica node.

[0108] An input stream request is sent to the first object storage device. After receiving the input stream request, the first object storage device reads data from the master node or the slave node according to the hot standby time point.

[0109] Among them, the replica node, the master node and the slave node are all object storage devices.

[0110] In this embodiment, the Ceph distributed storage device provided by the present invention, the data reading and writing module 1030 is specifically used for:

[0111] Determine whether the first object storage device reads the first target data from the primary node in response to the input stream request according to the hot standby time point.

[0112] The first target data is read from the master node first, and when the first target data does not exist in the master node, the first target data is read from the slave node.

[0113] The first target data is read from the slave node.

[0114] The master node includes a temporary master node, an original master node before a failure, and a second object storage device after becoming a master node.

[0115] In this embodiment, the Ceph distributed storage device provided by the present invention further includes a data reading and writing submodule, which is used to:

[0116] Determine whether the first object storage device reads the second target data from the slave node in response to the input stream request according to the hot standby time point.

[0117] The second target data is read from the slave node first, and when the second target data does not exist in the slave node, the second target data is read from the master node.

[0118] Read the second target data from the master node.

[0119] The master node includes a temporary master node, an original master node before a failure, and a second object storage device after becoming a master node.

[0120] In this embodiment, in the Ceph distributed storage device provided by the present invention, the master node switching module 1040 is specifically used for:

[0121] After the second object storage device joins the cluster, the data of the temporary master node is synchronized to the second object storage device, and the data in the slave node is synchronized to the second object storage device.

[0122] After the data of the temporary master node is synchronized to the second object storage device, the service of the temporary master node is exited and the storage data in the temporary master node is cleared, and the temporary master node is replaced with the second object storage device as the master node.

[0123] In this embodiment, the Ceph distributed storage device provided by the present invention further includes a new master node service module, which is used to:

[0124] After the second object storage device replaces the temporary master node as the master node, it receives IO read / write requests from the client, and establishes a read / write channel with the client in response to the IO read / write requests.

[0125] The output byte stream carried in the IO read / write request is written to the replica node through the read / write channel, and the input byte stream marked in the IO read / write request is read from the master node or the slave node through the read / write channel according to the hot standby time point, and the input byte stream is stored in the second object storage device.

[0126] Fig.11 An example of a physical structure diagram of an electronic device is shown. The electronic device may be a smart terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The electronic device includes a processor, an internal memory and a network interface connected through a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the Ceph distributed storage method is implemented, and the method includes:

[0127] Create a cluster process and read the configuration file, send a connection request to the monitor, and the monitor responds to the connection request to establish a connection channel with the client;

[0128] Sending an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishing a read / write channel with the client in response to the IO read / write request;

[0129] Writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, where the hot standby time point is the time point when the first object storage device is switched as the temporary master node;

[0130] When the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

[0131] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present invention, and does not constitute a limitation on the electronic device to which the scheme of the present invention is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0132] On the other hand, the present invention further provides a computer storage medium storing a computer program, wherein when the computer program is executed by a processor, a Ceph distributed storage method is implemented, the method comprising:

[0133] Create a cluster process and read the configuration file, send a connection request to the monitor, and the monitor responds to the connection request to establish a connection channel with the client;

[0134] Sending an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishing a read / write channel with the client in response to the IO read / write request;

[0135] Writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, where the hot standby time point is the time point when the first object storage device is switched as the temporary master node;

[0136] When the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

[0137] In another aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and when the processor executes the computer instructions, a Ceph distributed storage method is implemented, the method comprising:

[0138] Create a cluster process and read the configuration file, send a connection request to the monitor, and the monitor responds to the connection request to establish a connection channel with the client;

[0139] Sending an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishing a read / write channel with the client in response to the IO read / write request;

[0140] Writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, where the hot standby time point is the time point when the first object storage device is switched as the temporary master node;

[0141] When the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

[0142] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0143] By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0144] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0145] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A Ceph distributed storage method, characterized in that: The method comprises: Create a cluster process and read a configuration file, send a connection request to a monitor, and the monitor establishes a connection channel with a client in response to the connection request; Sending an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, wherein the first object storage device establishes a read / write channel with the client in response to the IO read / write request; Writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, wherein the hot standby time point is the time point of switching the first object storage device as a temporary master node; When the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node; The step of sending an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, wherein the first object storage device establishes a read / write channel with the client in response to the IO read / write request, comprises: When the original master node fails, the cluster PG data in the original master node is called out, and a temporary takeover request is sent to the monitor, and the monitor synchronizes the cluster PG data to the first object storage device in response to the temporary takeover request; After receiving the cluster PG data, the first object storage device acts as the temporary master node to replace the original master node and receive data synchronization from the slave node; The IO read / write request includes an input stream request and an output stream request; the IO read / write request is sent to a first object storage device of the hot standby non-volatile memory standard, and the first object storage device establishes a read / write channel with the client in response to the IO read / write request, and then includes: Sending the output stream request to the first object storage device, and after receiving the output stream request, the first object storage device writes the byte stream carried in the output stream request to the first replica node; Sending the input stream request to the first object storage device, and after receiving the input stream request, the first object storage device reads data from the master node or the slave node according to the hot standby time point; Wherein, the replica node, the master node and the slave node are all object storage devices; The step of writing the output byte stream to the replica node through the read-write channel, and reading the input byte stream from the master node or the slave node according to the hot standby time point through the read-write channel, comprises: Determine, according to the hot standby time point, whether the first object storage device reads the first target data from the master node in response to the input stream request; if so, The first target data is read from the master node first, and when the first target data does not exist in the master node, the first target data is read from the slave node; if not, then Read the first target data from the slave node; The master node includes a temporary master node, an original master node before failure, and the second object storage device after serving as the master node; The method further comprises: Determine, according to the hot standby time point, whether the first object storage device reads the second target data from the slave node in response to the input stream request; if so, The second target data is read from the slave node first, and when the second target data does not exist in the slave node, the second target data is read from the master node; if not, then Reading the second target data from the master node; The master node includes a temporary master node, an original master node before failure, and the second object storage device after serving as the master node; When the second object storage device joins the cluster, the first object storage device reversely synchronizes the stored data to the second object storage device, formats and clears the stored data after the data synchronization is completed, and the second object storage device serves as the master node, including: After the second object storage device joins the cluster, synchronizing the data of the temporary master node to the second object storage device, and synchronizing the data in the slave node to the second object storage device; After the data of the temporary master node is synchronized to the second object storage device, the service of the temporary master node is exited and the storage data in the temporary master node is cleared, and the temporary master node is replaced with the second object storage device as the master node.

2. The Ceph distributed storage method according to claim 1, characterized in that: The method further comprises: After the second object storage device replaces the temporary master node as the master node, it receives an IO read / write request from the client, and establishes a read / write channel with the client in response to the IO read / write request; The output byte stream carried in the IO read / write request is written to the replica node through the read / write channel, and the input byte stream marked in the IO read / write request is read from the master node or the slave node through the read / write channel according to the hot standby time point, and the input byte stream is stored in the second object storage device.

3. A Ceph distributed storage device, characterized in that: For implementing the Ceph distributed storage method according to any one of claims 1 to 2, the device comprises: A connection request module, used to create a cluster process and read a configuration file, send a connection request to a monitor, and the monitor establishes a connection channel with a client in response to the connection request; A read / write request module, configured to send an IO read / write request to a first object storage device of the hot standby non-volatile memory standard, wherein the first object storage device establishes a read / write channel with the client in response to the IO read / write request; A data read-write module, configured to write the output byte stream to the replica node through the read-write channel, and read the input byte stream from the master node or the slave node through the read-write channel according to the hot standby time point, wherein the hot standby time point is the time point of switching the first object storage device as a temporary master node; The master node switching module is used to synchronize the stored data from the first object storage device to the second object storage device after the second object storage device joins the cluster, and to format and clear the stored data after the data synchronization is completed, and the second object storage device serves as the master node.

4. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 2 are implemented.

5. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images