Data storage method and device, electronic equipment and distributed storage system

Abstract

The application provides a data storage method and device, electronic equipment and distributed storage system, and relates to the technical field of data processing. The method comprises the following steps: acquiring object storage device (OSD) distribution data corresponding to each storage node in a distributed storage system, wherein the OSD distribution data comprises the number of primary OSDs in the storage node; for each storage node, determining a node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node; determining a target storage node from each storage node in the distributed storage system based on the node selection parameter value corresponding to each storage node; and transmitting the acquired to-be-stored data to the primary OSD in the target storage node, so that the primary OSD in the target storage node forwards the to-be-stored data to the slave OSD, thereby solving the technical problem of how to better backup and store data in the prior art to reduce the cross-node data forwarding amount.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data storage method and device, electronic equipment and distributed storage system. BACKGROUND

[0002] In a distributed storage system, a storage cluster composed of multiple independent storage nodes is used to store data. In order to ensure high reliability of data, the data is copied into multiple identical copies, and each copy is saved to a different storage node to avoid data loss caused by failure of a single storage node.

[0003] In the prior art, based on the CPU (Central Processing Unit / Processor), memory, network busy degree, IO (Input / Output) delay and other conventional detection points of the device, the target storage node for AO (active / optimized) path selection is determined, and then the target storage node copies and forwards the data to other storage nodes. However, when selecting the target storage node, the cross-node distribution pressure brought by multiple copies is not considered, which can cause the problem of excessive cross-node data forwarding amount, and further cause the problems of longer IO processing path and increased delay.

[0004] Therefore, how to better backup and store data to reduce the cross-node data forwarding amount is a technical problem to be solved by the relevant technical personnel. SUMMARY

[0005] The present application provides a data storage method and device, electronic equipment and distributed storage system to solve the technical problem of how to better backup and store data to reduce the cross-node data forwarding amount in the prior art.

[0006] The present application provides a data storage method, comprising:

[0007] Obtaining object storage device (OSD) distribution data corresponding to each storage node in a distributed storage system, wherein the OSD distribution data comprises a number of primary OSDs in the storage node;

[0008] For each storage node, determining a node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node;

[0009] Based on the node selection parameter value corresponding to each storage node, determining a target storage node from each storage node in the distributed storage system;

[0010] transmit the obtained to-be-stored data to the master OSD in the target storage node, so that the master OSD in the target storage node forwards the to-be-stored data to the slave OSD.

[0011] According to the data storage method provided by the application, the node selection parameter value corresponding to the storage node is determined based on the number of master OSDs in the storage node, and the method comprises the following steps of:

[0012] The master OSD weight corresponding to the storage node is determined based on the number of master OSDs in the storage node.

[0013] The current weight coefficient corresponding to the storage node is determined, and the current weight coefficient is used to adjust the numerical value of the master OSD weight.

[0014] The node selection parameter value corresponding to the storage node is determined based on the master OSD weight and the current weight coefficient.

[0015] According to the data storage method provided by the application, the current weight coefficient corresponding to the storage node is determined, and the method comprises the following steps of:

[0016] The current weight coefficient corresponding to the storage node is determined based on the number of current AO connections corresponding to the storage node.

[0017] Alternatively, the current weight coefficient corresponding to the storage node is determined based on the front-end service network interface idle rate corresponding to the storage node.

[0018] Alternatively, the current weight coefficient corresponding to the storage node is determined based on the number of current AO connections corresponding to the storage node and the front-end service network interface idle rate.

[0019] According to the data storage method provided by the application, the master OSD weight is positively correlated with the number of master OSDs; the number of current AO connections is negatively correlated with the current weight coefficient; and the front-end service network interface idle rate is positively correlated with the current weight coefficient.

[0020] According to the data storage method provided by the application, the target storage node is determined from each storage node in the distributed storage system based on the node selection parameter value corresponding to each storage node, and the method comprises the following steps of:

[0021] The first maximum node selection parameter value in the node selection parameter values corresponding to each storage node is determined.

[0022] The storage node corresponding to the first maximum node selection parameter value is determined as the target storage node.

[0023] According to the data storage method provided by the application, the method further comprises the following steps of:

[0024] In a case where it is determined that the target storage node has a connection fault, determining a second maximum node selection parameter value in node selection parameter values corresponding to each remaining storage node;

[0025] Determining the remaining storage node corresponding to the second maximum node selection parameter value as a new target storage node.

[0026] The application further provides a data storage device, comprising:

[0027] A data acquisition module, configured to acquire object storage device (OSD) distribution data corresponding to each storage node in a distributed storage system, wherein the OSD distribution data comprises a number of primary OSDs in the storage node;

[0028] A parameter determination module, configured to determine, for each storage node, a node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node;

[0029] A node selection module, configured to determine a target storage node from each storage node in the distributed storage system based on the node selection parameter value corresponding to each storage node;

[0030] A data storage module, configured to transmit the acquired to-be-stored data to a primary OSD in the target storage node, so that the primary OSD in the target storage node forwards the to-be-stored data to a slave OSD.

[0031] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data storage method according to any one of the above when executing the program.

[0032] The application further provides a distributed storage system, comprising at least one storage node and a controller, wherein the controller is connected with a client device and each storage node respectively;

[0033] The controller is configured to receive to-be-stored data transmitted by the client device and execute the data storage method according to any one of the above.

[0034] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable on a processor to implement the data storage method according to any one of the above.

[0035] The application further provides a computer program product, comprising a computer program, wherein the computer program is executable on a processor to implement the data storage method according to any one of the above.

[0036] The data storage method, device, electronic equipment and distributed storage system provided by the application determine the node selection parameter value corresponding to each storage node based on the number of primary OSDs in each storage node, and determine the target storage node based on the node selection parameter value corresponding to each storage node. Since the number of primary OSDs in the storage node is related to the cross-node data forwarding amount, the cross-node distribution pressure is considered when selecting the target storage node, so that the data forwarding amount of the primary OSD in the target storage node can be reduced to avoid the problem of excessive cross-node data forwarding amount, thereby shortening the IO processing path and reducing the delay, and solving the technical problem of how to better backup and store data in the prior art to reduce the cross-node data forwarding amount. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0038] Figure 1 is one of the flowcharts of the data storage method provided by the embodiments of the application;

[0039] Figure 2 is the second flowchart of the data storage method provided by the embodiments of the application;

[0040] Figure 3 is the third flowchart of the data storage method provided by the embodiments of the application;

[0041] Figure 4 is the fourth flowchart of the data storage method provided by the embodiments of the application;

[0042] Figure 5 is the fifth flowchart of the data storage method provided by the embodiments of the application;

[0043] Figure 6 is the structural schematic diagram of the data storage device provided by the embodiments of the application;

[0044] Figure 7 is the structural schematic diagram of the electronic equipment provided by the embodiments of the application;

[0045] Figure 8 is the schematic diagram of the connection relationship between the client equipment and the storage node in the embodiments of the application. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, the following will clearly and completely describe the technical solutions in the present application with reference to the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0047] The following will be described with reference to the drawings Figures 1-5 The data storage method provided by the present application is described. As shown in the figure Figure 1 The present application provides a data storage method, which is applied to an AO (active / optimized) path selection scene based on ALUA (Asymmetric Logical Unit Access), and the method comprises the following steps:

[0048] In step 101, the OSD distribution data corresponding to each storage node in the distributed storage system is obtained, and the OSD distribution data comprises the number of master OSDs in the storage node.

[0049] In the distributed storage system, each storage node can be a storage server or a storage device, and can also be a storage service end. OSD (Object-based Storage Devices) represents an object storage device, and object storage (Object-based Storage) is also called "object-oriented storage". The number of master OSDs in the storage node represents the number of master OSDs in the resource group on the storage node OSD, and specifically, the number of master OSDs in the storage node can be understood as: on the OSD of the storage node, the number of master OSDs of the storage node in the resource group is summed up, that is, the number of existing master OSDs in the storage node.

[0050] Further, the object is mapped to the resource group, each resource group is mapped to n OSDs, that is, a master OSD and multiple slave OSDs are allocated for each resource group, based on the number of copies, the data is copied from the master OSD to multiple slave OSDs, and specifically, the erasure code mode is responsible for all write operations of the master OSD, encodes K+M blocks, and writes to other OSDs. In addition, each OSD will carry a large number of resource groups.

[0051] Further, ALUA is one of the multi-path mechanisms of the front-end controller. For a specific LUN (Logical Unit Number, namespace resource), in its path, the target port of one controller is in the active / optimized (AO) state, and the target port of another controller is in the active / unoptimized (AN) state. At a certain moment, a certain LUN belongs to only one controller. To achieve load balancing on both sides, task A is thrown to controller A, and task B is thrown to controller B. For the same task, only one controller is in control at any time.

[0052] In step 102, for each storage node, a node selection parameter value corresponding to the storage node is determined based on the number of master OSDs in the storage node.

[0053] Further, the node selection parameter value can be used to represent a probability value of the storage node being selected as the target storage node. Alternatively, the higher the node selection parameter value corresponding to the storage node, the higher the probability value of the storage node being selected as the target storage node.

[0054] In step 103, based on the node selection parameter value corresponding to each storage node, a target storage node is determined from the various storage nodes in the distributed storage system.

[0055] Further, each storage node includes at least one OSD, and the types of OSDs included in the storage node include master OSDs and slave OSDs.

[0056] In step 104, the obtained to-be-stored data is transmitted to the master OSD in the target storage node, so that the master OSD in the target storage node forwards the to-be-stored data to the slave OSD.

[0057] Further, the number of master OSDs in the target storage node includes at least one. The master OSD in the target storage node forwards the to-be-stored data to at least one slave OSD, and the at least one slave OSD can include a slave OSD in the target storage node and a slave OSD in another storage node.

[0058] In the above steps 101 to 104, the node selection parameter value corresponding to each storage node is determined based on the number of master OSDs in each storage node, and the target storage node is determined based on the node selection parameter value corresponding to each storage node. Since the number of master OSDs in the storage node is related to the amount of data forwarding across nodes, the cross-node distribution pressure is considered when selecting the target storage node, thereby reducing the amount of data forwarding of the master OSD in the target storage node to avoid the problem of excessive data forwarding across nodes, thereby shortening the IO processing path and reducing latency, and solving the technical problem in the prior art of how to better back up and store data to reduce the amount of data forwarding across nodes.

[0059] In one embodiment, when storage nodes are expanded, OSD resources are expanded, or even OSD anomalies occur, steps 101 to 104 are repeatedly executed to re-determine the target storage node in the current scenario, thereby achieving the purpose of adaptively selecting the optimal ALUA AO path.

[0060] In one embodiment, Figure 2 As shown, the step of determining the node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node in the above step 102 specifically includes:

[0061] Step 201: Determine the primary OSD weight corresponding to the storage node based on the number of primary OSDs in the storage node.

[0062] In one embodiment, the master OSD weight is positively correlated with the number of master OSDs. The greater the number of master OSDs, the greater the master OSD weight. Alternatively, the master OSD weight is the number of master OSDs multiplied by 100%.

[0063] For example, if the number of primary OSDs is 80, the primary OSD weight is 0.8. If the number of primary OSDs is 45, the primary OSD weight is 0.45. If the number of primary OSDs is 69, the primary OSD weight is 0.69. If the number of primary OSDs is 62, the primary OSD weight is 0.62.

[0064] Step 202: Determine the current weight coefficient corresponding to the storage node. The current weight coefficient is used to adjust the value of the master OSD weight.

[0065] Step 203: Determine the node selection parameter value corresponding to the storage node based on the master OSD weight and the current weight coefficient.

[0066] In one embodiment, the main OSD weight is added to the current weight coefficient to obtain the node selected parameter value corresponding to the storage node. Alternatively, the main OSD weight is subtracted from the current weight coefficient to obtain the node selected parameter value corresponding to the storage node. Alternatively, the main OSD weight is multiplied by the current weight coefficient to obtain the node selected parameter value corresponding to the storage node.

[0067] For example, in the case where the node selected parameter value is the product of the main OSD weight and the current weight coefficient, if the main OSD weight is 1 and the current weight coefficient is 1, the node selected parameter value is 1. If the main OSD weight is 1 and the current weight coefficient is 0.8, the node selected parameter value is 0.8. If the main OSD weight is 1 and the current weight coefficient is 0.6, the node selected parameter value is 0.6.

[0068] For example, in the case where the node selected parameter value is the sum of the main OSD weight and the current weight coefficient, if the main OSD weight is 1 and the current weight coefficient is -0.2, the node selected parameter value is 0.8. If the main OSD weight is 1 and the current weight coefficient is -0.4, the node selected parameter value is 0.6. If the main OSD weight is 1 and the current weight coefficient is -0.6, the node selected parameter value is 0.4.

[0069] It should be noted that the more the number of main OSDs in the storage node, the greater the main OSD weight of the storage node, and the higher the node selected parameter value corresponding to the storage node, and the greater the probability of the storage node being selected as the target storage node. As can be seen, the storage node with a larger number of main OSDs is more likely to be selected as the target storage node. Since the more the number of main OSDs in the storage node, the less the amount of data that needs to be forwarded by the main OSD in the storage node, therefore, selecting the storage node with a larger number of main OSDs as the target storage node can effectively reduce the amount of data forwarding across nodes, thereby effectively shortening the IO processing path and reducing the latency.

[0070] For example, the number of backup copies of the data to be stored is 100, if the number of main OSDs in the target storage node is 20, then the amount of data that needs to be forwarded by the main OSD in the target storage node is 80, if the number of main OSDs in the target storage node is 80, then the amount of data that needs to be forwarded by the main OSD in the target storage node is only 20, obviously, the more the number of main OSDs in the target storage node, the less the amount of data that needs to be forwarded by the main OSD in the target storage node.

[0071] The steps 201 to 203 determine the main OSD weight corresponding to the storage node based on the number of main OSDs in the storage node, determine the current weight coefficient corresponding to the storage node, the current weight coefficient is used to adjust the numerical value of the main OSD weight, determine the node selection parameter value corresponding to the storage node based on the main OSD weight and the current weight coefficient, so as to improve the storage node with more main OSDs as the target storage node, thereby improving the localization rate of the main OSD, effectively reducing the data forwarding amount and network transmission of data across nodes, shortening the IO processing path and reducing the delay.

[0072] In one embodiment, as shown in FIG. 3, the step of determining the current weight coefficient corresponding to the storage node in the step 202 includes one of the steps 301 to 303. Figure 3

[0073] The step 301 determines the current weight coefficient corresponding to the storage node based on the current AO connection number corresponding to the storage node.

[0074] In one embodiment, the current AO connection number and the current weight coefficient are in a negative correlation relationship. The more the current AO connection number is, the smaller the current weight coefficient is, and the smaller the main OSD weight is. The current AO connection number is the number of times that the storage node is selected as the target storage node of the AO path.

[0075] The step 301 determines the current weight coefficient corresponding to the storage node based on the current AO connection number corresponding to the storage node, so as to adjust the numerical value of the main OSD weight based on the current AO connection number corresponding to the storage node, so as to prevent the problem of crowded data transmission channels caused by too many AO connection times of the target storage node, thereby effectively improving the data transmission efficiency and the data storage efficiency.

[0076] The step 302 determines the current weight coefficient corresponding to the storage node based on the front-end service network port idle rate corresponding to the storage node.

[0077] In one embodiment, the front-end service network port idle rate and the current weight coefficient are in a positive correlation relationship. The higher the front-end service network port idle rate is, the larger the current weight coefficient is, and the larger the main OSD weight is.

[0078] ​The step 302 determines the current weight coefficient of the storage node based on the front-end service network port idle rate of the storage node, so as to adjust the value of the main OSD weight based on the front-end service network port idle rate of the storage node, thereby preferentially selecting the storage node with a higher front-end service network port idle rate as the target storage node, preventing the problem of crowded data transmission channels caused by the busy file and object storage services carried on the target storage node, and effectively improving the data transmission efficiency and data storage efficiency.

[0079] In step 303, the current weight coefficient of the storage node is determined based on the current AO connection number and the front-end service network port idle rate of the storage node.

[0080] In one embodiment, as shown in Figure 4 The step of determining the target storage node from the storage nodes in the distributed storage system based on the node selection parameter value of each storage node in the step 103 specifically includes:

[0081] In step 401, the first maximum node selection parameter value in the node selection parameter value of each storage node is determined.

[0082] In step 402, the storage node corresponding to the first maximum node selection parameter value is determined as the target storage node.

[0083] The steps 401 to 402 determine the first maximum node selection parameter value in the node selection parameter value of each storage node, and determine the storage node corresponding to the first maximum node selection parameter value as the target storage node, so as to reduce the data forwarding amount of the main OSD in the target storage node as much as possible, avoid the problem of excessive data forwarding amount across nodes, effectively shorten the IO processing path, and reduce the delay.

[0084] In one embodiment, as shown in Figure 5 The data storage method provided by the application further includes:

[0085] In step 501, in the case where the target storage node has a connection fault, the second maximum node selection parameter value in the node selection parameter value of each remaining storage node is determined.

[0086] The second maximum node selection parameter value is the second highest node selection parameter value in the node selection parameter value of the original storage nodes (including the original target storage node), and is also the highest node selection parameter value in the node selection parameter value of each remaining storage node after excluding the target storage node with the connection fault.

[0087] Step 502: Determine the remaining storage node corresponding to the second largest selected parameter value as a new target storage node.

[0088] Furthermore, the acquired data to be stored is transmitted to the master OSD in the new target storage node, so that the master OSD in the new target storage node forwards the data to be stored to the slave OSD.

[0089] In the above steps 501 to 502, when it is determined that there is a connection failure in the target storage node, the second maximum selected parameter value among the node selected parameter values ​​corresponding to each remaining storage node is determined, and the remaining storage nodes corresponding to the second maximum selected parameter value are determined as new target storage nodes. When there is a connection failure in the target storage node, a new connection is established by selecting the storage node with the second highest node selected parameter value, and then data is stored based on the new target storage node, so as to reduce the data forwarding amount of the main OSD in the target storage node, avoid the problem of excessive data forwarding across nodes, and thus shorten the IO processing path and reduce the delay.

[0090] In summary, the data storage method provided by the present invention uses the master OSD localization rate (i.e., the number of master OSDs) as an important condition for ALUA's AO path selection, that is, selects a relatively less busy storage node with the largest number of master OSDs as the target storage node, so as to reduce redundant data transmission and improve business write performance, thereby avoiding the problem of excessive cross-node data forwarding, thereby shortening the IO processing path and reducing latency, and reducing performance loss caused by network transmission.

[0091] The data storage device provided by the present invention is described below. The data storage device described below and the data storage method described above can be referenced to each other.

[0092] like Figure 6 As shown, the present invention provides a data storage device, the data storage device 100 includes:

[0093] The data acquisition module 1001 is used to acquire the object storage device OSD distribution data corresponding to each storage node in the distributed storage system. The OSD distribution data includes the number of master OSDs in the storage node.

[0094] The parameter determination module 1002 is configured to determine, for each storage node, a node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node.

[0095] The node selection module 1003 is configured to determine a target storage node from among the storage nodes in the distributed storage system based on a node selection parameter value corresponding to each storage node.

[0096] The data storage module 1004 is configured to transmit the obtained to-be-stored data to the master OSD in the target storage node, so that the master OSD in the target storage node forwards the to-be-stored data to the slave OSD.

[0097] In one embodiment, the parameter determination module 1002 is further configured to determine a master OSD weight corresponding to the storage node based on a number of master OSDs in the storage node; determine a current weight coefficient corresponding to the storage node, the current weight coefficient being used to adjust a numerical value of the master OSD weight; and determine the node selection parameter value corresponding to the storage node based on the master OSD weight and the current weight coefficient.

[0098] In one embodiment, the parameter determination module 1002 is further configured to determine the current weight coefficient corresponding to the storage node based on a current number of AO connections of the storage node; or determine the current weight coefficient corresponding to the storage node based on a front-end service network interface idle rate of the storage node; or determine the current weight coefficient corresponding to the storage node based on the current number of AO connections of the storage node and the front-end service network interface idle rate of the storage node.

[0099] In one embodiment, the master OSD weight is positively correlated with the number of master OSDs; the current number of AO connections is negatively correlated with the current weight coefficient; and the front-end service network interface idle rate is positively correlated with the current weight coefficient.

[0100] In one embodiment, the node selection module 1003 is further configured to determine a first maximum node selection parameter value in the node selection parameter values corresponding to the respective storage nodes; and determine the storage node corresponding to the first maximum node selection parameter value as the target storage node.

[0101] In one embodiment, the node selection module 1003 is further configured to, in a case where it is determined that the target storage node has a connection fault, determine a second maximum node selection parameter value in the node selection parameter values corresponding to the respective remaining storage nodes; and determine the remaining storage node corresponding to the second maximum node selection parameter value as a new target storage node.

[0102] Figure 7 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 7As shown, the electronic device can include a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 complete mutual communication through the communications bus 840. The processor 810 can invoke a logical instruction in the memory 830 to execute a data storage method provided by each of the above methods, and the method includes: obtaining object storage device (OSD) distribution data corresponding to each storage node in a distributed storage system, wherein the OSD distribution data includes a number of master OSDs in the storage node; for each storage node, determining a node selection parameter value corresponding to the storage node based on the number of master OSDs in the storage node; determining a target storage node from each storage node in the distributed storage system based on the node selection parameter value corresponding to each storage node; and transmitting the obtained to-be-stored data to a master OSD in the target storage node, so that the master OSD in the target storage node forwards the to-be-stored data to a slave OSD.

[0103] In addition, the logical instruction in the memory 830 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0104] On the other hand, the present application also provides a distributed storage system, which includes at least one storage node and a controller, wherein the controller is connected with a client device and each storage node respectively.

[0105] Further, each storage node is connected with the client device through a SCSI (Small Computer System Interface) device or a SCSI bus, wherein the SCSI device can be a SCSI gateway. Figure 8 An example of a case where each storage node is connected with the client device through the SCSI bus is shown.

[0106] Further, the storage node can be referred to as a storage server or a storage service end, and can also be referred to as an ISCSI (Internet Small Computer System Interface) target. The client device can be referred to as a storage client, and can also be referred to as an ISCSI initiator.

[0107] Further, the SCSI is a complete data transmission protocol, and a main function thereof is to transmit requests, states and block data between a host and a storage device, and is also a specification for interface transmission between a storage device and a client device. The SCSI data transmission protocol is used to implement communication between the storage service end and the storage client.

[0108] The controller is configured to receive the to-be-stored data transmitted by the client device, and execute the data storage method provided by each of the above methods. The method includes: obtaining object storage device (OSD) distribution data corresponding to each storage node in the distributed storage system, wherein the OSD distribution data includes a number of master OSDs in the storage node; determining, for each storage node, a node selection parameter value corresponding to the storage node based on the number of master OSDs in the storage node; determining a target storage node from the storage nodes in the distributed storage system based on the node selection parameter value corresponding to each storage node; and transmitting the obtained to-be-stored data to a master OSD in the target storage node, so that the master OSD in the target storage node forwards the to-be-stored data to a slave OSD.

[0109] In another aspect, the present application also provides a computer program product, which includes a computer program that can be stored on a computer readable storage medium. When the computer program is executed by a processor, the computer can execute the data storage method provided by each of the above methods. The method includes: obtaining object storage device (OSD) distribution data corresponding to each storage node in the distributed storage system, wherein the OSD distribution data includes a number of master OSDs in the storage node; determining, for each storage node, a node selection parameter value corresponding to the storage node based on the number of master OSDs in the storage node; determining a target storage node from the storage nodes in the distributed storage system based on the node selection parameter value corresponding to each storage node; and transmitting the obtained to-be-stored data to a master OSD in the target storage node, so that the master OSD in the target storage node forwards the to-be-stored data to a slave OSD.

[0110] In yet another aspect, the present application also provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements a data storage method provided by any of the above methods, the method comprising: obtaining object storage device (OSD) distribution data corresponding to each storage node in a distributed storage system, the OSD distribution data comprising a number of primary OSDs in the storage node; for each storage node, determining a node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node; determining a target storage node from the storage nodes in the distributed storage system based on the node selection parameter value corresponding to each storage node; and transmitting the obtained data to be stored to a primary OSD in the target storage node, so that the primary OSD in the target storage node forwards the data to be stored to a slave OSD.

[0111] The apparatus embodiments described above are merely illustrative, wherein the units shown as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0112] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method of each embodiment or some part of the embodiment.

[0113] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data storage method, characterized in that: include: Obtaining object storage device (OSD) distribution data corresponding to each storage node in a distributed storage system, wherein the OSD distribution data includes the number of primary OSDs in the storage node; For each of the storage nodes, determining a node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node; Determining a target storage node from each storage node in the distributed storage system based on a node selection parameter value corresponding to each storage node; The acquired data to be stored is transmitted to the master OSD in the target storage node, so that the master OSD in the target storage node forwards the data to be stored to the slave OSD.

2. The data storage method according to claim 1, wherein: The determining, based on the number of primary OSDs in the storage node, a node selection parameter value corresponding to the storage node includes: Determine the master OSD weight corresponding to the storage node based on the number of master OSDs in the storage node; Determine a current weight coefficient corresponding to the storage node, where the current weight coefficient is used to adjust the value of the master OSD weight; Based on the primary OSD weight and the current weight coefficient, a node selection parameter value corresponding to the storage node is determined.

3. The data storage method according to claim 2, wherein: The determining the current weight coefficient corresponding to the storage node includes: Determine a current weight coefficient corresponding to the storage node based on the current number of AO connections corresponding to the storage node; Alternatively, based on the idle rate of the front-end service network port corresponding to the storage node, determine the current weight coefficient corresponding to the storage node; Alternatively, the current weight coefficient corresponding to the storage node is determined based on the current number of AO connections corresponding to the storage node and the idle rate of the front-end service network port.

4. The data storage method according to claim 3, characterized in that: The main OSD weight is positively correlated with the number of the main OSDs; the current number of AO connections is negatively correlated with the current weight coefficient; and the front-end service network port idle rate is positively correlated with the current weight coefficient.

5. The data storage method according to any one of claims 1 to 4, characterized in that: The determining a target storage node from each storage node in the distributed storage system based on a node selection parameter value corresponding to each storage node includes: Determine a first maximum selected parameter value among the node selected parameter values ​​corresponding to each of the storage nodes; The storage node corresponding to the first maximum selected parameter value is determined as the target storage node.

6. The data storage method according to claim 5, characterized in that: The method further comprises: In the case where it is determined that the target storage node has a connection failure, determining a second maximum selected parameter value among the node selected parameter values ​​corresponding to the remaining storage nodes; The remaining storage nodes corresponding to the second maximum selected parameter value are determined as new target storage nodes.

7. A data storage device, characterized in that include: A data acquisition module is used to obtain object storage device (OSD) distribution data corresponding to each storage node in the distributed storage system, wherein the OSD distribution data includes the number of master OSDs in the storage node; a parameter determination module, configured to determine, for each storage node, a node selection parameter value corresponding to the storage node based on the number of primary OSDs in the storage node; a node selection module, configured to determine a target storage node from among the storage nodes in the distributed storage system based on a node selection parameter value corresponding to each of the storage nodes; The data storage module is used to transmit the acquired data to be stored to the master OSD in the target storage node, so that the master OSD in the target storage node forwards the data to be stored to the slave OSD.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data storage method according to any one of claims 1 to 6 is implemented.

9. A distributed storage system, characterized in that: include: At least one storage node and a controller, wherein: the controller is connected to the client device and each of the storage nodes respectively; The controller is configured to receive the data to be stored transmitted by the client device and execute the data storage method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data storage method according to any one of claims 1 to 6 is implemented.

Images