Data storage method and device, computer readable storage medium, and electronic device

Abstract

The present disclosure relates to a data storage method and device, computer readable storage medium and electronic equipment, and relates to the technical field of computers. The method comprises: in response to a data storage request sent by a client, analyzing the data storage request to obtain a data block to be stored included in the data storage request; determining the number of copies of the data block to be stored, and determining the current network address of the current rack where the master node device is located; obtaining other network addresses except the current network address from a preset rack cluster, and selecting target slave node devices not less than the number of copies from other racks corresponding to the other network addresses; and based on the other network addresses, copying the data block to be stored to the target slave node devices in the other racks in sequence, so as to store the data block to be stored through the target slave node devices. The present disclosure reduces the burden of the master node device.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of computer, and in particular, to a data storage method, a data storage device, a computer readable storage medium and an electronic device. BACKGROUND

[0002] In some data storage schemes, when the node where data is written is located inside the cluster, the storage of the first copy is located at the node (i.e. the node where the client is located). Therefore, when the amount of data written by this node becomes large, the data on this node becomes more and more, which makes the load of this node heavier and the load of other nodes less, thereby causing the load imbalance of the nodes.

[0003] Therefore, it is necessary to provide a new data storage method and device.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present disclosure is to provide a data storage method, a data storage device, a computer readable storage medium and an electronic device, thereby at least partially overcoming the problem of load imbalance of nodes caused by the limitations and defects of the related art.

[0006] According to one aspect of the present disclosure, a data storage method is provided, configured in a master node device where a client is located, the data storage method comprising:

[0007] In response to a data storage request sent by the client, the data storage request is parsed to obtain a to-be-stored data block included in the data storage request;

[0008] The number of copies of the to-be-stored data block is determined, and the current network address of the current rack where the master node device is located is determined;

[0009] From the preset rack cluster, other network addresses except the current network address are obtained, and target slave node devices not less than the number of copies are selected from other racks corresponding to the other network addresses;

[0010] Based on the other network addresses, the to-be-stored data block is copied to the target slave node devices in the other racks in sequence, so as to store the to-be-stored data block through the target slave node devices.

[0011] In an exemplary embodiment of the present disclosure, selecting the slave node devices not less than the number of copies from the other racks corresponding to the other network addresses comprises:

[0012] randomly selecting a first slave node device from other racks corresponding to the other network addresses, and determining whether the number of replicas is greater than one;

[0013] If the number of replicas is less than or equal to one, the first slave node device is taken as a target slave node device.

[0014] In an exemplary embodiment of the present disclosure, the data storage method further comprises:

[0015] If the number of replicas is greater than one, a first network address of a first rack where the first slave node device is located is determined, and a second slave node device is randomly selected from other racks corresponding to other network addresses except the current network address and the first network address in a preset rack cluster;

[0016] determining whether the number of replicas is greater than two, and taking the first slave node device and the second slave node device as the target slave node device when it is determined that the number of replicas is less than or equal to two.

[0017] In an exemplary embodiment of the present disclosure, the data storage method further comprises:

[0018] When it is determined that the number of replicas is greater than two, a third slave node device is randomly selected from other racks corresponding to the other network addresses, and it is determined whether the third slave node device is the same as the first slave node device and / or the second slave node device;

[0019] When it is determined that the third slave node device is the same as any of the first slave node device and the second slave node device, the selection of the third slave node device is repeated until the third slave node device is different from the first slave node device and the second slave node device, and the number of repetitions does not reach a preset threshold;

[0020] determining whether the number of replicas is greater than three, and taking the first slave node device, the second slave node device and the third slave node device as the target slave node device when it is determined that the number of replicas is less than or equal to three.

[0021] In an exemplary embodiment of the present disclosure, the data storage method further comprises:

[0022] When the number of repetitions reaches the preset threshold and the third slave node device is the same as the first slave node device or the second slave node device, the last selected slave node device is taken as the third slave node device.

[0023] In an exemplary embodiment of the present disclosure, the data storage method further comprises:

[0024] determining that the number of replicas is greater than three, randomly selecting one or more fourth slave node devices from the preset rack cluster; wherein the number of the fourth slave node devices is the difference between the number of replicas and the number of selected slave node devices;

[0025] selecting the first slave node device, the second slave node device, the third slave node device and the one or more fourth slave node devices as the target slave node devices.

[0026] In an exemplary embodiment of the present disclosure, after randomly selecting one or more fourth slave node devices from the preset rack cluster, the data storage method further comprises:

[0027] calculating the number of stored data blocks in the one or more fourth slave node devices, and calculating the current load rate of the one or more fourth slave node devices according to the number of stored data blocks;

[0028] determining that the current load rate of any fourth slave node device exceeds a preset load rate, randomly selecting another slave node device from the preset rack cluster to replace the fourth slave node device whose current load rate exceeds the preset load rate, until the current load rate of all fourth slave node devices is less than the preset load rate.

[0029] According to an aspect of the present disclosure, a data storage device is provided, configured in a master node device where a client is located, the data storage device comprising:

[0030] a data storage request analysis module, configured to analyze a data storage request sent by a client, to obtain a data block to be stored included in the data storage request;

[0031] a replica number determination module, configured to determine the number of replicas of the data block to be stored, and determine the current network address of the current rack where the master node device is located;

[0032] a slave node device selection module, configured to obtain other network addresses except the current network address from a preset rack cluster, and select target slave node devices not less than the number of replicas from other racks corresponding to the other network addresses;

[0033] a data storage module, configured to copy the data block to be stored to the target slave node devices in the other racks in sequence based on the other network addresses, to store the data block to be stored by the target slave node devices.

[0034] According to one aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the data storage method according to any one of the above.

[0035] According to one aspect of the present disclosure, an electronic device is provided, comprising:

[0036] a processor; and

[0037] a memory configured to store executable instructions of the processor;

[0038] wherein the processor is configured to implement the data storage method according to any one of the above by executing the executable instructions.

[0039] According to one aspect of the present disclosure, a data storage method is provided, which comprises: selecting, from other racks corresponding to other network addresses, target slave node devices not less than a number of replicas; and copying, based on the other network addresses, a data block to be stored to the target slave node devices in the other racks in sequence, so that the target slave node devices store the data block to be stored, thereby ensuring that a storage location of the data block to be stored is separated from a master node device where a client is located, and solving the problem of unbalanced load of nodes caused by the storage location of the data block to be stored being in the same rack as the client, thereby reducing a burden of the master node device.

[0040] It should be understood that the general description above and the detailed description below are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0042] Figure 1 A flow chart of a data storage method according to an example embodiment of the present disclosure is schematically shown.

[0043] Figure 2 An example diagram of a data storage system according to an example embodiment of the present disclosure is schematically shown.

[0044] Figure 3A flow chart of a method for selecting a node device according to an example embodiment of the present disclosure is schematically shown.

[0045] Figure 4 An example diagram of a storage scenario of a data block to be stored according to an example embodiment of the present disclosure is schematically shown.

[0046] Figure 5 A flow chart of another method for storing data according to an example embodiment of the present disclosure is schematically shown.

[0047] Figure 6 An example diagram of a storage result obtained by storing based on the data storage method according to an example embodiment of the present disclosure is schematically shown.

[0048] Figure 7 An example diagram of a storage result obtained by storing based on an existing data storage method is schematically shown.

[0049] Figure 8 A block diagram of a data storage apparatus according to an example embodiment of the present disclosure is schematically shown.

[0050] Figure 9 An electronic device for implementing the above-mentioned data storage method according to an example embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0051] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations can be implemented in any

[0052] Furthermore, the accompanying drawings are only schematic and are non-limiting. Specifically, the embodiments are not necessarily drawn to scale. Identical components have been given the same reference numerals in the various drawings and the same reference designators have been used throughout like components and / or components in different drawings and / or like functionality of components as the case can be. It will be understood that when a regular range, for example, 100 Hz to 200 Hz, is disclosed, this is intended to mean a range between 100 and 200, including any intervening values.

[0053] With the large-scale popularity and application of the Internet, the explosive growth of data marks the advent of the era of big data. The development of massive data brings many conveniences to people's life, such as cloud storage, electronic payment, online shopping, etc., but there are also severe challenges in storing massive data.

[0054] In the storage architecture of HaDoop distributed storage, the storage strategy for data replicas is as follows: if the write client is a node in the cluster, the first replica is stored in the node, the purpose is to save network resources and occupy less bandwidth; the second replica is stored in the node of other racks, and the last replica is stored in the node of the same rack as the second replica. The advantage of this replica storage strategy is that it can reduce data transmission between racks and improve the efficiency of write operations. Moreover, since the data blocks are only stored in two different racks, one third of the replicas are on one node and two thirds of the replicas are on another rack, if other replicas are randomly distributed in the remaining racks, this storage strategy can also reduce the total network transmission bandwidth required when reading data, improve write performance; and, since data is stored in different racks, the error of the rack is much less than the error of the node, so this strategy improves the reliability and availability of data.

[0055] In some storage strategies for data replicas, when the node where the data is written is located inside the cluster, the first replica is stored in the node (i.e. the node where the client is located); however, when the amount of data written by this node becomes large, there are more and more data on this node; when the client reads, since the first replica of the data is on this node, the data will be read from the node, which can easily cause the hot spot problem of data.

[0056] Based on this, in the example embodiment, a data storage method is first provided, which is configured in a master node device where a terminal device is located. The master node device can include a terminal device, a server, a server cluster, or a cloud server, etc. Of course, those skilled in the art can also run the method of the present disclosure on other platforms according to needs, which is not specially limited in the example embodiment. Referring to Figure 1 As shown, the data storage method can include the following steps:

[0057] Step S110. In response to the data storage request sent by the client, the data storage request is parsed to obtain the data block to be stored included in the data storage request;

[0058] Step S120. The number of copies of the data block to be stored is determined, and the current network address of the current rack where the master node device is located is determined.

[0059] Step S130. From the preset rack cluster, other network addresses except the current network address are obtained, and target slave node devices not less than the number of copies are selected from other racks corresponding to the other network addresses.

[0060] Step S140. Based on the other network addresses, the data block to be stored is copied to the target slave node devices in the other racks in sequence, so that the target slave node devices store the data block to be stored.

[0061] In the above data storage method, on the one hand, target slave node devices not less than the number of copies can be selected from other racks corresponding to other network addresses; then based on the other network addresses, the data block to be stored is copied to the target slave node devices in the other racks in sequence, so that the target slave node devices store the data block to be stored, thereby ensuring that the storage location of the data block to be stored is separate from the master node device where the client is located, solving the problem that in the prior art, the load of the node where the data block to be stored is stored is heavy and the load of other nodes is light, thereby causing load imbalance of the nodes, and reducing the burden of the master node device; on the other hand, the master node device can also be avoided to become a data hotspot, thereby causing the problem of low data access efficiency.

[0062] In the following, the data storage method of the example embodiments of the present disclosure will be explained and described in detail in combination with the drawings.

[0063] First, the invention purpose of the example embodiments of the present disclosure is explained and described. Specifically, the present disclosure is directed to the foregoing problem that the first copy of the data and the client are located on the same node, and the data is read from the node nearby, thereby easily causing the problem of data hotspot, a new write strategy and algorithm are adopted to disperse the data to different racks and nodes, thereby solving the problem of data hotspot. At the same time, when the data block is written to the cluster by the client, the data block can be evenly dispersed to different nodes of different racks. When reading these data blocks, the resources and network of multiple nodes can be effectively utilized, and the node (such as the node where the client is located) can be avoided to become a data hotspot.

[0064] Secondly, the data storage system involved in the exemplary embodiments of this disclosure will be explained and described. Specifically, refer to... Figure 2 As shown, the data storage system may include a rack cluster 201 composed of multiple racks; wherein, the rack cluster may include rack 1, rack 2, ..., rack N, where a rack refers to the physical rack where the node is located, or it can be a logical rack concept, which is not specifically limited in this example; each rack may include multiple node devices 202, such as node device 1, node device 2, ..., node device M, etc., where a node device refers to a storage or computing terminal, which may be a physical machine, virtual machine, or container, etc.; furthermore, node devices set on the same rack can communicate with each other, and node devices set on different racks can also communicate with each other; each node device can be used to store data blocks to be stored, and of course, the client 203 can also be set on any node device in any rack, and the node device and the client set on the node device can communicate with each other; at the same time, node devices set on the same rack can form a node cluster.

[0065] In a data storage method provided in the exemplary embodiments of this disclosure:

[0066] In step S110, in response to the data storage request sent by the client, the data storage request is parsed to obtain the data block to be stored included in the data storage request.

[0067] Specifically, when a client needs to store a data block, it can send a data storage request to the rack cluster. When any rack in the rack cluster receives the data storage request, it can respond to the request by allocating a target node device from the node cluster corresponding to that rack. Then, it can parse the data storage request based on the target node device to obtain the data block to be stored included in the data storage request. The data block to be stored can include file data, document data (documents, pictures, text, and symbols, etc.), or video data (image frames and / or audio frames), etc. Of course, it can also include other types of data, which are not specifically limited in this example.

[0068] In step S120, the number of replicas of the data block to be stored is determined, and the current network address of the rack where the master node device is located is determined.

[0069] Specifically, first, the number of copies of the to-be-stored data block is determined; wherein the number of copies can be used to represent the number of copies required for the to-be-stored data block, for example, one copy means that the to-be-stored data block needs to be stored one copy; two copies means that the to-be-stored data block needs to be stored two copies; wherein the number of copies of the to-be-stored data block can be determined according to the importance of the to-be-stored data block; the importance can be represented by the frequency of access of the to-be-stored data block or the confidentiality of the to-be-stored data block; of course, the number of copies can also be determined in other ways, such as the time interval between the generation time of the to-be-stored data block and the current time, etc., which is not limited in this example.

[0070] Secondly, the current network address of the current rack where the master node device is located also needs to be determined; for example, the current network address can be 192.168.8.15 (the current network address is only for exemplary purposes and does not represent the real network address). It needs to be supplemented here that by determining the current network address of the current rack where the master node device is located, the other node devices in the current rack except the master node device can be avoided in the subsequent process of selecting the target slave node device, thereby avoiding the problem of heavy burden caused by the current rack becoming a data hotspot.

[0071] In step S130, other network addresses except the current network address are obtained from the preset rack cluster, and target slave node devices not less than the number of copies are selected from other racks corresponding to the other network addresses.

[0072] In the example embodiment, first, other network addresses except the current network address are obtained from the preset rack cluster. By this method, it can be ensured that the rack where the slave node device storing the to-be-stored data block is different from the rack where the client is located, and the slave node device is also different from the master node device, thereby reducing the load of the rack where the master node device is located, avoiding the master node device becoming a data hotspot, and thereby reducing the problem of low storage efficiency and / or data reading efficiency.

[0073] Secondly, target slave node devices not less than the number of copies are selected from other racks corresponding to the other network addresses. Specifically, the selection of the target slave node device can include the following several ways:

[0074] Method one: when the number of copies is one, first, a first slave node device is randomly selected from the other racks corresponding to the other network addresses, and it is determined whether the number of copies is greater than one; secondly, if the number of copies is less than or equal to one, the first slave node device is taken as the target slave node device.

[0075] The second mode: the case of two copies: first, if the number of copies is greater than one, a first network address of a first rack where the first slave node device is located is determined, and a second slave node device is randomly selected from other racks corresponding to other network addresses except the current network address and the first network address in the preset rack cluster; second, it is determined whether the number of copies is greater than two, and when it is determined that the number of copies is less than or equal to two, the first slave node device and the second slave node device are taken as the target slave node device.

[0076] The third mode: the case of three copies: first, when it is determined that the number of copies is greater than two, a third slave node device is randomly selected from other racks corresponding to the other network addresses, and it is determined whether the third slave node device is the same as the first slave node device and / or the second slave node device; second, when it is determined that the third slave node device is the same as any one of the first slave node device and the second slave node device, the selection step of the third slave node device is repeated until the third slave node device is different from the first slave node device and the second slave node device, and the number of repeated selections does not reach a preset threshold; finally, it is determined whether the number of copies is greater than three, and when it is determined that the number of copies is less than or equal to three, the first slave node device, the second slave node device, and the third slave node device are taken as the target slave node device. Further, when the number of repeated selections reaches the preset threshold and the third slave node device is the same as the first slave node device or the second slave node device, the last selected slave node device is taken as the third slave node device.

[0077] The fourth mode: the case of four or more copies: first, when it is determined that the number of copies is greater than three, one or more fourth slave node devices are randomly selected from the preset rack cluster; wherein the number of fourth slave node devices is the difference between the number of copies and the number of selected slave node devices; second, the first slave node device, the second slave node device, the third slave node device, and one or more fourth slave node devices are taken as the target slave node device.

[0078] Hereinafter, the specific selection process of the target slave node device will be explained and described in combination with Figure 3 The specific selection process of the target slave node device will be explained and described in combination with Figure 3 The specific selection process of the target slave node device will be explained and described in combination with

[0079] Step S301, randomly selecting a first slave node device from other racks corresponding to other network addresses; for example, the slave node device is R-m-DN-n, where R is a preset rack cluster, m is the mth rack in the rack cluster R, DN is the node cluster of the mth rack, and n is the nth node device in the node cluster, i.e., the first slave node device;

[0080] Step S302, determining whether the number of copies is greater than one; if yes, jumping to step S303; if no, ending the selection;

[0081] Step S303, randomly selecting a second slave node device R-x-DN-y from other racks corresponding to other network addresses except the current network address and the first network address in the preset rack cluster; where x is the xth rack in the rack cluster R, DN is the node cluster of the xth rack, and y is the yth node device in the node cluster;

[0082] Step S304, determining whether the number of copies is greater than two; if yes, jumping to step S305; if no, ending the selection;

[0083] Step S305, randomly selecting a third slave node device R-p-DN-q from other racks corresponding to other network addresses; where p is the pth rack in the rack cluster R, DN is the node cluster of the pth rack, and q is the qth node device in the node cluster;

[0084] Step S306, determining whether the third slave node device is the same as the first slave node device and the second slave node device; if yes, deleting the third slave node device, and looping steps S305 and S306 until the third slave node device is different from the first slave node device and the second slave node device, or the number of loops reaches a preset number; if no, jumping to step S307;

[0085] Step S307, determining whether the number of copies is greater than three; if yes, jumping to step S308; if no, ending the selection;

[0086] Step S308, randomly selecting one or more fourth slave node devices from the preset rack cluster until the number of the selected fourth slave node devices is the same as the difference between the number of copies and the number of the selected slave node devices.

[0087] In Figure 3In the example embodiment shown, in step S306, the selected node R-p-DN-q is deleted, which is not to delete the actually existing data node in the node cluster corresponding to the rack, but to delete the record of the randomly selected node at the beginning of the loop. Since the selected node is the same as the node intended to store the current block data, the R-p-DN-q node is not selected as the selected node, and the loop needs to continue to search for a node that meets the different racks. On the other hand, when the number of replicas is three, the selection can continue based on the preset selection number M. Here, the random selection is to simplify the process. At the same time, since there are three different nodes on different racks, the reliability of the data can be guaranteed, and other replicas can randomly select nodes to store, so the fourth slave node device can randomly select.

[0088] Further, after obtaining the fourth slave node device, the data storage method can further include: first, calculating the number of stored data blocks in the one or more fourth slave node devices, and calculating the current load rate of the one or more fourth slave node devices according to the number of stored data blocks; second, when it is determined that the current load rate of any fourth slave node device exceeds the preset load rate, randomly selecting another slave node device from the preset rack cluster to replace the fourth slave node device whose current load rate exceeds the preset load rate, until the current load rate of all fourth slave node devices is less than the preset load rate. In the calculation of the current load rate, the current load rate can be calculated according to the number of stored data blocks and the total number of data blocks that can be stored in the fourth slave node device. When the current load rate is calculated, the fourth slave node device whose current load rate does not exceed the preset load rate can be determined. Through this method, the node with a lower load can be selected to store data according to the load of the node, thereby avoiding the problem of excessive load of each node device.

[0089] At this point, the first slave node device and / or the second slave node device and / or the third slave node device and / or the fourth slave node device have been obtained, and the target slave node device can be obtained according to the first slave node device and / or the second slave node device and / or the third slave node device and / or the fourth slave node device. In this embodiment, or can be understood as one replica number, two replica numbers, three replica numbers, and four or more replica numbers, that is, the number of corresponding slave node devices can be selected according to the specific number of replicas, and then the target slave node device is obtained. Of course, in actual application, most cases are three replicas, which can avoid the problem that one or two data replicas fail to access due to too few replicas, and can avoid the problem that the system has a heavy burden due to too much redundant data caused by too many replicas.

[0090] In step S140, based on the other network addresses, the to-be-stored data blocks are sequentially copied to the target slave node device in the other rack to store the to-be-stored data blocks by the target slave node device.

[0091] Specifically, when the target slave node device selection is completed, the to-be-stored data blocks can be sequentially copied to the target slave node device based on the other network addresses, so that the target slave node device stores the to-be-stored data blocks, for example, the to-be-stored data blocks can be stored in the disk where the target slave node device is located, and the like. For example, in a specific storage process, as shown in Figure 4 The to-be-stored data blocks in the client can be divided into a plurality of sub-data blocks, and the client can send the sub-data blocks of the to-be-stored data blocks to the target node device in an asynchronous manner, and the to-be-stored data blocks are contained in a data storage request; wherein the data storage request User Request Byte Buffer can contain a message header (request header) and a message body (request body), and both of them are DirectByteBuffer type data in off-heap memory, which can avoid the target node device copying the data blocks from the Java Virtual Machine (JVM) heap memory to the off-heap memory.

[0092] In the following, the data storage method disclosed in the example embodiments of the present disclosure will be further explained and described. Specifically, referring to Figure 5 The data storage method can include the following steps: Figure 5

[0093] Step S501, receiving a data storage request sent by a client and parsing the data storage request to obtain a to-be-stored data block;

[0094] Step S502, according to the data copy of the to-be-stored data block and the master node device where the client is located, selecting a target slave node device for the to-be-stored data block by an intelligent and reasonable algorithm and a suitable selection rule;

[0095] Step S503, storing the to-be-stored data block to the target slave node device.

[0096] In the data storage method shown in Figure 5 For the case of multiple copies and multiple racks of distributed storage, by using an intelligent and reasonable algorithm and a suitable selection rule, a relatively ideal data distribution state can be finally achieved, which provides a basis for uniform data reading and avoids data hotspots as much as possible, thereby improving the access rate during user access to data.

[0097] ​For example, assume that there are 3 racks, and data has 3 copies (i.e., the same 3 copies of data are stored in data nodes, and redundant data ensures that data can be provided by other nodes in the event of node failure). Among them, the data node 192.168.2.209 is a data write node, and the data storage rack result of the written data is as follows:

[0098] In one aspect, through the data storage method described in the example embodiment, i.e., in the case of an optimization algorithm, it can be seen that the saved data is evenly distributed in different nodes of different racks.

[0099] Test data 1: [ / DPL1-H-02 / 192.168.0.15, / DPL1-E-09 / 192.168.0.16, / DPL1-G-14 / 192.168.0.37];

[0100] Test data 2: [ / DPL1-E-09 / 192.168.2.209, / DPL1-H-02 / 192.168.2.210, / DPL1-G-14 / 192.168.0.9];

[0101] Test data 3: [ / DPL1-H-02 / 192.168.2.210, / DPL1-E-09 / 192.168.0.16, / DPL1-G-14 / 192.168.0.9];

[0102] Test data 4: [ / DPL1-G-14 / 192.168.0.9, / DPL1-H-02 / 192.168.2.210, / DPL1-E-05 / 192.168.0.16];

[0103] Test data 5: [ / DPL1-H-02 / 192.168.0.15, / DPL1-G-14 / 192.168.0.9, / DPL1-E-05 / 192.168.0.16]. The specific storage result can be referred to as shown in Figure 6

[0104] On the other hand, in the case of no optimization algorithm, it can be seen that the saved data has a default copy stored in a fixed rack / DPL1-E-09.

[0105] Test data 1: [ / DPL1-E-09 / 192.168.2.209, / DPL1-G-14 / 192.168.0.37, / DPL1-G-14 / 192.168.0.9];

[0106] ​Test data 2: [ / DPL1-E-09 / 192.168.2.209:, / DPL1-H-02 / 192.168.2.210, / DPL1-H-02 / 192.168.0.15]

[0107] Test data 3: [ / DPL1-E-09 / 192.168.2.209,, / DPL1-G-14 / 192.168.0.9, / DPL1-G-14 / 192.168.0.37].

[0108] Test data 4: [ / DPL1-E-09 / 192.168.2.209,, / DPL1-G-14 / 192.168.0.9, / DPL1-G-14 / 192.168.0.37].

[0109] Test data 5: [ / DPL1-E-09 / 192.168.2.209, / DPL1-H-02 / 192.168.2.210, / DPL1-H-02 / 192.168.0.15].

[0110] The specific storage result can be referred to as shown in Figure 7 .

[0111] The example embodiments of the present disclosure further provide a data storage device configured in a master node device where a client is located. As shown in Figure 8 , the data storage device can include a data storage request analysis module 810, a copy number determination module 820, a slave node device selection module 830, and a data storage module 840. Wherein:

[0112] The data storage request analysis module 810 can be used to analyze the data storage request sent by the client, and obtain the data block to be stored in the data storage request;

[0113] The copy number determination module 820 can be used to determine the number of copies of the data block to be stored, and determine the current network address of the current rack where the master node device is located;

[0114] The slave node device selection module 830 can be used to obtain other network addresses except the current network address from the preset rack cluster, and select target slave node devices not less than the number of copies from other racks corresponding to the other network addresses;

[0115] The data storage module 840 can be used to copy the data block to be stored to the target slave node devices in the other racks in sequence based on the other network addresses, so as to store the data block to be stored through the target slave node devices.

[0116] In an example embodiment of the present disclosure, the slave node device selection module 830 can be configured to randomly select a first slave node device from the other racks corresponding to the other network addresses, and determine whether the number of replicas is greater than one; if the number of replicas is less than or equal to one, the first slave node device is taken as the target slave node device.

[0117] In an example embodiment of the present disclosure, the slave node device selection module 830 can be further configured to, if the number of replicas is greater than one, determine a first network address of a first rack where the first slave node device is located, and randomly select a second slave node device from other racks corresponding to other network addresses except the current network address and the first network address in the preset rack cluster; determine whether the number of replicas is greater than two, and if the number of replicas is less than or equal to two, take the first slave node device and the second slave node device as the target slave node device.

[0118] In an example embodiment of the present disclosure, the slave node device selection module 830 can be further configured to, if the number of replicas is greater than two, randomly select a third slave node device from the other racks corresponding to the other network addresses, and determine whether the third slave node device is the same as the first slave node device and / or the second slave node device; if the third slave node device is the same as any of the first slave node device and the second slave node device, repeat the selection of the third slave node device until the third slave node device is different from the first slave node device and the second slave node device, and the number of repetitions does not reach a preset threshold; determine whether the number of replicas is greater than three, and if the number of replicas is less than or equal to three, take the first slave node device, the second slave node device and the third slave node device as the target slave node device.

[0119] In an example embodiment of the present disclosure, the slave node device selection module 830 can be further configured to, when the number of repetitions reaches the preset threshold and the third slave node device is the same as the first slave node device or the second slave node device, take the last selected slave node device as the third slave node device.

[0120] In an example embodiment of the present disclosure, the slave node device selecting module 830 can be further configured to: randomly select one or more fourth slave node devices from the preset rack cluster when it is determined that the number of replicas is greater than three; wherein the number of the fourth slave node devices is the difference between the number of replicas and the number of selected slave node devices; and take the first slave node device, the second slave node device, the third slave node device and the one or more fourth slave node devices as the target slave node devices.

[0121] In an example embodiment of the present disclosure, the data storage device further comprises:

[0122] The current load rate calculating module can be configured to calculate the number of stored data blocks in the one or more fourth slave node devices, and calculate the current load rate of the one or more fourth slave node devices according to the number of stored data blocks.

[0123] The fourth slave node device determining module can be configured to randomly select another slave node device from the preset rack cluster to replace any fourth slave node device whose current load rate exceeds the preset load rate until the current load rates of all fourth slave node devices are less than the preset load rate.

[0124] The specific details of the modules in the above data storage device have been described in detail in the corresponding data storage method, and thus will not be described here.

[0125] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into several modules or units.

[0126] In addition, although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. Additionally or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0127] In an example embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

[0128] Those skilled in the art can understand that the various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, the various aspects of the present disclosure can be embodied as a whole hardware implementation, a whole software implementation (including firmware, microcode, etc.), or an implementation combined with hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0129] The electronic device 900 according to this embodiment of the present disclosure will be described below with reference to Figure 9 Figure 9 The electronic device 900 shown is merely an example, and should not impose any limitation on the function and use range of the embodiments of the present disclosure.

[0130] As Figure 9 shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 can include, but are not limited to, the at least one processing unit 910 described above, the at least one storage unit 920 described above, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), and a display unit 940.

[0131] The storage unit stores program code that can be executed by the processing unit 910, so that the processing unit 910 performs the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification. For example, the processing unit 910 can perform the steps shown in Figure 1 S110: in response to a data storage request sent by a client, parsing the data storage request to obtain a data block to be stored included in the data storage request; S120: determining the number of copies of the data block to be stored, and determining the current network address of the current rack where the master node device is located; S130: obtaining other network addresses except the current network address from a preset rack cluster, and selecting target slave node devices not less than the number of copies from other racks corresponding to the other network addresses; S140: based on the other network addresses, copying the data block to be stored to the target slave node devices in the other racks in turn, so as to store the data block to be stored through the target slave node devices.

[0132] The storage unit 920 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 9201 and / or a cache memory 9202, and can further include a read-only memory (ROM) 9203.

[0133] ​The storage unit 920 can also include a program / utility 9204 having a set (at least one) of program modules 9205, examples of which include an operating system, one or more application programs, other program modules, and program data, each of which

[0134] The bus 930 can represent one or more of several types of bus structures, including a storage bus or bus controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures.

[0135] The electronic device 900 can also communicate with one or more external devices 1000 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices such as a storage device or an external effects device; and / or one or more devices that enable a user to interact with the electronic device 900 in some way, such as a user's eyes, ears, skin, etc. The communication can be through input / output (I / O) interface(s) 950. Additionally, the electronic device 900 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through a network adapter 960. As depicted, the network adapter 960 communicates with the other components of the electronic device 900 through the bus 930. It should be appreciated that although not shown, other hardware and / or software components could be used in conjunction with the electronic device 900. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0136] Those skilled in the art will readily understand that the example embodiments described herein can be implemented by software and / or by software in combination with the necessary hardware. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.

[0137] In exemplary embodiments of the present disclosure, a computer readable storage medium having stored thereon a program product capable of implementing the above-described methods of the present specification is also provided. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product including a program code that, when run on a terminal device, causes the terminal device to perform the steps described in the above "Exemplary Methods" section according to various exemplary embodiments of the present disclosure.

[0138] A program product for implementing the above-described methods according to embodiments of the present disclosure can take the form of a portable compact disc read-only memory (CD-ROM) and include a program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in the present document, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0139] The program product can take any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0140] The computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the readable program code is embodied. Such propagated data signal can take multiple forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The readable signal medium can also be any readable medium that is not a readable storage medium and that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0141] The program code contained on the readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, and the like, or any suitable combination thereof.

[0142] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0143] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0144] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention described herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not invented by this disclosure. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A data storage method, characterized by, The data storage method comprises the following steps of: In response to a data storage request sent by a client, the data storage request is parsed to obtain a data block to be stored included in the data storage request; The number of copies of the data block to be stored is determined, and the current network address of the rack where the master node device is located is determined; From the preset rack cluster, other network addresses except the current network address are obtained, and target slave node devices not less than the number of copies are selected from other racks corresponding to the other network addresses, including: a first slave node device is randomly selected from other racks corresponding to the other network addresses, and it is determined whether the number of copies is greater than one; wherein, If the number of copies is greater than one, the first network address of the first rack where the first slave node device is located is determined, and a second slave node device is randomly selected from other racks corresponding to other network addresses except the current network address and the first network address in the preset rack cluster; It is determined whether the number of copies is greater than two, and when it is determined that the number of copies is less than or equal to two, the first slave node device and the second slave node device are used as the target slave node devices; When it is determined that the number of copies is greater than two, a third slave node device is randomly selected from other racks corresponding to the other network addresses, and it is determined whether the third slave node device is the same as the first slave node device and / or the second slave node device; When it is determined that the third slave node device is the same as any one of the first slave node device and the second slave node device, the selection step of the third slave node device is repeated until the third slave node device is different from the first slave node device and the second slave node device, and the number of repeated selections does not reach a preset threshold; Based on the other network addresses, the data block to be stored is copied to the target slave node devices in the other racks in sequence, so that the data block to be stored is stored through the target slave node devices.

2. The data storage method of claim 1, wherein, From the other racks corresponding to the other network addresses, the slave node devices not less than the number of copies are selected, which also includes: If the number of copies is less than or equal to one, the first slave node device is used as the target slave node device.

3. The data storage method of claim 1, wherein, The data storage method further comprises: It is determined whether the number of copies is greater than three, and when it is determined that the number of copies is less than or equal to three, the first slave node device, the second slave node device and the third slave node device are used as the target slave node devices.

4. The data storage method of claim 3, wherein, The data storage method further comprises: When the number of repeated selections reaches the preset threshold and the third slave node device is the same as the first slave node device or the second slave node device, the last selected slave node device is used as the third slave node device.

5. The data storage method of claim 3, wherein, The data storage method further comprises: determining that the number of replicas is greater than three, randomly selecting one or more fourth slave node devices from the preset rack cluster; wherein the number of the fourth slave node devices is the difference between the number of replicas and the number of the selected slave node devices; taking the first slave node device, the second slave node device, the third slave node device and the one or more fourth slave node devices as the target slave node devices.

6. The data storage method of claim 5, wherein, After randomly selecting one or more fourth slave node devices from the preset rack cluster, the data storage method further comprises: calculating the number of stored data blocks in the one or more fourth slave node devices, and calculating the current load rate of the one or more fourth slave node devices according to the number of stored data blocks; determining that the current load rate of any fourth slave node device exceeds the preset load rate, randomly selecting another slave node device from the preset rack cluster to replace the fourth slave node device whose current load rate exceeds the preset load rate, until the current load rate of all fourth slave node devices is less than the preset load rate.

7. A data storage device, characterized by The data storage device is configured in the master node device where the client is located, and the data storage device comprises: a data storage request analysis module, configured to analyze a data storage request sent by a client, and obtain a data block to be stored in the data storage request; a replica number determination module, configured to determine the number of replicas of the data block to be stored, and determine the current network address of the current rack where the master node device is located; a slave node device selection module, configured to obtain other network addresses except the current network address from a preset rack cluster, and select target slave node devices not less than the number of replicas from other racks corresponding to the other network addresses, including: randomly selecting a first slave node device from other racks corresponding to the other network addresses, and determining whether the number of replicas is greater than one; wherein, if the number of replicas is greater than one, determining the first network address of the first rack where the first slave node device is located, and randomly selecting a second slave node device from other racks corresponding to other network addresses except the current network address and the first network address in the preset rack cluster; determining whether the number of replicas is greater than two, and taking the first slave node device and the second slave node device as the target slave node devices when the number of replicas is less than or equal to two; when the number of replicas is greater than two, randomly selecting a third slave node device from other racks corresponding to the other network addresses, and determining whether the third slave node device is the same as the first slave node device and / or the second slave node device; when the third slave node device is the same as any of the first slave node device and the second slave node device, repeating the selection step of the third slave node device until the third slave node device is different from the first slave node device and the second slave node device, and the number of repetitions does not reach a preset threshold. A data storage module is configured to copy the to-be-stored data block to a target slave node device in the other rack in sequence based on the other network address, so that the to-be-stored data block is stored by the target slave node device.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the data storage method of any one of claims 1-6.

9. An electronic device, comprising: Comprise: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to implement the data storage method of any one of claims 1-6 by executing the executable instructions.

Images