Data storage method, device and equipment and computer readable storage medium

Abstract

The invention discloses a data storage method, device and equipment and a computer readable storage medium, and is applied to the technical field of distributed storage, and the data storage method comprises the following steps: determining the weight of each collocation group according to the mapping relationship between the collocation groups and object storage equipment and the capacity use information of the object storage equipment; sending the weight of each collocation group to the client, so that the client determines a target collocation group according to the weight of each collocation group; and updating the weight of the at least one collocation group, so that the weight difference between the selected target collocation group and the unselected collocation group is increased. According to the invention, the weight of each collocation group is determined according to the mapping relation between the collocation group and the object storage device and the capacity use information of the object storage device, so that the client determines the target collocation group for data storage based on the weight of the collocation group and updates the weight of the target collocation group and the weight of other collocation groups; and other target collocation groups can be selected subsequently for data storage, so that the utilization rate balance of the disk is kept.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a data storage method, device, equipment and computer-readable storage medium. Background Art

[0002] At present, the most common distribution method is hashing (such as the crush algorithm). The crush algorithm uses the data unique identifier, the current cluster topology, and the data redundancy strategy as input. It can obtain the location of the underlying storage device (such as a disk) where the data is located through calculation anytime and anywhere and communicate directly with it. When using hashing operations, when the cluster scale is small, due to the limited number of nodes, different data items may be mapped to the same hash value, resulting in uneven data distribution. This conflict is more obvious when the number of nodes is small, because each node needs to bear more data load, thereby increasing the load pressure of some nodes, so hashing operations cannot guarantee balanced data distribution.

[0003] It can be seen that how to ensure the balance between disk capacity and disk utilization when performing distributed storage of data is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a data storage method, device, equipment and computer-readable storage medium, which solve the technical problem in the prior art that data storage cannot be balanced.

[0005] In order to solve the above technical problems, the present invention provides a data storage method, which is applied to a management module of a distributed storage system, wherein the distributed storage system includes a management module and a client, wherein the management module is connected to the client, and comprises:

[0006] Determine the weight of each placement group according to the mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device; wherein any placement group is mapped to multiple object storage devices;

[0007] Sending the weight of each placement group to the client, so that the client determines the target placement group according to the weight of each placement group;

[0008] The weight of at least one placement group is updated so that the weight difference between the selected target placement group and the unselected placement groups increases; and the step of sending the weight of each placement group to the client and subsequent steps are returned to execute.

[0009] Optionally, the weight of the placement group includes a data column read weight and a data column write weight; and determining the weight of each placement group according to a mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device includes:

[0010] Determining an initial weight of the object storage device based on the capacity usage information of the object storage device;

[0011] According to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy strategy, the data column read weight and the data column write weight corresponding to each placement group are determined.

[0012] Optionally, determining an initial weight of the object storage device based on the capacity usage information of the object storage device includes:

[0013] Determine the remaining capacity of the object storage device according to the capacity usage information of the object storage device;

[0014] The initial weight of the object storage device is determined according to the remaining capacity of the object storage device; wherein the size of the remaining capacity is positively correlated with the initial weight of the object storage device.

[0015] Optionally, updating the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases includes:

[0016] The weights of all placement groups are increased to obtain a first updated placement group weight, and the first updated placement group weight corresponding to the target placement group is decreased to obtain a second updated placement group weight corresponding to the target placement group.

[0017] Optionally, the step of increasing the weights of all placement groups to obtain a first updated placement group weight, and decreasing the first updated placement group weight corresponding to the target placement group to obtain a second updated placement group weight corresponding to the target placement group includes:

[0018] Based on the first preset parameter, weights of all placement groups are increased to obtain the first updated placement group weights;

[0019] Based on a second preset parameter, a first updated placement group weight corresponding to the target placement group is decreased to obtain the second updated placement group weight corresponding to the target placement group.

[0020] Optionally, the weight of the placement group includes a data column read weight and a data column write weight; and updating the weight of at least one placement group so as to increase the weight difference between the selected target placement group and the unselected placement groups includes:

[0021] Performing an increase process on the data column read weights and data column write weights of the object storage devices corresponding to all placement groups to obtain a first data column read weight and a first data column write weight corresponding to each object storage device;

[0022] Based on the first data column read weight and the first data column write weight corresponding to each object storage device, update the data column read weight and the data column write weight of each unselected placement group;

[0023] Performing a decreasing process on a first data column read weight and a first data column write weight of a target object storage device corresponding to the target placement group to obtain a second data column read weight and a second data column write weight corresponding to the target object storage device;

[0024] Based on the second data column read weight and the second data column write weight corresponding to the target object storage device, the data column read weight and the data column write weight corresponding to the target placement group are updated.

[0025] Optionally, performing an increase process on the data column read weight and the data column write weight of the object storage devices corresponding to all placement groups to obtain a first data column read weight and a first data column write weight corresponding to each object storage device includes:

[0026] Performing a logical operation based on the current data column read weight, the number of replicas, and the number of data columns to be read corresponding to the object storage device to determine the first data column read weight;

[0027] A logical operation is performed based on the current data column write weight corresponding to the object storage device and the number of data columns to be written to determine the first data column write weight.

[0028] An embodiment of the present invention further provides a method for determining a target placement group, which is applied to a client of a distributed storage system, and includes:

[0029] Receiving a weight of each placement group; wherein the weight of each placement group is a weight of the placement group determined based on the data storage method;

[0030] In response to a write operation of the data block, determining a target placement group based on a weight of each placement group;

[0031] The data block is written to the target placement group of the distributed storage system.

[0032] Optionally, in response to the write operation of the data block, determining the target placement group based on the weight of each placement group includes:

[0033] In response to a write operation of a data block, taking a placement group with a maximum read weight of a data column as the target placement group;

[0034] If the data column read weights of all placement groups are the same, the placement group with the largest data column write weight is used as the target placement group;

[0035] If the data column write weights of all placement groups are the same, the placement group with the largest placement group weight is used as the target placement group.

[0036] The embodiment of the present invention further provides a distributed storage system, including:

[0037] A management module, used to connect to a client and multiple object storage devices, and execute the steps of the above data storage method to obtain the weight of each placement group;

[0038] The client is used to execute the steps of the above-mentioned target placement group determination method.

[0039] An embodiment of the present invention further provides a data storage device, which is applied to a management module of a distributed storage system, wherein the distributed storage system includes a management module and a client, wherein the management module is connected to the client and includes:

[0040] A weight determination module for each placement group, used to determine the weight of each placement group according to a mapping relationship between the placement group and the object storage device and capacity usage information of the object storage device;

[0041] A weight sending module, used for sending the weight of each placement group to the client, so that the client determines the target placement group according to the weight of each placement group;

[0042] The placement group weight updating module is used to update the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases; and return to execute the step of sending the weight of each placement group to the client and subsequent steps.

[0043] An embodiment of the present invention further provides an electronic device, including:

[0044] Memory for storing computer programs;

[0045] A processor is used to execute the computer program to implement the steps of the data storage method as described above, or to execute the computer program to implement the steps of the target placement group determination method as described above.

[0046] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data storage method described above are implemented; or, when the computer program is executed by a processor, the steps of the target placement group determination method described above are implemented.

[0047] An embodiment of the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned data storage method, or, when executed by a processor, implements the steps of the above-mentioned target placement group determination method.

[0048] The embodiment of the present invention determines the weight of each placement group according to the mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device; wherein any placement group is mapped to multiple object storage devices; the weight of each placement group is sent to the client, so that the client determines the target placement group according to the weight of each placement group; the weight of at least one placement group is updated to increase the weight difference between the selected target placement group and the unselected placement groups; and returns to execute the step of sending the weight of each placement group to the client and the subsequent steps. The beneficial effects of the present invention are as follows: compared with the current use of hash operations that cannot ensure balanced data distribution, the present invention determines the weight of each placement group according to the mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device when performing data storage; so that the client can determine the target placement group for data storage based on the weight of each placement group, and after the client uses the target placement group for data storage, the weights of the target placement group and other placement groups are updated, so that the weight of the subsequent preferred target placement group is reduced, thereby maintaining the capacity balance of the cluster disk and the disk usage balance at the same time, taking into account the cluster write performance and read performance, and reducing the fluctuation caused by uneven data distribution.

[0049] In addition, the present invention also provides a data storage device, equipment and computer-readable storage medium, a target placement group determination method, equipment and computer-readable storage medium, and a distributed storage system also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0051] Figure 1 A flowchart of a data storage method provided by an embodiment of the present invention;

[0052] Figure 2 An example flowchart of a data storage method provided by an embodiment of the present invention;

[0053] Figure 3A schematic diagram for determining available placement groups provided by an embodiment of the present invention;

[0054] Figure 4 A flow chart of a method for determining a target placement group provided by an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the structure of a data storage device provided by an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of the structure of a device for determining a target placement group provided by an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of a distributed storage system provided by an embodiment of the present invention;

[0058] Figure 8 A specific schematic diagram of a distributed storage system provided by an embodiment of the present invention;

[0059] Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Some of the terms that appear in the description of the embodiments of the present invention may be interpreted as follows:

[0062] Data redundancy strategy: A data backup and redundant storage strategy adopted in computer systems. By creating redundant copies of data in the system, data reliability and availability can be improved to cope with hardware failures, data corruption or accidental deletion. Common data redundancy strategies include replicas and EC (erasure coding), which can help the system maintain data integrity in the event of a failure and quickly recover to normal working conditions.

[0063] File: A file is an ordered sequence of related information items with a symbolic name and a logically complete meaning.

[0064] Copy: A backup of the contents of a file.

[0065] Fault tolerance: The ability of a distributed system to tolerate failures, such as the number of disk failures and the number of host failures that can be tolerated.

[0066] Disk group: In a hybrid hard disk configuration, cache disks and data disks are paired in a disk group relationship. The SSD (solid state drive) in each disk group only provides cache acceleration capabilities for the HDD (mechanical hard disk) under its disk group.

[0067] Please refer to Figure 1 , Figure 1 A flowchart of a data storage method provided by an embodiment of the present invention. The method is applied to a management module of a distributed storage system, the distributed storage system includes a management module and a client, the management module and the client are connected, and may include:

[0068] S101, determining the weight of each placement group according to a mapping relationship between a placement group and an object storage device and capacity usage information of the object storage device; wherein any placement group is mapped to a plurality of object storage devices.

[0069] The placement group in this embodiment is a special term under the Ceph (unified storage system) distributed architecture. In the Ceph distributed storage system, the placement group (pg for short) is a fragment of the logical object pool, which places objects as a group in the osd (Object Storage Daemon). The object storage device (osd) in this embodiment is a disk specifically used to store a large amount of unstructured data. The main function of the placement group is to reduce the amount of metadata for each object that needs to be tracked and to reduce the number of processes that need to be run. Each object will be fixedly mapped into a pg, so when looking for an object, you only need to find the pg to which the object belongs, and then traverse this pg, without traversing all objects. Objects in pg are mapped to multiple osds to achieve data redundancy and reliability. If an osd fails, Ceph will remap the pg data copy on the osd to other healthy osds. This embodiment does not impose any restrictions on the applicable distributed architecture, and this embodiment can be applied to distributed architectures such as Ceph and HDFS (master-slave distributed architecture). In this embodiment, the mapping relationship between the placement group and the object storage device is calculated based on the hash operation method. That is, the data block slog corresponds to a placement group, and the pg is mapped to multiple osds according to the redundant information. The mapping relationship between pg and osd still adopts the hash operation method. In this embodiment, the initial weight can be determined according to the remaining capacity of the object storage device (capacity usage information). The larger the remaining capacity, the greater the weight. Alternatively, this embodiment can also determine the initial weight of each object storage device according to the used capacity of the object storage device (capacity usage information). The larger the used capacity, the smaller the initial weight. For example, taking ec4+2 as an example, it is assumed that there are pg0-pg31 (32 pgs) and osd0-osd29 (30 osds), where pg1 corresponds to osd0-osd5. The first four osds of all pgs are data columns, and the last two osds are check columns. Assume that all osds have 30% remaining capacity. The osd weight is related to the remaining capacity. To simplify the processing, the initial weight of all osds is 3. Among them, ec is the erasure code strategy, and pg represents the placement group (page group). This embodiment can set the initial weight of each placement group to a set value. For example, the initial weight of each placement group in this embodiment can be 1; or the initial weight of each placement group in this embodiment can be 2. This embodiment does not limit the specific placement group weight. For example, the placement group weight in this embodiment can be the weight of the placement group itself; or the placement group weight in this embodiment can also be the weight of the placement group itself, as well as the data column read weight and data column write weight of the placement group determined based on the data column read weight and data column write weight of the object storage device. This embodiment does not limit the specific timing of the placement group update.For example, this embodiment can use the monitoring device to periodically obtain the osd capacity information and update the placement group weight; or this embodiment can also update the placement group weight after the client determines the target placement group. That is, this embodiment updates the placement group weight at two times: 1. Obtain the osd capacity usage information regularly according to the preset period, reset the initial weight of the object storage device, and reset the placement group weight so that the placement group weight adapts to the change of disk capacity; 2. Obtain the osd capacity usage information only once, and continuously update the placement group weight according to the placement group weight update method until a reset signal is received.

[0070] It should be further explained that the weight of the above-mentioned placement group includes a data column read weight and a data column write weight; determining the weight of each placement group according to the mapping relationship between the placement group and the object storage device, and the capacity usage information of the object storage device, may include: determining the initial weight of the object storage device based on the capacity usage information of the object storage device; determining the data column read weight and data column write weight corresponding to each placement group according to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy strategy. In this embodiment, when the weight of the placement group is determined for the first time, the initial weight of the object storage device can be directly determined according to the capacity usage information of the object storage device. This embodiment does not limit the specific data redundancy strategy. The data redundancy strategy in this embodiment can be a multi-copy strategy; or the data redundancy strategy in this embodiment can also be an erasure code strategy. The multi-copy data redundancy strategy refers to copying the data into multiple identical copies, and distributing these copies in different storage nodes or geographical locations. At this time, the data column read weight and the data column write weight are not distinguished, which is equivalent to the data column read weight and the data column write weight being the same. When the data redundancy strategy in this embodiment is the erasure code strategy, if the erasure code strategy is EC2+1, the data column read weight is only related to 2 data columns, and the data column write weight is related to 2 data columns and 1 check column. For ease of understanding, taking ec4+2 as an example, when the initial weight of all osds is determined to be 6, the data column write weight of all osds is determined to be osd_write_weight=6, and the data column read weight osd_read_weight=6 / 6=1 (the denominator 6 is the threshold, and the threshold is determined according to the number of copies. For example, if there are three copies, the threshold is 3, and if it is EC4+2, the threshold is 6). Then the total_write of each pg=sum(osd)=6x6=36, and the total_read of each pg=sum(osd read weight)=1x4=4. It can be understood that the data column read weight and data column write weight corresponding to each placement group in this embodiment need to be determined by determining the data column read weight and data column write weight of the object storage device corresponding to each placement group. This embodiment does not limit the specific method of determining the data column read weight and data column write weight corresponding to each placement group according to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy policy.For example, this embodiment can directly determine the object storage device corresponding to the placement group according to the mapping relationship between the placement group and the object storage device, and the data redundancy strategy, add up the data column read weights of all object storage devices to obtain the data column read weight corresponding to each placement group, and directly add up the data column write weights of all object storage devices to obtain the data column write weight corresponding to each placement group; or in order to improve the feasibility of subsequent selection of the target placement group based on the data column write weight, the data column read weight and the placement group weight, this embodiment can also adopt a certain strategy to adjust the data column read weight so that the data column read weight is not easy to be the same, for example, the read weight of each object storage device can be divided by a set value, for example, the set value can be 3; or the set value can also be 4.

[0071] It should be further explained that the above-mentioned determination of the initial weight of the object storage device based on the capacity usage information of the object storage device may include: determining the remaining capacity of the object storage device according to the capacity usage information of the object storage device; determining the initial weight of the object storage device according to the remaining capacity of the object storage device; wherein the size of the remaining capacity is positively correlated with the initial weight of the object storage device. This embodiment can directly determine the initial weight of the object storage device based on the remaining capacity of the object storage device, thereby improving the efficiency of determining the initial weight of the object storage device.

[0072] It should be further explained that, when the data redundancy strategy is an erasure coding strategy, the above-mentioned determination of the data column read weight and the data column write weight corresponding to each placement group according to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy strategy may include:

[0073] S1021, determine the number of data columns and the number of check columns according to the erasure coding strategy.

[0074] The check column in this embodiment is a method of improving data reliability by adding redundant check information. In the erasure code strategy, the check column generally refers to a column used to store check information, which can be used to detect and correct errors. The data column in this embodiment refers to the column that actually stores user data. In the erasure code strategy, the data column contains the original data, while the check column contains additional information for data recovery. For ease of understanding, taking ec4+2 as an example, assume that there are pg0-pg31 (32 pgs), osd0-osd29 (30 osds), where pg1 corresponds to osd0-osd5. Then the first four osds of each pg are data columns, and the last two osds are check columns. Taking ec2+1 as an example, assume that there are pg0-pg31 (32 pgs), osd0-osd29 (30 osds), where pg1 corresponds to osd0-osd2. Then the first two osds of each pg are data columns, and the last osd is a check column.

[0075] S1022, based on the mapping relationship between the placement group and the object storage device, and the initial weight of the object storage device, determine that the data column read weight corresponding to each placement group is the product of the read weight and the number of data columns; wherein the read weight is the ratio of the initial weight of the object storage device to the sum of the number of data columns and the number of check columns.

[0076] For ease of understanding, when the write weight of all osds is osd_write_weight=3, the read weight osd_read_weight=3 / 3=1, 3 is the threshold for ignoring decimals, that is, weights less than 3 will be ignored. The larger the ignore threshold, the easier it is for the data column read weights to be the same, and the harder it is to select a PG by the data column read weight. This is equivalent to a controllable parameter. The data column read weight of each placement group = read weight x (number of data columns) = 1x4=4, because the erasure code strategy is ec4+2; where x represents a multiplication logic operation.

[0077] S1023, according to the mapping relationship between the placement group and the object storage device, and the initial weight of the object storage device, determine that the data column write weight corresponding to each placement group is the product of the initial weight of the object storage device and the sum of the number of data columns and the number of check columns.

[0078] In this embodiment, the initial weight of the object storage device is 3, then the data column write weight of each placement group = sum(osd) = 3x6 = 18, where 3 is the weight corresponding to each object storage device, and 6 is the sum of the number of data columns and the number of check columns. This embodiment provides a specific method for determining the data column write weight and data column read weight corresponding to each placement group, which improves the accuracy of selecting the target placement group.

[0079] It should be further explained that the above-mentioned determination of the data column read weight and data column write weight corresponding to each placement group according to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy strategy may include: determining some available placement groups; determining the data column read weight and data column write weight corresponding to some available placement groups according to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy strategy. This embodiment does not limit the method for determining the available placement groups. For example, the available placement groups in this embodiment are completely healthy placement groups; or the available placement groups in this embodiment are completely healthy placement groups and can be used, but the consistency of the data in the placement groups is weak (weak consistency means that after some osds in the placement group are offline (restarted, network failure, etc.) and reloaded, some objects in the pg have not been fully recovered. That is, the pg is in the state of recovering, backfilling, etc.); or the available placement groups in this embodiment can be used, but the disk (osd) corresponding to the placement group pg that is being migrated has changed. This embodiment only considers some of the available placement groups when selecting the target placement group, but does not consider all the available placement groups, thereby reducing the time for traversing the placement groups and improving the efficiency of determining the target placement group.

[0080] S102, sending the weight of each placement group to the client, so that the client determines the target placement group according to the weight of each placement group.

[0081] The purpose of the present invention is to select a placement group for a data block. The client in this embodiment may be a client on the user side. This embodiment determines a target placement group according to the weight of each placement group, and may be to use the placement group with the largest weight as the target placement group.

[0082] S103, updating the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases; and returning to execute the step of sending the weight of each placement group to the client and subsequent steps.

[0083] In this embodiment, updating the weight of at least one placement group may be to update only the weight of the target placement group; or in this embodiment, updating the weight of at least one placement group may be to update the weights of all unselected placement groups and the selected target placement group. In this embodiment, the increase in the weight difference between the selected target placement group and the unselected placement groups refers to decreasing the weight of the target placement group and increasing the weight of the unselected placement groups, so that the weight difference between the selected target placement group and the unselected placement groups increases. In this embodiment, returning to execute the step of sending the weight of each placement group to the client and the subsequent steps refers to, after updating the weight of the placement group, executing the step of sending the weight of each placement group to the client in step S102, so that the client determines the target placement group according to the weight of each placement group.

[0084] It should be further explained that, in order to improve the accuracy of the placement group weight update, the above-mentioned updating of the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement group increases can include: increasing the weights of all placement groups to obtain a first updated placement group weight, and decreasing the first updated placement group weight corresponding to the target placement group to obtain a second updated placement group weight corresponding to the target placement group. This embodiment reduces the probability of the target placement group being selected subsequently by decreasing the weight of the target placement group. This embodiment provides a specific method for updating the placement group weights, thereby improving the accuracy of the placement group weight update.

[0085] It should be further explained that the above-mentioned increasing the weights of all placement groups to obtain the first updated placement group weight, and decreasing the first updated placement group weight corresponding to the target placement group to obtain the second updated placement group weight corresponding to the target placement group, may include:

[0086] S1031: Based on a first preset parameter, increase the weights of all placement groups to obtain first updated placement group weights.

[0087] This embodiment does not limit the specific first preset parameter. For example, the first preset parameter in this embodiment may be 1, or the first preset parameter in this embodiment may be 2, or the first preset parameter in this embodiment may be the weight of the placement group during update.

[0088] S1032: Based on the second preset parameter, decrease the first updated placement group weight corresponding to the target placement group to obtain a second updated placement group weight corresponding to the target placement group.

[0089] This embodiment does not limit the specific second preset parameter. For example, the second preset parameter in this embodiment may be 1, or the second preset parameter in this embodiment may be 2, or the second preset parameter in this embodiment may be the weight of the target placement group during the update. This embodiment updates the placement group weight, so that further selection can be made based on the weight, polling is implemented, and the capacity balance of the cluster disk is maintained.

[0090] It should be further explained that the above-mentioned increasing the weights of all placement groups to obtain the first updated placement group weight, and decreasing the first updated placement group weight corresponding to the target placement group to obtain the second updated placement group weight corresponding to the target placement group, may include:

[0091] S1: Increase the weights of all placement groups to obtain a first updated placement group weight; wherein the first updated placement group weight=current placement group weight+placement group initial weight.

[0092] In this embodiment, all placement group weights are updated (initial value is 1), new_pg_weight=pg_weight+1. Here, new_pg_weight represents the first updated placement group weight, and pg_weight represents the current placement group weight. Assuming there are 3 placement groups, total_pgs_weight (total weight) = 1+1+1=3. The weight of pg1 +=1 (initial weight) = 2, and similarly the weight of pg2 = 2, and the weight of pg3 = 2.

[0093] S2: Decrease the weight of the target placement group to obtain a second updated placement group weight; wherein the second updated placement group weight=the first updated placement group weight corresponding to the target placement group-the total placement group weight.

[0094] This embodiment updates the weight of the target placement group for the selected target placement group, pg_weight=pg_weight-total_pgs_weight; wherein pg_weight represents the second updated placement group weight, and pg_weight represents the first updated placement group weight corresponding to the target placement group. For example, pg1 is selected, so the latest pg1 weight=pg1 weight-the weight of all pgs=2-3=-1. After this step, the total weight=sum(pg)=-1+2+2=3. It can be seen that the total weight is unchanged, but the weight of pg1 becomes -1, and it will not be selected later.

[0095] It should be further explained that, in order to improve the comprehensiveness of the placement group weight update, the above-mentioned placement group weight includes the data column read weight and the data column write weight; updating the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases may include:

[0096] Step 1: Perform an increase process on the data column read weight and the data column write weight of the object storage devices corresponding to all placement groups to obtain a first data column read weight and a first data column write weight corresponding to each object storage device.

[0097] Step 2: Based on the first data column read weight and the first data column write weight corresponding to each object storage device, update the data column read weight and the data column write weight of each unselected placement group.

[0098] Step 3: Reduce the first data column read weight and the first data column write weight of the target object storage device corresponding to the target placement group to obtain the second data column read weight and the second data column write weight corresponding to the target object storage device.

[0099] Step 4: Based on the second data column read weight and the second data column write weight corresponding to the target object storage device, update the data column read weight and the data column write weight corresponding to the target placement group.

[0100] This embodiment improves the comprehensiveness of the weight update of the placement group by updating the weight corresponding to the object storage device and the data column read weight and data column write weight corresponding to the target placement group.

[0101] It should be further explained that the above-mentioned increasing processing of the data column read weight and the data column write weight of the object storage devices corresponding to all placement groups to obtain the first data column read weight and the first data column write weight corresponding to each object storage device may include: performing a logical operation based on the current data column read weight, the number of replicas and the number of data columns to be read corresponding to the object storage device to determine the first data column read weight; performing a logical operation based on the current data column write weight and the number of data columns to be written corresponding to the object storage device to determine the first data column write weight. This embodiment determines the read weight of each object storage device as the first data column read weight; wherein the logical operation is: the first data column read weight += (the data column read weight of the current object storage device / (the number of data columns and the number of check columns)) x the number of data columns to be read; determining the write weight of each object storage device as the first data column write weight; wherein the logical operation is: the first data column write weight += the data column write weight of the current object storage device x the number of data columns to be written. wherein each logical operation is to increase the value after the equation on the basis of the original weight.

[0102] This embodiment updates the read weight (data column read weight) of each object storage device (the initial value is the weight calculated according to the capacity of the object storage device), new_read_weight+=(initial osd_weight / 3)x number of osd data columns (the number of osds that need to be read for pg mapping). Among them, new_read_weight represents the latest object storage device read weight, that is, the first data column read weight, initial osd_weight represents the read weight of the current object storage device, when it is the first time, the read weight of the object storage device is the initial weight of the object storage device, when it is the second time, it is the read weight of the object storage device determined last time, 3 is the threshold, and the threshold is determined according to the number of copies. For example, if there are three copies, the threshold is 3, if it is EC4+2, the threshold is 6-the 3 here is used to ignore decimals, that is, weights less than 3 will be ignored. The larger the ignore threshold, the easier it is for the read weights to be the same, and the less likely it is to select a placement group by the read weight. This is equivalent to a controllable parameter used to adjust the balance of osd capacity. The read weights of all object storage devices are updated (including selected and unselected). For ease of understanding, new_read_weight=1+3 / 3x4=5, where the first 3 is the initial weight of the object storage device, 3 is the threshold, and the read weight of the object storage device is 1, so on the basis of (initial osd_weight / 3)x the number of osd data columns, add 1. After this step, the read weight of all object storage devices is 5. This embodiment updates the write weight (data column write weight) of each object storage device (the initial value is the weight calculated according to the osd capacity), new_write_weight+=initial osd_weightxpg mapped osd number, where new_write_weight represents the latest object storage device write weight, that is, the first data column write weight, the initial osd_weight represents the write weight of the current object storage device, and the number of pg mapped osds represents the number of data columns to be written. This embodiment updates the write_weight of all osds (including selected and unselected). new_write_weight=3+3x6=21. After this step, the write weight of all osds is 21.

[0103] It should be further explained that the above-mentioned lowering processing of the first data column read weight and the first data column write weight of the target object storage device corresponding to the target placement group to obtain the second data column read weight and the second data column write weight corresponding to the target object storage device may include: determining the target object storage device corresponding to the target placement group, lowering the data column read weight and the data column write weight of the target object storage device to obtain the second data column read weight and the second data column write weight corresponding to the target object storage device; wherein, the second data column read weight -= (total weight of the object storage device / (the number of data columns and the number of check columns)), the second data column write weight -= the total weight of the object storage device.

[0104] In this embodiment, the second data column read weight -= (total weight of the object storage device / (the number of data columns and the number of check columns)) can be understood as the second data column read weight = the first data column read weight - (total weight of the object storage device / (the number of data columns and the number of check columns)); in this embodiment, the second data column read weight -= (total weight of the object storage device / (the number of data columns and the number of check columns)) can be understood as the second data column read weight = the first data column write weight - (total weight of the object storage device / (the number of data columns and the number of check columns)). For the selected target placement group, this embodiment updates the read_weight data column read weight corresponding to the osd mapped thereto, read_weight-=total_osds_read_weight, toatl_osds_read_weight=sum(osd_weight / 3), where 3 is a threshold for ignoring decimals, read_weight represents the data column read weight of the target object storage device corresponding to the target placement group, total_osds_read_weight represents the total data column read weight of the object storage device, and osd_weight represents the total weight of the object storage device. For ease of understanding, for example, total_osds_read_weight=sum(osd_read_weight) / 3=3x30 / 3=30 (rounded), 3 represents the data column read weight, and 30 represents the number of data columns. The data column read weight of osd0 read_weight=5 (data column read weight before update)-30=-25, 5 is the data column read weight of the target object storage device after update, and similarly, the weights of the other osd read slices of the selected pg are all -25. For the selected placement group, update the write_weight corresponding to the mapped osd, write_weight-=total_write_weight, total_osds_write_weight=sum(osd_weight). total_osds_write_weight=sum(osd_weight)=3x30=90 (rounded); where write_weight represents the latest data column write weight, and total_osds_write_weight represents the total write weight of all object storage devices. The data column write weight of osd0 is write_weight=21-90=-69, where 21 is the data column write weight of the target object storage device after the previous unified update, and 90 represents the total data column write weight; similarly, the write weights of other osd shards of the selected pg are calculated to be -69. The present invention improves the accuracy of the update by providing a specific method for updating the data column write weight and data column read weight corresponding to the placement group.

[0105] An embodiment of the present invention provides a data storage method, which may include: S101, determining the weight of each placement group according to the mapping relationship between the placement group and the object storage device, and the capacity usage information of the object storage device; wherein any placement group is mapped to multiple object storage devices; S102, sending the weight of each placement group to the client, so that the client determines the target placement group according to the weight of each placement group; S103, updating the weight of at least one placement group to increase the weight difference between the selected target placement group and the unselected placement groups; and returning to execute the step of sending the weight of each placement group to the client and subsequent steps. Compared with the current use of hash operations that cannot ensure balanced data distribution, the present invention determines the weight of each placement group according to the mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device when performing data storage; so that the client can determine the target placement group for data storage based on the weight of each placement group, and after the client uses the target placement group for data storage, the weights of the target placement group and other placement groups are updated, so that the weight of the subsequent preferred target placement group is reduced, thereby maintaining the capacity balance of the cluster disk and the disk usage balance at the same time, taking into account the cluster write performance and read performance, and reducing the fluctuation caused by uneven data distribution.

[0106] In order to make the present invention easier to understand, please refer to Figure 2 , Figure 2 A flow chart of a data storage method provided by an embodiment of the present invention is applied to a management module of a distributed storage system, wherein the distributed storage system includes a management module and a client, and the management module and the client are connected, and specifically may include:

[0107] In this embodiment, the data block slog corresponds to a placement group pg, and the pg is mapped to multiple object storage devices osd (disks) by hashing. When selecting a placement group for a data block, the present invention determines the available placement groups according to the status of the placement group, determines the partially available placement groups from the available placement groups, and selects from the partially available placement groups. There are currently five types of placement group status, namely 1. Completely healthy pg; 2. Unusable pg; 3. Usable, but the data consistency in the pg is weak; 4. Temporarily unusable pg; 5. Usable, but the pg is being migrated (the disk (osd) corresponding to the pg has changed). When selecting pg, the status of the pg will be judged. If the pg status is unavailable (for example, the shard is missing, the pg corresponds to 6 osds, but one of the osds is offline), this osd will be avoided. For ease of understanding, please refer to Figure 3 , Figure 3 A schematic diagram for determining an available placement group is provided in an embodiment of the present invention. pg2 is in an unavailable state, so pg2 is not selected.

[0108] In order to select a suitable pg for a data block, it is necessary to obtain the capacity information of the osd in the cluster and the status information of the placement group. Only according to the current information status of the cluster can a suitable pg be selected. The osd will report information to the monitoring component, and the client will periodically obtain the information required to select the pg from the monitoring component and store it in memory. By periodically obtaining the capacity information of each osd in the storage pool and the status information of the pg, the current cluster status information can be perceived when writing data, which serves as the basis for selecting which osd to write to.

[0109] S201, calculate the osd weights corresponding to all available osds according to the osd capacity information, and determine that the initial weight values ​​of the available pgs are all 1.

[0110] This embodiment determines the weight of each osd based on the water level information (capacity information) of the osd. The purpose of the pg weight in this embodiment is to poll the pg to avoid repeatedly selecting the same pg. Taking ec4+2 as an example, assume that there are pg0-pg31 (32 pgs), osd0-osd29 (30 osds), where pg1 corresponds to osd0-osd5. The first four osds of all pgs are data columns, and the last two osds are check columns. Assume that all osds have 30% remaining capacity. The osd weight is related to the remaining capacity. To simplify the processing, the weight of all osds is 3.

[0111] S202, traverse some available pgs, and calculate the data column read weight and data column write weight of the pg according to the osd weight information mapped by the pg; wherein the osd write weight corresponding to each pg is equal to the osd weight, and the osd read weight corresponding to each pg is equal to the ratio of the osd weight to the set threshold, and the set threshold is determined according to the data redundancy strategy.

[0112] According to the weight of osd, calculate the weight of each type of pg. In this embodiment, the write weight of all osds is osd_write_weight=3, and the read weight of osd is osd_read_weight=3 / 3=1. Then the data column write weight of each pg = sum(osd)=3x6=18, and the data column read weight of each pg = sum(osd read weight)=1x4=4. The initial weight of pg is 1.

[0113] S203, sending the weight information of the PG to the client, so that the client determines the target PG based on the weight information of the PG.

[0114] The weight information of the PG in this embodiment includes the weight of the available PG, and the data column read weight and data column write weight corresponding to the available PG. In this embodiment, when the client determines the target PG, it can first compare the data column read weight. If the data column read weights of all PGs are the same, then compare the data column write weights. If the data column write weights are still the same, then compare the PG weights. Since the weights calculated in S202 are all the same, it is assumed that PG0 is selected here.

[0115] S204, update the read weight of each osd, the latest read weight of osd += (initial osd weight / 3) x number of osd data columns, where 3 is the set threshold, and the initial value of the read weight of osd is the weight calculated based on the osd capacity.

[0116] In this embodiment, S204 and S205 update the weights of all osds, thereby affecting the read weight and write weight (upward processing). This embodiment updates the read weights of all osds (including selected and unselected ones). The latest read weight of osd = 1 + 3 / 3x4 = 5. After this step, the read weight of all osds is 5.

[0117] S205, update the write weight of each osd, the latest write weight of osd += the initial osd weight xpg maps the number of osds; the initial value of the write weight of osd is the weight calculated according to the osd capacity.

[0118] This embodiment updates the write weights of all osds (including selected and unselected ones). The latest write weight of osd = 3 + 3 x 6 = 21. After this step, the write weight of all osds is 21.

[0119] S206, update the available pg weight, the latest pg weight = current pg weight + 1.

[0120] This embodiment updates the weights of all pgs, affecting the capacity weight (increase processing). In this embodiment, it is assumed that total_pgs_weight (total weight) = 1+1+1 = 3 (assuming 3 partially available pgs). The weight of pg1 += 1 (initial weight) = 2, similarly the weight of pg2 = 2, and the weight of pg3 = 2.

[0121] S207, update the weight of the target PG, the latest weight of the target PG = the latest PG weight after update - the total weight of PG.

[0122] This embodiment updates the weight of the selected pg, affecting the capacity weight (downward processing). In this embodiment, pg0 is selected, so pg0 weight = pg0 weight - weight of all pgs = 2-3 = -1. After this step, the total weight = sum(pg) = -1+2+2=3. It can be seen that the total weight is unchanged, but the weight of pg0 becomes -1, and it will not be selected in the future.

[0123] S208, update the read weight corresponding to the data column osd mapped by the target pg, the latest data column read weight of the target pg -= total data column read weight, total data column read weight = sum(osd weight t / 3).

[0124] In this example, the total data column read weight = sum(osd weight) / 3 = 3x30 / 3 = 30 (rounded). The read weight of osd0 read_weight = 5-30 = -25. Similarly, the read weights of other osds in the selected pg are calculated to be -25.

[0125] S209, update the write weight corresponding to the data column osd mapped by the target pg, the latest data column write weight of the target pg -= the total data column write weight, the total data column write weight = the sum of all data column write weights.

[0126] This embodiment updates the osd weight of the selected pg, thereby affecting the read weight and write weight (downward processing). The total data column write weight of this embodiment is t=sum(osd_weight)=3x30=90 (rounded). The write weight of osd0 is write_weight=21-90=-69. Similarly, the write weights of other osd shards of the selected pg are calculated to be -69.

[0127] The embodiment of the present invention achieves balanced utilization of the disks as much as possible by polling and traversing the disks, so as to improve the overall performance of the cluster. The algorithm takes into account both data writing performance and data reading performance.

[0128] For ease of understanding, assume a three-node cluster, with 10 disks on each node (that is, 10 osds), and a total of 30 disks in the cluster. Assume the cluster's storage pool has redundancy information of ec4+2, with 32 pgs on it, and each pg is mapped to 6 disks (osds). The 32 pgs will be mapped to the 30 disks in the cluster. Different pgs can be mapped to the same osd, and the same osd can carry multiple pgs.

[0129] The desired effect is as follows: 1. During the reading and writing process of the cluster, the usage capacity of the 30 disks (osd) can remain the same (for example, all disks use 10% or 20% of the capacity, etc.). The purpose of capacity balancing is to make full use of the disk space. If the space of a single disk is used slowly, the other osds may not be able to write anymore (due to EC4+2, data needs to be written to 6 disks at the same time). At the same time, if the space of a single disk is used too high, the performance of the high-water mark disk will often decrease, and the disk will become a bottleneck that blocks the performance of the cluster. Therefore, it is best for all disks to be balanced. 2. During the reading and writing process, the utilization of the disks is basically the same (if a disk has a utilization rate of 100%, and some disks have a utilization rate of 10%, the overall performance of the cluster cannot be brought into play, that is, the disk with a utilization rate of 100% is a hot disk. The best is that all disk utilization rates remain the same, and the performance of the cluster will be maximized). It should be noted that for EC4+2, when reading data, only 4 disks need to be read, while writing data requires (4+2) = 6 disks. Calculation method:

[0130] 1. Assume that among the 32 pgs, due to operations such as capacity expansion, 10 pgs are currently unavailable, then select from the remaining 22 pgs. Assuming that the current capacity of all disks is 40%, the weight is 60. For calculation purposes, 60 is changed to 3. The weight of each osd is 3, and the initial weight of each pg is 1 (this embodiment does not limit the initial weight of the placement group. For example, the initial weight of the placement group in this embodiment can be 1; or the initial weight of the placement group in this embodiment can also be 2, and the initial weight in this embodiment is 1). The above calculation is in the monitor (monitoring component), and the cluster status and usage information can be obtained in the monitor. Through this information in the monitor, the relevant information of the disk and pg can be obtained when the data block is written. Use negative feedback information for writing data blocks. The relevant calculation for selecting pg by data blocks is:

[0131] Assume that the osds mapped to pg2 are 28, 14, 18, 6, 23, and 2. The write weight of each osd is 3, and the read weight is 1 (the read weight needs to be the osd weight / 3. The weight of the osd corresponding to the read shard is divided by 3. It is essential to balance it, because the capacity gap within three is ignored after division by 3, resulting in a large number of pgs with the same total_read. Then, the osd with high weight (low capacity) is selected through total_write to achieve capacity balance). The total data column write weight is 3x30 = 90, and the total data column read weight is 1x30=30. Iterate the calculation once: the calculation of the data column write weight is 3x6 + 3 = 21. Since osds 28, 14, 18, 6, 23, and 2 are selected, their weights need to be subtracted by 90, that is, -69. The calculation of data column read weight is 1x4+1=5. Since 28, 14, 18, and 6 are the data columns selected by read, their weights need to be subtracted by 30, that is, -25. After iteration: Data column write weight: osd0 weight 21, osd1 weight 21, osd2 weight -69.......osd29 weight 21 (the weights of the 6 osds 28, 14, 18, 6, 2, 2 are -69, and the rest are 21); Data column read weight: osd0 weight 5, osd1 weight 5, osd2 weight 5.......osd29 weight 5 (the weights of the 4 osds 28, 14, 18, and 6 are -25, and the rest are 5). In this way, the data column write weight and data column read weight corresponding to the selected pg will become smaller, and the probability of selecting this pg and the osd corresponding to the pg will become very small next time, thereby achieving smooth polling.

[0132] The embodiment of the present invention takes into account the capacity of the osd, the status of the pg, the status of the osd, etc., and can avoid faulty osds, etc., thereby enhancing the reliability of data and the performance of the cluster. By polling according to weights, the capacity of the cluster disk can be balanced, and at the same time, the utilization rate of the disk can be balanced, taking into account the improvement of the cluster write performance and read performance, and the performance of the cluster can be smooth, reducing the fluctuation caused by uneven data distribution.

[0133] Please refer to Figure 4 , Figure 4 A flow chart of a method for determining a target placement group provided by an embodiment of the present invention. Applied to a client of a distributed storage system, the method may include:

[0134] S401, receiving a weight of each placement group; wherein the weight of each placement group is a weight of the placement group determined based on the above data storage method.

[0135] The execution subject of this embodiment is the client.

[0136] S402 : In response to a write operation of a data block, determine a target placement group based on a weight of each placement group.

[0137] This embodiment does not limit the specific process of determining the target placement group based on the weight of each placement group. For example, this embodiment can determine the total weight according to the placement group weight, the data column read weight and the data column write weight corresponding to each placement group, and use the placement group with the highest total weight as the target placement group; or this embodiment can also use the data column read weight as the first priority parameter, the data column write weight as the second priority parameter, and the placement group weight as the third priority parameter. When the data column read weights are different, the placement group with the largest data column read weight is used as the target placement group; when the data column read weights are the same, the second priority parameter is given priority, and the placement group with the largest data column write weight is used as the target placement group; when the data column read weight and the write weight are the same, the third priority weight is considered, and the placement group with the largest placement group weight is used as the target placement group (when the first selection is made, the initial weights of all placement groups are the same. When it is not the first selection, the weights of all placement groups are the weights after the last update. At this time, the weights of the placement groups are different, so selection can be made); or this embodiment can also use the data column write weight as the first priority parameter, the data column read weight as the second priority parameter, and the placement group weight as the third priority parameter.

[0138] It should be further explained that, in response to the write operation of the data block, determining the target placement group based on the weight of each placement group may include:

[0139] S4021, in response to the write operation of the data block, the placement group with the largest data column read weight is used as the target placement group.

[0140] S4022: If the data column read weights of all placement groups are the same, the placement group with the largest data column write weight is used as the target placement group.

[0141] S4023: If the data column write weights of all placement groups are the same, the placement group with the largest placement group weight is used as the target placement group.

[0142] S403: Write the data block into a target placement group of the distributed storage system.

[0143] This embodiment first compares the data column read weights. If the data column read weights of all placement groups are the same, the data column write weights are compared. If the data column write weights are still the same, the placement group weights are compared. The specific execution steps may include: determining whether the data column read weights corresponding to each placement group are the same; when the data column read weights corresponding to each placement group are different, the placement group with the largest data column read weight is used as the target placement group; when the data column read weights corresponding to each placement group are the same, determining whether the data column write weights corresponding to each placement group are the same; when the data column write weights corresponding to each placement group are different, the placement group with the largest data column write weight is used as the target placement group; when the data column write weights corresponding to each placement group are the same, the placement group with the largest placement group weight is used as the target placement group. In this embodiment, considering that read operations are usually more frequent than write operations and have higher performance requirements, the priority of read weight is higher than that of write weight. According to the performance requirements, the embodiment of the present invention improves the accuracy of selecting the target placement group by taking the data column read weight as the first priority, the data column write weight as the second priority, and the placement group weight as the third priority.

[0144] The target placement group determination method provided by an embodiment of the present invention may include: S401, receiving the weight of each placement group; wherein the weight of each placement group is the weight of the placement group determined based on the above-mentioned data storage method. S402, in response to the write operation of the data block, determining the target placement group based on the weight of each placement group. S403, writing the data block to the target placement group of the distributed storage system. Compared with the current inability to poll, the embodiment of the present invention obtains the weight of each placement group, so that the target placement group can be determined based on the weight of each placement group, thereby realizing polling of the placement groups according to the weights, thereby maintaining the capacity balance of the cluster disks.

[0145] The data storage device provided in the embodiment of the present invention is introduced below. The data storage device described below and the data storage method described above can be referred to each other.

[0146] Please refer to Figure 5 , Figure 5 A structural diagram of a data storage device provided in an embodiment of the present invention may include:

[0147] A weight determination module 100 for each placement group, configured to determine the weight of each placement group according to a mapping relationship between the placement group and the object storage device and capacity usage information of the object storage device;

[0148] A weight sending module 200, configured to send the weight of each placement group to a client, so that the client determines a target placement group according to the weight of each placement group;

[0149] The placement group weight updating module 300 is used to update the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases; and return to execute the step of sending the weight of each placement group to the client and subsequent steps.

[0150] Further, based on the above embodiment, the weight of the placement group includes a data column read weight and a data column write weight; the weight determination module 100 of each placement group may include:

[0151] An initial weight determination unit for an object storage device, configured to determine an initial weight of the object storage device based on capacity usage information of the object storage device;

[0152] The data column read weight and data column write weight determination unit corresponding to the placement group is used to determine the data column read weight and data column write weight corresponding to each placement group according to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy strategy.

[0153] Further, based on the above embodiment, the initial weight determination module of the above object storage device may include:

[0154] The remaining capacity determination subunit of the object storage device is used to determine the remaining capacity of the object storage device according to the capacity usage information of the object storage device;

[0155] The initial weight determination subunit of the object storage device is used to determine the initial weight of the object storage device according to the remaining capacity of the object storage device; wherein the size of the remaining capacity is positively correlated with the initial weight of the object storage device.

[0156] Further, based on any of the above embodiments, the weight updating module 300 of the placement group may include:

[0157] The placement group weight increasing unit is used to increase the weights of all placement groups to obtain a first updated placement group weight, and to decrease the first updated placement group weight corresponding to the target placement group to obtain a second updated placement group weight corresponding to the target placement group.

[0158] Further, based on the above embodiment, the weight increasing unit of the above placement group may include:

[0159] A weight increasing subunit of all placement groups, configured to increase the weights of all placement groups based on a first preset parameter to obtain the first updated placement group weights;

[0160] The target placement group weight decreasing subunit is configured to decrease the first updated placement group weight corresponding to the target placement group based on a second preset parameter to obtain the second updated placement group weight corresponding to the target placement group.

[0161] Further, based on any of the above embodiments, the weight of the placement group includes a data column read weight and a data column write weight; the weight update module 300 of the placement group may include:

[0162] The data column weight increasing unit of the object storage device is used to increase the data column read weight and the data column write weight of the object storage devices corresponding to all placement groups to obtain the first data column read weight and the first data column write weight corresponding to each object storage device;

[0163] The data column weight updating unit corresponding to the unselected placement group is used to update the data column read weight and the data column write weight of each unselected placement group based on the first data column read weight and the first data column write weight corresponding to each object storage device;

[0164] A data column weight decreasing unit of a target placement group is used to decrease a first data column read weight and a first data column write weight of a target object storage device corresponding to the target placement group to obtain a second data column read weight and a second data column write weight corresponding to the target object storage device;

[0165] The data column updating unit corresponding to the target placement group is used to update the data column read weight and the data column write weight corresponding to the target placement group based on the second data column read weight and the second data column write weight corresponding to the target object storage device.

[0166] Further, based on the above embodiment, the data column weight increasing unit of the above object storage device may include:

[0167] A first data column read weight determination subunit, configured to perform a logical operation based on the current data column read weight, the number of replicas, and the number of data columns to be read corresponding to the object storage device, to determine the first data column read weight;

[0168] The first data column write weight determination subunit is used to perform a logical operation based on the current data column write weight corresponding to the object storage device and the number of data columns to be written, so as to determine the first data column write weight.

[0169] It should be noted that the order of the modules and units in the above data storage device can be changed without affecting the logic.

[0170] The data storage device provided by the embodiment of the present invention may include: a weight determination module 100 for each placement group, which is used to determine the weight of each placement group according to the mapping relationship between the placement group and the object storage device, and the capacity usage information of the object storage device; a weight sending module 200, which is used to send the weight of each placement group to the client, so that the client determines the target placement group according to the weight of each placement group; a placement group weight update module 300, which is used to update the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases; and return to execute the step of sending the weight of each placement group to the client and subsequent steps. Compared with the current use of hash operations that cannot ensure balanced data distribution, the present invention determines the weight of each placement group according to the mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device when performing data storage; so that the client can determine the target placement group for data storage based on the weight of each placement group, and after the client uses the target placement group for data storage, the weights of the target placement group and other placement groups are updated, so that the weight of the subsequent preferred target placement group is reduced, thereby maintaining the capacity balance of the cluster disk and the disk usage balance at the same time, taking into account the cluster write performance and read performance, and reducing the fluctuation caused by uneven data distribution.

[0171] The following is an introduction to a target placement group determination device provided by an embodiment of the present invention. The target placement group determination device described below and the target placement group determination method described above can be referenced to each other.

[0172] Please refer to Figure 6 , Figure 6 A schematic diagram of a target placement group determination device provided by an embodiment of the present invention may include:

[0173] A placement group weight determination module 400 is used to receive the weight of each placement group; wherein the weight of each placement group is the weight of the placement group determined based on the above data storage method;

[0174] A target placement group determination module 500, configured to determine a target placement group based on a weight of each placement group in response to a write operation of a data block;

[0175] The data block writing module 600 is configured to write the data block into the target placement group of the distributed storage system.

[0176] Further, based on the above embodiment, the target placement group determination module 500 may include:

[0177] A first target placement group determining unit, configured to respond to a write operation of a data block and use a placement group with a maximum data column read weight as the target placement group;

[0178] A second target placement group determination unit, configured to, if the data column read weights of all placement groups are the same, use the placement group with the largest data column write weight as the target placement group;

[0179] The third target placement group determining unit is configured to, if the data column write weights of all the placement groups are the same, use the placement group with the largest placement group weight as the target placement group.

[0180] It should be noted that the order of the modules and units in the above-mentioned target placement group determination device can be changed without affecting the logic.

[0181] The target placement group determination device provided by the embodiment of the present invention may include: a placement group weight determination module 400, which is used to receive the weight of each placement group; wherein the weight of each placement group is the weight of the placement group determined based on the above-mentioned data storage method; a target placement group determination module 500, which is used to determine the target placement group based on the weight of each placement group in response to the write operation of the data block; and a data block writing module 600, which is used to write the data block to the target placement group of the distributed storage system. Compared with the current inability to poll, the embodiment of the present invention obtains the weight of each placement group, so that the target placement group can be determined based on the weight of each placement group, thereby realizing polling of the placement group according to the weight, thereby maintaining the capacity balance of the cluster disk.

[0182] A distributed storage system provided by an embodiment of the present invention is introduced below. The distributed storage system described below and the data storage method and target placement group determination method described above can refer to each other.

[0183] Please refer to Figure 7 , Figure 7 A schematic diagram of a distributed storage system provided for an embodiment of the present invention may include:

[0184] A management module, used to connect to a client and multiple object storage devices, and execute the steps of the above data storage method to obtain the weight of each placement group;

[0185] The client is used to execute the steps of the above-mentioned target placement group determination method.

[0186] The management module in this embodiment may be a monitor (monitoring module), and relies on the monitor to implement related functions. This application does not impose any restrictions on the specific implementation of the management module. For ease of understanding, please refer to Figure 8 , Figure 8 A distribution provided in an embodiment of the present invention is a specific schematic diagram of a storage system, from Figure 8It can be seen that the client (software) belongs to the user side, and the management module (software) and object storage device (hardware) belong to the service side. Figure 8 The object storage device in the management module is used to report the information required by the management module to the management module. The management module is used to perform the steps of the above data storage method based on the information obtained from the object storage device, obtain the weight of each placement group, and send the weight of each placement group to the client.

[0187] It should be noted that Figure 7 and Figure 8 The inclusion of one client is merely exemplary. In a distributed storage system, multiple clients may be deployed on the user side, and the number of clients is determined according to user needs. The service side may respond to multiple clients simultaneously and provide distributed storage services.

[0188] An electronic device provided by an embodiment of the present invention is introduced below. The data storage device described below and the data storage method and target placement group determination method described above can refer to each other.

[0189] Please refer to Fig. 9 , Fig. 9 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention may include:

[0190] A memory 10, used for storing computer programs;

[0191] The processor 20 is configured to execute a computer program to implement the above-mentioned data storage method and target placement group determination method.

[0192] The memory 10 , the processor 20 , and the communication interface 30 all communicate with each other via a communication bus 40 .

[0193] In the embodiment of the present invention, the memory 10 is used to store one or more programs, the program may include program code, the program code includes computer operation instructions, and in the embodiment of the present invention, the memory 10 may store programs for implementing the following functions.

[0194] In a possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0195] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0196] The processor 20 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, a microprocessor or any conventional processor, etc. The processor 20 may call a program stored in the memory 10 .

[0197] The communication interface 30 may be an interface of a communication module, and is used to connect to other devices or systems.

[0198] Of course, it should be noted that Fig. 9 The structure shown does not constitute a limitation on the data storage device in the embodiment of the present invention. In actual applications, the data storage device may include Fig. 9 More or fewer components than shown, or combinations of certain components.

[0199] The computer-readable storage medium provided in an embodiment of the present invention is introduced below. The computer-readable storage medium described below and the data storage method described above can be referenced to each other.

[0200] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned data storage method are implemented.

[0201] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0202] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0203] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0204] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0205] The above is a detailed introduction to a data storage method, device, equipment and computer-readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A data storage method, characterized in that: A management module applied to a distributed storage system, the distributed storage system comprising a management module and a client, the management module and the client being connected, comprising: Determine the weight of each placement group according to the mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device; wherein any placement group is mapped to multiple object storage devices; Sending the weight of each placement group to the client, so that the client determines the target placement group according to the weight of each placement group; The weight of at least one placement group is updated so that the weight difference between the selected target placement group and the unselected placement groups increases; and the step of sending the weight of each placement group to the client and subsequent steps are returned to execute.

2. The data storage method according to claim 1, characterized in that: The weight of the placement group includes a data column read weight and a data column write weight; The step of determining the weight of each placement group according to the mapping relationship between the placement group and the object storage device and the capacity usage information of the object storage device includes: Determining an initial weight of the object storage device based on the capacity usage information of the object storage device; According to the mapping relationship between the placement group and the object storage device, the initial weight of the object storage device and the data redundancy strategy, the data column read weight and the data column write weight corresponding to each placement group are determined.

3. The data storage method according to claim 2, characterized in that: Determining an initial weight of the object storage device based on the capacity usage information of the object storage device includes: Determine the remaining capacity of the object storage device according to the capacity usage information of the object storage device; The initial weight of the object storage device is determined according to the remaining capacity of the object storage device; wherein the size of the remaining capacity is positively correlated with the initial weight of the object storage device.

4. The data storage method according to any one of claims 1 to 3, characterized in that: The updating of the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases includes: The weights of all placement groups are increased to obtain a first updated placement group weight, and the first updated placement group weight corresponding to the target placement group is decreased to obtain a second updated placement group weight corresponding to the target placement group.

5. The data storage method according to claim 4, characterized in that: The step of increasing the weights of all placement groups to obtain a first updated placement group weight, and decreasing the first updated placement group weight corresponding to the target placement group to obtain a second updated placement group weight corresponding to the target placement group, includes: Based on the first preset parameter, weights of all placement groups are increased to obtain the first updated placement group weights; Based on a second preset parameter, a first updated placement group weight corresponding to the target placement group is decreased to obtain the second updated placement group weight corresponding to the target placement group.

6. The data storage method according to claim 1, characterized in that: The weight of the placement group includes a data column read weight and a data column write weight; The updating of the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases includes: Performing an increase process on the data column read weights and data column write weights of the object storage devices corresponding to all placement groups to obtain a first data column read weight and a first data column write weight corresponding to each object storage device; Based on the first data column read weight and the first data column write weight corresponding to each object storage device, update the data column read weight and the data column write weight of each unselected placement group; Performing a decreasing process on a first data column read weight and a first data column write weight of a target object storage device corresponding to the target placement group to obtain a second data column read weight and a second data column write weight corresponding to the target object storage device; Based on the second data column read weight and the second data column write weight corresponding to the target object storage device, the data column read weight and the data column write weight corresponding to the target placement group are updated.

7. The data storage method according to claim 6, characterized in that: The data column read weights and data column write weights of the object storage devices corresponding to all placement groups are increased to obtain the first data column read weight and the first data column write weight corresponding to each object storage device, including: Performing a logical operation based on the current data column read weight, the number of replicas, and the number of data columns to be read corresponding to the object storage device to determine the first data column read weight; A logical operation is performed based on the current data column write weight corresponding to the object storage device and the number of data columns to be written to determine the first data column write weight.

8. A method for determining a target placement group, characterized in that: Clients used in distributed storage systems include: Receiving a weight of each placement group; wherein the weight of each placement group is a weight of the placement group determined based on the data storage method according to any one of claims 1 to 9; In response to a write operation of the data block, determining a target placement group based on a weight of each placement group; The data block is written to the target placement group of the distributed storage system.

9. The method for determining a target placement group according to claim 8, characterized in that: In response to the write operation of the data block, determining the target placement group based on the weight of each placement group includes: In response to a write operation of a data block, taking a placement group with a maximum read weight of a data column as the target placement group; If the data column read weights of all placement groups are the same, the placement group with the largest data column write weight is used as the target placement group; If the data column write weights of all placement groups are the same, the placement group with the largest placement group weight is used as the target placement group.

10. A distributed storage system, characterized in that: include: A management module, configured to connect to a client and to connect to a plurality of object storage devices, and to execute the steps of the data storage method according to any one of claims 1 to 7 to obtain a weight of each placement group; A client, used to execute the steps of the target placement group determination method described in claim 8 or claim 9.

11. A data storage device, characterized in that: A management module applied to a distributed storage system, the distributed storage system comprising a management module and a client, the management module and the client being connected, comprising: A weight determination module for each placement group, used to determine the weight of each placement group according to a mapping relationship between the placement group and the object storage device and capacity usage information of the object storage device; A weight sending module, used for sending the weight of each placement group to the client, so that the client determines the target placement group according to the weight of each placement group; The placement group weight updating module is used to update the weight of at least one placement group so that the weight difference between the selected target placement group and the unselected placement groups increases; and return to execute the step of sending the weight of each placement group to the client and subsequent steps.

12. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, used to execute the computer program to implement the steps of the data storage method according to any one of claims 1 to 7, or used to execute the computer program to implement the steps of the target placement group determination method according to claim 8 or claim 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the data storage method according to any one of claims 1 to 7, or, when executed by a processor, implements the steps of the target placement group determination method according to claim 8 or 9.

Images