Cluster construction method and device, equipment, medium and product

Abstract

The invention relates to the technical field of distributed storage, and discloses a cluster construction method and device, equipment, a medium and a product, and the method comprises the steps: calculating the OSD affinity weight of each node according to the performance parameter of each node in a target cluster when the target cluster is initialized; according to the number of the to-be-stored object data sets and the OSD affinity weight of each node, determining the unit weight of each node; and for any to-be-stored object data set, selecting a main OSD from the target node with the maximum OSD affinity weight in each node, and updating the affinity weight of the target node according to the unit weight of the target node. According to the scheme, factors such as computing power in the heterogeneous distributed storage cluster are comprehensively calculated, it is ensured that the nodes with high performance bear more main OSDs, and then the overall performance of the cluster is improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a cluster construction method, device, equipment, medium and product. Background Art

[0002] With the continuous development of informatization, the explosive growth of data volume has brought new challenges to storage systems. How to build a high-performance, scalable, and stable storage system to carry massive data storage.

[0003] The distributed storage system based on the Ceph open source architecture is currently a mainstream distributed storage system. In Ceph's distributed storage system, data striping becomes multiple objects, and the objects are mapped to PGs through a hash algorithm, and then mapped to OSDs by PGs. The mapping is determined by the CRUSH algorithm. The PG ID is used as the input of the algorithm, and a set of 3 (3 copies) OSDs is obtained. The first OSD in the set is used as the master OSD, and the others are used as slave OSDs in turn.

[0004] However, in the above solution, the computing power of chips with different architectures varies greatly, and the effect of determining the selection of the primary OSD according to the general OSD affinity algorithm is poor, which will lead to low performance utilization of the distributed storage cluster. Summary of the invention

[0005] In view of this, the present application provides a cluster construction method, device, equipment, medium and product to improve the performance of the cluster. The technical solution is as follows.

[0006] In a first aspect, a cluster construction method is provided, the method comprising:

[0007] When the target cluster is initialized, the OSD affinity weight of each node is calculated based on the performance parameters of each node in the target cluster;

[0008] Determine the unit weight of each node based on the number of object data sets to be stored and the OSD affinity weight of each node;

[0009] For any set of object data to be stored, a primary OSD is selected from the target node with the largest OSD affinity weight among the nodes, and the affinity weight of the target node is updated according to the unit weight of the target node.

[0010] In a possible implementation manner, updating the affinity weight of the target node according to the unit weight of the target node includes:

[0011] The difference between the affinity weight of the target node and the unit weight of the target node is updated as a new affinity weight of the target node.

[0012] In a possible implementation, determining the unit weight of each node according to the number of object data sets to be stored and the OSD affinity weight of each node includes:

[0013] According to the number of the object data sets to be stored and the number of storage nodes in the target cluster, obtaining the average number of primary OSDs carried by the storage nodes;

[0014] The ratio of the OSD affinity weight of each node to the average number of primary OSDs is determined as the unit weight of each node.

[0015] In a possible implementation, the performance parameters include processor computing power, actual memory, and network bandwidth;

[0016] The step of calculating the OSD affinity weight of each node according to the performance parameters of each node in the target cluster includes:

[0017] Obtaining the performance weight corresponding to each performance parameter of each node; the sum of the performance weights of each performance parameter is 1;

[0018] The OSD affinity weight of each node is obtained according to each performance parameter of each node and the corresponding performance weight.

[0019] In a possible implementation, the method further includes:

[0020] When a first node of the target cluster changes, determining each second node associated with the first node data and a first set of object data to be stored on the first node;

[0021] Obtaining a current affinity weight of each of the second nodes;

[0022] According to the current affinity weight of each of the second nodes, a primary OSD corresponding to the first set of object data to be stored is selected from each of the second nodes.

[0023] In a possible implementation manner, the obtaining the current affinity weight of each second node includes:

[0024] Obtaining the processor occupancy rate, memory occupancy rate and network bandwidth occupancy rate of each of the second nodes;

[0025] The current affinity weight of the second node is calculated according to the processor occupancy, the memory occupancy, the network bandwidth occupancy and the performance weights corresponding to the various performance parameters of the second nodes.

[0026] In a second aspect, a cluster construction device is provided, the device comprising:

[0027] The affinity weight calculation module is used to calculate the OSD affinity weight of each node according to the performance parameters of each node in the target cluster when the target cluster is initialized;

[0028] The unit weight acquisition module is used to determine the unit weight of each node according to the number of object data sets to be stored and the affinity weight of each node OSD;

[0029] The node selection module is used to select the main OSD from the target node with the largest OSD affinity weight among the nodes for any set of object data to be stored, and update the affinity weight of the target node according to the unit weight of the target node.

[0030] In a third aspect, a computer device is provided, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above-mentioned cluster construction method by executing the computer instructions.

[0031] In a fourth aspect, a computer-readable storage medium is provided, on which computer instructions are stored, and the computer instructions are used to enable a computer to execute the above-mentioned cluster construction method.

[0032] In a fifth aspect, a computer program product or a computer program is provided, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the above-mentioned cluster construction method.

[0033] The technical solution provided by this application may have the following beneficial effects:

[0034] When the target cluster is initialized, the OSD affinity weight of each node can be calculated according to the performance parameters of each node in the target cluster; then the unit weight of each node is determined according to the number of object data sets to be stored and the OSD affinity weight of each node; for any object data set to be stored, the master OSD is selected from the target node with the largest OSD affinity weight among all nodes, and the affinity weight of the target node is updated according to the unit weight of the target node. At this time, the next object data set to be stored will select the OSD according to the updated affinity weight of the target node and the affinity weights of other nodes. The above scheme realizes the coordinated calculation of factors such as computing power in heterogeneous distributed storage clusters, ensuring that nodes with strong performance carry more master OSDs, thereby improving the overall performance of the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A schematic diagram of the mapping relationship from PG to OSD of a heterogeneous distributed storage cluster involved in an embodiment of the present application is shown.

[0037] Figure 2 The present invention is a flowchart of a cluster construction method according to an exemplary embodiment.

[0038] Figure 3 The present invention is a flowchart of a cluster construction method according to an exemplary embodiment.

[0039] Figure 4 A schematic diagram of a heterogeneous distributed storage cluster change involved in an embodiment of the present application is shown.

[0040] Figure 5 It is a structural diagram of a cluster building device provided in an embodiment of the present application.

[0041] Figure 6 It is a structural schematic diagram of a computer device provided by an optional embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0043] In the description of the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between two items, or an association relationship between the two items, or a relationship between indication and being indicated, configuration and being configured, and the like.

[0044] In Ceph's distributed storage system, data is striped into multiple objects. Objects are mapped to PGs through a hash algorithm, and then mapped to OSDs by PGs. The mapping is determined by the CRUSH algorithm. The PG ID is used as the input of the algorithm, and a set of 3 (3 copies) OSDs is obtained. The first OSD in the set is used as the primary OSD, and the others are used as slave OSDs in turn. The primary OSD carries the main IO reads and writes of the PG and has the greatest impact on the overall performance of distributed storage. In actual production scenarios, the selection of the primary OSD needs to consider factors such as the affinity and fault domain of the OSD. The greater the affinity, the greater the possibility of being selected as the primary OSD. In heterogeneous computing power scenarios, determining the affinity of OSDs in Ceph distributed storage may be more challenging because different types of hardware and storage devices may differ in performance, capacity, and other aspects. In this case, you need to carefully consider the following factors to determine OSD affinity:

[0045] Hardware performance: Evaluate the hardware performance of different OSD nodes, including CPU, memory, storage speed, and network bandwidth. Based on the hardware specifications, you can create different CRUSH rules for different types of hardware devices.

[0046] CRUSH rules: Use CRUSH rules to define OSD affinity. Create different CRUSH rules for different types of hardware devices to ensure that data objects are mapped to OSDs that suit their needs.

[0047] Dynamic Adjustment: Monitors the performance and data access patterns of the cluster and dynamically adjusts OSD affinity as needed. This can help optimize performance and balance loads across hardware devices.

[0048] Please refer to Figure 1 , which shows a schematic diagram of the mapping relationship between PG and OSD of a heterogeneous distributed storage cluster involved in an embodiment of the present application. Figure 1As shown, the cluster includes several heterogeneous resources, that is, several nodes including Node1, Node2, Node3, Node4, etc. In distributed storage, data is striped and mapped to PG in the form of object, and the final data storage of PG needs to be mapped to OSD through the crush algorithm. Taking multi-copy storage as an example, a single PG will be mapped to N OSDs. The first OSD in the set will be used as the master OSD, and the other OSDs will be used as slave OSDs in turn. N is the number of copies, and N is generally 3 in the production environment. Among them, the master OSD is the node with the most IO processing in data reading and writing, and requires more resource consumption. Therefore, for distributed storage clusters with different computing power and configuration, it is necessary to use the computing power weight as an important factor in the calculation of OSD affinity during initialization, and then combine the fault domain factor to select PG to OSDs with different affinities. The greater the affinity, the greater the possibility of selecting it as the master OSD to improve the overall performance of the cluster.

[0049] Figure 2 FIG. 1 is a flow chart of a cluster construction method according to an exemplary embodiment. The method is executed by a computer device. Figure 2 As shown, the cluster construction method may include the following steps:

[0050] Step 201 : when the target cluster is initialized, the OSD affinity weight of each node is calculated according to the performance parameters of each node in the target cluster.

[0051] When initializing the target cluster, you need to consider the performance parameters of each node, including CPU, memory, storage speed, network bandwidth, etc. The hardware performance of each node directly affects its load capacity and efficiency in processing storage operations.

[0052] Perform a hardware performance evaluation on each OSD node to obtain the hardware performance indicators of each node (such as CPU performance, memory size, storage medium speed, network bandwidth, etc.). Then, assign an "affinity weight" to each node based on these performance indicators. The weight reflects the adaptability of the node when processing specific data. The larger the weight, the stronger the performance of the node, and the more suitable it is for storing and processing the data.

[0053] Step 202: Determine the unit weight of each node according to the number of object data sets to be stored and the OSD affinity weight of each node.

[0054] The unit weight of each node is determined based on the number of object data sets to be stored (that is, the number of PGs) and the OSD affinity weight of each node to optimize load balancing and ensure the efficiency of each node in processing storage requests.

[0055] Specifically, the number of object data sets to be stored represents the total amount of data to be stored. Typically, data sets may have data objects of different sizes and different storage requirements. Unit weights can be used to combine the affinity weight of each node with the number of data sets to be stored to derive the load capacity of a node in a specific situation. Nodes with higher unit weights can handle more storage requests and have stronger performance to handle these data.

[0056] Step 203 , for any set of object data to be stored, select a primary OSD from the target node with the largest OSD affinity weight among the nodes, and update the affinity weight of the target node according to the unit weight of the target node.

[0057] For each data set to be stored, an OSD is selected as the primary OSD. The primary OSD will bear most of the read and write load during data storage and access. At the same time, the affinity weight of the target node is updated to reflect the selection of the primary OSD and the change in weight. Specifically, for each object data set to be stored, the target node with the largest weight and the best performance is selected as the primary OSD according to the affinity weight of each node. The primary OSD will be responsible for the main read and write operations of the data set. Once the primary OSD is selected and the data set is assigned, the system dynamically adjusts the affinity weight of the node according to the unit weight of the target node. For example, if a node becomes the primary OSD and bears more load, its affinity weight may change (for example, decrease), which helps to distribute subsequent data to nodes with lower load, thereby maintaining the load balance of the storage cluster, avoiding overload of a single node, and maximizing the performance of the entire system.

[0058] To summarize, when the target cluster is initialized, the OSD affinity weight of each node can be calculated according to the performance parameters of each node in the target cluster; then, the unit weight of each node is determined according to the number of object data sets to be stored and the OSD affinity weight of each node; for any object data set to be stored, the master OSD is selected from the target node with the largest OSD affinity weight among all nodes, and the affinity weight of the target node is updated according to the unit weight of the target node. At this time, the next object data set to be stored will select the OSD according to the updated affinity weight of the target node and the affinity weights of other nodes. The above scheme realizes the coordinated calculation of factors such as computing power in heterogeneous distributed storage clusters, ensuring that nodes with strong performance carry more master OSDs, thereby improving the overall performance of the cluster.

[0059] Figure 3 The method flow chart of a cluster construction method according to an exemplary embodiment is shown. The method is executed by a computer device, and the cluster construction method may include the following steps:

[0060] Step 301: when the target cluster is initialized, the OSD affinity weight of each node in the target cluster is calculated according to the performance parameters of each node.

[0061] Optionally, the performance parameters include processor computing power Flops, actual memory Mem, and network bandwidth Net; the performance weights corresponding to the performance parameters of each node are obtained; the sum of the performance weights of the performance parameters is 1; according to the performance parameters of each node and the corresponding performance weights, the OSD affinity weight of each node is obtained.

[0062] The number of PGs in a common distributed storage cluster is M, which is generally an integer power of 2, such as 4096 PGs. When the corresponding cluster is initialized, there are M primary OSDs to choose from.

[0063] Assuming that the cluster has N distributed storage servers, considering that the resource consumption of the host node in distributed storage mainly includes three factors: CPU, memory, and network, the node OSD affinity weight value is set to:

[0064] W=Flops*W cpu +Mem*W mem +Net*W net

[0065] Among them, the computing power Flops is calculated according to different CPU computing power, architecture, threads, etc., the memory calculates the actual memory value, and the network mainly calculates the bandwidth. cpu 、Memory weight W mem , network weight W net Determine according to the actual scenario, W cpu / W mem / W net The sum of the three weight values ​​is 1.

[0066] In distributed storage, data striping, hash value calculation, and core Crush addressing algorithms consume a lot of CPU resources. The CPU computing power under different architectures varies greatly and has a great impact on the performance of distributed storage clusters. A unified computing power evaluation method is needed to evaluate the computing power performance of nodes, and design and divide OSD affinity based on this. This patent proposes a computing power evaluation method for CPU resources of different architectures. The computing power of the CPU is affected by factors such as CPU architecture, main frequency, number of cores, cache capacity and level, instruction set parallelism, hyperthreading, heat dissipation, and power consumption. For distributed storage scenarios, the focus is on architectural performance factors, main frequency, number of cores, and hyperthreading. The computing power evaluation function is as follows:

[0067] Flops=F*Core*Thread*Arch*HT

[0068] in:

[0069] Flops represents the final computing power result.

[0070] Architecture factor Arch: It is strongly related to the CPU architecture and the degree of optimization of the instruction set. This parameter is different for different architectures and different CPU manufacturers. It can be quantitatively evaluated based on the specific CPU model.

[0071] Thread: CPU thread.

[0072] Main frequency F: CPU core frequency, in Hertz (Hz).

[0073] Core number: The number of CPU cores, indicating the parallel processing capability of multiple cores.

[0074] Hyperthreading factor HT: If hyperthreading is supported, it is 2; if not, it is 1.

[0075] Step 302: According to the number of the object data set to be stored and the number of storage nodes in the target cluster, obtain the average number of primary OSDs carried by the storage nodes.

[0076] The number of object data sets to be stored indicates the amount of data that the system needs to store. The number of storage nodes refers to the total number of OSD nodes available in the entire cluster. The more nodes there are, the more evenly the load will be distributed, avoiding overload of individual nodes. The master OSD is the node responsible for the main read and write operations. In the cluster, each storage node will carry a certain number of master OSDs. By calculating the average number of master OSDs carried by each node, it can help determine the load situation of each node. If the number of master OSDs carried by some nodes is higher than the average, load balancing or redistribution may be required.

[0077] Step 303: Determine the ratio of the OSD affinity weight of each node to the average value of the number of primary OSDs as the unit weight of each node.

[0078] Specifically, the affinity weight values ​​of the nodes are calculated as W1W2W3…W N , usually for the balance of distributed storage data, the number of OSDs on each storage node is the same, because there are a total of M PGs that need to map M logical primary OSDs, and the average number of primary OSDs that need to be carried on each storage node is M / N, so the affinity weight of each primary OSD is W osd-pg It can be calculated as:

[0079] W osd-pg =W N / (M / N)=W N *N / M

[0080] Step 304 : for any set of object data to be stored, select a primary OSD from the target nodes with the largest OSD affinity weight among the nodes.

[0081] For each data set, the system will select the node with the largest OSD affinity weight as the primary OSD. The selection is based on the node's hardware performance and load balancing requirements. Therefore, this step can select the node with the best performance so that the cluster can handle data read and write operations more efficiently and improve the overall system performance.

[0082] Step 305: Update the difference between the affinity weight of the target node and the unit weight of the target node to a new affinity weight of the target node.

[0083] Once the primary OSD is selected, the system needs to update the affinity weight of the target node based on the primary OSD selection result. This is to enable dynamic adjustment the next time the primary OSD is selected to ensure load balance. The updated affinity weight should take into account the current node load and unit weight differences. For example, if a node takes on more primary OSDs, its affinity weight may need to be reduced accordingly to avoid it becoming a bottleneck for cluster performance.

[0084] Optionally, when the first node of the target cluster changes, determine each second node associated with the first node data and the first set of object data to be stored on the first node; obtain the current affinity weight of each second node; and select the main OSD corresponding to the first set of object data to be stored in each second node based on the current affinity weight of each second node.

[0085] Optionally, the processor occupancy, memory occupancy and network bandwidth occupancy of each of the second nodes are obtained; and the current affinity weight of the second node is calculated based on the processor occupancy, memory occupancy, network bandwidth occupancy and performance weights corresponding to each performance parameter of each second node.

[0086] When a cluster has a node addition or deletion, it will involve data synchronization of some PGs or new OSD master election, but most PGs in the cluster will not be affected, thus maintaining the stability of the cluster. When the cluster changes, the affinity of the OSD is determined based on the node real-time computing power evaluation factor, and then the master OSD of the affected PG is selected.

[0087] The computing power evaluation of the nodes affected by the cluster changes uses the real-time CPU utilization, memory utilization, and network bandwidth occupancy to calculate the current affinity weight value. The formula is as follows:

[0088] W = Flops*(1-Cpu%)*W cpu +Mem*(1-Mem%)*W mem +Net*(1-Net%)*W net

[0089] The focus is on calculating the usage percentage of CPU, memory, and network, and then comparing the affinity weight values ​​of the remaining computing power and the newly added nodes.

[0090] The affinity algorithm for re-selecting the primary OSD for PGs affected by cluster changes is consistent with the above method. Dynamic weight value comparison is performed to determine the OSD with the largest weight affinity as the primary OSD to ensure the maximum performance of the cluster. After all affected PGs have completed the primary election and data migration or synchronization, the cluster will be stable again and provide the highest cluster performance.

[0091] Please refer to Figure 4 , which shows a schematic diagram of a heterogeneous distributed storage cluster change involved in an embodiment of the present application. Figure 4 As shown, when the cluster is expanded, some PGs involved in the expansion process are determined. For any of the PGs, the expanded new node NodeX is taken into account, and the master OSD of each PG is re-determined according to the affinity weight value.

[0092] To summarize, when the target cluster is initialized, the OSD affinity weight of each node can be calculated according to the performance parameters of each node in the target cluster; then, the unit weight of each node is determined according to the number of object data sets to be stored and the OSD affinity weight of each node; for any object data set to be stored, the master OSD is selected from the target node with the largest OSD affinity weight among all nodes, and the affinity weight of the target node is updated according to the unit weight of the target node. At this time, the next object data set to be stored will select the OSD according to the updated affinity weight of the target node and the affinity weights of other nodes. The above scheme realizes the coordinated calculation of factors such as computing power in heterogeneous distributed storage clusters, ensuring that nodes with strong performance carry more master OSDs, thereby improving the overall performance of the cluster.

[0093] In the embodiments of the present application, a cluster construction device is also provided, which is used to implement the above embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0094] The present application embodiment provides a cluster construction device, Figure 5: is a schematic diagram of a cluster building device provided in an embodiment of the present application, the device comprising:

[0095] Affinity weight calculation module 501, used to calculate the OSD affinity weight of each node according to the performance parameters of each node in the target cluster when the target cluster is initialized;

[0096] The unit weight acquisition module 502 is used to determine the unit weight of each node according to the number of object data sets to be stored and the affinity weight of each node OSD;

[0097] The node selection module 503 is used to select the primary OSD from the target node with the largest OSD affinity weight among the nodes for any set of object data to be stored, and update the affinity weight of the target node according to the unit weight of the target node.

[0098] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0099] The cluster building device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0100] The embodiment of the present invention also provides a computer device having the above Figure 5 The cluster building device shown.

[0101] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphic information in a graphical user interface on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6A processor 10 is taken as an example.

[0102] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0103] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0104] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0106] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0107] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0108] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0109] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the present invention.

Claims

1. A cluster construction method, characterized in that: The method comprises: When the target cluster is initialized, the OSD affinity weight of each node is calculated based on the performance parameters of each node in the target cluster; Determine the unit weight of each node based on the number of object data sets to be stored and the OSD affinity weight of each node; For any set of object data to be stored, a primary OSD is selected from the target node with the largest OSD affinity weight among the nodes, and the affinity weight of the target node is updated according to the unit weight of the target node.

2. The method according to claim 1, characterized in that: The updating the affinity weight of the target node according to the unit weight of the target node includes: The difference between the affinity weight of the target node and the unit weight of the target node is updated as a new affinity weight of the target node.

3. The method according to claim 2, characterized in that Determining the unit weight of each node according to the number of object data sets to be stored and the OSD affinity weight of each node includes: According to the number of the object data sets to be stored and the number of storage nodes in the target cluster, obtaining the average number of primary OSDs carried by the storage nodes; The ratio of the OSD affinity weight of each node to the average number of primary OSDs is determined as the unit weight of each node.

4. The method according to any one of claims 1 to 3, characterized in that: The performance parameters include processor computing power, actual memory and network bandwidth; The step of calculating the OSD affinity weight of each node according to the performance parameters of each node in the target cluster includes: Obtaining the performance weight corresponding to each performance parameter of each node; the sum of the performance weights of each performance parameter is 1; The OSD affinity weight of each node is obtained according to each performance parameter of each node and the corresponding performance weight.

5. The method according to claim 4, characterized in that The method further comprises: When a first node of the target cluster changes, determining each second node associated with the first node data and a first set of object data to be stored on the first node; Obtaining a current affinity weight of each of the second nodes; According to the current affinity weight of each of the second nodes, a primary OSD corresponding to the first set of object data to be stored is selected from each of the second nodes.

6. The method according to claim 5, characterized in that The obtaining the current affinity weight of each second node includes: Obtaining the processor occupancy rate, memory occupancy rate and network bandwidth occupancy rate of each of the second nodes; The current affinity weight of the second node is calculated according to the processor occupancy, the memory occupancy, the network bandwidth occupancy and the performance weights corresponding to the various performance parameters of the second nodes.

7. A cluster building device, characterized in that: The device comprises: The affinity weight calculation module is used to calculate the OSD affinity weight of each node according to the performance parameters of each node in the target cluster when the target cluster is initialized; The unit weight acquisition module is used to determine the unit weight of each node according to the number of object data sets to be stored and the affinity weight of each node OSD; The node selection module is used to select the main OSD from the target node with the largest OSD affinity weight among the nodes for any set of object data to be stored, and update the affinity weight of the target node according to the unit weight of the target node.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the cluster construction method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the cluster construction method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the cluster building method according to any one of claims 1 to 6.

Images