Data distributed deployment method and apparatus, computing device, and computer storage medium
Patent Information
- Application Number
- CN202211333356.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-28
- Publication Date
- 2026-10-30
- Estimated Expiration
- 2042-10-28
AI Technical Summary
[0004]随着云计算的发展,算力时代对云存储有着更加泛在的需求,存储节点的部署在地域上更加广泛,面临的部署环境更为复杂,存储服务器在空间、温度、承重、电源系统等很多方面面临着更加严峻的考验,系统的运维管理也更具难度
[0021]根据本发明上述实施例提供的方案,实现了数据在机架层面的分布式冗余部署,存储集群内任何一个机架掉电,都不会造成数据丢失;存储数据会均匀分布在不同机架的不同服务器上,各服务器利用率、负荷均匀,资源均得到充分利用。
Smart Images

Figure CN116954992B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and specifically to a data distributed deployment method, apparatus, computing device, and computer storage medium. Background Technology
[0002] In existing technologies, distributed cloud storage mostly uses storage servers to form storage clusters, with each server serving as a storage node. Each storage node has the same number and capacity of hard drives as storage media. To ensure data security, a storage redundancy mechanism of "triple replication" is used. That is, the same data is duplicated three times on three different storage nodes to ensure that data is not lost in the event of a storage node failure.
[0003] like Figure 1 As shown, based on the power consumption and rack capacity of the storage servers, each rack contains several storage servers. Data is distributed across these storage servers in a three-replica manner. When writing data, it is written to all three storage nodes simultaneously, depending on their availability. During normal data retrieval, only one replica is needed. If one compute node fails, the other two nodes can still provide the same data. Even if up to two nodes fail, service will not be interrupted, as there is still one node available to support the system.
[0004] With the development of cloud computing, the computing power era has a more ubiquitous demand for cloud storage. Storage nodes are deployed in more geographical areas and face more complex deployment environments. Storage servers face more severe challenges in many aspects such as space, temperature, load-bearing capacity, and power systems, and the operation and maintenance management of the system is also more difficult.
[0005] Existing distributed cloud storage server racking only considers server power consumption and rack capacity. Data deployment also selects available storage nodes based on the predetermined node distribution after racking, lacking consideration for the entire lifecycle of distributed cloud storage, and has the following problems:
[0006] (1) High availability of data redundancy distribution at the rack level was not considered. Although the data has three-replica redundancy, all three replicas may be located in the same rack, such as... Figure 1 Data D, with its three replicas residing on storage nodes within a single rack, will lose all three replicas of data D if that rack fails, such as due to a power outage.
[0007] (2) The number of storage nodes installed in each rack may be inconsistent. If you want the data to be distributed across multiple storage nodes, the storage resources cannot be fully utilized due to the "bucket effect". Summary of the Invention
[0008] In view of the above problems, embodiments of the present invention are proposed to provide a data distributed deployment method, apparatus, computing device, and computer storage medium that overcomes or at least partially solves the above problems.
[0009] According to one aspect of the present invention, a data distributed deployment method is provided, comprising:
[0010] Obtain the number of servers to be configured, and determine the server quantity model based on the number of servers and the preset number of data replicas;
[0011] The corresponding rack quantity model is determined based on the server quantity model. The rack quantity model and the maximum number of racks that can be installed are used to calculate the number of racks. The servers are then installed into the racks with the corresponding number of racks. The number of racks with the same number of servers after installation is greater than or equal to the preset number of data replicas.
[0012] Select a rack with a preset number of data replicas from racks that have the same number of servers, and distribute the data to the servers in the selected rack.
[0013] According to another aspect of the present invention, a data distributed deployment apparatus is provided, comprising:
[0014] The module is suitable for obtaining the number of servers to be configured and determining the server quantity model based on the number of servers and the preset number of data replicas.
[0015] The computing module is suitable for determining the corresponding rack quantity model based on the server quantity model, and calculating the rack quantity based on the rack quantity model and the maximum number of machines that can be installed in the rack.
[0016] The installation module is suitable for installing servers into racks of a corresponding number of racks, wherein the number of racks with the same number of servers after installation is greater than or equal to the preset number of data copies.
[0017] The deployment module is suitable for selecting a rack with a preset number of data replicas from racks with the same number of servers, and distributing the data to the servers in the selected rack.
[0018] According to another aspect of the present invention, a computing device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;
[0019] The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described data distributed deployment method.
[0020] According to another aspect of the present invention, a computer storage medium is provided, wherein at least one executable instruction is stored in the storage medium, the executable instruction causing a processor to perform an operation corresponding to the data distributed deployment method described above.
[0021] According to the solution provided in the above embodiments of the present invention, distributed redundant deployment of data at the rack level is realized. If any rack in the storage cluster loses power, no data loss will occur. The stored data will be evenly distributed on different servers in different racks, and the utilization and load of each server will be uniform, so that resources are fully utilized.
[0022] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific implementation methods of the embodiments of the present invention are described below. Attached Figure Description
[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0024] Figure 1 This is a schematic diagram of distributed data storage in existing technologies;
[0025] Figure 2A A flowchart of a data distributed deployment method provided in an embodiment of the present invention is shown;
[0026] Figure 2B A diagram of distributed storage with three replicas. Figure 1 ;
[0027] Figure 2C Schematic diagram 2 for distributed storage of three-replica data;
[0028] Figure 3 A schematic diagram of the structure of the data distributed deployment device provided in an embodiment of the present invention is shown;
[0029] Figure 4 A schematic diagram of the structure of a computing device provided in an embodiment of the present invention is shown. Detailed Implementation
[0030] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.
[0031] Figure 2A A flowchart of a data distributed deployment method provided by an embodiment of the present invention is shown. Figure 2A As shown, the method includes the following steps:
[0032] Step S201: Obtain the number of servers to be configured, and determine the server quantity model based on the number of servers and the preset number of data replicas.
[0033] When deploying data in a distributed manner, the number of servers to be configured is determined based on business needs. For example, the business side may specify the required storage space for its data, and the number of servers to be configured is then determined based on the required storage space. Next, the server quantity model is determined based on the number of servers and the preset number of data replicas. The preset number of data replicas refers to the number of copies of a single set of data that need to be stored. Typically, three replicas are used to store data, but other numbers of replicas are also possible. The server quantity model is determined based on the preset number of data replicas and the server redundancy level.
[0034] Here, we take a three-replica server as an example to illustrate the point. The number of servers (x) is not necessarily a multiple of 3. Generally, there are three cases: multiples of 3, 1 more than a multiple of 3, and 2 more than a multiple of 3.
[0035] Considering the redundancy characteristics of distributed storage with three replicas and the utilization rate of storage resources, the number of servers is summarized into six mathematical models: 6n+2, 6n+1, 6n, 6n-1, 6n-2, 6n-3 (n=1,2,3,...).
[0036] That is, the server quantity model can be expressed as:
[0037] Step S202: Determine the corresponding rack quantity model based on the server quantity model, calculate the rack quantity based on the rack quantity model and the maximum number of racks that can be installed, and install the servers into the racks of the corresponding number of racks.
[0038] For each server quantity model, there is a corresponding rack quantity model. The rack quantity model is a model function for calculating the number of racks. The rack quantity model is related to one or more of the following: the number of servers, the number of racks installed, and the number of preset data replicas.
[0039] Considering the available capacity of a rack (physical space, power load), the number of storage nodes that can be installed in each rack is also limited. Let the maximum number of nodes that can be installed in a rack be k, and the number of nodes that can be installed in a single rack is: k i (i = 1, 2, 3, ...), 1 ≤ k i ≤k.
[0040] Continuing with the explanation using the three-replica model, the rack quantity model corresponding to the server quantity model can be expressed as:
[0041]
[0042] Where n = 1, 2, 3, ...; k i (i = 1, 2, 3, ...), 1 ≤ k i ≤k; [] indicates that the result is rounded to the nearest integer.
[0043] Therefore, the rack quantity model can be summarized in Table 1.
[0044] Table 1:
[0045]
[0046] The number of racks in Model 1 is any number greater than 3, but not an integer multiple of 3, and the number of servers in each rack is the same.
[0047] Model 2 has a number of racks that are multiples of 3, and at least 3 racks contain the same number of servers.
[0048] After determining the server quantity model, the corresponding rack quantity model is determined based on the server quantity model. For example, if the server quantity model is 6n+2, then the corresponding rack quantity model is [(6n+2) / k]. i After determining the rack quantity model, the servers are installed into racks of the corresponding number of racks according to the rack quantity model and the maximum number of racks that can be installed. The number of racks with the same number of servers in the installed racks is greater than or equal to the preset number of data replicas.
[0049] The maximum number of servers that can be installed in a rack is limited. When determining the number of racks, the maximum number of servers that can be installed in a rack is used as the upper limit. A suitable number of servers can be installed in a rack is selected. The number of servers that can be installed in a rack is less than or equal to the maximum number of servers that can be installed. The number of servers that can be installed in a rack is determined in combination with the number of servers, the number of preset data replicas, and from the perspective of economic cost.
[0050] In an optional embodiment of the present invention, the number of computer racks, based on the rack quantity model and the maximum number of computers that can be installed in a rack, further includes:
[0051] If the number of servers meets the first condition, the rack installation quantity for each rack is determined based on the maximum number of racks that can be installed. The determined rack installation quantity should be a value between 1 and the maximum number of racks that can be installed. In order to meet the requirements of distributed data deployment on racks and make full use of the characteristics of servers, the determined rack installation quantity should be divisible by the number of servers, and the result of the divisibility should be greater than the preset number of data replicas, but not an integer multiple of the preset number of data replicas.
[0052] The number of racks is calculated using a rack quantity model based on the number of servers and the number of racks installed.
[0053] For a three-replica server configuration, the first condition is that the number of servers is a multiple of 3 with a remainder of 2, for example, 32 servers. Alternatively, the number of servers can be one multiple of 3 less than 3, for example, 29 servers. Or, the number of servers can be two multiples of 3 less than 3, for example, 28 servers. These are just examples and have no limiting effect. The rack capacity is determined based on the maximum rack capacity. The rack capacity should be any number greater than 3, but not a multiple of 3, and the number of servers in each rack should be consistent. For example, with 32 servers and a maximum rack capacity of 6, the rack capacity can be set to 4, with 4 servers per rack. After determining the rack capacity, the total number of racks can be calculated using a rack capacity model based on the number of servers and racks. For a server count of 32, the rack capacity model is f(6n+2)=[(6n+2) / k i Substituting the number of servers and the number of racks installed into the model function, we get a rack count of 8.
[0054] In an optional embodiment of the present invention, the number of computer racks, based on the rack quantity model and the maximum number of computers that can be installed in a rack, further includes:
[0055] If the number of servers meets the second condition, the servers are divided into equal groups according to the preset number of data replicas to determine the number of servers in each group.
[0056] Based on the number of servers in each group and the maximum number of racks that can be installed, the rack installation quantity is combined and the number of combinations is determined. Among them, the rack installation quantity in the rack installation quantity combination is less than or equal to the maximum number of racks that can be installed.
[0057] The number of computer racks is calculated based on the number of combinations and the number of racks.
[0058] Specifically, the number of servers may not be an integer multiple of the preset number of data replicas. To enable equal division, when the number of servers is not an integer multiple of the preset number of data replicas, it is necessary to subtract a specific value from the number of servers so that the result is exactly an integer multiple of the preset number of data replicas. By performing equal division (the number of equal parts is the preset number of data replicas), the number of servers in each group can be determined. The number of servers in each group and the maximum installable number of servers per rack are used to perform intra-group division and combination of the number of servers installed in racks. The number after division and combination shall be equal to the number of servers in each group, and the number of servers installed in the racks selected during combination shall be less than or equal to the maximum installable number of servers per rack. The number of servers installed in the racks is preferentially considered from the perspective of economic cost to minimize the configuration of the number of racks. After the combination is determined, the number of combined divisions can be determined, and the number of racks is equal to the product of the number of combined divisions and the preset number of data replicas.
[0059] Taking three replicas as an example, first divide the servers into 3 equal parts DIV 1,2,3 , taking the server quantity model as 6n as an example, the number of servers in each equal part is: DIV 1,2,3 =6n / 3, where n=1,2,3,...
[0060] Then according to the maximum installable number of servers per rack k, divide DIV1,2,3 into:
[0061] DIV 1,2,3 =k1+k2+...+k i , where k i <k,i=1,2,3,... that is, the sum of k1+k2+...+k i is equal to the number of servers in each equal part, so that the number of combinations can be determined, and then the number of racks is equal to i*3
[0062] If the server quantity model is 6n+1, the following formula can be used to calculate the number of servers in each equal part: (6n+1-1) / 3, which will not be repeated here.
[0063] Step S203: Select racks of the preset number of data replicas from racks with the same number of servers, and store data in a distributed manner to the servers in the selected racks.
[0064] Specifically, if the number of racks corresponds to a first preset rack quantity model, racks of the preset number of data replicas are randomly selected from all racks, and data is stored in a distributed manner to the servers in the selected racks, wherein in the first preset model, the number of racks is any number greater than the preset number of data replicas and is not an integer multiple of the preset number of data replicas, and each rack has the same number of servers.
[0065] If the number of racks corresponds to the second preset rack number model, then a rack with a preset number of data replicas is selected from the racks with the same number of servers, and the data is distributed and stored in the servers of the selected racks. In the second preset model, the number of racks is an integer multiple of the preset number of data replicas, and the number of racks with the same number of servers is greater than or equal to the preset number of data replicas.
[0066] Continuing with the three-replica example, the first preset rack quantity model is Model 1. For Model 1, a global data distribution approach is adopted. From all racks, three racks with the same number of storage nodes are randomly selected as three logical racks. The total number of logical racks provided is: 3*C 3 f(x) Here, f(x) is the calculated number of racks.
[0067] Each time data is stored, 3 racks are randomly selected from f(x) racks, and the same number of servers are selected for each rack to form 3 logical racks. The data is stored in these 3 racks and is physically located within them. If any 1 or 2 racks lose power, the data will not be lost.
[0068] Figure 2B A diagram of distributed storage with three replicas. Figure 1 ,like Figure 2B As shown, with a total of 4 racks, the three replicas of data ABCDE are stored in 3 different racks, ensuring high availability of the three-replica data at the rack level. If any two racks lose power, no data will be lost.
[0069] Figure 2C The second diagram illustrates a distributed storage system with three replicas of data. Figure 2C As shown, assuming there are 30 storage servers, divided into 3 equal groups of 10 servers each, and each rack can hold a maximum of 6 servers, choosing a combination of 4 and 6, we can determine the total number of racks to be 6. This results in two triple-replica storage rack groups: 3 racks with 4 servers each, and 3 racks with 6 servers each. Data A and C are stored in the 3 racks with 4 servers each, and data B and D are stored in the 3 racks with 6 servers each.
[0070] Therefore, by combining three replicas, the above scheme (1) achieves true distributed storage of cloud resource pool data with three replicas. The three replicas of the data are not only distributed on three storage nodes, but also deployed on three racks respectively. If any two racks lose power, the data will not be lost, achieving rack-level redundancy and high availability of the three replica data, improving the reliability of the resource pool, and enhancing the environmental adaptability of ubiquitous resource deployment in the computing power era.
[0071] (2) Maximizing the utilization of storage resources. Triple-replica storage requires three replicas of data to occupy three storage spaces of the same size. Regardless of the number of storage nodes, they can be deployed on three physical or logical racks of the same size, with identical nodes in each rack. At most, one or two spare nodes will be temporarily idle, while the remaining storage nodes can be fully utilized, maximizing the computing power of resources and improving computing efficiency.
[0072] (3) Data security and resistance to loss are guaranteed from the source. From the very beginning of the installation of storage devices, the reasonable deployment of three copies of data is considered, and the high availability of distributed cloud storage throughout its entire lifecycle is taken into account, thus building a cloud resource pool environment with high data availability and high resource efficiency.
[0073] According to the solution provided in the above embodiments of the present invention, distributed redundant deployment of data at the rack level is achieved. A power outage in any rack within the storage cluster will not result in data loss. Stored data is evenly distributed across different servers in different racks, ensuring uniform server utilization and load, and full utilization of resources. The use of this method can improve the availability of cloud storage, thereby enhancing the resilience of cloud computing resource pools. In the era of computing power and ubiquitous data, deploying distributed storage using the method of this invention in the construction of various cloud computing resource pools can provide users with more reliable and available cloud computing products, ensuring the security and reliability of user data assets and improving user satisfaction.
[0074] Figure 3 A schematic diagram of the structure of a data distributed deployment device provided in an embodiment of the present invention is shown. Figure 3 As shown, the device includes:
[0075] The module 301 is adapted to obtain the number of servers to be configured and determine the server quantity model based on the number of servers and the preset number of data replicas.
[0076] The calculation module 302 is adapted to determine the corresponding rack quantity model based on the server quantity model, and to calculate the rack quantity based on the rack quantity model and the maximum number of machines that can be installed in the rack.
[0077] The installation module 303 is suitable for installing servers into racks of a corresponding number of racks, wherein the number of racks with the same number of servers after installation is greater than or equal to the preset number of data copies.
[0078] Deployment module 304 is adapted to select a rack with a preset number of data replicas from racks with the same number of servers, and to distribute the data to the servers in the selected rack.
[0079] Optionally, the deployment module is further adapted to: if the number of racks corresponds to a first preset rack number model, then randomly select a rack with a preset number of data replicas from all racks, and distribute the data to the servers in the selected racks, wherein the number of racks in the first preset model is any number greater than the preset number of data replicas, and is not an integer multiple of the preset number of data replicas, and each rack has the same number of servers.
[0080] Optionally, the deployment module is further adapted to: if the number of racks corresponds to a second preset rack number model, then select a rack with a preset number of data replicas from racks with the same number of servers, and distribute the data to the servers in the selected racks, wherein the number of racks in the second preset model is an integer multiple of the preset number of data replicas, and the number of racks with the same number of servers is greater than or equal to the preset number of data replicas.
[0081] Optionally, the computing module is further adapted to: if the number of servers meets the first condition, determine the rack capacity of each rack based on the maximum number of racks that can be installed;
[0082] The number of racks is calculated using a rack quantity model based on the number of servers and the number of racks installed.
[0083] Optionally, the calculation module is further adapted to: if the number of servers meets the second condition, then divide the servers equally according to the preset number of data replicas to determine the number of servers in each group;
[0084] Based on the number of servers in each group and the maximum number of racks that can be installed, the rack installation quantity is combined and the number of combinations is determined. Among them, the rack installation quantity in the rack installation quantity combination is less than or equal to the maximum number of racks that can be installed.
[0085] The number of computer racks is calculated based on the number of combinations and the number of racks.
[0086] Optionally, the server quantity model is determined based on the preset number of data replicas and the number of server redundancies.
[0087] According to the solution provided in the above embodiments of the present invention, distributed redundant deployment of data at the rack level is achieved. A power outage in any rack within the storage cluster will not result in data loss. Stored data is evenly distributed across different servers in different racks, ensuring uniform server utilization and load, and full utilization of resources. The use of this method can improve the availability of cloud storage, thereby enhancing the resilience of cloud computing resource pools. In the era of computing power and ubiquitous data, deploying distributed storage using the method of this invention in the construction of various cloud computing resource pools can provide users with more reliable and available cloud computing products, ensuring the security and reliability of user data assets and improving user satisfaction.
[0088] This invention provides a non-volatile computer storage medium storing at least one executable instruction that can execute the data distributed deployment method in any of the above method embodiments.
[0089] Figure 4 The diagram shows a structural schematic of a computing device provided in an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.
[0090] like Figure 4 As shown, the computing device may include a processor, a communications interface, memory, and a communications bus.
[0091] The processor, communication interface, and memory communicate with each other via a communication bus. The communication interface is used to communicate with other network elements, such as clients or other servers. The processor executes programs, specifically the steps described in the embodiment of the data distributed deployment method for computing devices.
[0092] Specifically, the program may include program code, which includes computer operation instructions.
[0093] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.
[0094] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.
[0095] Specifically, the program can be used to cause the processor to execute the data distributed deployment method in any of the above method embodiments. The specific implementation of each step in the program can be found in the corresponding descriptions of the steps and units in the above data distributed deployment embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.
[0096] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the embodiments of the present invention described herein can be implemented using various programming languages, and the above description of specific languages is for the purpose of disclosing the best mode of implementation of the embodiments of the present invention.
[0097] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.
[0098] Similarly, it should be understood that, in order to streamline the embodiments of the invention and aid in understanding one or more of the various inventive aspects, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.
[0099] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.
[0100] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.
[0101] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The embodiments of the present invention can also be implemented as device or apparatus programs (e.g., computer programs and computer program products) for performing part or all of the methods described herein. Such programs implementing the embodiments of the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.
[0102] It should be noted that the above embodiments are illustrative of the present invention and not restrictive of the invention, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of the present invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.
Claims
1. A method for distributed data deployment, comprising: Obtain the number of servers to be configured, and determine the server number model based on the number of servers and the preset number of data replicas, wherein the server number model is a model function for calculating the number of servers; The corresponding rack quantity model is determined based on the server quantity model. The number of racks is calculated based on the rack quantity model and the maximum number of racks that can be installed. Servers are then installed into racks of the corresponding number of racks. The number of racks with the same number of servers in the installed racks is greater than or equal to the preset number of data replicas. The rack quantity model is a model function of the number of racks. The rack quantity model is related to one or more of the number of servers, the number of racks installed, and the preset number of data replicas. Select a rack with a preset number of data replicas from racks that have the same number of servers, and distribute the data to the servers in the selected rack.
2. The method according to claim 1, wherein, The step of selecting a rack with a preset number of data replicas from racks having the same number of servers, and distributing the data to the servers in the selected rack, further includes: If the number of racks corresponds to the first preset rack number model, then a rack with a preset number of data replicas is randomly selected from all racks, and the data is distributed and stored in the server of the selected rack. In the first preset rack number model, the number of racks is any number greater than the preset number of data replicas, and is not an integer multiple of the preset number of data replicas. Each rack has the same number of servers.
3. The method according to claim 1 or 2, wherein, Selecting a rack with a preset number of data replicas from racks having the same number of servers, and distributing the data to the servers in the selected racks, further includes: If the number of racks corresponds to the second preset rack number model, then a rack with a preset number of data replicas is selected from the racks with the same number of servers, and the data is distributed and stored in the servers of the selected racks. In the second preset rack number model, the number of racks is an integer multiple of the preset number of data replicas, and the number of racks with the same number of servers is greater than or equal to the preset number of data replicas.
4. The method according to claim 1 or 2, wherein, The number of computer racks based on the rack quantity model and the maximum number of computers that can be installed in a rack further includes: If the number of servers meets the first condition, the rack installation quantity for each rack is determined according to the maximum number of racks that can be installed. The first condition is that the number of servers is a multiple of 3 with a remainder of 2, or the number of servers is a multiple of 3 that is one less than 3, or the number of servers is a multiple of 3 that is two less than 3. The determined rack installation quantity satisfies that it is divisible by the number of servers, and the result of the divisor is greater than the preset number of data replicas, but is not a multiple of the preset number of data replicas. The number of racks is calculated using the rack quantity model based on the number of servers and the number of racks installed.
5. The method according to claim 1 or 2, wherein, The number of computer racks based on the rack quantity model and the maximum number of computers that can be installed in a rack further includes: If the number of servers meets the second condition, the servers are divided into equal groups according to the preset number of data replicas to determine the number of servers in each group; the second condition is: the number of servers is an integer multiple of the preset number of data replicas, or the number of servers is not an integer multiple of the preset number of data replicas but after subtracting a specific value, it is an integer multiple of the preset number of data replicas; Based on the number of servers in each group and the maximum number of racks that can be installed, the rack installation quantity is combined and the number of combinations is determined. Among them, the rack installation quantity in the rack installation quantity combination is less than or equal to the maximum number of racks that can be installed. The number of computer racks is calculated based on the number of combinations and the number of racks.
6. The method according to claim 1 or 2, wherein, The server quantity model is determined based on the preset number of data replicas and the number of server redundancies.
7. A data distributed deployment device, comprising: The determination module is adapted to obtain the number of servers to be configured, and determine the server number model based on the number of servers and the preset number of data replicas, wherein the server number model is a model function for calculating the number of servers; The calculation module is adapted to determine the corresponding rack quantity model based on the server quantity model, and to calculate the rack quantity based on the rack quantity model and the maximum number of racks that can be installed, wherein the rack quantity model is a model function of the number of racks, and the rack quantity model is related to one or more of the number of servers, the number of racks installed, and the number of preset data replicas; The installation module is suitable for installing servers into racks of a corresponding number of racks, wherein the number of racks with the same number of servers after installation is greater than or equal to the preset number of data copies. The deployment module is suitable for selecting a rack with a preset number of data replicas from racks with the same number of servers, and distributing the data to the servers in the selected rack.
8. The apparatus according to claim 7, wherein, The deployment module is further adapted to: if the number of racks corresponds to a first preset rack number model, then randomly select a rack with a preset number of data replicas from all racks, and distribute the data to the servers in the selected racks, wherein the number of racks in the first preset rack number model is any number greater than the preset number of data replicas, and is not an integer multiple of the preset number of data replicas, and each rack has the same number of servers.
9. A computing device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the data distributed deployment method as described in any one of claims 1-6.
10. A computer storage medium storing at least one executable instruction that causes a processor to perform an operation corresponding to the data distributed deployment method as described in any one of claims 1-6.
Citation Information
Patent Citations
Data storing method and device
CN103077197A
Distribution-based task scheduling method and system
CN105335219A
