Server cluster deployment optimization method and apparatus, and electronic device
By building a hybrid integer quadratic constraint model and branch delimiting method, the problem of poor node deployment in large-scale server clusters is solved, and efficient data processing and performance improvement is achieved.
Patent Information
- Application Number
- CN202310658553.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-05
- Publication Date
- 2024-12-06
AI Technical Summary
In large-scale server clusters, how to perform reasonable node deployment optimization to reduce the cost pressure of data exchange and improve computing performance, existing methods such as greedy algorithms and genetic algorithms are difficult to obtain global optimal solutions, resulting in poor deployment results.
By building a hybrid integer quadratic constraint model, combining data traversal traffic and storage volume upper and lower limits, server cluster deployment is optimized, branch bounding method is used to solve to obtain the global optimal solution, and business nodes are migrated based on the optimization results to realize the server cluster Reasonable deployment.
It improves the data processing speed and system performance of the server cluster, reduces the total bandwidth, improves the accuracy and stability of deployment, and solves the two-way problem of data shuttle.
Smart Images

Figure CN119094345A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to an optimization method, device and electronic device for server cluster deployment. Background Art
[0002] As a type of computer, servers generally have the ability to respond to service requests, provide services, and guarantee services. Servers have high-speed CPU (Central Processing Unit) computing power, powerful I / O (Input / Output) external data throughput capacity, and better scalability. Therefore, compared with ordinary computers, servers run faster, have higher loads, and are more expensive. Although the computing power of a single server is limited, after hundreds or thousands of servers are formed into a server cluster, the entire system can have powerful computing power. With the increase in data processing volume, server clusters came into being. Server clusters are a solution to improve the overall computing power of servers. Multiple computers can be used for parallel computing to obtain higher computing speeds, and multiple computers can also be used for backup. Through a parallel or distributed system composed of a group of servers connected to each other, it generally refers to concentrating many servers to provide the same service.
[0003] There are multiple business nodes in the server cluster, and different business nodes may be deployed in different areas. Reasonable deployment of each node in the server cluster can ensure that the server cluster has good performance and data transmission efficiency. For such a large-scale cluster deployment scenario in multiple locations, a large amount of cross-city access bandwidth and business access data may be generated between business nodes. When the business scenario changes, such as: business volume changes or new business nodes are added, it may be necessary to readjust each node in the originally deployed server cluster according to the needs of the business scenario to reduce the cost pressure of data exchange and improve the performance of the server cluster.
[0004] How to reasonably deploy and optimize each node in a server cluster is a technical problem that urgently needs to be solved in this field. Summary of the invention
[0005] The embodiments of this specification provide a method, apparatus, device, storage medium, and computer program product for optimizing server cluster deployment, which can achieve reasonable deployment of each node in the server cluster and improve the data processing efficiency and performance of the server cluster.
[0006] On the one hand, an embodiment of this specification provides a method for optimizing server cluster deployment, the method comprising:
[0007] According to the directed graph of the service nodes corresponding to the target area, traffic data of the target area is counted, the target area includes at least two sub-areas, and the traffic data includes: data traversal traffic between different service nodes, data access traffic of different sub-areas, and at least one of upper and lower limits of storage corresponding to different sub-areas;
[0008] The minimum value of the upper limit of the total amount of data traversal between the sub-areas within a specified time is used as the objective function, and the constraint condition of the objective function is set according to at least one of the data traversal flow between the different service nodes, the data access flow of the different sub-areas, and the upper and lower limits of the storage volume corresponding to the different sub-areas, and the first server cluster deployment optimization model is constructed by the objective function and the constraint condition;
[0009] Solving the first server cluster deployment optimization model to obtain a server cluster deployment optimization result, wherein the server cluster deployment optimization result includes a target sub-region corresponding to each service node;
[0010] Based on the server cluster deployment optimization result, the service nodes in the target area are migrated to the servers in the target sub-area corresponding to the service nodes.
[0011] On the other hand, a device for optimizing server cluster deployment is provided, the device comprising:
[0012] A data collection module, used to collect statistics of traffic data of the target area according to a directed graph of service nodes corresponding to the target area, wherein the target area includes at least two sub-areas, and the traffic data includes at least one of: data traversal traffic between different service nodes, data access traffic of different sub-areas, and upper and lower limits of storage corresponding to different sub-areas;
[0013] an optimization model building module, used to take the minimum value of the upper limit of the total amount of data traversal between different sub-areas within a specified time as an objective function, set a constraint condition of the objective function according to at least one of the data traversal flow between the different business nodes, the data access flow of the different areas, and the upper and lower limits of the storage volume corresponding to the different sub-areas, and build a first server cluster deployment optimization model based on the objective function and the constraint condition;
[0014] A model solving module, used to solve the first server cluster deployment optimization model to obtain a server cluster deployment optimization result, wherein the server cluster deployment optimization result includes a target sub-region corresponding to each business node;
[0015] A node migration module is used to migrate the service nodes in the target area to the servers in the target sub-area corresponding to the service nodes based on the server cluster deployment optimization result.
[0016] Another aspect provides an electronic device, including: a processor;
[0017] a memory for storing instructions executable by the processor;
[0018] The processor is configured to execute the instructions to implement any of the above-mentioned methods for optimizing server cluster deployment.
[0019] On the other hand, a computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned optimization methods for server cluster deployment.
[0020] On the other hand, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the optimization method for server cluster deployment provided in the above various optional implementations.
[0021] The server cluster deployment optimization method, device, equipment, storage medium, and computer program product provided in the embodiments of this specification have the following technical effects:
[0022] The server cluster deployment optimization method provided in the embodiment of this specification collects the data traversal traffic, data access traffic, and upper and lower limits of storage between each business node in the target area, introduces the upper limit value of the total amount of data traversal between each sub-area within a specified time as an auxiliary variable, and constructs a first server cluster deployment optimization model. The first server cluster deployment optimization model fully considers the two-way data interaction between business nodes, improves the accuracy of server cluster deployment optimization, and solves the two-way problem of data shuttling. By solving the first server cluster deployment optimization model, the server cluster deployment optimization result in the target area can be obtained, thereby performing migration optimization on the business nodes in the target area. The optimization of server cluster deployment in the target area is achieved, making the deployment of business nodes more reasonable and improving the system performance of the server cluster. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions and advantages in the embodiments of this specification or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0024] Figure 1 It is a schematic diagram of an application environment of an optimization method for server cluster deployment provided in an embodiment of this specification;
[0025] Figure 2 It is a flow chart of a method for optimizing server cluster deployment provided by an embodiment of this specification;
[0026] Figure 3 It is a schematic diagram of the structure of a directed graph of service nodes in one embodiment of this specification;
[0027] Figure 4 It is a schematic diagram of the principle of server cluster deployment optimization in another embodiment of this specification;
[0028] Figure 5 It is a structural diagram of an optimization device for server cluster deployment provided in an embodiment of this specification;
[0029] Figure 6 is a block diagram of an optimized electronic device for server cluster deployment provided by an embodiment of this specification;
[0030] Figure 7 It is a block diagram of another optimized electronic device for server cluster deployment provided in an embodiment of this specification. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this specification.
[0032] It should be noted that the terms "first", "second", etc. in the description and claims of the embodiments of this specification and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of this specification described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.
[0033] See also Figure 1 , Figure 1It is a schematic diagram of an application environment of an optimization method for server cluster deployment provided in an embodiment of this specification, and the application environment may include at least a server 100 and a terminal 200.
[0034] In an optional embodiment, server 100 can be used to optimize the deployment of a server cluster. Server 100 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.
[0035] In an optional embodiment, the terminal 200 may be a client that sends a server cluster deployment request to the server 100. Specifically, the terminal 200 may include but is not limited to smart phones, desktop computers, tablet computers, laptop computers, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, vehicle terminals, smart TVs and other types of electronic devices; it may also be software running on the above electronic devices, such as applications, applets, etc. The operating system running on the electronic device in the embodiments of this specification may include but is not limited to Android system, IOS system, Linux, Windows, etc.
[0036] In addition, it should be noted that Figure 1 What is shown is merely an application environment of an optimization method for server cluster deployment, and the embodiments of this specification are not limited to the above.
[0037] In the embodiments of this specification, the server 100 and the terminal 200 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of this specification.
[0038] As the amount of data processing continues to increase, the requirements for computing power are also getting higher and higher. Generally, server clusters can be used to process data and improve the data processing speed. Server clusters refer to the collection of many servers to perform the same service together. Usually, many server clusters may be required to process services. The servers in the server cluster can correspond to multiple services, so each server cluster corresponds to multiple service nodes, and different service nodes may be deployed in different regions. For this large-scale cluster deployment scenario in multiple locations, a large amount of cross-city access bandwidth and service access data may be generated between service nodes. When there are business changes, such as: adding a new service node to process a new service or the business volume corresponding to different service nodes changes, it may be necessary to readjust the nodes in the originally deployed server cluster so that each node in the server cluster can be reasonably deployed to reduce the cost pressure of data exchange. In particular, with the rapid development of cloud computing technology in recent years, a new technology that can be used to provide multimedia services and applications has been born, multimedia cloud computing. Its new cloud-based computing model has greatly promoted the dissemination and sharing of media information among network users. This large-scale cross-regional access model will greatly promote the development of the information industry.
[0039] Generally, the deployment or optimization of nodes in a server cluster can be done through machine learning algorithms to obtain the optimal deployment plan. However, the algorithms generally used are greedy algorithms, such as genetic algorithms. The gap between the local optimal solution and the global optimal solution is too large, and the cost converted into bandwidth further magnifies this gap.
[0040] The embodiments of this specification provide an optimization method for server cluster deployment, which comprehensively considers the two-way data interaction between each node in the server cluster, and improves the accuracy of the creation of the server cluster optimization deployment model. In addition, the embodiments of this specification transform the original scheduling problem of multi-location deployment of the server cluster into an optimization mixed integer quadratic constraint problem, and solve the mixed integer quadratic constraint model to quickly obtain the optimized deployment plan of the server cluster, thereby improving the data processing speed of the server cluster deployment optimization. Based on the optimized deployment plan, each node in the server cluster is migrated and adjusted, effectively reducing the total bandwidth, so that the server cluster can have stronger stability and scalability, and improve the system performance of the server cluster.
[0041] The following describes an optimization method for server cluster deployment in an embodiment of this specification. Figure 2This is a flow chart of an optimization method for server cluster deployment provided by an embodiment of this specification. This specification provides method operation steps such as the embodiment or flow chart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). This method can be applied to terminal devices such as computers, tablet computers, and smart phones, and can also be applied to servers. Specifically, Figure 2 As shown, the method may include:
[0042] S202: Count the traffic data of the target area according to the directed graph of the business nodes corresponding to the target area, where the target area includes at least two sub-areas, and the traffic data includes: data traversal traffic between different business nodes, data access traffic between different sub-areas, and at least one of the upper and lower limits of storage capacity corresponding to different sub-areas.
[0043] In the specific implementation process, when the amount of business processed by each service node in the server cluster in the target area changes, or a new server cluster is deployed in the target area, or a new service node is deployed in the target area, at this time, the deployment of the service node of the original server cluster may not meet the business processing requirements, and it is necessary to optimize the deployment. When it is necessary to optimize the deployment of the service nodes in the server cluster of a region, the region can be used as the target area. The scope of the target area can be determined according to actual needs, and can be divided according to the location of the service node, or according to the service content corresponding to the service node, or according to other methods such as the service deployment area, which is not specifically limited in this embodiment of the specification. Generally, the scope of the target area in the embodiment of this specification is relatively large, and can include 2 or more sub-areas. The division method of the sub-area can be divided according to geographical location, such as: the East China region and the South China region are respectively used as two sub-areas, or a city or a county can be used as a sub-area. It can be divided according to actual needs, and the embodiment of this specification is not specifically limited. Generally, one or more server clusters can be deployed in the target area, and each server cluster can be used to process traffic data of one or more types of services.
[0044] Referring to the records of the above embodiments, it can be known that there can be multiple servers in a server cluster, each server can be used to process data related to one or more services, and a service node can be understood as a node with a server as a carrier, that is, one server can correspond to one or more service nodes. A directed graph of service nodes corresponding to service nodes in the target area can be created in advance based on the dependency relationship between service nodes corresponding to each server cluster in the target area, such as whether service A has scheduling and access to service B.
[0045] A business node directed graph may contain multiple business nodes, and corresponding directed edges may be drawn between each business node based on business transaction relationships, etc. A directed edge may indicate that there is cross-traffic between two business nodes. Figure 3 is a schematic diagram of the structure of a directed graph of service nodes in one embodiment of this specification, such as Figure 3 As shown, Figure 3 The dots in the graph represent business nodes, and the lines between the dots represent directed edges between the nodes. In addition, nodes belonging to different businesses can be marked with different colors in the directed graph, so that the locations of nodes processing the same business can be seen. Figure 3 It can be seen that there is complex data interaction between the various business nodes in the server cluster. The directed graph can clearly and intuitively show the direct business relationship between the business nodes, which facilitates data processing.
[0046] When optimizing the server cluster deployment in the target area, the traffic data of the target area can be counted based on the directed graph of the business nodes in the target area, such as: counting the data traversal traffic between different business nodes in the target area, the data access traffic of different sub-areas, and at least one of the upper and lower limits of the storage volume corresponding to different sub-areas. When counting the traffic data of the target area, the granularity of the statistics can be set according to actual needs, such as: counting the data traversal traffic between different business nodes every day, or counting the data traversal traffic between different business nodes every minute or every hour, which is not specifically limited in the embodiments of this specification.
[0047] In some embodiments of this specification, statistics on traffic data of the target area are collected according to the directed graph of service nodes corresponding to the target area, including:
[0048] According to the connection relationship between each node in the business node directed graph, the business relationship between each business node is obtained;
[0049] Based on the business relationship between each business node, count the data traversal traffic between different business nodes;
[0050] According to the service relationship between the service nodes in each sub-area in the target area and the data traversal flow between the service nodes, at least one of the data access flow of different sub-areas and the upper and lower limits of storage capacity corresponding to different sub-areas is counted.
[0051] In the specific implementation process, the data access flow in the sub-region can be calculated based on the sum of the data access flow of each service node in the sub-region. The upper and lower limits of the storage volume corresponding to each sub-region can be understood as: based on the number of service nodes in the sub-region, the business volume, etc., the estimated range of storage space, the range of storage space can be obtained based on the configuration parameters of the server cluster in the target region, etc., and the embodiments of this specification are not specifically limited. According to the directed graph of the service nodes, it can be intuitively seen whether each service node has a connection relationship, and based on the connection relationship, it can be determined whether the service relationship between each service node, that is, whether there is data interaction between each service node. Based on the service relationship between each service node, the data traversal flow between each service node can be counted, such as: counting the access data volume between service node A and service node B, that is, the data traversal flow between service node A and service node B can be obtained. Based on the service relationship between each service node and the data traversal flow between each service node, the data access flow of all service nodes in each sub-region can be counted, and then the data access flow in the sub-region can be obtained. Based on the number of service nodes in the sub-region, the business volume, etc., the upper and lower limits of the storage volume corresponding to each sub-region are estimated. Through the directed graph, we can more intuitively see the business relationship between each business node, and then quickly count the required traffic data.
[0052] Among them, data traversal traffic and data access traffic refer to the data traffic transmitted by the network, which is mainly distinguished from the module. For example, in some scenarios, access traffic can be generated by tdbank (real-time data access platform) and traversal traffic can be generated by tdw (offline computing platform). Access traffic can be understood as the business distribution that will generate a large amount of cross-city access bandwidth, and traversal traffic can be understood as the business mutual access data that generates a large amount of cross-city traversal bandwidth.
[0053] S204: Taking the minimum upper limit of the total amount of data traversal between sub-regions within a specified time as the objective function, setting constraints of the objective function according to at least one of the data traversal traffic between different business nodes, the data access traffic of different sub-regions, and the upper and lower limits of the storage volume corresponding to different sub-regions, and constructing a first server cluster deployment optimization model based on the objective function and the constraints.
[0054] In the specific implementation process, after collecting and counting relevant traffic data based on the directed graph of the business nodes in the target area, an optimization model for server cluster deployment is constructed based on the collected traffic data. In the embodiment of this specification, the minimum value of the upper limit of the total amount of data crossing between each sub-area within a specified time is used as the objective function, and then the corresponding constraints are set based on the statistical data crossing traffic between different business nodes, the data access traffic of different sub-areas, and the upper and lower limits of the storage volume corresponding to different sub-areas. The first server cluster deployment optimization model is constructed based on the objective function and the constraints. That is to say, the first server cluster deployment optimization model is composed of the objective function and the constraints. The target variable of the first server cluster deployment optimization model is a binary variable that specifies whether the business node is placed in the specified sub-area, such as: whether business node A is placed in sub-area 1. If so, the target variable corresponding to business node A takes the value of 1, otherwise, the target variable corresponding to business node A takes the value of 0.
[0055] Among them, the upper limit of the total amount of data traversal between sub-areas within a specified time is an auxiliary variable introduced based on the data interaction of each business node between two sub-areas and the scenario of two-way data traversal, and can be calculated based on the data traversal traffic between business nodes between sub-areas. The number of visits from business node A to business node B and the number of visits from business node B to business node A are likely to be different. The embodiments of this specification fully consider the two-way interaction of data by introducing the upper limit of the total amount of data traversal.
[0056] For example: based on the data crossing traffic between the business nodes in each sub-area, the maximum value of the data crossing traffic between each sub-area can be calculated; based on the maximum value of the data crossing traffic between each sub-area in the target area, the upper limit of the total amount of data crossing between every two sub-areas within a specified time is calculated. Constraints can be set according to actual business needs, such as: all business volumes in the sub-area need to meet the upper and lower limits of the regional storage volume, that is, all business volumes in the sub-area need to meet the upper and lower limits of the corresponding sub-area storage volume. The values of the upper and lower limits of the storage volume of each sub-area can be obtained when the traffic data is statistically calculated based on the directed graph. The specific content of other constraints can be set according to business needs, and the embodiments of this specification are not specifically limited here. The specified time can be set according to the calculation granularity required by the actual scenario, such as: it can be every day, every week, or every hour, etc., and the embodiments of this specification are not specifically limited.
[0057] In general, the server cluster deployment optimization model can choose a greedy algorithm or a genetic algorithm structure, but this model generally cannot obtain a global optimal solution, which may lead to poor deployment results of the server cluster. In the embodiment of this specification, the first server cluster deployment optimization model can adopt a mixed integer quadratic constraint model structure, wherein the MIP (Mixed Integer Programming) model with quadratic constraints can be called a mixed integer quadratic constraint model (Mixed Integer Quadratically Constrained Programming, MIQCP), and there are quadratic terms in the constraints of the mixed integer quadratic constraint model. By introducing the variable of the upper limit value of the total amount of data crossing, the optimal problem of the amount of data crossing between clusters when the server cluster is deployed is abstracted into a mixed integer quadratic constraint problem, and a mixed integer quadratic constraint model is constructed as a server cluster deployment optimization model. The mixed integer quadratic constraint model is relatively stable, the data processing speed is relatively fast, and the global optimal solution can be obtained. The mixed integer quadratic constraint model is used to optimize the server cluster deployment, and the optimal deployment mode of the service node can be quickly obtained, which lays a data foundation for the optimization of the server cluster deployment.
[0058] In some embodiments of the present specification, the sub-area includes a first sub-area and a second sub-area, and setting constraints according to the data traversal traffic between different service nodes, the data access traffic of different sub-areas, and the upper and lower limits of storage corresponding to different sub-areas includes:
[0059] According to the data transit traffic between different service nodes and the data access traffic of different sub-areas, the sum of the first data transit traffic from the first sub-area to the second sub-area within the specified time is calculated, and the sum of the second data transit traffic from the second sub-area to the first sub-area within the specified time is calculated;
[0060] The constraints are set as:
[0061] A business node is placed in a sub-area;
[0062] The total amount of data traversed by each service node in each sub-region is between the upper and lower limits of the storage capacity of the sub-region;
[0063] Within a specified time, an upper limit value of the total amount of data transit between the first sub-area and the second sub-area is greater than or equal to the sum of the first data transit traffic, and greater than or equal to the sum of the second data transit traffic.
[0064] In the specific implementation process, in a scenario example, based on the principles of naive solution, greedy algorithm, dynamic programming, etc., a model for optimizing server cluster deployment with a calculation granularity of one day can be obtained, which can be expressed as follows:
[0065]
[0066]
[0067] In the above formula, formula (1) can be understood as the objective function of the original optimization problem, such as: It can represent the sum of data traversal traffic from sub-area k to sub-area l, that is, the sum of first data traversal traffic, It can be expressed as the sum of the data traversal flow from sub-area l to sub-area k, that is, the sum of the second data traversal flow. The optimization goal is to find the minimum traversal flow of the sum of the pairwise traversal flows of all sub-areas. Formula (2) represents the constraint condition of the objective function.
[0068] It can be seen from the above formula (1) that the objective function of the original optimization problem of server cluster deployment contains two optimization problems, max and min, which cannot be solved directly. In the optimization problem of server cluster deployment represented by the above formula (1)-formula (2), the embodiment of this specification introduces an auxiliary variable, i.e., the upper limit value y of the total amount of data traversal between each sub-area within a specified time. kl , convert the above optimization problem into a mixed integer quadratic constraint problem, set the calculation granularity to every day, and obtain the first server cluster deployment optimization model constructed according to the crossing volume / day, which can be expressed as follows:
[0069] min∑ k<l,k,l=1…m y kl (3)
[0070]
[0071] In the above formula, formula (3) can be understood as the objective function of the first server cluster deployment optimization model, and formula (4) can represent the constraint condition of the first server cluster deployment optimization model. kl represents the upper limit of the total amount of data traversal from region k to region l. The optimization goal is to find the minimum amount of data traversal between all sub-regions, that is, to find y kl It can be seen that the embodiment of this specification introduces the auxiliary variable y kl , transform the original optimization problem of formula (1)-formula (2) into a MIQCP problem.
[0072] In the above formula, X i =[x i1, x i2 ,…,x in ],i=1,2...m,X i is a variable where each component x ijis a binary variable indicating whether the jth service node is placed in the ith sub-area, x ij =1 means yes, x ij =0 means no. The quantitative data read in the above formula includes:
[0073] n represents the total number of sub-service nodes;
[0074] m represents the total number of sub-regions;
[0075] N j Indicates the data volume of the j-th service node, that is, the data traversal flow of the j-th service node;
[0076] F represents the data traversal matrix between business nodes, that is, the data traversal volume of each business node every day. Each element f in F pq It can represent the data traversal traffic from the pth service node to the qth service node;
[0077] B represents the data access traffic of the business node in the specified sub-area, such as: B k represents the data access traffic of the service node in sub-area k, B k It can represent a matrix where the element b kj represents the data access traffic of the jth service node in the kth sub-area;
[0078] U represents the storage capacity upper limit matrix of each sub-region, where the element u i It can represent the upper limit of storage capacity of the ith sub-region;
[0079] D represents the storage limit matrix of each sub-region, where the element d i It can represent the lower limit of the storage capacity of the i-th sub-area.
[0080] It can be seen from the above formula that the sum of the first data crossing traffic from the first sub-area to the second sub-area and the sum of the second data crossing traffic from the second sub-area to the first sub-area in any two sub-areas in the target area can be calculated in sequence according to the collected traffic data. After obtaining the sum of the first data crossing traffic and the sum of the second data crossing traffic between all sub-areas, the constraint conditions can be set.
[0081] It can be seen that in formula (4), the first constraint can be understood as the same service node can only be assigned to one sub-area, the second constraint can be understood as the indicator target variable, indicating that the j-th service node is placed in the i-th sub-area, the third constraint can be understood as the sum of the data traversal traffic of each service node in each sub-area is between the upper and lower limits of the storage capacity of the sub-area, and the fourth constraint can be understood as the introduced auxiliary variable, that is, the upper limit of the total amount of data traversal between any two sub-areas per day must be greater than or equal to the sum of the data traversal traffic between the two sub-areas. For example: y in formula (4) kl It must be greater than the sum of the first data traffic from sub-area k to sub-area l, and also greater than the sum of the second data traffic from sub-area l to sub-area k.
[0082] The embodiment of this specification introduces an auxiliary variable, which is the upper limit of the total amount of data crossing between any two sub-regions within a specified time. The target variable in the first server cluster deployment optimization model is a binary variable, and the constraint conditions are converted into quadratic constraints, thereby converting the original server cluster deployment optimization problem into a mixed integer quadratic constraint problem, which can improve the subsequent data processing speed and lay a data foundation for the deployment optimization of the server cluster.
[0083] S206: Solve the first server cluster deployment optimization model to obtain a server cluster deployment optimization result, where the server cluster deployment optimization result includes a target sub-region corresponding to each business node.
[0084] In the specific implementation process, after constructing the first server cluster deployment optimization model, the first server cluster deployment optimization model can be solved to obtain the server cluster deployment optimization result, which can include the target sub-area corresponding to each business node in the target area, such as: the value of the target variable X in the above formula can be obtained, and the value (0 or 1) corresponding to each element in X can show the sub-area where each business node should be deployed. Among them, the solution method for the first server cluster deployment optimization model can be selected according to actual needs, such as: selecting a suitable machine learning algorithm or optimal algorithm, such as: gradient solution method, etc., to solve the above objective function, which is not specifically limited in the embodiments of this specification.
[0085] In some embodiments of the present specification, when solving the first server cluster deployment optimization model, the branch-and-cut method can be selected for solution. Specifically, the CPLEX (a tool for solving optimization problems) standard package can be selected for corresponding solution. The main idea is to transform the MIQCP problem into an LP relaxation problem (continuous optimization). The continuous optimization problem can be solved by the simplex method, and finally an integer solution is obtained by multiple rounds of branch-and-bound plus pruning. Finally, multiple rounds of iterations can obtain a global optimal solution with good stability. Among them, the branch-and-bound method is the most commonly used algorithm for solving integer programming problems. Its core is to repeatedly divide the entire feasible solution space into smaller and smaller subsets, called branches; and calculate a target lower bound (for minimum value problems) for the solution set in each subset, which is called delimitation. After each branch, those subsets whose bounds exceed the target value of the known feasible solution set will no longer branch further. In this way, many subsets can be ignored, which is called pruning.
[0086] By using the branch and bound method to solve the server cluster deployment optimization model of the mixed integer quadratic constraint structure, the global optimal solution of the server cluster deployment optimization can be obtained, which improves the accuracy of the server cluster deployment optimization. In addition, through multiple rounds of solving by the branch and bound method, the stability is better, which improves the stability of the server cluster system.
[0087] S208. Migrate the service nodes in the target area to the servers in the target sub-area corresponding to the service nodes based on the server cluster deployment optimization result.
[0088] In the specific implementation process, after obtaining the server cluster deployment optimization results, the business nodes in the target area can be migrated according to the target sub-areas corresponding to each business node in the server cluster deployment optimization results. For example, according to the target sub-areas corresponding to each business node, the business nodes are migrated from the original area to the servers in the corresponding target sub-areas, thereby achieving the optimization of the server cluster deployment in the target area.
[0089] The server cluster deployment optimization method provided in the embodiment of this specification collects the data traversal flow, data access flow, and upper and lower limits of storage between each business node in the target area, introduces the upper limit value of the total amount of data traversal between each sub-area within a specified time as an auxiliary variable, and constructs a first server cluster deployment optimization model. The first server cluster deployment optimization model fully considers the two-way data interaction between business nodes, improves the accuracy of server cluster deployment optimization, and solves the two-way problem of data shuttling. By solving the first server cluster deployment optimization model, the server cluster deployment optimization result in the target area can be obtained, thereby performing migration optimization on the business nodes in the target area. The optimization of the server cluster deployment in the target area is achieved, making the deployment of business nodes more reasonable and improving the system performance of the server cluster. In addition, the server cluster deployment optimization model in the embodiment of this specification can adopt a mixed integer quadratic constraint model, which has a fast data processing speed and improves the time for server cluster deployment optimization.
[0090] In some embodiments of this specification, after obtaining the server cluster deployment optimization result, the method further includes:
[0091] Substituting the server deployment optimization result into the pre-built second server cluster deployment optimization model, verifying whether the server deployment optimization result satisfies the constraint conditions of the objective function in the second server cluster deployment optimization model, and if so, determining that the server deployment optimization result is accurate, and migrating the service nodes in the target area based on the server cluster deployment optimization result;
[0092] If not, solving the second server cluster deployment optimization model, taking the server deployment optimization result corresponding to the second server cluster deployment optimization model as the target server deployment optimization result, and migrating the service nodes in the target area based on the preferred server cluster deployment optimization result;
[0093] The second server cluster deployment optimization model has the same model structure as the first server cluster deployment optimization model, and the calculation granularity of the second server cluster deployment optimization model is smaller than that of the first server cluster deployment optimization model.
[0094] In the specific implementation process, after obtaining the optimization result of the first server cluster deployment optimization model, the embodiments of this specification can also verify the optimization result. Specifically, a second server cluster deployment optimization model with a smaller calculation granularity than the first server cluster deployment optimization model can be constructed, and the optimization result of the first server cluster deployment optimization model can be substituted into the second server cluster deployment optimization model to verify whether the optimization result meets the constraint conditions of the objective function in the second server cluster deployment optimization model. If the verification is successful, the optimization result is directly used to migrate the business nodes in the target area. If the verification fails, the constructed second server cluster deployment optimization model can be solved to obtain the optimization result of the second server cluster deployment optimization model, which is the target server deployment optimization result. The business nodes in the target area are migrated according to the target server deployment optimization result. Among them, the method for solving the second server cluster deployment optimization model can be selected from the method for solving the first server cluster deployment optimization model, such as: branch and bound method, which is not specifically limited in the embodiments of this specification.
[0095] It should be noted that the second server cluster deployment optimization model has the same model structure as the first server cluster deployment optimization model, for example, both are mixed integer quadratic constraint models, but the calculation granularity of the second server cluster deployment optimization model is smaller than that of the first server cluster deployment optimization model, for example, if the calculation granularity of the first server cluster deployment optimization model is day, the calculation granularity of the second server cluster deployment optimization model can be set to hour or minute, which can be determined according to actual needs, and the embodiments of this specification do not make specific limitations.
[0096] In some embodiments of this specification, the second server cluster deployment optimization model can be created with the calculation granularity of hours. The specific process can be referred to as follows:
[0097] The original optimization problem of calculating the data traversal traffic between business nodes on an hourly basis can be referred to as follows:
[0098]
[0099]
[0100] Introducing the auxiliary variable z, the optimization problem calculated according to the crossing flow / hour is as follows:
[0101] min z (7)
[0102]
[0103] In the above formula, z can represent the hourly data traffic between two service nodes, M h =(M0, M1, M2, ...M23 ,) The data traffic per hour between business nodes, M h The elements in can be represented as m hpq It represents the data traversal flow from the pth service node to the qth service node in the hth hour / hour. The meanings of other parameters or mathematical formulas in the formula are the same as those of the above formulas (1) to (4), and will not be repeated here.
[0104] Among them, formula (5)-formula (6) is the original optimization problem of the server cluster expressed with the calculation granularity of hours, and formula (7)-formula (8) is the second server cluster deployment optimization model obtained with the calculation granularity of hours by introducing auxiliary variables on the basis of formula (5)-formula (6). It can be seen that the second server cluster deployment optimization model is similar to the structure and parameters of the first server cluster deployment optimization model corresponding to formula (3)-formula (4), except that the calculation granularity (hours) is smaller than the calculation granularity (days) of the first server cluster deployment optimization model.
[0105] The server cluster deployment optimization problem in the embodiment of this specification is actually a generalized knapsack problem, and therefore is also an NP-hard problem. If the scale of the problem is not large enough (the solution provided in the embodiment of this specification can handle thousands of business), it can be solved within a limited time. In the embodiment of this specification, a relatively large calculation granularity (such as day-level) can be used for optimization of data traversal traffic, and after quickly obtaining the optimal solution, a smaller calculation granularity such as hour-level or minute-level traversal volume is brought in for verification calculation, and the total bandwidth consumption and peak traversal at the hour-level or minute-level are obtained to verify whether physical constraints and budgets are met, etc. If met, it means that the optimization result obtained at the day-level calculation granularity is accurate and can be used, which improves the data processing speed of server cluster deployment optimization. If not met, the optimization model of smaller granularity is solved to improve the accuracy of server cluster deployment optimization.
[0106] In some embodiments of the present specification, before collecting statistics on traffic data of the target area according to the directed graph of service nodes corresponding to the target area, the method further includes:
[0107] According to the calling relationship between each business node in the business node directed graph, the business node directed graph is merged;
[0108] The traffic data of the target area is counted based on the directed graph of business nodes after node fusion.
[0109] In the specific implementation process, after obtaining the directed graph of business nodes corresponding to the target area, the business nodes can be merged based on the calling relationship between the business nodes in the directed graph, such as whether there is business dealings, etc. For example, business nodes belonging to the same business can be merged into one node. Then, the relevant data is counted based on the merged directed graph of business nodes to create the corresponding server cluster deployment optimization model.
[0110] By integrating business nodes, the amount of data processing can be reduced, the complexity of server cluster deployment optimization can be reduced, and the data processing speed can be improved.
[0111] In some embodiments of the present specification, according to the calling relationship between each service node in the service node directed graph, node fusion is performed on the service node directed graph, including:
[0112] Filtering out independent subgraphs from the business node directed graph according to the call relationship between each business node in the business node directed graph, and merging the business nodes in the independent subgraphs into one business node;
[0113] And / or, performing node fusion on the business node directed graph according to the in-and-out degrees corresponding to each business node in the business node directed graph.
[0114] In the specific implementation process, the method of business node fusion can be based on the out-degree and in-degree screening of the nodes and independent subgraphs, and the appropriate business nodes can be merged into one node. For example, independent subgraphs can be selected from the entire business node directed graph for merging. Independent subgraphs will not have data access with other business nodes. After merging, the amount of data traversal between all nodes in the independent subgraph can be reduced to 0, which can reduce the scale of the problem without affecting the final solution. Independent subgraphs can be understood as directed graphs composed of multiple business nodes in the business node directed graph that have no data interaction with other business nodes. In the entire business node directed graph, each business node in the independent subgraph has no data access with other business nodes, but the business nodes in the independent subgraph have direct data access.
[0115] In addition, the sum of (out-degree + in-degree) of each business node can be sorted from small to large, and the values of all directed edges can be sorted, and the business nodes with a small sum of in-degree and a large directed edge value can be merged. For example, the business nodes with a sum of in-degree and in-degree less than the preset sum value but a directed edge value greater than the preset edge value can be merged into one node.
[0116] By fusing nodes based on independent subgraphs and in-and-out degrees in directed graphs, related nodes can be merged into one. For large-scale scenarios, this greatly reduces the complexity of data processing and improves the data processing speed.
[0117] In some embodiments of this specification, after obtaining the server cluster deployment optimization result, the method further includes:
[0118] Adjust the server cluster deployment optimization results based on the business rules corresponding to each business node in the target area;
[0119] Migrate the business nodes in the target area based on the adjusted server cluster deployment optimization results.
[0120] In the specific implementation process, after obtaining the server cluster deployment optimization results, the server cluster deployment optimization results can also be adjusted according to the business rules corresponding to different business nodes, wherein the business rules may include the definition of the positional relationship of the business nodes that process related businesses. For example: some business rules stipulate that the business nodes belonging to business A need to be deployed in the same area, then the business nodes belonging to A can be adjusted to one area in the server cluster deployment optimization results, and the specific adjustment method can be determined according to actual needs, and the embodiments of this specification do not make specific limitations. For example: the business rules stipulate that the business nodes belonging to business A need to be deployed in the same area, and the obtained server cluster deployment optimization results show that the business nodes belonging to business A are deployed in both sub-area 1 and sub-area 2, wherein there are 3 business nodes in sub-area 1 and 6 business nodes in sub-area 2, then the 3 business nodes in sub-area 1 can be adjusted to sub-area 2, which can reduce the workload.
[0121] After adjusting the server cluster deployment optimization result based on the business rules, the business nodes in the target area can be migrated based on the adjusted server cluster deployment optimization result, and each business node can be migrated to its corresponding target sub-area.
[0122] The obtained server cluster deployment optimization results are adjusted based on business rules, so that the deployment of each business node in the server cluster meets different business needs, thereby improving the optimization accuracy and applicability of the server cluster deployment.
[0123] In some embodiments of the present specification, migrating a service node in a target area to a server in a target sub-area corresponding to the service node based on the server cluster deployment optimization result includes:
[0124] Determine a first target sub-region and a second target sub-region in the target region according to the server cluster deployment optimization result, the first target sub-region has a service node to be migrated to the second target sub-region, and / or the second target sub-region has a service node to be migrated to the first target sub-region;
[0125] Migrating the service nodes in the first target sub-area and the second target sub-area according to the server cluster deployment optimization result;
[0126] After the migration of business nodes in the first target sub-region and the second target sub-region is completed, a new first target sub-region and a new second target sub-region are determined in the target region based on the server cluster deployment optimization result, and the business nodes in the new first target sub-region and the new second target sub-region are replaced and migrated until all business nodes in the server cluster deployment optimization result are migrated.
[0127] In the specific implementation process, when migrating business nodes based on the server cluster deployment optimization results, a batch migration method can be adopted, mainly considering that after each migration, the amount of data shuttle between the new and old areas meets the bandwidth limit, and the amount of each migration is limited by the replacement machine. Specifically, the sub-areas with business node migration relationships can be used as the first target sub-area and the second target sub-area according to the server cluster deployment optimization results. For example, two sub-areas with business node bidirectional migration or unidirectional migration can be used as the first target sub-area and the second target sub-area. Specifically, if there are nodes in sub-area A to migrate to sub-area B, and there are nodes in sub-area B to migrate to sub-area A, that is, there is bidirectional migration of nodes between sub-areas A and B, then sub-area A can be used as the first target sub-area and sub-area B as the second target sub-area; or, if there are nodes in sub-area A to migrate to sub-area B, and there are no nodes in sub-area B to migrate to sub-area A, that is, there is unidirectional migration of nodes between sub-areas A and B, then sub-area A can be used as the first target sub-area and sub-area B as the second target sub-area; or, if there are nodes in sub-area A to migrate to sub-area B, and there are no nodes in sub-area B to migrate to sub-area A, that is, there is unidirectional migration of nodes between sub-areas A and B, then sub-area A can also be used as the first target sub-area and sub-area B as the second target sub-area.
[0128] After determining the first target sub-region and the second target sub-region, the required business nodes in the first target sub-region and the second target sub-region can be migrated to the corresponding target sub-region based on the server cluster deployment optimization result. When the business nodes in the first target sub-region and the second target sub-region are migrated, a new first target sub-region and a new second target sub-region can be selected, and the above process can be repeated until all the business nodes in the server cluster deployment optimization result are migrated.
[0129] In the embodiments of this specification, the scheduling problem of multiple locations is solved into two locations, and a batch migration strategy is adopted to simplify the problem of multiple business nodes needing to be migrated. Each migration process fully considers the bandwidth limitations of the two areas, thereby improving the accuracy of business node migration and deployment optimization.
[0130] In some embodiments of the present specification, migrating service nodes in a first target sub-area and a second target sub-area includes:
[0131] If there are available servers in both the first target sub-region and the second target sub-region, migrate the service nodes in the first target sub-region to be migrated to the second target sub-region to the available servers in the second target sub-region according to the server cluster deployment optimization result, and migrate the service nodes in the second target sub-region to be migrated to the first target sub-region to the available servers in the first target sub-region;
[0132] If there is no available server in at least one of the first target sub-region or the second target sub-region, the service nodes in the first target sub-region and the second target sub-region are replaced and migrated.
[0133] In the specific implementation process, when the idle capacity of the servers in the first target sub-area and the second target sub-area is sufficient and there are available servers, the business nodes in the first target sub-area and the second target sub-area that need to be migrated to the other area can be migrated to the available servers in the corresponding sub-area according to the server cluster deployment optimization results. The migration process can be bidirectional synchronization to improve the data processing speed, or the business nodes that need to be migrated in one area can be migrated first, and then the business nodes in another area can be migrated. The embodiments of this specification do not make specific limitations.
[0134] Among them, the available server can be understood as a server with sufficient capacity and can also perform other data processing. The new service node that has been migrated can be deployed in the available server. If the server capacity in the first target sub-area or the second target sub-area is insufficient and there is no available server, then a replacement migration strategy can be adopted to migrate the service nodes in the first target sub-area and the second target sub-area. The replacement migration strategy can be understood as exchanging the service nodes in the two areas to achieve the purpose of business node migration. This replacement migration can avoid the problem of insufficient data storage in the first target sub-area or the second target sub-area and the inability to migrate nodes.
[0135] Of course, if the scenario of one-way node migration is between the first target sub-region and the second target sub-region, if there is an available server in the target sub-region of the node to be migrated, and there is no available server in the source region of the node to be migrated, then the node to be migrated can be directly migrated from the source region to the available server in the other region, and the migration of the business node can be completed without adopting the replacement migration strategy.
[0136] In some embodiments of the present specification, performing replacement migration on service nodes in a first target sub-area and a second target sub-area includes:
[0137] If there are available servers in the first target sub-region or the second target sub-region, then according to the server cluster deployment optimization result, the service nodes in the sub-regions without available servers in the first target sub-region and the second target sub-region that are to be migrated to the sub-region with available servers are migrated to the available servers in the sub-region with available servers, and then the service nodes in the sub-region with available servers that are to be migrated to the sub-region without available servers are migrated to the available servers replaced in the sub-region without available servers;
[0138] If there are no available servers in the first target sub-area and the second target sub-area, if one or more available servers are deployed in the first target sub-area or the second target sub-area, the business nodes in the sub-area where the available servers are not deployed and are to be migrated to the sub-area where the available servers are deployed are migrated to the deployed available servers according to the server cluster deployment optimization result, and then the business nodes in the sub-area where the available servers are deployed and are to be migrated to the sub-area where the available servers are not deployed are migrated to the available servers replaced in the sub-area where the available servers are not deployed.
[0139] In a specific implementation process, if there is an available server in one of the first target sub-region or the second target sub-region, such as: if there is an available server in the first target sub-region and there is no available server in the second target sub-region, then the service nodes in the second target sub-region where there is no available server and need to be migrated to the first target sub-region can be migrated to the available servers in the first target sub-region based on the server cluster deployment optimization result. At this time, the available server is replaced in the second target sub-region, and then the service nodes in the first target sub-region that need to be migrated to the second target sub-region are migrated to the replaced available servers in the second target sub-region based on the server cluster deployment optimization result.
[0140] If there are no available servers in the first target sub-region and the second target sub-region, that is, the servers in the two regions are occupied, then one or more available servers can be deployed in the first target sub-region or the second target sub-region. Assuming that an available server is deployed in the first target sub-region, then the service nodes in the second target sub-region that need to be migrated to the first target sub-region can be migrated to the newly deployed available server based on the server cluster deployment optimization result. In this way, the available server is replaced in the second target sub-region, and then the service nodes in the first target sub-region that need to be migrated to the second target sub-region are migrated to the newly replaced available server in the second target sub-region based on the server cluster deployment optimization result.
[0141] Through the above replacement migration method, the migration problem of business nodes can be solved if the servers in the area are occupied, so that the migration of business nodes is not restricted by the machines, and the optimized deployment of business nodes in the server cluster can be realized.
[0142] In addition, in the embodiments of this specification, when solving the scheduling problem of changing multiple locations into two locations, a greedy strategy can be appropriately introduced in the specific implementation process to simplify the replacement and migration process of business nodes, because the bandwidth consumption here is temporary and does not necessarily need to be the optimal solution. At the same time, considering that the scale of data nodes is in the millions, the model needs to be simplified. The embodiments of this specification will not be expanded in detail.
[0143] Figure 4 FIG. 1 is a schematic diagram showing the principle of optimizing the deployment of a server cluster in another embodiment of the present specification. Figure 4 As shown, the optimization of server cluster deployment in the embodiment of this specification mainly includes 5 steps:
[0144] 1) Data collation and preprocessing, including the calculation and statistics of hourly and daily data traversal between nodes, regional data access volume, and merging of nodes with the same business. Specifically, each business can contain multiple business nodes. It is necessary to sort out the call relationship between business nodes and perform node fusion according to the node in-and-out degree to reduce the magnitude and scale of the problem. After obtaining a complete processed business node, sort out the traversal data aggregated by day or hour according to the data traversal volume between the calculation nodes, similar to the in-node, out-node, and data traversal volume. The access traversal volume between regions needs to be calculated comprehensively based on the region. Sort out the data such as regional storage capacity limit, fusion traversal volume between business nodes, and regional access traversal volume, and read and process them through automated programs for subsequent optimization.
[0145] 2) Determine the optimization strategy and abstract the optimization problem into a mixed integer quadratic constraint problem, which can ensure that the global optimal solution is obtained while the time complexity is relatively reduced.
[0146] 3) Select a solution and use the branch-and-solve method to solve the optimization problem in 2).
[0147] 4) Secondary optimization and adjustment: sort out the maximum bandwidth penetration at the hourly level and the maximum bandwidth penetration at the daily level to obtain the optimal total bandwidth consumption, and make appropriate optimization adjustments based on business constraints.
[0148] 5) Replacement migration: Use the scheduling plan to perform batch migration. The old and new areas are replaced to solve the bandwidth consumption problem after a single migration.
[0149] The specific content of each step can be referred to the description of the above embodiment and will not be repeated here.
[0150] The server cluster deployment optimization method provided in the embodiments of this specification uses a directed graph to simulate the server cluster distribution, and abstracts the optimal problem of the amount of data shuttled between clusters into a mixed integer quadratic constraint problem for solution. Based on this, the embodiments of this specification propose a set of automated cluster multi-location deployment optimization solutions, and provide the optimal deployment solution based on the capacity constraints of each location, so that the amount of data shuttled between locations is minimized, greatly saving bandwidth costs.
[0151] Table 1 shows the effects of different optimization algorithms in an embodiment of this specification. It can be seen from Table 1 that the server cluster deployment solution of MIQCP based on branch-and-cut proposed in the embodiment of this specification is better in terms of time consumption and results than other methods, and has stronger stability and scalability. At the same time, the broadband cost saving of the final multi-location scheduling solution is much higher than other methods.
[0152] Table 1
[0153]
[0154] Based on the above-mentioned optimization method for server cluster deployment, one or more embodiments of this specification also provide a client and a server for optimizing the deployment of a server cluster. The client and the server may include a device (including a distributed system), software (application), module, component, server, client, etc. using the method described in the embodiment of this specification and a device in combination with necessary implementation hardware. Based on the same innovative concept, the device in one or more embodiments provided in the embodiment of this specification is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the device are similar, the implementation of the specific device in the embodiment of this specification can refer to the implementation of the aforementioned method, and the repetitions will not be repeated. As used below, the term "unit" or "module" can implement a combination of software and / or hardware for predetermined functions. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.
[0155] It can be seen from the technical solutions provided in the above embodiments of this specification that the embodiments of this specification also provide an optimization device for server cluster deployment. Figure 5 is a schematic diagram of a server cluster deployment optimization device provided in an embodiment of this specification, such as Figure 5 As shown, the above device comprises:
[0156] The data collection module 510 is used to collect statistics on the traffic data of the target area according to the directed graph of the service nodes corresponding to the target area, wherein the target area includes at least two sub-areas, and the traffic data includes at least one of: data traversal traffic between different service nodes, data access traffic of different sub-areas, and upper and lower limits of storage corresponding to different sub-areas;
[0157] The optimization model building module 520 is used to take the minimum value of the upper limit of the total amount of data traversal between different sub-areas within a specified time as the objective function, set the constraint condition of the objective function according to at least one of the data traversal flow between the different business nodes, the data access flow of the different areas, and the upper and lower limits of the storage volume corresponding to the different sub-areas, and build a first server cluster deployment optimization model based on the objective function and the constraint condition;
[0158] A model solving module 530, configured to solve the first server cluster deployment optimization model to obtain a server cluster deployment optimization result, wherein the server cluster deployment optimization result includes a target sub-region corresponding to each service node;
[0159] The node migration module 540 is used to migrate the service nodes in the target area to the servers in the target sub-area corresponding to the service nodes based on the server cluster deployment optimization result.
[0160] Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated in detail here. The device in the above embodiment may also include other implementation methods according to the description of the method embodiment, and the specific implementation method can refer to the description of the relevant method embodiment, which will not be repeated here one by one.
[0161] Figure 6 is a block diagram of an optimized electronic device for server cluster deployment provided by an embodiment of this specification. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 6As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an optimization method for server cluster deployment is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.
[0162] Figure 7 is a block diagram of another optimized electronic device for server cluster deployment provided by an embodiment of this specification. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an optimization method for server cluster deployment is implemented.
[0163] Those skilled in the art will understand that Figure 6 or Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the embodiment scheme of this specification, and does not constitute a limitation on the electronic device to which the embodiment scheme of this specification is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0164] In an exemplary embodiment, an electronic device is also provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the optimization method for server cluster deployment as in the embodiment of this specification.
[0165] In an exemplary embodiment, a computer-readable storage medium is also provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the optimization method for server cluster deployment in the embodiment of this specification.
[0166] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program includes a computer instruction, the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the optimization method for server cluster deployment provided in the above various optional implementations.
[0167] It is understandable that in the specific implementation of this specification, user-related data is involved. When the above embodiments of this specification are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this specification can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0169] Those skilled in the art will readily appreciate other implementations of the embodiments of the present specification after considering the specification and practicing the invention disclosed herein. The present specification is intended to cover any variations, uses, or adaptations of the embodiments of the present specification, which follow the general principles of the embodiments of the present specification and include common knowledge or customary technical means in the art that are not disclosed in the embodiments of the present specification. The description and embodiments are to be considered merely as exemplary, and the true scope and spirit of the embodiments of the present specification are indicated by the following claims.
[0170] It should be understood that the embodiments of the present specification are not limited to the precise structures described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the embodiments of the present specification is limited only by the appended claims.
Claims
1. A method for optimizing server cluster deployment, characterized in that: The method comprises: According to the directed graph of the service nodes corresponding to the target area, traffic data of the target area is counted, the target area includes at least two sub-areas, and the traffic data includes: data traversal traffic between different service nodes, data access traffic of different sub-areas, and at least one of upper and lower limits of storage corresponding to different sub-areas; The minimum value of the upper limit of the total amount of data traversal between the sub-areas within a specified time is used as the objective function, and the constraint condition of the objective function is set according to at least one of the data traversal flow between the different service nodes, the data access flow of the different sub-areas, and the upper and lower limits of the storage volume corresponding to the different sub-areas, and the first server cluster deployment optimization model is constructed by the objective function and the constraint condition; Solving the first server cluster deployment optimization model to obtain a server cluster deployment optimization result, wherein the server cluster deployment optimization result includes a target sub-region corresponding to each service node; Based on the server cluster deployment optimization result, the service nodes in the target area are migrated to the servers in the target sub-area corresponding to the service nodes.
2. The method according to claim 1, characterized in that The sub-area includes a first sub-area and a second sub-area, and the constraint condition of the objective function is set according to at least one of the data traversal flow between the different service nodes, the data access flow of different sub-areas, and the upper and lower limits of the storage volume corresponding to different sub-areas, including: According to the data transit traffic between different service nodes and the data access traffic of different sub-areas, the sum of the first data transit traffic from the first sub-area to the second sub-area within the specified time is calculated, and the sum of the second data transit traffic from the second sub-area to the first sub-area within the specified time is calculated; The constraints are set as: A business node is placed in a sub-area; The total amount of data traversed by each service node in each sub-region is between the upper and lower limits of the storage capacity of the sub-region; Within the specified time, an upper limit value of the total amount of data transiting between the first sub-area and the second sub-area is greater than or equal to the sum of the first data transiting traffic, and greater than or equal to the sum of the second data transiting traffic.
3. The method according to claim 1, characterized in that: After obtaining the server cluster deployment optimization result, the method further includes: Substituting the server deployment optimization result into a pre-built second server cluster deployment optimization model to verify whether the server deployment optimization result satisfies the constraint condition of the objective function in the second server cluster deployment optimization model; If satisfied, determining that the server deployment optimization result is accurate, and migrating the service nodes in the target area based on the server cluster deployment optimization result; If not, solving the second server cluster deployment optimization model, taking the server deployment optimization result corresponding to the second server cluster deployment optimization model as the target server deployment optimization result, and migrating the service nodes in the target area based on the target server deployment optimization result; The second server cluster deployment optimization model has the same model structure as the first server cluster deployment optimization model, and the calculation granularity of the second server cluster deployment optimization model is smaller than that of the first server cluster deployment optimization model.
4. The method according to claim 1, characterized in that: Before counting the traffic data of the target area according to the directed graph of the service nodes corresponding to the target area, the method further includes: According to the calling relationship between each service node in the service node directed graph, performing node fusion on the service node directed graph; The traffic data of the target area is counted based on the directed graph of the service nodes after the node fusion.
5. The method according to claim 1, characterized in that The solving of the first server cluster deployment optimization model includes: The first server cluster deployment optimization model is solved using a branch and bound method.
6. The method according to claim 1, characterized in that After obtaining the server cluster deployment optimization result, the method further includes: Adjusting the server cluster deployment optimization result based on the business rules corresponding to each business node in the target area; The service nodes in the target area are migrated based on the adjusted server cluster deployment optimization result.
7. The method according to claim 1, characterized in that Migrating the service node in the target area to a server in a target sub-area corresponding to the service node based on the server cluster deployment optimization result includes: Determine a first target sub-region and a second target sub-region in the target region according to the server cluster deployment optimization result, wherein the first target sub-region has a service node to be migrated to the second target sub-region, and / or the second target sub-region has a service node to be migrated to the first target sub-region; Migrating the service nodes within the first target sub-area and the second target sub-area according to the server cluster deployment optimization result; After the migration of business nodes in the first target sub-region and the second target sub-region is completed, a new first target sub-region and a new second target sub-region are determined within the target region based on the server cluster deployment optimization result, and the business nodes within the new first target sub-region and the new second target sub-region are replaced and migrated until all business nodes in the server cluster deployment optimization result are migrated.
8. The method according to claim 7, characterized in that The migrating the service nodes in the first target sub-area and the second target sub-area includes: If there are available servers in both the first target sub-area and the second target sub-area, migrate the service nodes in the first target sub-area to be migrated to the second target sub-area to the available servers in the second target sub-area according to the server cluster deployment optimization result, and migrate the service nodes in the second target sub-area to be migrated to the first target sub-area to the available servers in the first target sub-area; If there is no available server in at least one of the first target sub-region or the second target sub-region, the service nodes in the first target sub-region and the second target sub-region are replaced and migrated.
9. The method according to claim 8, characterized in that The replacing and migrating the service nodes in the first target sub-area and the second target sub-area includes: If there are available servers in the first target sub-region or the second target sub-region, then according to the server cluster deployment optimization result, the service nodes in the sub-regions without available servers in the first target sub-region and the second target sub-region that are to be migrated to the sub-region with available servers are migrated to the available servers in the sub-region with available servers, and then the service nodes in the sub-region with available servers that are to be migrated to the sub-region without available servers are migrated to the available servers replaced in the sub-region without available servers; If there are no available servers in the first target sub-area and the second target sub-area, if one or more available servers are deployed in the first target sub-area or the second target sub-area, the business nodes in the sub-area where the available servers are not deployed and are to be migrated to the sub-area where the available servers are deployed are migrated to the deployed available servers according to the server cluster deployment optimization result, and the business nodes in the sub-area where the available servers are deployed and are to be migrated to the sub-area where the available servers are not deployed are migrated to the available servers replaced in the sub-area where the available servers are not deployed.
10. The method according to claim 1, characterized in that The counting of traffic data of the target area according to the directed graph of service nodes corresponding to the target area includes: Acquire the service relationship between the service nodes according to the connection relationship between the nodes in the service node directed graph; Based on the business relationship between each business node, count the data traversal traffic between different business nodes; According to the service relationship between the service nodes in each sub-area in the target area and the data traversal traffic between the service nodes, at least one of the data access traffic of different sub-areas and the upper and lower limits of storage capacity corresponding to different sub-areas is counted.
11. An optimization device for server cluster deployment, characterized in that: The device comprises: A data collection module, used to collect statistics of traffic data of the target area according to a directed graph of service nodes corresponding to the target area, wherein the target area includes at least two sub-areas, and the traffic data includes at least one of: data traversal traffic between different service nodes, data access traffic of different sub-areas, and upper and lower limits of storage corresponding to different sub-areas; an optimization model building module, configured to take the minimum value of the upper limit of the total amount of data traversal between different sub-regions within a specified time as an objective function, set a constraint condition of the objective function according to at least one of the data traversal flow between the different service nodes, the data access flow of the different regions, and the upper and lower limits of the storage volume corresponding to the different sub-regions, and build a first server cluster deployment optimization model based on the objective function and the constraint condition; A model solving module, used to solve the first server cluster deployment optimization model to obtain a server cluster deployment optimization result, wherein the server cluster deployment optimization result includes a target sub-region corresponding to each business node; A node migration module is used to migrate the service nodes in the target area to the servers in the target sub-area corresponding to the service nodes based on the server cluster deployment optimization result.
12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the server cluster deployment optimization method as described in any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the server cluster deployment optimization method as described in any one of claims 1 to 10.
