Method and system for solving decomposable problems based on a distributed computing system

Abstract

Embodiments of the present specification provide a method and system for solving decomposable problems based on a distributed computing system. In the method for solving decomposable problems based on a distributed computing system, via each computing node, optimization variable values and dual variable values of a previous iteration round for solving a sub-problem shard corresponding to the computing node are obtained from a parameter node, and the sub-problem is solved based on the optimization variable values and the dual variable values, to obtain optimization variable values of a current iteration round corresponding to the sub-problem and corresponding constraint calculation values; via the parameter node, optimization variable values of the current iteration round corresponding to the sub-problem shard and constraint calculation values corresponding to each sub-problem shard sent by each computing node are aggregated and saved respectively, and dual variable values of the current iteration round are determined and saved; via a computing control node, constraint residuals are determined according to the aggregated optimization variable values and dual variable values of the current iteration round, and whether to continue iteration is determined based on the constraint residuals.

Description

Technical Field

[0001] The embodiments in this specification generally relate to the field of computer technology, and in particular to methods and systems for solving decomposable problems based on distributed computing systems. Background Technology

[0002] With the rapid development of internet technology, the scale of data processing is also increasing. For example, there is a need for large-scale linear optimization solutions in decision-making scenarios such as machine learning system optimization and resource allocation. One existing technology is to decompose large-scale optimization problems into several solvable subproblems, and then use a distributed computing system to solve each subproblem in parallel and coordinate the solutions of the subproblems to obtain the global solution of the original problem. However, when faced with real-world needs such as billions of decision variables and the need to obtain solutions within time constraints, the stability and reliability of distributed computing systems are often greatly challenged. Therefore, a more stable and reliable method suitable for solving decomposable problems with large-scale data is needed. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a method and system for solving decomposable problems based on a distributed computing system. Using this method and system, more stable and reliable solutions to decomposable problems applicable to large-scale data can be achieved.

[0004] According to one aspect of an embodiment of this specification, a method for solving a decomposable problem based on a distributed computing system is provided. The distributed computing system includes a computing control node, parameter nodes, and multiple computing nodes. Each computing node is used to compute a shard corresponding to the decomposable problem. The method includes: obtaining, via each computing node, the optimization variable values ​​and dual variable values ​​of the previous iteration for solving the shard corresponding to that computing node from the parameter node; solving the shard based on the obtained optimization variable values ​​and dual variable values ​​of the previous iteration and a pre-acquired constraint coefficient matrix to obtain the optimization variable values ​​of the current iteration corresponding to the shard; and solving the shard based on the current iteration value of the shard corresponding to the shard. The optimization variable values ​​of each iteration and the constraint coefficient matrix corresponding to the subproblem piece determine the corresponding constraint calculation values; the calculated constraint calculation values ​​are sent to the parameter node; through the parameter node, the optimization variable values ​​of the current iteration and the constraint calculation values ​​corresponding to each subproblem piece are aggregated and saved respectively; the dual variable values ​​of the current iteration are determined based on the aggregated optimization variable values ​​of the current iteration, the dual variable values ​​of the previous iteration, and the aggregated constraint calculation values; the determined dual variable values ​​of the current iteration are saved; and through the calculation control node, the constraint residuals are determined based on the aggregated optimization variable values ​​of the current iteration and the determined dual variable values ​​of the current iteration; the constraint residuals are used to determine whether to continue iterating.

[0005] According to another aspect of the embodiments of this specification, a distributed computing system for solving decomposable problems is provided. The distributed computing system includes a computing control node, a parameter node, and multiple computing nodes. Each computing node is used to compute a subproblem piece corresponding to the decomposable problem. Each computing node includes: an optimization variable value determination unit, configured to obtain optimization variable values ​​and dual variable values ​​from the parameter node for solving the subproblem piece corresponding to the computing node in the previous iteration; and to solve the subproblem based on the obtained optimization variable values ​​and dual variable values ​​from the previous iteration and a pre-obtained constraint coefficient matrix to obtain the optimization variable value for the current iteration corresponding to the subproblem piece. A sub-result sending unit is configured to send the optimization variable value based on the optimization variable values ​​of the previous iteration and the dual variable values, as well as the pre-obtained constraint coefficient matrix, to obtain the optimization variable value for the current iteration corresponding to the subproblem piece. The variable values ​​and the constraint coefficient matrix corresponding to the subproblem piece determine the corresponding constraint calculation values, and the calculated constraint calculation values ​​are sent to the parameter node. The parameter node includes: an aggregation unit configured to aggregate and save the optimization variable values ​​and constraint calculation values ​​corresponding to the current iteration round for each subproblem piece; a dual variable value saving unit configured to determine the dual variable value of the current iteration round based on the aggregated optimization variable value of the current iteration round, the dual variable value of the previous iteration round, and the aggregated constraint calculation values, and save the determined dual variable value of the current iteration round; and a calculation control node configured to determine the constraint residual based on the aggregated optimization variable value of the current iteration round and the determined dual variable value of the current iteration round, and determine whether to continue iterating based on the constraint residual.

[0006] According to another aspect of the embodiments of this specification, a distributed computing system for solving decomposable problems is provided. The distributed computing system includes a computing control device, a parameter device, and a plurality of computing devices. Each of the plurality of computing devices is used to compute a subproblem slice corresponding to the decomposable problem. The computing control device, the parameter device, and the plurality of computing devices each include at least one processor, a memory coupled to the at least one processor, and a computer program stored in the memory. The at least one processor executes the computer program to implement the corresponding operations in the method for solving decomposable problems based on the distributed computing system as described above.

[0007] According to another aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the method described above for solving decomposable problems based on a distributed computing system.

[0008] According to another aspect of the embodiments of this specification, a computer program product is provided, including a computer program that is executed by a processor to implement the method described above for solving decomposable problems based on a distributed computing system. Attached Figure Description

[0009] A further understanding of the nature and advantages of this specification can be achieved by referring to the following figures. In the figures, similar components or features may have the same reference numerals.

[0010] Figure 1 An exemplary architecture of a method and system for solving decomposable problems based on a distributed computing system, according to embodiments of this specification, is shown.

[0011] Figure 2 A signaling diagram illustrating an example of a method for solving decomposable problems based on a distributed computing system according to an embodiment of this specification is shown.

[0012] Figure 3 A flowchart illustrating an example of the process for obtaining the values ​​of the optimization variable and the dual variable in the previous iteration according to an embodiment of this specification is shown.

[0013] Figure 4 A schematic diagram illustrating an example of the process for obtaining the constraint coefficient matrix corresponding to a subproblem segment according to an embodiment of this specification is shown.

[0014] Figure 5 This diagram illustrates an example application scenario of the method for solving decomposable problems based on a distributed computing system according to embodiments of this specification.

[0015] Figure 6 This diagram illustrates yet another example of an application scenario for the method of solving decomposable problems based on a distributed computing system according to an embodiment of this specification.

[0016] Figure 7 This diagram illustrates yet another example of an application scenario of the method for solving decomposable problems based on a distributed computing system according to an embodiment of this specification.

[0017] Figure 8 This diagram illustrates another example of an application scenario for the method of solving decomposable problems based on a distributed computing system according to an embodiment of this specification.

[0018] Figure 9 A block diagram of an example of a distributed computing system for solving decomposable problems, according to an embodiment of this specification, is shown.

[0019] Figure 10A block diagram illustrating an example of various devices in a distributed computing system for solving decomposable problems, according to embodiments of this specification, is shown. Detailed Implementation

[0020] The subject matter described herein will be discussed below with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of the embodiments described herein. Various processes or components may be omitted, substituted, or added as needed in the various examples. Furthermore, features described in some examples may be combined in other examples.

[0021] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0022] The method and system for solving decomposable problems based on a distributed computing system according to embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0023] Figure 1 An exemplary architecture 100 of a method and system for solving decomposable problems based on a distributed computing system according to embodiments of this specification is shown.

[0024] exist Figure 1 In this distributed computing system 100, a computing control node 110, a parameter node 120, and multiple computing nodes 130 may be included. Each of the multiple computing nodes is used to compute the subproblem fragments corresponding to the decomposable problem. The computing control node 110, the parameter node 120, and the multiple computing nodes 130 can be interconnected via a network.

[0025] The aforementioned network can be any type of network capable of interconnecting network entities; for example, it can be a single network or a combination of various networks. In terms of coverage, the network can be a Local Area Network (LAN), a Wide Area Network (WAN), etc. Regarding the carrying medium, the network can be a wired network, a wireless network, etc. In terms of data switching technology, the network can be a circuit-switched network, a packet-switched network, etc.

[0026] The computation control node 110 can control whether the iterative solution process continues. The parameter node 120 can aggregate and save the computation results for each subproblem obtained by each computation node 130, and calculate the update result of the dual variable value accordingly. Each computation node 130 can solve each subproblem and pass the obtained computation results for each subproblem to the parameter node 120.

[0027] It should be understood that Figure 1 All network entities shown are exemplary, and any other network entities may be involved in Architecture 100 depending on the specific application requirements.

[0028] Figure 2 A signaling diagram of a method 200 for solving decomposable problems based on a distributed computing system according to an embodiment of this specification is shown.

[0029] like Figure 2 As shown, steps 210-240 are performed via each compute node. Steps 250-270 are performed via the parameter node. Steps 280-290 are performed via the compute control node.

[0030] In step 210, the optimization variable values ​​and dual variable values ​​from the parameter node are obtained for solving the subproblem slice corresponding to that computation node in the previous iteration.

[0031] In this embodiment, for each computing node, the optimization variable values ​​and dual variable values ​​used to solve the subproblem piece corresponding to that computing node in the previous iteration can be obtained from the aforementioned parameter node. In this embodiment, the aforementioned parameter node can be used to store the optimization variable values ​​and dual variable values ​​calculated by each computing node in each iteration. Optionally, in one example, the stored optimization variable values ​​and dual variable values ​​can have identifiers that match the subproblem piece corresponding to each computing node.

[0032] In one example, a decomposable problem can include a convex optimization problem. In another example, a decomposable problem can be a linear programming problem, which can be represented as:

[0033]

[0034] stAx=b

[0035] Gx≤h

[0036] in, It can be used to represent n optimization variables. A and G can be used to represent the coefficient vector of the objective function. A and G can be used to represent the constraint coefficient matrix. This can correspond to equality constraints. This can correspond to inequality constraints. and These can be used to represent the corresponding constraint limits. `m` and `l` can represent the number of equality constraints and the number of inequality constraints, respectively. It is understood that only the above-mentioned equality constraints or inequality constraints are allowed; the subsequent calculation method can be adjusted accordingly.

[0037] The inequality constraints above can be relaxed using the slack variable ξ, decomposing the linear programming problem into K subproblems, where k represents the identifier of each subproblem subproblem. The decomposed linear programming problem can be expressed as:

[0038]

[0039]

[0040]

[0041] Where, x k ξ and ξ can be used to represent the vector formed by the optimization variables of each subproblem segment and the vector formed by the relaxation variables corresponding to each inequality constraint, respectively. k A k and G k These can be used to represent the coefficient vector, equality constraint coefficient matrix, and inequality constraint coefficient matrix of the objective function corresponding to the k-th subproblem segment, respectively.

[0042] Accordingly, the augmented Lagrangian form of the decomposed linear programming problem can be expressed as:

[0043]

[0044] Here, ρ can be used to represent the penalty parameter, and ρ is usually greater than 0. λ and μ can be used to represent the vectors formed by the dual variables corresponding to each equality constraint and the vectors formed by the dual variables corresponding to each inequality constraint, respectively. It can be used to represent the sum of squares of the elements in a vector. The meanings of the other symbols are as described above.

[0045] Optionally, to ensure algorithm convergence, a proximal term can be added to the Lagrangian function during the iterative solution of each subproblem piece, denoted as: in, In one example It can represent hyperparameters greater than 0. This can be used to represent the optimization variable value of the previous iteration corresponding to the k-th subproblem slice. The meanings of the other symbols are as described above.

[0046] Optionally, in one example, the distributed computing system described above may include at least two parameter nodes, and the data stored on these two parameter nodes can be mutually backed up. For a computing node used to solve the k-th subproblem partition, the optimization variable value from the previous iteration can be obtained from either of the at least two parameter nodes (e.g., x). t (representation) and dual variable values ​​(e.g., can be represented by λ) t ,μ t (Representation). It can be understood that when inequality constraints exist, the values ​​of the slack variables from the previous iteration can also be obtained (for example, using ξ). t express).

[0047] Optionally, see [link to relevant documentation] Figure 3 , Figure 3 A flowchart illustrating an example of a process 300 for obtaining the values ​​of the optimization variable and the dual variable in the previous iteration according to an embodiment of this specification is shown.

[0048] In 310, select the target parameter node from at least two parameter nodes.

[0049] In one example, the target parameter node can be randomly selected. In another example, a relatively idle parameter node can be identified as the target parameter node based on the load situation.

[0050] Optionally, the parameter node that matches the identifier of the subproblem piece corresponding to the computation node among at least two parameter nodes can be identified as the target parameter node.

[0051] In one example, if there are two parameter nodes, the target parameter node can be determined by the parity of the identifier of the subproblem partition corresponding to the computation node (e.g., represented by k). In another example, if there are multiple parameter nodes, the target parameter node can be determined by taking the remainder of the number of parameter nodes with respect to the identifier of the subproblem partition corresponding to the computation node (e.g., represented by k).

[0052] In step 320, the optimization variable values ​​and dual variable values ​​from the target parameter node are obtained from the previous iteration for solving the subproblem slice corresponding to that computation node.

[0053] Based on this, this solution sets at least two parameter nodes and selects one to obtain the required optimized variable value and dual variable value from the previous iteration. This allows for a more balanced distribution of a large number of requests arriving within a similar time period while ensuring the stability of the parameter nodes in providing services to the outside world. This avoids excessive load on a single parameter node, which could cause response delays or failures, thereby improving the overall availability of the system.

[0054] In step 220, the subproblem is solved based on the obtained optimization variable values ​​and dual variable values ​​from the previous iteration and the pre-obtained constraint coefficient matrix, to obtain the optimization variable values ​​for the current iteration corresponding to the subproblem.

[0055] In one example, for the computation node used to solve the k-th subproblem slice, the optimization variable value for the current iteration of that subproblem (e.g., x) k t+1 (This can be obtained in the following ways:) In one example, x k t+1 It can also be obtained in the following ways: In one example, x k t+1 It can also be obtained in the following ways: in, g k t ·x k It can be used to indicate g k t and x k The inner product between them. The meanings of the remaining symbols are as described above.

[0056] In step 230, the corresponding constraint calculation value is determined based on the optimization variable value of the current iteration corresponding to the subproblem segment and the constraint coefficient matrix corresponding to the subproblem segment.

[0057] In one example, the constraint calculation value can be used to indicate the optimization variable value for the current iteration corresponding to the subproblem slice (e.g., x). k t+1 (represented by A) and the corresponding constraint coefficient matrix (e.g., can be represented by A) k G k The product of (represented by A). In one example, for a computation node used to solve the k-th subproblem piece, the calculated constraint value corresponding to that subproblem piece may include A. k x k t+1 and G k x k t+1 .

[0058] Optionally, continue to refer to Figure 4 , Figure 4 A schematic diagram illustrating an example of a process 400 for obtaining the constraint coefficient matrix corresponding to a subproblem piece according to an embodiment of this specification is shown.

[0059] like Figure 4As shown, the parameter node stores the constraint coefficient matrix A corresponding to the decomposable problem. k G k It is possible Figure 4 As shown in 410 and 420, for each computation node used to solve the partitions of the 1st, 2nd, ..., kth subproblems, the constraint coefficient matrix A can be used as a reference. k G k Extract the portion that matches the identifier (e.g., 1, 2, ..., k) of the subproblem piece corresponding to the computation node as the constraint coefficient matrix corresponding to that subproblem piece, such as A1, A k G1, G k .

[0060] At 240, the calculated constraint values ​​are sent to the parameter node.

[0061] In one example, the calculated constraint values ​​can be sent to the parameter node. Alternatively, the calculated optimization variable values ​​for the current iteration can also be sent to the parameter node.

[0062] Optionally, when at least two parameter nodes exist, the calculated constraint values ​​can be sent to either of the at least two parameter nodes, which then synchronize via a synchronization mechanism. Alternatively, the calculated optimization variable values ​​for the current iteration can also be sent to the at least two parameter nodes. Alternatively, the calculated constraint values ​​can be sent to each of the at least two parameter nodes separately. This allows for mutual backup between at least two parameter nodes through a simple architecture.

[0063] In step 250, the optimization variable values ​​for the current iteration round and the constraint calculation values ​​for each subproblem piece are aggregated and saved respectively.

[0064] In one example, the optimization variable value x for the current iteration corresponding to each subproblem can be divided into pieces. k t+1 Arrange the subproblems in the order they were partitioned to obtain the aggregated optimization variable value x for the current iteration round. t+1 The aggregated constraint calculation value for the current iteration can be expressed as: and

[0065] In 260, the dual variable value of the current iteration is determined based on the aggregated optimization variable value of the current iteration, the dual variable value of the previous iteration, and the aggregated constraint calculation value.

[0066] In one example, the dual variable value λ of the current iteration round t+1 and μ t+1 It can be obtained in the following ways: In one example, the slack variable ξ for the current iteration corresponding to the inequality constraint. t+1 Obtained through the following method: ξ t+1 =argmin ξ L ρ (x t+1 ,ξ,λ t ,μ t ).

[0067] At 270, store the dual variable values ​​for the current iteration round as determined.

[0068] Optionally, if at least two parameter nodes are in an abnormal state, the other parameter nodes in a normal state shall take over the subsequent services. In one example, if steps 250-270 as described above were originally performed by two parameter nodes, and one of the parameter nodes becomes abnormal and unable to continue providing the relevant services, then the other parameter node shall take over to provide the aforementioned services.

[0069] See below. Figure 5 , Figure 5 A schematic diagram illustrating an example application scenario 500 of the method for solving decomposable problems based on a distributed computing system according to an embodiment of this specification.

[0070] like Figure 5 As shown in section 510, computation node 1 and computation node 2 obtain the optimization variable values ​​and dual variable values ​​from parameter node 1 for solving the subproblem piece corresponding to their respective computation node in the previous iteration round, while computation node 3 obtains the optimization variable values ​​and dual variable values ​​from parameter node 2 for solving the subproblem piece corresponding to its respective computation node. Each computation node sends its calculated intermediate results to parameter node 1 and parameter node 2, respectively. The computation control node obtains the aggregated optimization variable values ​​and the determined dual variable values ​​for the current iteration round from parameter node 1, and sends the calculated constraint residuals to parameter node 1 and parameter node 2, respectively. In section 520, if parameter node 1 experiences a state anomaly, computation node 1, computation node 2, and computation node 3 all obtain the optimization variable values ​​and dual variable values ​​from parameter node 2 for solving the subproblem piece corresponding to their respective computation node. During the period when parameter node 1 experiences a state anomaly, each computation node can send its calculated intermediate results to parameter node 2. The calculation control node obtains the aggregated optimization variable values ​​and the determined dual variable values ​​of the current iteration from parameter node 2, and sends the calculated constraint residuals to parameter node 2.

[0071] Based on this, this solution can achieve backup of critical nodes through at least two parameter nodes, which helps to improve the stability of the entire system.

[0072] In 280, the constraint residuals are determined based on the aggregated optimization variable values ​​of the current iteration round and the determined dual variable values ​​of the current iteration round.

[0073] In one example, the constraint residuals may include at least one of equality constraint residuals and inequality constraint residuals. The individual equality constraint residuals can be expressed, for example, as follows: Among them, b i It can be used to represent the constraint limit value corresponding to the i-th equation. A ki , can be used to represent the i-th row of the equality constraint matrix corresponding to the k-th subproblem segment. Correspondingly, the residuals of each inequality constraint can be expressed, for example, as: Among them, h j and ξ j It can be used to represent the constraint limit and slack variable corresponding to the j-th inequality. kj , can be used to represent the j-th row of the inequality constraint matrix corresponding to the k-th subproblem segment.

[0074] At 290, determine whether to continue iterating based on the constraint residuals.

[0075] In this embodiment, the decision to continue iteration can be determined based on the constraint residuals using algorithms such as the Primal-Dual Hybrid Gradient (PDHG) and the Alternating Direction Method of Multipliers (ADMM). If iteration continues, steps 210-240 above are executed through each computation node. If iteration stops, the optimization variable values ​​for the current iteration corresponding to each subproblem piece stored in the parameter node are determined as the solution result of the decomposable problem.

[0076] See below. Figure 6 , Figure 6 This diagram illustrates yet another example of an application scenario 600 of the method for solving decomposable problems based on a distributed computing system according to an embodiment of this specification.

[0077] like Figure 6 As shown, parameter node 610 can be used to monitor whether the state of the aforementioned computation control node 620 is abnormal. If abnormal, parameter node 610 can record the current iteration round (e.g., "t+1" above) and start a new computation control node 630 so that the new computation control node 630 re-executes the computation of the current iteration round (e.g., determining the constraint residual and determining whether to continue iterating based on the constraint residual).

[0078] In one example, the computing control node, parameter nodes, and multiple computing nodes included in the above distributed computing system can serve as workload nodes (distinct from the master node) within the Kubernetes framework. Parameter node 610 can use the Kubernetes framework to launch a new computing control node 630.

[0079] Based on this, this solution can effectively address the risk of abnormalities in the computing control node, thereby enabling failover.

[0080] See below. Figure 7 , Figure 7 A schematic diagram illustrates yet another example of an application scenario 700 of the method for solving decomposable problems based on a distributed computing system according to an embodiment of this specification.

[0081] like Figure 7 As shown, parameter node 710 can be used to monitor whether the status of the above-mentioned multiple computing nodes is abnormal. If an abnormality is found, parameter node 710 can record the computing node with the abnormal status (e.g., Figure 7 The identifier of the subproblem partition corresponding to 720 shown (e.g., k=3) and the current iteration round (e.g., "t+1" above), and the determination of alternative computation nodes (e.g. Figure 7 As shown in 730), the aforementioned alternative computing node re-executes the relevant calculations of the subproblem in the current iteration round (e.g., calculating the optimization variable values ​​of the current iteration round corresponding to the subproblem piece and determining the constraint calculation values ​​of the current iteration round corresponding to the subproblem piece).

[0082] Based on this, this solution can effectively address the risk of computing node anomalies, thereby enabling failover.

[0083] See below. Figure 8 , Figure 8 A schematic diagram illustrates another example of an application scenario 800 of the method for solving decomposable problems based on a distributed computing system according to an embodiment of this specification.

[0084] In one example, the computing control node, parameter nodes, and multiple computing nodes included in the above distributed computing system can serve as workload nodes (distinct from the master node) within the Kubernetes framework. For example... Figure 8 As shown, workload nodes can each run their own corresponding workload container group (pod) (e.g.) Figure 8 (e.g., 810, 820, etc. shown). Within each compute node running the corresponding load container group, virtual compute nodes (virtual workers, such as...) can be created based on multiple processes (executors). Figure 8(e.g., 811, 812, 813, etc. shown in the diagram). If the parameter node 830 running the corresponding load container group detects an abnormal status of a compute node, it can determine whether there are any idle compute nodes among the current virtual compute nodes. If so, the parameter node 830 can determine a replacement compute node from the idle compute nodes. If not, the parameter node 830 can launch a new workload node (e.g., ...). Figure 8 (as shown in 840). In one example, similar to workload node 810, multiple virtual compute nodes can run on the aforementioned new workload node. Parameter node 830 can determine the alternative compute node from among the virtual compute nodes running on the aforementioned new workload node.

[0085] Based on this, this solution can control the number of machines simultaneously launched to solve the decomposable problem through a multi-process running mode, avoiding slow processing caused by a large number of machines launched at the same time. Moreover, since all virtual computing nodes corresponding to a workload node are launched when a workload node is launched, a considerable number of idle virtual computing nodes can be reserved (for example, if the original decomposable problem is divided into 398 subproblem pieces, and each of the 100 launched machines runs 4 virtual computing nodes, there are 2 idle virtual computing nodes that have not been assigned to subproblems), thereby saving the time required to launch machines again during failover.

[0086] use Figures 1-8 The method disclosed in this paper for solving decomposable problems based on a distributed computing system can perform the computation of each subproblem segment by computing nodes, aggregate and save the computation results obtained by each computing node by parameter nodes, and determine the constraint residuals of the decomposable problem by computing control nodes based on the aggregation results and determine whether to continue iterative computation accordingly. This provides a more stable and reliable method for solving decomposable problems suitable for large-scale data.

[0087] Continue to refer to Figure 9 , Figure 9 A block diagram of an example of a distributed computing system 900 for solving decomposable problems according to an embodiment of this specification is shown. This system embodiment can be compared with... Figures 2-8 The method embodiments shown correspond to those described.

[0088] like Figure 9 As shown, a distributed computing system 900 for solving decomposable problems may include a computing control node 910, a parameter node 920, and multiple computing nodes 930. Each of the multiple computing nodes is used to compute the subproblem fragments corresponding to the decomposable problem.

[0089] The computation control node 910 is configured to: determine the constraint residual based on the aggregated optimization variable values ​​of the current iteration round and the determined dual variable values ​​of the current iteration round, and determine whether to continue iterating based on the constraint residual. The operation of the computation control node 910 can be referenced above. Figure 2 The operation described in section 290-290.

[0090] Parameter node 920 includes: an aggregation unit 921, configured to aggregate and save the optimization variable values ​​and constraint calculation values ​​corresponding to the current iteration round for each subproblem piece; and a dual variable value storage unit 922, configured to determine the dual variable value of the current iteration round based on the aggregated optimization variable value of the current iteration round, the dual variable value of the previous iteration round, and the aggregated constraint calculation value, and save the determined dual variable value of the current iteration round. The above operations of parameter node 920 can be referenced above. Figure 2 The operation described is 250-270.

[0091] In one example, parameter node 920 further includes a node recovery unit 923, configured to monitor whether the state of the computation control node is abnormal. If abnormal, it records the current iteration round, starts a new computation control node, and enables the new computation control node to re-execute the computation of the current iteration round. The operation of node recovery unit 923 can be referred to above. Figure 6 The described operation.

[0092] In one example, the node recovery unit 923 is further configured to monitor whether the states of the plurality of computing nodes are abnormal. If abnormal, it records the identifier of the sub-problem slice corresponding to the computing node with the abnormal state and the current iteration round, determines a replacement computing node, and enables the replacement computing node to re-execute the relevant calculations of the sub-problem in the current iteration round. The above operation of the node recovery unit 923 can be referred to above. Figure 7 The described operation.

[0093] Optionally, in one example, the compute node includes virtual compute nodes running on a multi-process basis, with multiple virtual compute nodes running on the workload node. The node recovery unit 923 is further configured to determine whether there is an idle compute node among the current virtual compute nodes; if so, to determine a replacement compute node from the idle compute nodes; if not, to launch a new workload node, wherein multiple virtual compute nodes are running on the new workload node, and to determine a replacement compute node from the virtual compute nodes running on the new workload node. The above operations of the node recovery unit 923 can be referred to the above. Figure 8 The described operation.

[0094] In one example, the node recovery unit 923 is further configured to, if any of the at least two parameter nodes is in an abnormal state, have the other parameter nodes in a normal state take over the subsequent services. The operation of the node recovery unit 923 described above can be referenced. Figure 5 The described related operations.

[0095] The computation node 930 includes: an optimization variable value determination unit 931, configured to obtain the optimization variable values ​​and dual variable values ​​from the parameter node for solving the subproblem piece corresponding to the computation node in the previous iteration, and solve the subproblem based on the obtained optimization variable values ​​and dual variable values ​​from the previous iteration and a pre-obtained constraint coefficient matrix to obtain the optimization variable values ​​for the current iteration corresponding to the subproblem; and a sub-result sending unit 932, configured to determine the corresponding constraint calculation values ​​based on the optimization variable values ​​for the current iteration corresponding to the subproblem piece and the constraint coefficient matrix corresponding to the subproblem piece, and send the calculated constraint calculation values ​​to the parameter node. The above operations of the computation node 930 can be referred to the above. Figure 2 The operations described are 210-240.

[0096] In one example, the optimization variable value determination unit 931 is further configured to select a target parameter node from the at least two parameter nodes; and to obtain from the target parameter node the optimization variable values ​​and dual variable values ​​from the previous iteration used to solve the subproblem slice corresponding to that computation node. The above operations of the optimization variable value determination unit 931 can be referred to above. Figure 3 The operation described in 310-320.

[0097] In one example, the optimization variable value determination unit 931 is further configured to determine the parameter node that matches the identifier of the subproblem fragment corresponding to the computation node among the at least two parameter nodes as the target parameter node. The above operation of the optimization variable value determination unit 931 can be referred to the above. Figure 3 The operation of the optional implementation of 310 described.

[0098] In one example, the sub-result sending unit 932 is further configured to send the calculated constraint values ​​to the at least two parameter nodes respectively. The operation of the sub-result sending unit 932 can be referred to above. Figure 2 The operation of the optional implementation of 240 is described.

[0099] In one example, computation node 930 further includes a coefficient extraction unit 933, configured to extract, via each computation node, the portion of the constraint coefficient matrix corresponding to the decomposable problem stored in the parameter node that matches the identifier of the subproblem piece corresponding to that computation node, as the constraint coefficient matrix corresponding to that subproblem piece. The operation of the coefficient extraction unit 933 can be referred to above. Figure 4 The described operation.

[0100] Reference above Figures 1 to 9 An embodiment of a method for solving decomposable problems based on a distributed computing system according to embodiments of this specification, and an example block diagram of a distributed computing system for solving decomposable problems, are described.

[0101] The distributed computing system for solving decomposable problems described in this specification can be implemented in hardware, software, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of its host device reading the corresponding computer program instructions from memory into RAM and executing them. In the embodiments of this specification, the distributed computing system for solving decomposable problems can, for example, be implemented using electronic devices.

[0102] Figure 10 A schematic diagram of various devices 1000 in a distributed computing system for solving decomposable problems, according to an embodiment of this specification, is shown.

[0103] like Figure 10 As shown, the computational control device, parameter device, or computing device 1000 in a distributed computing system for solving decomposable problems may include at least one processor 1010, a memory (e.g., non-volatile memory) 1020, a main memory 1030, and a communication interface 1040, and the at least one processor 1010, memory 1020, main memory 1030, and communication interface 1040 are connected together via a bus 1050. At least one processor 1010 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0104] In one embodiment, computer-executable instructions are stored in memory, which, when executed, cause at least one processor 1010 to: obtain from a parameter node the optimization variable values ​​and dual variable values ​​of the previous iteration for solving a subproblem piece corresponding to that computing node; solve the subproblem based on the obtained optimization variable values ​​and dual variable values ​​of the previous iteration and a pre-obtained constraint coefficient matrix to obtain the optimization variable values ​​of the current iteration corresponding to the subproblem; determine the corresponding constraint calculation values ​​based on the optimization variable values ​​of the current iteration corresponding to the subproblem piece and the constraint coefficient matrix corresponding to the subproblem piece; and calculate the constraint calculation values. The value is sent to the parameter node; or, via the parameter node, the optimization variable values ​​and constraint calculation values ​​corresponding to the current iteration round for each subproblem piece are aggregated and saved respectively, and the dual variable value of the current iteration round is determined based on the aggregated optimization variable value of the current iteration round, the dual variable value of the previous iteration round, and the aggregated constraint calculation value; the determined dual variable value of the current iteration round is saved; or, via the calculation control node, the constraint residual is determined based on the aggregated optimization variable value of the current iteration round and the determined dual variable value of the current iteration round, and whether to continue iterating is determined based on the constraint residual.

[0105] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1010 to perform the above-described combinations in the various embodiments of this specification. Figure 1-8 The description includes various operations and functions.

[0106] According to one embodiment, a program product, such as a computer-readable medium, is provided. The computer-readable medium may have instructions (i.e., the elements implemented in software as described above), which, when executed by a computer, cause the computer to perform the above-described combinations of the various embodiments of this specification. Figure 1-8 The description includes various operations and functions.

[0107] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0108] In this case, the program code itself, which can be read from a readable medium, can perform the functions of any of the above embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0109] The computer program code required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB, .NET, and Python; conventional procedural programming languages ​​such as C, Visual Basic 2003, Perl, COBOL 2002, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service, such as Software as a Service (SaaS).

[0110] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0111] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0112] Not all steps and units in the above process and system structure diagrams are mandatory; some steps or units can be omitted as needed. The execution order of each step is not fixed and can be determined as required. The device structure described in the above embodiments can be a physical structure or a logical structure. That is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0113] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" over other embodiments. Detailed descriptions are included for the purpose of providing an understanding of the described techniques. However, these techniques may be practiced without these detailed descriptions. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0114] The optional embodiments of the present specification have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present specification are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present specification, various simple modifications can be made to the technical solutions of the embodiments of the present specification, and these simple modifications all fall within the protection scope of the embodiments of the present specification.

[0115] The foregoing description of this specification is provided to enable any person skilled in the art to implement or use the content of this specification. Various modifications to the content of this specification will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of protection of this specification. Therefore, this specification is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for solving a decomposable problem based on a distributed computing system, the distributed computing system comprising a computing control node, parameter nodes, and multiple computing nodes, each of the multiple computing nodes being used to compute a subproblem partition corresponding to the decomposable problem, the method comprising: Through each computing node, the optimization variable values ​​and dual variable values ​​of the previous iteration for solving the subproblem piece corresponding to that computing node are obtained from the parameter node. The subproblem is solved based on the obtained optimization variable values ​​and dual variable values ​​of the previous iteration and the pre-obtained constraint coefficient matrix to obtain the optimization variable values ​​of the current iteration corresponding to the subproblem. The corresponding constraint calculation values ​​are determined based on the optimization variable values ​​of the current iteration corresponding to the subproblem piece and the constraint coefficient matrix corresponding to the subproblem piece. The calculated constraint values ​​are sent to the parameter node; Through the parameter nodes, the optimization variable values ​​and constraint calculation values ​​corresponding to the current iteration rounds of each subproblem piece are aggregated and saved respectively. The dual variable values ​​of the current iteration round are determined based on the aggregated optimization variable values ​​of the current iteration round, the dual variable values ​​of the previous iteration round, and the aggregated constraint calculation values. The determined dual variable values ​​of the current iteration round are saved. as well as The constraint residual is determined by the calculation control node based on the aggregated optimization variable value of the current iteration round and the determined dual variable value of the current iteration round, and the decision to continue iteration is made based on the constraint residual.

2. The method as described in claim 1, wherein, The distributed computing system includes at least two parameter nodes, and obtaining the optimization variable values ​​and dual variable values ​​from the parameter nodes for solving the subproblem partition corresponding to that computing node in the previous iteration includes: Select a target parameter node from the at least two parameter nodes; and Obtain the optimization variable values ​​and dual variable values ​​from the previous iteration of the target parameter node for solving the subproblem piece corresponding to that computation node. Sending the calculated constraint values ​​to the parameter node includes: The calculated constraint values ​​are sent to the at least two parameter nodes respectively.

3. The method as described in claim 2, wherein, Selecting the target parameter node from the at least two parameter nodes includes: The parameter node that matches the identifier of the subproblem segment corresponding to the computation node among the at least two parameter nodes is determined as the target parameter node.

4. The method of claim 1, wherein, The method further includes: Through each computing node, the part that matches the identifier of the subproblem segment corresponding to that computing node is extracted from the constraint coefficient matrix corresponding to the decomposable problem stored in the parameter node, and used as the constraint coefficient matrix corresponding to that subproblem segment.

5. The method of claim 1, wherein, The method further includes: The parameter node monitors whether the state of the computation control node is abnormal. If it is abnormal, the current iteration round is recorded, and a new computation control node is started so that the new computation control node re-executes the computation of the current iteration round.

6. The method of claim 1, wherein, The method further includes: The parameter node is used to monitor whether the status of the multiple computing nodes is abnormal. If abnormal, the identifier of the subproblem slice and the current iteration round corresponding to the computing node with abnormal status are recorded, and a replacement computing node is determined so that the replacement computing node can re-execute the relevant calculations of the subproblem in the current iteration round.

7. The method of claim 5, wherein, Compute nodes include virtual compute nodes that run on a multi-process basis, with multiple virtual compute nodes running on workload nodes. The determination of alternative computing nodes includes: Determine if there are any idle computing nodes among the current virtual computing nodes; If an alternative computing node exists, determine it from the available computing nodes; If it does not exist, a new workload node is started, wherein multiple virtual computing nodes are running on the new workload node, and a replacement computing node is determined from the virtual computing nodes running on the new workload node.

8. The method of claim 2, wherein, The method further includes: If any of the at least two parameter nodes is in an abnormal state, the other parameter nodes in a normal state shall take over the subsequent services.

9. A distributed computing system for solving decomposable problems, the distributed computing system comprising a computing control node, parameter nodes, and multiple computing nodes, each of the multiple computing nodes being used to compute a subproblem segment corresponding to the decomposable problem. The computing nodes include: The optimization variable value determination unit is configured to obtain the optimization variable value and dual variable value of the previous iteration for solving the subproblem piece corresponding to the computing node from the parameter node, and solve the subproblem based on the obtained optimization variable value and dual variable value of the previous iteration and the pre-obtained constraint coefficient matrix to obtain the optimization variable value of the current iteration corresponding to the subproblem piece. The sub-result sending unit is configured to determine the corresponding constraint calculation value based on the optimization variable value of the current iteration round corresponding to the sub-problem segment and the constraint coefficient matrix corresponding to the sub-problem segment, and send the calculated constraint calculation value to the parameter node. The parameter node includes: an aggregation unit, configured to aggregate and save the optimization variable values ​​and constraint calculation values ​​corresponding to the current iteration round for each subproblem piece; and a dual variable value saving unit, configured to determine the dual variable value of the current iteration round based on the aggregated optimization variable value of the current iteration round, the dual variable value of the previous iteration round, and the aggregated constraint calculation value, and save the determined dual variable value of the current iteration round. The computation control node is configured to determine the constraint residual based on the aggregated optimized variable value of the current iteration round and the determined dual variable value of the current iteration round, and to determine whether to continue iterating based on the constraint residual.

10. A distributed computing system for solving decomposable problems, the distributed computing system comprising a computing control device, a parameter device, and a plurality of computing devices, each of the plurality of computing devices being used to compute a subproblem slice corresponding to the decomposable problem, the computing control device, the parameter device, and the plurality of computing devices each comprising at least one processor, a memory coupled to the at least one processor, and a computer program stored in the memory, the at least one processor executing the computer program to implement a corresponding operation in the method for solving decomposable problems based on a distributed computing system as described in any one of claims 1 to 8.

Images