A distributed double-layer optimization method and device for a time-varying directed graph, a terminal and a medium

Abstract

This invention discloses a method, apparatus, terminal, and medium for distributed bi-level optimization on time-varying directed graphs, comprising: acquiring a time-varying directed graph sequence; solving a bi-level optimization problem for a multi-agent system based on the time-varying directed graph sequence; updating the decision variables of each agent using a row random matrix and calculating the local gradient of the decision variables of each agent using a penalty function; updating the gradient tracking variables of each agent using a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence; and outputting the bi-level optimization results for the multi-agent system. This invention solves the problem of bi-level optimization failure under dynamic topology, eliminates the risk of numerical instability, overcomes the second-order computational bottleneck, and reduces resource overhead.

Description

Technical Field

[0001] This invention relates to the field of distributed optimization technology, and in particular to a distributed two-layer optimization method, apparatus, terminal and medium for time-varying directed graphs. Background Technology

[0002] Distributed optimization techniques minimize the global objective function through collaborative local neighbor communication among multiple agents, avoiding dependence on a central coordinator and the sharing of raw privacy data. They have wide applications in multi-agent learning systems, satellite networks, and drone swarms. In these real-world scenarios, communication networks typically exhibit time-varying directed topologies, which presents significant challenges such as data heterogeneity, intermittent link interruptions, and asymmetric information flow.

[0003] For distributed optimization problems, early static undirected graph methods (such as the EXTRA method, Exact-Diffusion method, and gradient tracking method) often fail in dynamic directed environments due to their inability to eliminate consensus bias caused by directed communication. To address directed graph problems, researchers have developed algorithms based on the Push-Sum mechanism (such as the Subgradient-Push algorithm and Push-DIGing algorithm), which utilize asymmetric communication protocols to handle directed connections. However, in dynamic topologies, their reliance on a single push-based weight matrix leads to low mixing efficiency, limiting convergence speed. Although the latest Push-Pull (AB) communication mechanism effectively improves convergence performance by using two independent matrices for pushing and pulling information respectively, existing research on Push-Pull extensions for time-varying directed networks is mainly limited to strongly convex objective functions, and most works focus only on single-layer optimization problems.

[0004] However, many key machine learning tasks (such as hyperparameter optimization, meta-learning, and reinforcement learning) inherently possess a nested structure of bi-layer optimization. Current research on bi-layer optimization mainly focuses on centralized settings or static network environments. Achieving fully distributed bi-layer optimization on time-varying directed graphs remains in the exploratory stage, facing multiple technical bottlenecks: first, how to effectively correct topological deviations caused by dynamic unbalanced communication; second, bi-layer optimization typically involves second-order Hessian matrix approximation, resulting in huge computational overhead and making it difficult to deploy on resource-constrained distributed nodes.

[0005] Therefore, existing technologies still need improvement. Summary of the Invention

[0006] The technical problem to be solved by this invention is to address the shortcomings of existing technologies by providing a distributed two-layer optimization method, device, terminal, and medium for time-varying directed graphs, thereby solving the problems of two-layer optimization failure and high computational resource requirements under dynamic topology.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows: In a first aspect, the present invention provides a distributed two-layer optimization method for time-varying directed graphs, comprising: Obtain the time-varying directed graph sequence; Based on the time-varying directed graph sequence, solve the bi-level optimization problem of multiple agent systems; The decision variables of each agent are updated by a row random matrix, and the local gradient of the decision variables of each agent is calculated using a penalty function; The gradient tracking variable of each agent is updated by a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence; Output the two-level optimization results of multiple intelligent agent systems.

[0008] In one implementation, the step of solving the bi-level optimization problem of a multi-agent system based on the time-varying directed graph sequence includes, prior to: Set the number of iterations Penalty parameters and learning rate ; Input initial variables Row random matrix and column random matrix ; Calculate the initial gradient .

[0009] In one implementation, solving the bi-level optimization problem of a multi-agent system based on the time-varying directed graph sequence includes: Based on auxiliary variables and penalty parameters, the upper-level target will be included. and lower-level constraints The bi-level optimization problem is transformed into a single-level minima-maximum problem: ; in, For the auxiliary variable; Let be the penalty parameter.

[0010] In one implementation, in the single-layer minimax problem, each agent... The local penalty function is defined as: .

[0011] In one implementation, updating the decision variables of each agent via a row random matrix includes: For each intelligent agent Using the row random matrix Obtain status information from the first neighbor and update its own decision variables. : ; in, Represents the gradient tracking variable. This represents the learning rate.

[0012] In one implementation, calculating the local gradient of the decision variables for each agent using a penalty function includes: For each intelligent agent The gradient of the local penalty function is calculated based on the latest state:

[0013] in, Represents each intelligent agent Update its own decision variables The latest status.

[0014] In one implementation, updating the gradient tracking variable of each agent using a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence includes: For each intelligent agent Using column random matrices Push the state information to the second neighbor and update the gradient tracking variables: ; in, This represents the gradient tracking variable.

[0015] In one implementation, the method further includes: A decentralized distributed two-layer optimization method is used to solve the two-layer optimization problem of the multiple intelligent agent system. Alternatively, a distributed two-level optimization method based on a second-order approximation algorithm can be used to solve the two-level optimization problem of the multiple intelligent agent system.

[0016] In one implementation, the decentralized distributed two-level optimization method for solving the two-level optimization problem of the multiple intelligent agent system includes: For each intelligent agent The weight variables are updated using the same communication topology and column random update rules as the decision variables. Divide the updated decision variable by its corresponding weight variable to offset the homologous topological distortion factors in the numerator and denominator; The penalty function is used to calculate the local gradient of the decision variables for each agent; The gradient tracking variable of each agent is updated by the column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence.

[0017] In one implementation, the distributed two-level optimization method based on a second-order approximation algorithm solves the two-level optimization problem of the multiple intelligent agent system, including: By introducing auxiliary variables The inverse vector product of the approximate Hessian matrix is ​​used to calculate the local gradient of the decision variable for each agent. The gradient tracking variable of each agent is updated through the column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence.

[0018] Secondly, the present invention provides a distributed two-layer optimization device for time-varying directed graphs, comprising: The directed graph acquisition module is used to acquire time-varying directed graph sequences; A two-level optimization problem module is used to solve the two-level optimization problem of a multi-agent system based on the time-varying directed graph sequence. The decision variable update module is used to update the decision variables of each agent through a row random matrix. The local gradient calculation module is used to calculate the local gradient of the decision variables of each agent using a penalty function; The gradient tracking variable update module is used to update the gradient tracking variable of each agent through a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence. The optimization result output module is used to output the two-level optimization results of multiple intelligent agent systems.

[0019] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a distributed bi-level optimization program for time-varying directed graphs, and the distributed bi-level optimization program for time-varying directed graphs is executed by the processor to implement the operation of the distributed bi-level optimization method for time-varying directed graphs as described in the first aspect.

[0020] Fourthly, the present invention also provides a medium, which is a computer-readable storage medium, storing a distributed two-level optimization program for time-varying directed graphs, which, when executed by a processor, is used to implement the operation of the distributed two-level optimization method for time-varying directed graphs as described in the first aspect.

[0021] The present invention, by employing the above technical solution, has the following effects: This invention introduces row-based random matrices for the "pull" operation on decision variables and column-based random matrices for the "push" operation on gradient tracking variables. State consensus is achieved through the row-based random matrices, and gradient estimation bias caused by directed graphs is eliminated through the column-based random matrices. Furthermore, the penalty function method transforms the bi-level optimization problem into a single-level minima-maximum problem, thus avoiding the calculation of second derivatives and achieving convergence using only first-order gradients. This invention solves the failure problem of bi-level optimization under dynamic topology, eliminates the risk of numerical instability, overcomes the second-order computational bottleneck, and reduces resource consumption. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a distributed two-layer optimization method for time-varying directed graphs in one implementation of the present invention.

[0024] Figure 2 This is a schematic diagram of Push-Pull information interaction under a dynamic time-varying directed graph in one implementation of the present invention.

[0025] Figure 3 This is a comparison of the test accuracy and regularization parameters between a single-layer benchmark algorithm and the FAB algorithm of this invention. A schematic diagram of the change curve.

[0026] Figure 4 This diagram illustrates the performance comparison of data cleaning in a distributed IMDB sentiment analysis task based on the BERT model.

[0027] Figure 5 This is a schematic diagram illustrating the performance ablation comparison between the FAB algorithm of this invention and four algorithm variants in a multi-agent reinforcement learning task.

[0028] Figure 6 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0029] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0031] Exemplary methods For distributed optimization problems, early static undirected graph methods (such as the EXTRA method, Exact-Diffusion method, and gradient tracking method) often fail in dynamic directed environments due to their inability to eliminate consensus bias caused by directed communication. To address directed graph problems, researchers have developed algorithms based on the Push-Sum mechanism (such as the Subgradient-Push algorithm and Push-DIGing algorithm), which utilize asymmetric communication protocols to handle directed connections. However, in dynamic topologies, their reliance on a single push-based weight matrix leads to low mixing efficiency, limiting convergence speed. Although the latest Push-Pull (AB) communication mechanism effectively improves convergence performance by using two independent matrices for pushing and pulling information respectively, existing research on Push-Pull extensions for time-varying directed networks is mainly limited to strongly convex objective functions, and most works focus only on single-layer optimization problems.

[0032] However, many key machine learning tasks (such as hyperparameter optimization, meta-learning, and reinforcement learning) inherently possess a nested structure of bi-layer optimization. Current research on bi-layer optimization mainly focuses on centralized settings or static network environments. Achieving fully distributed bi-layer optimization on time-varying directed graphs remains in the exploratory stage, facing multiple technical bottlenecks: first, how to effectively correct topological deviations caused by dynamic unbalanced communication; second, bi-layer optimization typically involves second-order Hessian matrix approximation, resulting in huge computational overhead and making it difficult to deploy on resource-constrained distributed nodes.

[0033] For time-varying directed graph scenarios, the most popular single-layer algorithms are Push-Sum and Push-Pull (also known as AB algorithms). However, existing two-layer optimization algorithms require bidirectional interaction to construct a double random matrix in a distributed environment, making them unsuitable for direct networks that only support unidirectional communication. Therefore, existing technical solutions suffer from the following significant drawbacks: 1. Single-layer optimization algorithms cannot handle nested tasks and have low tuning efficiency; existing distributed algorithms are all designed for single-layer optimization. When faced with tasks that inherently have a double-layer nested structure, such as hyperparameter optimization, meta-learning, or reinforcement learning, single-layer algorithms cannot automatically optimize upper-layer variables (such as regularization parameters). Users are forced to use brute-force search methods such as grid search to find the optimal hyperparameters. This leads to extremely wasteful computational resources and makes it difficult to find the global optimum when the parameter space is large. Experiments show that its convergence accuracy is much lower than that of double-layer optimization algorithms.

[0034] 2. Existing two-layer optimization algorithms are ill-suited for time-varying directed networks. Although some distributed two-layer optimization algorithms exist in academia, most assume that the communication network is static or undirected. In time-varying directed graphs, the network topology changes continuously, and the lack of time-invariant eigenvectors causes traditional consensus mechanisms to fail, resulting in severe consensus bias or drift. Directly applying existing two-layer algorithms to dynamic networks can lead to models failing to converge or diverging.

[0035] 3. High computational complexity and difficult deployment (reliance on second-order information): Existing two-layer optimization methods heavily rely on second-order derivative information (such as the calculation of the Hessian matrix and its inverse) to compute hypergradients. In optimization problems involving high-dimensional parameter spaces such as deep neural networks, their computational complexity and storage overhead increase exponentially. Furthermore, in distributed computing environments, frequent and extensive data exchanges between nodes are necessary to approximate global second-order information, leading to a surge in communication load. These dual bottlenecks of computation and communication make it difficult to effectively deploy existing algorithms on edge computing devices with limited computing power and bandwidth.

[0036] To address the aforementioned technical problems, this embodiment provides a distributed bi-level optimization method for time-varying directed graphs, comprising: acquiring a time-varying directed graph sequence; solving a bi-level optimization problem for a multi-agent system based on the time-varying directed graph sequence; updating the decision variables of each agent using a row random matrix and calculating the local gradient of the decision variables of each agent using a penalty function; updating the gradient tracking variables of each agent using a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence; and outputting the bi-level optimization results for the multi-agent system. This invention solves the failure problem of bi-level optimization under dynamic topology, eliminates the risk of numerical instability, overcomes the second-order computational bottleneck, and reduces resource overhead.

[0037] like Figure 1 As shown, this embodiment of the invention provides a distributed two-layer optimization method for time-varying directed graphs, including the following steps: Step S100: Obtain the time-varying directed graph sequence.

[0038] In this embodiment, a time-varying directed graph distributed bilevel optimization method based on the first-order Push-Pull / AB algorithm (FAB algorithm for short) is proposed. This method aims to solve the bilevel optimization problem with nested structures in dynamically changing unbalanced directed networks.

[0039] The FAB algorithm is designed and developed based on the organic combination of the penalty function method and the Push-Pull (AB) communication strategy. Existing distributed two-layer optimization algorithms typically assume the network is static and undirected, or require the computation of complex second-order Hessian matrix information, resulting in huge computational costs and inability to adapt to dynamic networks. In time-varying directed graphs, traditional single-layer optimization algorithms (such as Push-Sum) often suffer from slow convergence speed and poor numerical stability (relying on division operations to eliminate weights). To address these issues, this invention introduces two independent time-varying hybrid matrices: a row random matrix... Sum of random matrices .in The "pull" operation is used for decision variables to achieve state consensus; The "push" operation used for gradient tracking variables eliminates gradient estimation bias caused by directed graphs. Furthermore, the penalty function method transforms the bi-level optimization problem into a single-level minima-maximum problem, thus avoiding the calculation of the second derivative and achieving convergence using only the first-order gradient.

[0040] In this embodiment, in order to implement the above-mentioned FAB algorithm, consider a... An agent in a time-varying directed graph sequence This is a decentralized bilevel optimization problem to be solved. Its objective is to minimize the global upper-level objective function. And it is limited by the lower-level variables. The global lower-level objective function must be minimized. This constraint. It is usually assumed that the lower-level objective function... Regarding lower-level variables yes - Strongly convex, specifically in the following form: ; Among them, intelligent agents The upper and lower level functions are defined as follows: ,here and These are the upper and lower level variables, respectively.

[0041] Regarding the above-mentioned An agent in a time-varying directed graph sequence The decentralized two-layer optimization problem is solved by constructing a time-varying directed network simulation environment containing 100 agents, obtaining the MNIST dataset with contaminated labels, and using the MNIST dataset as a time-varying directed graph sequence to solve the two-layer optimization problem of the multi-agent system.

[0042] like Figure 1 As shown, in one implementation of this invention, the distributed two-layer optimization method for time-varying directed graphs further includes the following steps: Step S200: Based on the time-varying directed graph sequence, solve the bi-level optimization problem of multiple intelligent agent systems.

[0043] In this embodiment, after obtaining the time-varying directed graph sequence, before solving the two-layer optimization problem of multiple intelligent agent systems, it is also necessary to configure the corresponding learning parameters and initialize the gradient parameters of the entire network.

[0044] Specifically, in one implementation of this embodiment, the following steps are included before step S200: Step S201a, set parameters: number of iterations Penalty parameters and learning rate ; Step S201b, Input: Initial variables Time-varying network topology: row random matrix and column random matrix ; Step S201c, Initialization: Calculate the initial gradient .

[0045] In this embodiment, the initial gradient is calculated during the initialization process in the following manner. : .

[0046] After performing the above parameter settings and initialization operations, the FAB algorithm of this embodiment is executed. First, the problem is reconstructed and the penalty function is transformed, transforming the original two-level optimization problem into a form suitable for distributed solution.

[0047] Specifically, in one implementation of this embodiment, step S200 includes the following steps: Step S201, based on auxiliary variables and penalty parameters, includes the upper-level target and lower-level constraints The bi-level optimization problem is transformed into a single-level minima-maximum problem: ; in, For the auxiliary variable; Let be the penalty parameter.

[0048] In this embodiment, in the single-layer minimax problem, each agent... The local penalty function is defined as: .

[0049] This embodiment transforms complex nested optimization into a single variable-specific optimization. The problem of finding saddle points.

[0050] like Figure 1 As shown, in one implementation of this invention, the distributed two-layer optimization method for time-varying directed graphs further includes the following steps: In step S300, the decision variables of each agent are updated by a row random matrix, and the local gradient of the decision variables of each agent is calculated using a penalty function.

[0051] In this embodiment, after transforming the original bi-level optimization problem into a form suitable for distributed solution, the first interaction step of the algorithm is executed: decision variable update (Pull Step).

[0052] Specifically, in one implementation of this embodiment, step S300 includes the following steps: Step S301, for each agent Using the row random matrix Obtain state information from the first neighbor (in-neighbor) and update its own decision variables. The calculation formula is as follows: ; in, Represents the gradient tracking variable. This represents the learning rate.

[0053] In this embodiment, the first step of the interaction mainly utilizes a matrix. The properties of the variables make the decision variables of each node tend to be consistent, eliminating the influence of row imbalance; after updating the decision variables, the local gradient is calculated.

[0054] Step S302, for each agent The gradient of the local penalty function is calculated based on the latest state, and the calculation formula is as follows: ; in, Represents each intelligent agent Update its own decision variables The latest status.

[0055] In the above calculation method, for auxiliary variables Since this is a maximization problem, the gradient direction is reversed (or considered as gradient ascent). This step only involves the calculation of the first derivative, and the computational cost is minimal.

[0056] like Figure 1 As shown, in one implementation of this invention, the distributed two-layer optimization method for time-varying directed graphs further includes the following steps: Step S400: Update the gradient tracking variable of each agent by using a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence; Step S500: Output the two-layer optimization results of multiple intelligent agent systems.

[0057] In this embodiment, after calculating the gradient of the local penalty function for each agent, a gradient tracking variable update (Push Step) is performed. This step is the most critical communication step in this embodiment, used for global information correction. Each agent utilizes a column random matrix. Push the information to the second neighbor (out-neighbors) and update the gradient tracking variable. .

[0058] Specifically, in one implementation of this embodiment, step S400 includes the following steps: Step S401, for each agent Using column random matrices The state information is pushed to the second neighbor, and the gradient tracking variable is updated. The calculation formula is as follows: ; in, This represents the gradient tracking variable.

[0059] In this embodiment, gradient tracking technology is used during the gradient tracking variable update process, namely, "new tracking value = neighbor weighted sum + new gradient - old gradient". This is achieved using a matrix. The randomness of the columns ensures that the gradient information of the entire network can be correctly tracked, eliminates the column imbalance of the directed graph, and does not require division operations like the Push-Sum algorithm, thus guaranteeing numerical stability.

[0060] like Figure 2 As shown, Figure 2 The diagram illustrates the Push-Pull information interaction in a dynamic time-varying directed graph, as shown in this embodiment. Figure 2Based on the above process, the complete calculation flow of the new algorithm (FAB) in this embodiment is as follows: Setting parameters: Number of iterations Penalty parameters Learning rate: ; Input: Initial variables Time-varying network topology: row random matrix Column random matrix ; Initialization: Calculate the initial gradient The following formula is used: ; for to ;do for each agent Parallel execution; do Step 1: Update decision variables (Pull); ; Step 2: Local gradient calculation; ; Step 3: Gradient tracking variable update (Push); ; end for end for Output: .

[0061] It is worth noting that the distributed two-level optimization method (FAB) based on time-varying directed graphs proposed in this embodiment of the invention is not limited to the preferred embodiments described above. Without departing from the concept of this embodiment, modifications or substitutions can be made to the communication protocol or gradient estimation algorithm, including at least: a distributed two-level optimization method based on the Push-Sum communication mechanism, and a distributed two-level optimization method combined with a second-order approximation algorithm.

[0062] In one implementation of this invention, the distributed two-layer optimization method for time-varying directed graphs further includes the following steps: Step S101: Solve the bi-level optimization problem of the multiple intelligent agent system using a decentralized distributed bi-level optimization method.

[0063] Specifically, in one implementation of this embodiment, step S101 includes the following steps: Step S101a, for each agent The weight variables are updated using the same communication topology and column random update rules as the decision variables. Step S101b: Divide the updated decision variable by its corresponding weight variable to offset the homologous topological distortion factors in the numerator and denominator. Step S101c: Calculate the local gradient of the decision variables of each agent using the penalty function; Step S101d: Update the gradient tracking variable of each agent using the column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence.

[0064] In this embodiment, alternative solution 1: distributed two-layer optimization based on the Push-Sum communication mechanism (PushSum_FAB).

[0065] The preferred embodiment (FAB) described above employs a Push-Pull (AB) communication architecture, which uses both row-randomized and column-randomized matrices. As an alternative, a Push-Sum communication protocol can be used for information exchange. In this alternative, the agent communicates using only column-randomized matrices, but requires maintaining an additional set of weight variables and correcting for biases through "ratio extraction."

[0066] Specifically: Since using only column random matrices can lead to uneven information reception among nodes, resulting in consensus bias, this scheme initializes corresponding scalar weight variables for each decision variable. In step 1 (decision variable update), the weight variables are forced to undergo the exact same communication topology and column random update rules as the decision variables, perfectly "replicating" and recording the degree of network imbalance distortion accumulated by the nodes; subsequently, in step 2 (bias correction), "ratio extraction" is performed, that is, the decision variables are divided by their corresponding weight variables (such as...). This ensures that the topological distortion factors that are homogeneous in the numerator and denominator are strictly canceled out, thereby extracting the unbiased global average state that is unaffected by network asymmetry, which is then used in step 3 to calculate the local gradient. The specific algorithm is as follows: Setting parameters: Number of iterations Penalty parameters Learning rate ; Input: Initial variables Initial weights Time-varying network topology: row random matrix ; Initialization: Calculate the initial gradient The following formula is used: ; for to ;do for each agent Parallel execution; do Step 1: Update decision variables; ; Weight update: ; Step 2: Deviation correction (ratio extraction); ; Step 3: Local gradient calculation; ; Step 4: Gradient tracking variable update (Push);

[0067] end for end for Output: .

[0068] Although the PushSum_FAB algorithm can utilize a one-sided communication matrix (only a random matrix is ​​listed) This achieves bi-level optimization on a directed graph, but it relies on the "ratio extraction" operation in step 2 (i.e. To eliminate consensus bias. In time-varying directed networks, weight variables... The value may fluctuate drastically with changes in network topology, and may even approach zero. In this situation, division operations can lead to severe numerical instability or even computational overflow, thereby compromising the convergence of the algorithm.

[0069] In contrast, the preferred embodiment of the present invention (FAB algorithm) adopts a Push-Pull (AB) architecture, which completely avoids such division operations, and therefore has stronger robustness in sparse or dynamic networks.

[0070] In one implementation of this invention, the distributed two-layer optimization method for time-varying directed graphs further includes the following steps: Step S102: Solve the bi-level optimization problem of the multiple intelligent agent system using a distributed bi-level optimization method based on a second-order approximation algorithm.

[0071] Specifically, in one implementation of this embodiment, step S102 includes the following steps: Step S102a, by introducing auxiliary variables The inverse vector product of the approximate Hessian matrix is ​​used to calculate the local gradient of the decision variable for each agent. The gradient tracking variable of each agent is updated through the column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence.

[0072] In this embodiment, alternative solution two: distributed two-layer optimization (PushPull_SOBA) combined with a second-order approximation algorithm.

[0073] As another alternative embodiment of the present invention, gradient estimation for two-layer optimization can be performed by combining a SOBA strategy while maintaining the Push-Pull (AB) communication architecture unchanged. This scheme does not employ a penalty function method, but instead introduces auxiliary variables. We iterate through the inverse vector product (HVP) of the approximate Hessian matrix to directly calculate the hypergradient. The specific technical solution is as follows: Setting parameters: Number of iterations Learning rate: ; Input: Initial variables Time-varying network topology: row random matrix , row random matrix ; Initialization: Calculate the initial gradient The formula is as follows: ; for to ;do for each agent Parallel execution; do Step 1: Update decision variables (Pull); ; Step 2: Local gradient calculation (HVP); ; Step 3: Gradient tracking variable update (Push); ; end for end for Output: .

[0074] Although the algorithm PushPull_SOBA introduces auxiliary variables While avoiding the issue of selecting penalty parameters, its core drawback lies in the need to calculate second-order derivative information in step 2. Specifically, this algorithm updates auxiliary variables... When calculating the hypergradient $d_x$, it is necessary to calculate the product of the Hessian matrix and the vector (HVP), which is the part of the local gradient calculation formula. and Based on the local gradient calculation logic in step 2 above, in order to obtain the auxiliary variables... Tracking direction It is necessary to calculate the product of the Hessian matrix of the lower-level objective function with respect to the lower-level variables and the auxiliary variables. Regarding upper-level variables hypergradient The specific form is This means that the algorithm must additionally calculate the product of the mixed partial derivative Hessian matrix and the auxiliary variables. After obtaining the aforementioned local directions that rely on expensive second-order computation, the algorithm utilizes a column random matrix in step 3 above to... and Inject global communication stream for gradient tracking correction and update unbiased tracking variables. and Finally, in step 1 of the next iteration, a row random matrix is ​​used to achieve state consensus, and auxiliary variables are completed along the tracked globally unbiased direction. and upper-level decision variables This represents a substantial update. For high-dimensional parametric models such as deep neural networks (e.g., BERT), the Hessian matrix has a huge dimension, and directly calculating or approximating the HVP will result in extremely high computational overhead and storage requirements.

[0075] In contrast, the preferred embodiment (FAB) of this invention employs a fully first-order computation strategy, involving only gradient calculation, and its computational complexity is linearly related to the parameter dimension. It is more suitable for large-scale deep learning tasks.

[0076] This invention transforms a complex bi-level optimization problem into a single-level minima-maximum problem using the penalty function method, and reduces the computational complexity to linear level by utilizing first-order gradient estimation, which involves only basic vector operations. Meanwhile, this invention constructs a dual-matrix communication mechanism based on the Push-Pull strategy, using row random matrices for state consensus and column random matrices for gradient tracking. Without performing the numerically unstable "ratio extraction (division)" operation, it overcomes the convergence problem in time-varying directed networks and ensures the robustness of the algorithm in environments with frequent changes in communication links.

[0077] The three algorithms provided in this invention can all be implemented in parallel on various heterogeneous computing devices (including but not limited to graphics processing units, tensor processing units, and dedicated neural network processors). The FAB scheme, in particular, effectively avoids the risk of memory overflow (OOM) during the training of large-scale deep neural networks (such as the Transformer model and its variants) by utilizing the low memory usage of all first-order computations. Specific applications implemented through this method, such as automated hyperparameter optimization, multi-agent reinforcement learning (MARL) policy evaluation, and data privacy cleaning, aim to provide a high-efficiency, universal, and proprietary underlying optimization engine for large-scale distributed intelligent systems.

[0078] To verify the feasibility of the technical solutions provided in the embodiments of the present invention, the following verification is carried out through corresponding experiments.

[0079] The embodiments of this invention have undergone systematic implementation and testing, demonstrating excellent computational performance and convergence results in distributed tasks across multiple fields, including computer vision (CV), natural language processing (NLP), and reinforcement learning (RL). This embodiment has completed algorithm deployment on NVIDIA GPU processors, verifying its feasibility and potential application.

[0080] 1. Distributed hyperparameter automated optimization experiment.

[0081] In this embodiment, a time-varying directed network simulation environment with 100 agents was constructed, and an automated hyperparameter optimization (HPO) experiment for a linear classification task was conducted on the MNIST dataset with contaminated labels. The experiment compared the FAB algorithm of this invention with classic single-layer distributed benchmark algorithms such as Subgradient-Push, Push-DIGing, and Push-Pull (comparison results are shown in the figure). Figure 3 (As shown).

[0082] Among them, the single-layer benchmark algorithm uses a grid search strategy to select fixed regularization parameters. The training and validation sets are merged for training; while the FAB algorithm utilizes a two-layer optimization architecture, using the clean validation set loss as the upper-layer objective and the noisy training set loss as the lower-layer objective, thus achieving control over the regularization parameter. The adaptive dynamic optimization is described below. Specifically, the data processing and training process of this experiment, combined with the steps of the FAB algorithm, is explained as follows: The MNIST dataset used in the experiment was obtained directly from public networks. In the preprocessing stage, the images were first flattened and normalized. Subsequently, to simulate low-quality data in a real distributed environment, noise was injected into the training set by randomly flipping the real labels at a specific contamination rate, while the validation set remained strictly clean and uncontaminated. The processed data was non-independently identically distributed (Non-IID) and assigned to 100 agents, with each agent receiving a fixed 25 samples (including 20 noisy training samples and 5 clean validation samples). In the training initialization phase of the two-layer architecture, each agent first loaded its locally assigned data and initialized its local model network weights (corresponding to lower-layer variables) and regularization hyperparameters (corresponding to upper-layer variables). In step 1 (Pull Variable Update), each agent receives the states of its neighboring nodes through a row-random network topology and aggregates consensus on its local model network weights and regularization hyperparameters. Next, in step 2 (Local Gradient Calculation), each agent calculates the lower-level gradients of the model network weights using its local contaminated training set, while simultaneously calculating the upper-level hypergradients of the regularization hyperparameters based strictly on performance feedback from its local clean validation set. This mechanism incorporates feedback from the clean validation set into the dynamic optimization of the hyperparameters. Finally, in step 3 (Push Gradient Tracking), each agent pushes the latest gradient information obtained above to its outgoing neighboring nodes via a column-random matrix to correct gradient estimation biases caused by the time-varying directed network. This process iterates until the model network weights converge, ultimately achieving adaptive dynamic optimization of the regularization parameters.

[0083] Experimental results show that at different label contamination rates ( and Under these conditions, the FAB algorithm achieved significantly higher peak test accuracy than the benchmark algorithm. This is because single-layer algorithms lack a dynamic feedback mechanism and cannot achieve the desired regularization effect. In contrast, the embodiments of this invention can adaptively adjust hyperparameters through a two-layer structure and alleviate overfitting caused by label noise, fully demonstrating its adaptability and robustness in handling complex optimization problems under time-varying directed topologies.

[0084] 2. Distributed data super-cleaning experiments for large-scale pre-trained models.

[0085] To further verify the potential of this invention in supporting large-scale deep neural network training, the FAB algorithm was applied to a sentiment analysis task based on the IMDB dataset. Distributed fine-tuning was performed on a BERT pre-trained model (bert-base-uncased) with approximately 110 million parameters, and the results were compared with a single-layer stochastic benchmark algorithm. This task was modeled as a distributed two-layer optimization (DBO) problem, aiming to guide the lower layer in solving a specific training objective on cleaned data by optimizing the data cleaning strategy (i.e., sample weighting parameters) in the upper layer. The specific training process, combined with the FAB algorithm steps, is explained below: First, each agent initializes its local BERT model parameters (corresponding to lower-level variables) and sample weighting parameters (corresponding to upper-level variables), and loads the assigned noisy IMDB training set and a small amount of clean validation set. In step 1 (Pull Variable Update), each agent receives the state of its neighbor nodes through a row-random network topology and performs consensus aggregation on its local BERT model parameters and sample weighting parameters. Next, in step 2 (Local Gradient Calculation), the agent calculates the lower-level gradient of the BERT model parameters using its local contaminated training set, and simultaneously calculates the upper-level hypergradient of the sample weighting parameters based strictly on performance feedback from the local clean validation set. This mechanism automatically identifies and reduces the weight of noisy samples. Finally, in step 3 (Push Gradient Tracking), each agent pushes the gradient information calculated above to its outgoing neighbor nodes through a column-random matrix to correct the global bias caused by the time-varying directed network. This process is iterated until robust fine-tuning of the BERT model is completed.

[0086] The experiment constructed a rigorous distributed environment with 10 agents, using a Dirichlet non-independent identically distributed (Non-IID) scheme to partition the data, and introducing asymmetric label noise of up to 30%-40%. Of particular note is the severe label skew challenge in the distributed environment, where some agents have only samples of a single class, causing the initial test accuracy of traditional algorithms to approach 0% and accompanied by severe fluctuations. Under different network connectivity conditions (…),… The smaller the value, the sparser the topology and the degree of data heterogeneity. (The larger the value, the greater the category difference) and label noise rate ( Under various combinations of configurations, the present invention (FAB) was comprehensively compared with benchmark algorithms in terms of test accuracy and running efficiency. For example... Figure 4As shown, the experimental results demonstrate that the FAB algorithm successfully overcomes the problem of local training failure and verifies its excellent robustness in handling large-scale network parameters and severe label skew. Meanwhile, the efficiency analysis shows that although the simple benchmark algorithm has a slight speed advantage under low noise conditions, the FAB algorithm achieves better overall performance in complex scenarios with high noise and large-scale parameters while ensuring controllable GPU memory, fully demonstrating its practical value in industrial-grade large model cleaning tasks.

[0087] 3. Multi-agent reinforcement learning (MARL) and algorithm variant ablation experiments.

[0088] To further verify the core technical advantages of this invention in dynamic environments, this embodiment conducted detailed ablation comparison experiments on high-frequency time-varying topology scenarios in a multi-agent reinforcement learning simulation environment based on Bellman equations. The invention (FAB) was compared with a centralized ideal benchmark (Centralized_FAB), a static topology variant (Static_FAB), a communication mechanism variant (PushSum_FAB), and a second-order optimization policy variant (PushPull_SOBA). Experimental results are as follows: Figure 5 As shown, the specific analysis is as follows: like Figure 5 As shown in (a), Static_FAB (orange dashed line) which ignores topology changes fails to converge, confirming the necessity of dynamic adaptation mechanisms in time-varying networks; while the convergence trajectory of the present invention (FAB, red solid line) is closest to the theoretically optimal centralized benchmark (blue dashed line), and its convergence speed is significantly faster than PushSum and SOBA variants.

[0089] like Figure 5 As shown in (b), compared to PushPull_SOBA (outermost purple line) which suffers from high memory usage due to the introduction of second-order approximation, and PushSum_FAB (green line) which has a heavy computational burden, this invention (blue line) is located in the innermost circle of the radar chart thanks to the organic combination of "all first-order computation + Push-Pull communication". This indicates that FAB achieves the lowest peak memory usage and the least computation time while ensuring high-precision convergence, fully demonstrating its engineering value as an optimal solution for resource-constrained distributed systems.

[0090] This embodiment achieves the following technical effects through the above technical solution: This embodiment proposes a distributed first-order bi-layer optimization algorithm (FAB) based on time-varying directed graphs. Through technical breakthroughs in "full first-order gradient calculation" and "Push-Pull communication strategy", it brings significant benefits in terms of computational efficiency, network adaptability, and hardware compatibility. 1. Achieving automated hyperparameter optimization for large-scale deep neural networks (based on breakthroughs in computing power using first-order theory): For deep networks with a large number of parameters, such as ResNet50 and Transformer, this embodiment addresses the pain points of time-consuming and resource-intensive traditional grid search and memory overflow (OOM) in traditional second-order two-layer algorithms. By utilizing a fully first-order two-layer architecture to model hyperparameter search as an upper-level optimization problem, this embodiment achieves a computational complexity of linear $O(d)$, completely eliminating the overhead of Hessian matrix inversion. Theoretical analysis shows that in ResNet50-level image classification tasks, this embodiment can reduce memory usage by more than 50% compared to algorithms based on second-order gradients, and has the ability to support automatic optimization of large models with limited computing power; at the same time, compared with traditional grid search methods, the gradient-guided mechanism is expected to save 60%-80% of search time, significantly improving R&D efficiency.

[0091] 2. Empowering multi-agent reinforcement learning in time-varying networks (solving the dynamic topology convergence problem): In scenarios with frequently changing communication topologies (time-varying directed graphs), such as drone swarms or robot collaborations, this embodiment addresses the failure of existing two-layer algorithms due to the assumption of a static network. This embodiment utilizes a push-pull communication mechanism to eliminate dynamic consensus bias, perfectly aligning with the Actor-Critic architecture in reinforcement learning. Simulation experiments demonstrate that even in simulated swarms with 100+ nodes, under extreme conditions of high topology change frequency and asymmetric communication, this algorithm still guarantees stable convergence of the agent's policy, while traditional push-sum algorithms exhibit error divergence in this scenario, proving the strong robustness of this embodiment in dynamic environments.

[0092] 3. Automated cleaning and sample reweighting for noisy data (supporting privacy-preserving collaborative cleaning): To address the common label noise problem in industrial sensor or crowdsourced data, this embodiment constructs a two-layer mechanism of "optimizing training set weights based on validation set feedback." This mechanism can automatically identify and reduce the weights of noisy samples, achieving "automatic denoising" during model training. Particularly in distributed federated learning scenarios, this embodiment allows nodes to collaboratively clean data without exchanging raw data to protect privacy. Experimental results show that even with a label noise rate as high as 40%, the model trained using this algorithm maintains extremely high prediction accuracy, significantly outperforming traditional methods without two-layer optimization cleaning.

[0093] In summary, this embodiment, through its innovative algorithm architecture, has for the first time overcome the computational bottleneck of two-layer optimization in time-varying directed networks. It has opened up an automated path for hyperparameter selection, policy learning, and data cleaning in deep learning with extremely low resource overhead, providing an efficient and universal solution for the engineering implementation of large-scale distributed intelligent systems.

[0094] Exemplary device Based on the above embodiments, the present invention also provides a distributed two-layer optimization device for time-varying directed graphs, comprising: The directed graph acquisition module is used to acquire time-varying directed graph sequences; A two-level optimization problem module is used to solve the two-level optimization problem of a multi-agent system based on the time-varying directed graph sequence. The decision variable update module is used to update the decision variables of each agent through a row random matrix. The local gradient calculation module is used to calculate the local gradient of the decision variables of each agent using a penalty function; The gradient tracking variable update module is used to update the gradient tracking variable of each agent through a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence. The optimization result output module is used to output the two-level optimization results of multiple intelligent agent systems.

[0095] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 6 As shown.

[0096] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a storage medium and internal memory; the storage medium stores the operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices, such as mobile terminals and computers; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or mobile terminal.

[0097] When executed by a processor, this computer program is used to implement a distributed two-level optimization method for time-varying directed graphs.

[0098] It will be understood by those skilled in the art that Figure 6 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0099] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a distributed bi-level optimization program for time-varying directed graphs, which, when executed by the processor, is used to implement the operation of the distributed bi-level optimization method for time-varying directed graphs as described above.

[0100] In one embodiment, a storage medium is provided, wherein the storage medium stores a distributed bi-level optimization program for time-varying directed graphs, which, when executed by a processor, is used to implement the operations of the distributed bi-level optimization method for time-varying directed graphs as described above.

[0101] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory.

[0102] In summary, this invention provides a method, apparatus, terminal, and medium for distributed bi-level optimization on time-varying directed graphs. The method includes: acquiring a sequence of time-varying directed graphs; solving a bi-level optimization problem for a multi-agent system based on the time-varying directed graph sequence; updating the decision variables of each agent using a row random matrix and calculating the local gradient of the decision variables of each agent using a penalty function; updating the gradient tracking variables of each agent using a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence; and outputting the bi-level optimization results for the multi-agent system. This invention solves the problem of bi-level optimization failure under dynamic topology, eliminates the risk of numerical instability, overcomes the second-order computational bottleneck, and reduces resource overhead.

[0103] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A distributed two-layer optimization method for time-varying directed graphs, characterized in that, include: Obtain a time-varying directed graph sequence; Based on the time-varying directed graph sequence, solve the bi-level optimization problem of multiple agent systems; The decision variables of each agent are updated by a row random matrix, and the local gradient of the decision variables of each agent is calculated using a penalty function; The gradient tracking variable of each agent is updated by a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence; Output the two-level optimization results of multiple intelligent agent systems.

2. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The process of solving a two-level optimization problem for a multi-agent system based on the time-varying directed graph sequence includes the following steps: Set the number of iterations Penalty parameters and learning rate ; Input initial variables Row random matrix and column random matrix ; Calculate the initial gradient .

3. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The method of solving a two-level optimization problem for a multi-agent system based on the time-varying directed graph sequence includes: Based on auxiliary variables and penalty parameters, the upper-level target will be included. and lower-level constraints The bi-level optimization problem is transformed into a single-level minima-maximum problem: ； in, For the auxiliary variable; Let be the penalty parameter.

4. The distributed two-layer optimization method for time-varying directed graphs according to claim 3, characterized in that, In the single-layer minimax problem, each agent The local penalty function is defined as: 。 5. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The process of updating the decision variables of each agent using a row random matrix includes: For each intelligent agent Using the row random matrix Obtain status information from the first neighbor and update its own decision variables. : ； in, Represents the gradient tracking variable. This represents the learning rate.

6. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The calculation of the local gradient of the decision variables of each agent using the penalty function includes: For each intelligent agent The gradient of the local penalty function is calculated based on the latest state: in, Represents each intelligent agent Update its own decision variables The latest status.

7. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The step of updating the gradient tracking variable of each agent through a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence includes: For each intelligent agent Using column random matrices Push the state information to the second neighbor and update the gradient tracking variables: ； in, This represents the gradient tracking variable.

8. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The method further includes: A decentralized distributed two-layer optimization method is used to solve the two-layer optimization problem of the multiple intelligent agent system. Alternatively, a distributed two-level optimization method based on a second-order approximation algorithm can be used to solve the two-level optimization problem of the multiple intelligent agent system.

9. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The decentralized distributed two-level optimization method for solving the two-level optimization problem of the multiple agent system includes: For each intelligent agent The weight variables are updated using the same communication topology and column random update rules as the decision variables. Divide the updated decision variable by its corresponding weight variable to offset the homologous topological distortion factors in the numerator and denominator; The penalty function is used to calculate the local gradient of the decision variables for each agent; The gradient tracking variable of each agent is updated by the column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence.

10. The distributed two-layer optimization method for time-varying directed graphs according to claim 1, characterized in that, The distributed two-level optimization method based on the second-order approximation algorithm solves the two-level optimization problem of the multiple intelligent agent system, including: By introducing auxiliary variables The inverse vector product of the approximate Hessian matrix is ​​used to calculate the local gradient of the decision variable for each agent. The gradient tracking variable of each agent is updated through the column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence.

11. A distributed two-layer optimization device for time-varying directed graphs, characterized in that, include: The directed graph acquisition module is used to acquire time-varying directed graph sequences; A two-level optimization problem module is used to solve the two-level optimization problem of a multi-agent system based on the time-varying directed graph sequence. The decision variable update module is used to update the decision variables of each agent through a row random matrix. The local gradient calculation module is used to calculate the local gradient of the decision variables of each agent using a penalty function; The gradient tracking variable update module is used to update the gradient tracking variable of each agent through a column random matrix to eliminate gradient estimation bias in the time-varying directed graph sequence. The optimization result output module is used to output the two-level optimization results of multiple intelligent agent systems.

12. A terminal, characterized in that, include: The processor and memory, wherein the memory stores a distributed bi-level optimization program for time-varying directed graphs, which, when executed by the processor, is used to implement the operation of the distributed bi-level optimization method for time-varying directed graphs as described in any one of claims 1-10.

13. A medium, characterized in that, The medium is a computer-readable storage medium that stores a distributed two-level optimization program for time-varying directed graphs. When executed by a processor, the distributed two-level optimization program for time-varying directed graphs is used to implement the operation of the distributed two-level optimization method for time-varying directed graphs as described in any one of claims 1-10.

Images