A data aggregation method based on a Spark platform classification problem optimization algorithm
By designing the ARAggregate data aggregation pattern and applying AllReduce and ADMM algorithms on the Spark platform, the problems of excessive load on the master node and resource waste in the Spark platform were solved, and the data processing efficiency and algorithm selectivity were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI UNIV
- Filing Date
- 2022-07-06
- Publication Date
- 2026-07-31
AI Technical Summary
When processing big data, the Spark platform experiences excessive performance load on the master node as the number of cluster nodes increases, leading to bottleneck issues. Furthermore, other nodes waste resources significantly during data aggregation, impacting performance.
Design a non-intrusive ARAggregate data aggregation mode that enables data aggregation to occur between worker nodes, reducing the load on the driver node, and uses the AllReduce and ADMM algorithms to perform data synchronization and model updates between worker nodes.
It alleviates the performance bottleneck of the Driver node, reduces resource waste, improves the data processing efficiency of the Spark platform, and enriches the optimization algorithm library, providing more optimization options.
Smart Images
Figure CN115309541B_ABST
Abstract
Description
Technical Field
[0001] This invention proposes a data aggregation method based on the Spark platform classification problem optimization algorithm, and applies it to the Spark platform classification problem optimization algorithm. It involves machine learning and distributed computing, and belongs to the field of computer technology. Background Technology
[0002] With the development of mobile internet, the popularization of smart devices, and the networking of various aspects of the material world, the amount of data generated by human activities is experiencing explosive growth. [1] And the technology for processing such massive amounts of data is becoming increasingly important. Traditional single-node data processing methods are no longer sufficient to meet today's demands, as exemplified by Spark. [2][3] Distributed big data processing frameworks, exemplified by Spark, have been widely adopted. For instance, well-known internet companies such as Tencent, Baidu, and Yahoo all have their own Spark clusters to support data processing. [4] .
[0003] However, with the continuous development of the big data industry, some bottlenecks of the Spark platform have gradually emerged. For example, as the number of Spark cluster nodes increases, the master node, responsible for coordinating the global data and aggregating data from various slave nodes, becomes overwhelmed, thus becoming the bottleneck of the entire system. Although there are currently parameter servers with better performance... [5][6][7] While the technology is advanced, migrating data processing tasks from the Spark platform to the parameter server platform undoubtedly requires users to undergo a data and program migration process, incurring significant human and time costs.
[0004] The core of the original distributed solution implementation for Spark training models is centralized models and distributed computation. In distributed computation, the model for solving the problem is concentrated on only one server. During each iteration, the loss and gradient values corresponding to the training data on each child node are computed in parallel, and this information is aggregated to the node where the model resides, then processed using SGD. [8] The L-BFGS optimization algorithm updates the parameters. Then, the parameters are distributed to each child node for the next iteration until convergence. For example, when training a logistic regression model, the process of one iteration can be summarized as follows: Figure 1 The iterative process diagram shows the four steps:
[0005] 1) Model Broadcast: The Driver process broadcasts model parameters to all Executors.
[0006] 2) Gradient calculation: Each Executor calculates the loss function value and gradient value corresponding to the local dataset in parallel.
[0007] 3) Gradient aggregation: The Driver process uses treeAggregate [9]
[10] The operator aggregates the gradients and loss values computed by all Executors.
[0008] 4) Model update: The Driver process updates the model locally using aggregated gradients and loss values.
[0009] Spark uses two optimization algorithms, SGD and L-BFGS, to find the optimal model. In each iteration, the loss and gradient values corresponding to the dataset on each child node are computed in parallel using the Executor. These values are then aggregated to the Driver node using the `treeAggregate` function, and the model parameters are updated. The model parameters are then broadcast to all child nodes for the next iteration until convergence.
[0010] The core of Spark's classification optimization algorithm is to use the `treeAggregate` function on the Driver side to aggregate information. The `treeAggregate` data aggregation process is as follows: Figure 2 As shown. The `treeAggregate` function requires defining `seqOp` and `combOp` operations. `seqOp` adds a training sample to existing statistics, updating the loss function and gradient values based on the dataset. `combOp` merges two sets of statistical information. The former defines the computational logic within each RDD, while the latter defines the data merging logic between RDDs. The `treeAggregate` function aggregates the information obtained from each iteration at each node and then summarizes it at the Driver where the model resides.
[0011] pass Figure 3 It can be seen that Spark requires two treeAggregate processes during data aggregation: one is aggregation within the Executor, and the other is data aggregation between the Executor and the Driver. Furthermore, due to Spark's BSP overall synchronous parallelism strategy, two bottlenecks arise: (1) When aggregation occurs between Executors, other non-participating nodes must wait; when aggregation occurs between the Executor and the Driver, other non-participating nodes must wait. This process results in resource idleness and waste, affecting performance. (2) When the Driver aggregates data from Executor nodes, if the aggregated data is too large, the Driver's load becomes excessive, easily becoming a bottleneck for the entire system.
[0012] Therefore, optimizing or improving the Spark platform to mitigate or even eliminate the bottleneck caused by the master node of the Spark cluster would be of great practical significance. AllReduce...
[11]
[12] As a commonly used algorithm in the field of parallel computing, AllReduce can achieve model synchronization, maintain distributed datasets, and perform distributed operations. Many parallel programs require all processes to access the results of a data aggregation (reduced), not just the main process. To address this, the AllReduce algorithm distributes the aggregated results to various processes or child nodes, ensuring that the aggregation results are not limited to the main process. Based on these characteristics of AllReduce, its principles can be applied to the Spark platform to design a new data aggregation mode suitable for Spark—the ARAggregate data aggregation mode. This alleviates the performance bottleneck caused by the main node and improves the performance of linear model training on the Spark platform.
[0013] In the Spark platform, only SGD and L-BFGS algorithms are available for solving linear classification problems, limiting the range of optimization algorithms available to users. After implementing the ARAggregate data aggregation pattern described above, a new version of the SGD algorithm can be implemented based on this pattern. This version calculates a complete local model on each Executor based on local data, instead of simply calculating gradients from local data and then having the Driver node aggregate the data from each Executor and perform global model variable calculations. This alleviates the performance bottleneck problem caused by the treeAggregate aggregation method mentioned earlier on the Spark platform. Furthermore, due to the alternating direction multiplier method...
[13]
[14] Alternating Direction Method of Multipliers (ADMM), as a distributed optimization algorithm, can be used to solve large-scale machine learning problems. Its main idea is a decomposition-coordination process: by breaking down a large global problem into multiple smaller, easily solvable local subproblems (transforming the original problem into a globally consistent problem), the solution to the global problem is obtained by coordinating the solutions to the subproblems. In this process, the solutions to the subproblems can be solved in parallel, and then the solutions to the synchronized subproblems are integrated to obtain the solution to the global problem. Therefore, it can also be implemented in a distributed manner on the Spark platform based on the ARAggregate data aggregation pattern, thus enriching Spark's optimization algorithm library for solving linear classification problems.
[0014] Current research on Spark mainly falls into four categories: (1) Based on the characteristics of different application areas, Spark is combined with the Spark platform to make Spark better applicable to those areas. (Literature)
[15] This paper proposes a method for partitioned processing of large-scale data in power distribution networks based on the Spark computing engine. By leveraging Spark, the method better processes partitioned data from power distribution networks, extracting power indicators with practical value for power distribution network planning and management. (References)
[16] A sea surface temperature prediction method based on the Spark framework is proposed. The Dynamic Time Correction (DTW) algorithm is started under the Spark framework and the data preprocessing is completed. Then, the Spark working cluster is called to calculate the Dynamic Time Correction Distance (DTW) between the analog and reference patterns. After sorting by the size of the DTW distance, the top k analog patterns are taken as the training set to train a model with predictive ability. This type of patent does not involve any changes to the Spark platform, but only uses the Spark platform as a complete tool. The innovation of the patent lies in the processing of business logic in different fields, that is, how to use the Spark platform to process data in this industry, rather than optimizing the Spark platform to improve or enhance the efficiency of Spark in data processing. In this type of patent, Spark is simply used as a data processing tool, and there is no optimization or modification. (2) Research and optimization of Spark SQL. Li Huarong et al.
[17] A column computation optimization method based on Spark SQL was invented. By unifying the scheduling of heterogeneous computing resources, the optimizer and plugins in Spark SQL are extended to implement a heterogeneous resource scheduling mechanism based on data characteristics. Rule matching is performed on the logical execution plan of Spark SQL to generate a physical execution plan based on a unified memory structure (Arrow) and a hybrid CPU and GPU orchestration. This avoids the JVM memory computation GC overhead and accelerates computational efficiency. (Song Aibo et al.)
[18] A method to improve the query efficiency of Spark SQL was invented. First, a query pre-analysis module was built. Through an estimation model, the size of the intermediate data generated by Shuffle was calculated, thereby calculating the total size of the intermediate data cache layer used to cache the intermediate data. Based on the total size of the intermediate data cache layer calculated above, combined with the distribution of input data of each node in the cluster, a reasonable memory space size was set for each node through the cache layer allocation module. This effectively solves the problem of high disk I / O overhead in Spark SQL queries. (3) Research and optimization of Sparkstreaming. Max Petrov
[19] M. Reza Hoseiny and Farahabady proposed an adaptive performance model that significantly improves the efficiency of executing streaming tasks on Spark through dynamic resource allocation.
[20] Researchers proposed a flexible automatic tuning strategy called Spark-Tuner to address resource contention issues in Spark streaming programs. Wang Yiwu...
[21] To address the shortcomings of the k-means algorithm, an improved k-means clustering algorithm based on Markov chain Monte Carlo method and ring region search was designed and implemented on Spark Streaming. (4) Research and optimization of Sparkmllib. Li Zhongzheng
[22] Analysts like Lu Baoguo introduced parameter servers into Spark, reducing the computational load on the Driver nodes by adding server nodes and mitigating the impact of slow nodes on algorithm performance through asynchronous transmission.
[23] A researcher proposed an optimization method for Spark mllib machine learning algorithms, which obtains an auxiliary matrix through augmented matrix operations on the training dataset, and then obtains the machine learning algorithm parameters. This invention pertains to the research and optimization of Spark mllib, with a hierarchical structure as follows: Figure 4 As shown. Summary of the Invention
[0015] Because the current Spark classification problem optimization algorithm uses the treeAggregate data aggregation mode, this aggregation mode will bring excessive performance load to the driver node as the number of Spark cluster nodes increases. Furthermore, due to the synchronization mechanism of Spark's BSP (Block Synchronous Parallelism), other non-participating nodes can only wait during the treeAggregate data aggregation process, resulting in a certain degree of performance waste.
[0016] To overcome the aforementioned problems, this invention provides a data aggregation method based on the Spark platform's classification problem optimization algorithm. Employing a non-intrusive programming approach, a novel data aggregation method, ARAggregate, is designed and implemented. This method allows data aggregation to occur between worker nodes in the Spark cluster, extracting the Driver node from specific data aggregation tasks. The Driver node is then responsible only for task partitioning and scheduling, thus mitigating the performance bottleneck caused by the Driver node and also reducing the cluster performance waste associated with the original treeAggregate data aggregation mode.
[0017] To achieve the above objectives, the present invention adopts the following technical solution:
[0018] A data aggregation method based on the Spark platform for optimizing classification algorithms includes the following steps:
[0019] (1) Design and implement the ARAggregate data aggregation pattern on the Spark platform;
[0020] (2) Applying the ARAggregate data aggregation pattern to the Spark platform's SGD algorithm;
[0021] (3) Implement the ADMM algorithm based on the Spark platform and apply the ARAggregate data aggregation mode to the ADMM algorithm.
[0022] The specific steps of step (1) are as follows:
[0023] (1.1) Model segmentation: The local model variables maintained by each worker node are divided into segments according to the number of nodes. For example, if there are N nodes, each local model variable is divided into N segments, and each segment has a segment number, which takes the values 0, 1, 2, ... N-1 in sequence; forming a key-value data type with the segment number as the key and the model segment data as the value.
[0024] (1.2) Reduce the model data after segmentation: Using the reduceByKey operator provided by Spark, the segment number is used as the key value to perform a shuffle operation on the local model segmented data maintained by each worker node. This ensures that the data with the same segment number maintained on different worker nodes are distributed to the same worker node. Then, the data with the same segment number is reduced to obtain the model value corresponding to the segment number of the new global model.
[0025] (1.3) Model segment copying after reduction: Copy each model segment that has completed data aggregation N times, and add new tags to the copied data, namely 0, 1...N-1; so that the updated model segment data can be distributed to other worker nodes.
[0026] (1.4) Distribution of new model segments: Using the groupbykey operator, model segments with the same tag number calculated on each worker node are collected, so that model segments with the same tag number are aggregated on the same worker node.
[0027] (1.5) Assemble model segments into new local model variables: Since each node now has model segment data with the same tag number, after removing the tag number, we get the updated global model with the segment number as the key and the model segment data as the value. Arrange the segment data according to the segment number to form the updated local model variables.
[0028] The detailed steps of step (2) are as follows:
[0029] (2.1) Set initialization parameters in the Driver node, including the path to the input dataset, the number of iterations, the iteration step size, the regularization parameter, and the number of partitions;
[0030] (2.2) The worker node performs local model calculation based on local data. Through sampling algorithm, a portion of the data is randomly sampled and the stochastic gradient descent method is used to solve the model variables, instead of just calculating the stochastic gradient of the local data and sending it to the driver node for aggregation operation when the original data aggregation mode based on treeAggregate is used.
[0031] (2.3) After each worker node calculates its local model variables, the ARAggregate data aggregation mode is used to perform data aggregation operations between each worker node, thereby updating the local model variables of each worker node and preparing for the next iteration calculation.
[0032] (2.4) Determine whether to proceed with the next iteration based on the set number of iterations. If the set number of iterations is reached, return the model variable; otherwise, continue with steps (2.2) to (2.4).
[0033] The detailed steps of step (3) are as follows:
[0034] (3.1) Setting the storage structure for intermediate variables: Since the ADMM algorithm needs to save local model variables, dual variables and global model variables during the solution process, ADMMState is defined to save these three intermediate variables calculated in each iteration.
[0035] (3.2) Set initialization parameters in the Driver node, including the path to the input dataset, the number of iterations, the regularization parameter, and the number of partitions;
[0036] (3.3) The worker node calculates new local model variables based on local data and intermediate variables stored in local ADMMState, and saves them in ADMMState. The process of solving local model variables is called subproblem solving, which uses various optimization algorithms, such as stochastic gradient descent or dual coordinate descent.
[0037] (3.4) After each worker node calculates the new local model variables, the ARAggregate data aggregation mode is used to perform data aggregation operations between each worker node, so that the global model variables stored in the ADMMState of each worker node are updated, in preparation for the next iteration calculation.
[0038] (3.5) After the global model variables are updated, continue to update the dual variables in each ADMMState; at this point, one round of iterative updates is completed;
[0039] (3.6) Determine whether to proceed to the next iteration based on the set number of iterations or other termination conditions. If the termination condition is reached, return the model variables; otherwise, continue with steps (3.3) to (3.6).
[0040] Compared with the prior art, the present invention has the following advantages:
[0041] This invention primarily addresses the computational load on Spark's Driver nodes during model training and the performance waste caused by waiting for other nodes during data aggregation by designing and implementing a novel data aggregation pattern within Spark. Furthermore, by applying this new data aggregation pattern to the SGD and ADMM algorithms, it enriches Spark's mllib optimization algorithm library, providing users with more optimization options. Compared to introducing a parameter server, this invention achieves compatibility with the existing Spark platform through a non-intrusive design, reducing the cost of program migration and user learning. Attached Figure Description
[0042] Figure 1 This is a schematic diagram of the iterative process of a Spark classification problem.
[0043] Figure 2 This is a diagram illustrating data aggregation in SparkTreeAggregate.
[0044] Figure 3 This is a diagram illustrating data aggregation during the Spark iteration process.
[0045] Figure 4 This is a system hierarchy diagram of Spark for processing classification problems.
[0046] Figure 5 This is a schematic diagram of the ARAggregate data aggregation mode.
[0047] Figure 6 This is a diagram of the Spark iteration process based on the ARAggregate aggregation pattern.
[0048] Figure 7 This is a schematic diagram of the iterative process of the ADMM algorithm implemented based on Spark.
[0049] Figure 8 This is a schematic diagram of the iterative process of the Spark ADMM algorithm based on ARAggregate. Detailed Implementation
[0050] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0051] A data aggregation method based on the Spark platform for optimizing classification algorithms includes the following steps:
[0052] (1) Design and implement the ARAggregate data aggregation pattern on the Spark platform, such as Figure 5 As shown;
[0053] This data aggregation method eliminates the need for the `treeAggregate` method to aggregate all data to the Spark cluster's Driver node. The Driver node is no longer responsible for specific data computation, thus improving the performance bottleneck caused by excessive Driver node load and reducing the performance waste caused by other non-participating nodes having to wait during `treeAggregate` aggregation. The specific steps are as follows:
[0054] (1.1) Model Segmentation: The local model variables maintained by each working node are divided into segments based on the number of nodes. For example, if there are N nodes, each local model variable can be divided into N segments, each segment having a segment number with values of 0, 1, 2, ..., N-1. This forms a key-value data type with the segment number as the key and the model segment data as the value.
[0055] (1.2) Reduce the model data after segmentation: Using the reduceByKey operator provided by Spark, the segment number is used as the key value to perform a shuffle operation on the local model segmented data maintained by each worker node. This ensures that data with the same segment number maintained on different worker nodes are distributed to the same worker node. Then, the data with the same segment number is reduced to obtain the model value corresponding to the segment number of the new global model.
[0056] (1.3) Model segment replication after reduction: Each model segment that has completed data aggregation is replicated N times, and a new tag number is added to the replicated data, which is 0, 1...N-1. This is to prepare for distributing the updated model segment data to other working nodes.
[0057] (1.4) Distribution of new model segments: Using the groupbykey operator, model segments with the same label number calculated on each working node are collected, so that model segments with the same label number are aggregated on the same working node.
[0058] (1.5) Assemble model segments into new local model variables: Since each node currently contains model segment data with the same tag number, removing the tag number results in the updated global model, which uses the segment number as the key and the model segment data as the value. The segmented data is then arranged according to the segment number to form the updated local model variables.
[0059] (2) Applying the ARAggregate data aggregation pattern to the Spark platform's SGD algorithm;
[0060] (2.1) Set initialization parameters in the Driver node, including the path to the input dataset, the number of iterations, the iteration step size, the regularization parameter, the number of partitions, etc.
[0061] (2.2) The worker node performs local model calculations based on local data. Through a sampling algorithm, a portion of the data is randomly selected for stochastic gradient descent to solve for the model variables. This is not simply a matter of calculating the stochastic gradient of the local data and sending it to the driver node for aggregation, as was done in the original treeAggregate data aggregation mode.
[0062] (2.3) After calculating the local model variables at each worker node, the ARAggregate data aggregation mode is used to perform data aggregation operations between the worker nodes, thereby updating the local model variables of each worker node and preparing for the next iteration calculation.
[0063] (2.4) Determine whether to proceed with the next iteration based on the set number of iterations. If the set number of iterations has been reached, return the model variables; otherwise, continue with steps (2.2) to (2.4). Figure 6 As shown.
[0064] (3) Implement the ADMM algorithm based on the Spark platform and apply the ARAggregate data aggregation mode to the ADMM algorithm.
[0065] (3.1) Setting the storage structure for intermediate variables: Since the ADMM algorithm needs to save local model variables, dual variables and global model variables during the solution process, ADMMState is defined to save these three intermediate variables calculated in each iteration.
[0066] (3.2) such as Figure 7 As shown, the Driver node sets initialization parameters, including the input dataset path, number of iterations, regularization parameters, number of partitions, etc.
[0067] (3.3) The worker node calculates new local model variables based on local data and intermediate variables stored in the local ADMMState, and saves them to ADMMState. The process of solving for local model variables is called subproblem solving, which can utilize various optimization algorithms, such as stochastic gradient descent and dual coordinate descent.
[0068] (3.4) After calculating the new local model variables at each worker node, the ARAggregate data aggregation mode is used to perform data aggregation operations between each worker node, thereby updating the global model variables stored in the ADMMState of each worker node and preparing for the next iteration calculation.
[0069] (3.5) After the global model variables are updated, continue to update the dual variables in each ADMMState. At this point, one round of iterative updates is complete.
[0070] (3.6) Determine whether to proceed to the next iteration based on the set number of iterations or other termination conditions. If the termination condition is reached, return the model variables; otherwise, continue with steps (3.3) to (3.6). Figure 8 As shown.
[0071] The above embodiments of this invention are based on a data aggregation method for classification problem optimization algorithms on the Spark platform. An ARAggregate data aggregation mode is proposed, allowing model data updates to occur between worker nodes, eliminating the need for the Driver node to aggregate data from each worker node and then broadcast it to all worker nodes to complete the model data update. This extracts the data aggregation function from the Driver node, allowing it to focus solely on task scheduling and other functions, thereby reducing the performance bottleneck caused by excessive Driver node load on the Spark platform. The above embodiments of this invention apply the ARAggregate data aggregation mode to the SGD algorithm in the Spark platform, enabling worker nodes to calculate a complete local model based on local data, and then use the ARAggregate data aggregation mode to update the global model among worker nodes. The above embodiments of this invention implement the ADMM algorithm on the Spark platform and apply the ARAggregate data aggregation mode to it, realizing an ADMM algorithm based on a new data aggregation mode on the Spark platform.
[0072] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made according to the purpose of the invention. Any changes, modifications, substitutions, combinations or simplifications made based on the spirit and principle of the technical solution of the present invention shall be equivalent substitutions. As long as they meet the purpose of the invention and do not deviate from the technical principle and inventive concept of the present invention, they shall fall within the protection scope of the present invention.
Claims
1. A data aggregation method based on a Spark platform classification problem optimization algorithm, characterized in that, Includes the following steps: (1) Design and implement the ARAggregate data aggregation pattern on the Spark platform; (2) Applying the ARAggregate data aggregation pattern to the Spark platform's SGD algorithm; (3) Implement the ADMM algorithm based on the Spark platform and apply the ARAggregate data aggregation mode to the ADMM algorithm; The specific steps of step (1) are as follows: (1.1) Model segmentation: The local model variables maintained by each worker node are divided into segments according to the number of nodes. If there are N nodes, each local model variable is divided into N segments. Each segment has a segment number, which takes values of 0, 1, 2, ..., N-1. This forms a key-value data type with the segment number as the key and the model segment data as the value. (1.2) Reduce the segmented model data: Using the reduceByKey operator provided by Spark, the segment number is used as the key value to perform a shuffle operation on the segmented data of the local model maintained by each worker node. This ensures that the data with the same segment number maintained on different worker nodes are distributed to the same worker node. Then, the data with the same segment number is reduced to obtain the model value corresponding to the segment number of the new global model. (1.3) Model segment copying after reduction: Copy each model segment after data aggregation N times, and add new labels to the copied data, namely 0, 1...N-1; In preparation for distributing the updated model segmented data to other worker nodes; (1.4) Distribution of new model segments: Using the groupbykey operator, model segments with the same tag number calculated on each worker node are collected, so that model segments with the same tag number are aggregated on the same worker node. (1.5) Assemble model segments into new local model variables: Since each node now has model segment data with the same tag number, after removing the tag number, we get the updated global model with the segment number as the key and the model segment data as the value. Based on the segment number, the segmented data is arranged to form the updated local model variables. 2.The data aggregation method for optimizing Spark platform classification problem algorithm according to claim 1, wherein, The detailed steps of step (2) are as follows: (2.1) Set initialization parameters in the Driver node, including the path to the input dataset, the number of iterations, the iteration step size, the regularization parameter, and the number of partitions; (2.2) The worker node performs local model calculation based on local data. Through sampling algorithm, a portion of the data is randomly sampled and the stochastic gradient descent method is used to solve the model variables. Instead of simply calculating the stochastic gradient of the local data and sending it to the driver node for aggregation operation when the original data aggregation mode is based on treeAggregate. (2.3) After each worker node calculates its local model variables, the ARAggregate data aggregation mode is used to perform data aggregation operations between each worker node, thereby updating the local model variables of each worker node and preparing for the next iteration calculation. (2.4) Determine whether to proceed with the next iteration based on the set number of iterations. If the set number of iterations is reached, return the model variable; otherwise, continue with steps (2.2) to (2.4).
3. The data aggregation method based on the Spark platform classification problem optimization algorithm according to claim 1, characterized in that, The detailed steps of step (3) are as follows: (3.1) Setting the storage structure of intermediate variables: Since the ADMM algorithm needs to save local model variables, dual variables and global model variables during the solution process, ADMMState is defined to save these three intermediate variables calculated in each iteration. (3.2) Set the initialization parameters in the Driver node, including the path to the input dataset, the number of iterations, the regularization parameters, and the number of partitions; (3.3) The worker node calculates new local model variables based on local data and intermediate variables stored in local ADMMState, and saves them in ADMMState. The process of solving local model variables is called subproblem solving, which uses various optimization algorithms, such as stochastic gradient descent or dual coordinate descent. (3.4) After each worker node calculates the new local model variables, the ARAggregate data aggregation mode is used to perform data aggregation operations between each worker node, so that the global model variables stored in the ADMMState of each worker node are updated, in preparation for the next iteration calculation. (3.5) After the global model variables are updated, continue to update the dual variables in each ADMMState; This completes one round of iterative updates; (3.6) Determine whether to proceed to the next iteration based on the set number of iterations or other termination conditions. If the termination condition is reached, return the model variables; otherwise, continue with steps (3.3) to (3.6).