A distributed model training algorithm for fast image classification problems

Abstract

The present invention discloses a distributed model training algorithm for fast image classification problems, including: preparation and distribution of image data; model initialization; distributed model training; model evaluation and performance optimization. The trained distributed model is an adaptive moment estimation algorithm based on Nesterov acceleration, namely the NGTAdam algorithm, including: initializing parameters; calculating gradients and updating the gradient estimator based on the gradient tracking method; updating the first-order moment estimation vector and the second-order moment estimation vector; adaptively updating the learning rate; updating the temporary variable and the Nesterov momentum variable; updating the estimated solution based on the gradient descent method. In addition, the present invention also discloses a theoretical convergence upper bound of the NGTAdam algorithm to ensure algorithm convergence. The method proposed by the present invention focuses on real-time decision-making and dynamic optimization, can efficiently solve large-scale online optimization problems, is applicable to various scenarios, and solves the problems of slow speed and poor accuracy of existing image classification algorithms when training large-scale image data.

Description

Technical Field

[0001] The present invention relates to the technical fields of distributed optimization and machine learning, and particularly to a distributed model training algorithm for fast image classification problems. Background Art

[0002] Image classification is one of the core tasks in the field of computer vision, and image classification technology is a key factor driving the development of the fields of computer vision and machine learning. Image classification technology is widely used in various industries. In scenarios such as medical diagnosis and emergency response systems that require quick decision-making, image classification can quickly provide decision support to help professionals make accurate judgments in the shortest time; in scientific research fields such as biology, geography, and astronomy, image classification technology can help researchers automatically identify and classify a large amount of image data to promote scientific discoveries; with the explosive growth of digital images and video content, image classification technology has become an effective tool for managing large-scale data. With the continuous progress of technology and algorithms, image classification will play a greater role in more fields.

[0003] Currently, there is an urgent need to accelerate the processing speed of image classification tasks and improve classification accuracy, and various image classification methods are constantly evolving. Deep neural networks, such as convolutional neural networks, perform excellently in image recognition and classification tasks. However, training these complex networks requires a large amount of computing resources and optimization algorithms. Traditional optimization algorithms, such as the gradient descent method, are less efficient when dealing with large-scale and high-dimensional image data. Therefore, it is necessary to seek more efficient optimization methods to accelerate the training process and improve the performance of the model. The Adam algorithm, as an adaptive learning rate optimization algorithm, combines the momentum method and adaptive learning rate technology, effectively improving the convergence speed and performance of deep learning models. However, with the growth of the dataset and model scale, the traditional Adam algorithm may still have problems with low optimization efficiency when dealing with large-scale image classification tasks.

[0004] To solve the above problems, distributed optimization algorithms have emerged. They improve computing efficiency by processing data in parallel on multiple computing nodes. In recent years, distributed optimization technology has been widely applied in fields such as machine learning, image processing, network optimization, and intelligent control.

[0005] However, existing distributed optimization algorithms often face problems of slow update speed and poor convergence in dynamic environments, especially in online learning scenarios. To improve the performance of distributed optimization algorithms in image classification tasks, there is an urgent need for a new algorithm that can quickly adapt to data changes, effectively utilize local information, and maintain algorithm stability. Summary of the Invention

[0006] In view of the deficiencies of the above-mentioned existing technologies, the present invention provides a distributed model training algorithm for fast image classification problems: the NGTAdam (Nesterov Accelerated Gradient Tracking with Adaptive Momentum) algorithm. This algorithm is a new distributed online optimization algorithm that accelerates and improves the commonly used Adam algorithm for image classification in the existing technologies based on the Nesterov momentum variable, increases the adaptability to dynamic changes in the network structure, accelerates the processing speed of the image classification task, and improves the classification accuracy.

[0007] To achieve the above technical objectives, the present invention provides the following technical solutions:

[0008] A distributed model training algorithm for fast image classification problems, including:

[0009] S1. Initialize the parameters of the NGTAdam algorithm; import the image data set and perform data preprocessing. On the decentralized network, distribute the training data to all nodes;

[0010] S2. On each node, update the temporary variable using the estimated solution and the Nesterov momentum variable, and calculate the current gradient using the temporary variable;

[0011] S3. On each node, update the gradient estimator based on the gradient tracking method;

[0012] S4. On each node, update the first-order moment estimate and the second-order matrix estimate using the exponential moving average;

[0013] S5. On each node, adaptively update the gradient descent direction of the estimated solution using the first-order moment estimate and the second-order moment estimate updated in step S4;

[0014] S6. Combine the Nesterov momentum variables of the neighbor nodes and update the Nesterov momentum variable of the local node based on the gradient descent method;

[0015] S7. Combine the temporary variables of the neighbor nodes and update the estimated solution of the local node based on the gradient descent method;

[0016] S8. Repeat S2 - S7 for multiple iterations until the algorithm termination condition is met.

[0017] Preferably, step S2 is specifically:

[0018] S21. Update the temporary variable; use the estimated solution and the Nesterov momentum variable to update the temporary variable of node i at time t The formula expression is:

[0019]

[0020] Among them, θ t represents the decreasing step size, is the estimated solution of node i at time t, is the estimated Nesterov momentum variable of node i at time t;

[0021] S22. Calculate the current gradient; Use the temporary variable obtained in step S21 to calculate the gradient of node i at time t The formula is expressed as:

[0022]

[0023] Among them, is the gradient calculation function.

[0024] Preferably, step S3 is specifically:

[0025] The gradient estimator of node i at time t is denoted as to represent the update descent direction of the Nesterov momentum variable, and its update expression is:

[0026]

[0027] Among them, N i represents the neighborhood of node i, and j is the neighbor node of node i in the neighborhood N i ; are the gradients of node i at time t and t - 1 respectively, w ij is an element in the weight matrix W of the time-varying network, reflecting the weight of node i to the information of its neighbor node j.

[0028] Preferably, step S4 specifically includes:

[0029] S41. Update the first moment estimate; Node i updates the first moment estimate using exponential moving average. The first moment estimate of node i at time t + 1 is denoted as The update expression is:

[0030]

[0031] Among them, β 1 represents the exponential decay rate of the first moment estimate, is the first moment estimate of node i at time t, is the gradient estimator of node i at time t;

[0032] S42. Update the second moment estimate; Node i updates the second moment estimate using exponential moving average. The second moment estimate of node i at time t + 1 is denoted as Its update expression is:

[0033]

[0034] where β 2 represents the exponential decay rate of the second - moment estimate value, the symbol ⊙ represents the Hadamard product of matrices, V m > 0 is the saturation parameter to ensure the upper bound of the estimate value; is the second - moment estimate value of node i at time t, is the gradient estimator of node i at time t.

[0035] Preferably, step S5 is specifically:

[0036] Adaptively update the descent direction of the estimated solution using the first - moment estimate value and the second - moment estimate value updated in step S4. The gradient descent direction of the estimated solution of node i at time t + 1 is denoted as Its update expression is:

[0037]

[0038] where the parameter ∈> 0 is used to ensure the stability of numerical operations and avoid gradient explosion; are respectively the first - moment estimate value and the second - moment estimate value of node i at time t + 1.

[0039] Preferably, step S6 is specifically:

[0040] Node i combines the Nesterov momentum variables of its neighbor nodes and updates its own Nesterov momentum variable based on the gradient descent method. The Nesterov momentum variable of node i at time t + 1 is denoted as Its update expression is:

[0041]

[0042] where N i is the neighborhood of node i, j is the neighbor node of node i in the neighborhood N i inside, is the Nesterov momentum variable of neighbor node j of node i at time t; α is a fixed step size, θ t is a decreasing step size, w ij is an element in the weight matrix W of the time - varying network, reflecting the weight of node i for the information of its neighbor node j; the gradient estimator represents the updated descent direction.

[0043] Preferably, step S7 is specifically:

[0044] Node i combines the temporary variable of neighbor node j and updates the estimated solution based on the gradient descent method; the estimated solution of node i at time t+1 is denoted as Its update expression is:

[0045]

[0046] where N i is the neighborhood of node i, j is the neighbor node of node i within the neighborhood N i , w ij is an element in the weight matrix W of the time-varying network, reflecting the weight of node i on the information of its neighbor node j; is the temporary variable of neighbor node j of i at time t, α is a fixed step size, is the gradient descent direction of the estimated solution of node i at time t+1.

[0047] Preferably, step S8 is specifically:

[0048] Each time steps S2-S7 are repeated is recorded as one iteration. Each time an iteration is completed, it is necessary to evaluate the optimal error of the loss function corresponding to the currently output estimated solution. If the set error range is not reached, repeat the iteration until the set error range or the set maximum number of iterations is reached, and end the algorithm training.

[0049] More specifically, the decreasing step size θ in steps S2 and S6 above t is specifically:

[0050]

[0051] where t represents time t;

[0052] The fixed step size α in steps S6 and S7 selects different specific values for different classification problems, and the specific selection range is: α∈{0.001, 0.01, 0.1}.

[0053] Furthermore, the above NGTAdam algorithm is used to achieve fast classification of images, specifically including the following steps:

[0054] (1) Establish an image dataset, use the collected real image data to establish a dataset, mark the training samples and test samples in the image dataset, and form a training set and a test set;

[0055] (2) Import the dataset to be distributed and perform data preprocessing. On the decentralized network, distribute the data to all nodes;

[0056] (3) Initialize the model parameters of the NGTAdam algorithm on each node;

[0057] (4) Construct a distributed optimization problem based on the NGTAdam algorithm. Define the objective function of the optimization problem as the combination of the average hinge loss sum of all data points and the regularization term, i.e.:

[0058]

[0059] where w represents the weight vector to be optimized, corresponding to the estimated solution finally output by the NGTAdam algorithm; c i represents the class or classification target to which the data point b i belongs, N represents the total number of nodes in the distributed system, and λ is the regularization parameter; the loss of the i-th node is calculated through the function f i (w) to evaluate the optimization performance.

[0060] (5) Import the image dataset and use the NGTAdam algorithm for training under the framework of the distributed optimization problem;

[0061] (6) Output the classification result and evaluate the accuracy of image classification;

[0062] (7) Deploy the trained model to the actual environment for actual image classification tasks.

[0063] Based on the above technical solutions, the present invention has the following beneficial effects:

[0064] The distributed model training algorithm (NGTAdam algorithm) proposed by the present invention improves the commonly used Adam algorithm for image classification, introduces the Nesterov momentum and gradient tracking mechanism, and effectively solves the problems such as large computational resources required and low optimization efficiency when existing optimization algorithms handle large-scale image classification tasks. In addition, the present invention provides a theoretical support for the NGTAdam algorithm to solve the online optimization problem, ensuring its convergence.

[0065] Compared with several existing image classification technologies, the present invention has the following further advantages:

[0066] (1) Compared with the centralized algorithm, the NGTAdam algorithm provided by the present invention can process data in parallel on multiple computing nodes, breaking through the computational bottleneck of the centralized algorithm, which makes it more efficient in processing large-scale image datasets.

[0067] (2) Compared with existing distributed algorithms, the NGTAdam algorithm provided by the present invention combines the advantages of the Nesterov momentum, gradient tracking mechanism, and adaptive moment estimation method, greatly improving the speed of image classification while ensuring the accuracy of image classification, and having better performance and efficiency in processing large-scale image datasets.

[0068] (3) The NGTAdam algorithm provided by the present invention has scalability and can be applied not only to image classification tasks but also directly to all online optimization scenarios where the objective function is convex. In addition, the NGTAdam algorithm can adjust the number of iterations and the convergence threshold according to the complexity of the data model, and adjust the step size according to the specific scenario to ensure optimal performance.

[0069] (4) The convergence of the NGTAdam algorithm provided by the present invention is theoretically guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is the flowchart of the NGTAdam algorithm of the present invention;

[0071] Figure 2 is the flowchart of the NGTAdam algorithm performing an image classification task;

[0072] Figure 3 is the result comparison diagram of the NGTAdam algorithm performing multi-type image classification tasks;

[0073] Figure 4 is the result comparison diagram of the NGTAdam algorithm performing binary image classification tasks. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0074] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0075] Although the steps in the present invention are numbered, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It can be understood that the term "and / or" used herein relates to and encompasses any and all possible combinations of one or more of the associated listed items.

[0076] A distributed model training algorithm for fast image classification problems proposed by the present invention has its core in the Nesterov accelerated gradient tracking adaptive moment estimation algorithm: NGTAdam. The NGTAdam algorithm is a distributed online optimization algorithm that adds the Nesterov momentum and gradient tracking mechanism to accelerate the Adam algorithm to accelerate the completion of image classification tasks.

[0077] As Figure 1 shown, a distributed model training algorithm for fast image classification problems proposed by the present invention specifically includes:

[0078] S1. Initialize the parameters of the NGTAdam algorithm; import the image dataset and perform data preprocessing. On the decentralized network, distribute the training data to all nodes;

[0079] In this embodiment, the image classification problem is divided into: multi-type image classification and binary image classification;

[0080] The parameters for initializing the NGTAdam algorithm include: for the number of nodes N, decay rates β 1 , β 2 , decreasing step size θ t , parameter ∈, first moment estimator second moment estimator gradient estimator gradient and estimated solution Assign different initial values to different image problems:

[0081] For multi-type image classification: set the number of nodes N = 40, decay rates β 1 = 0.9, β 2 = 0.999, the dimension of model parameters q = 100, regularization parameter λ = 0.1; after parameter adjustment, set the fixed step size α = 0.01, decreasing step size It should be noted that the specific step size is adjusted according to the classification task of the image dataset; set the first moment estimator second moment estimator Nesterov momentum variable temporary variable All elements of are 0, gradient estimator where the estimated solution is set arbitrarily; parameter ∈ = 10 -4 , convergence threshold tol_optgap = 10 -24 , mini-batch size is set to 10, and the maximum number of iterations is set to 100.

[0082] For binary image classification: set the number of nodes N = 10, fixed step size α = 0.1, and the remaining parameters are the same as those in the multi-type image classification task.

[0083] In addition, in this embodiment, use the seed command of the random number generator: rng(238) to ensure that the same random number sequence is obtained each time the program runs, so as to ensure that repeatable random optimization experiments can be carried out and improve the rigor of the experimental results.

[0084] After initializing the parameters, perform data preprocessing. Different problems still adopt different preprocessing methods:

[0085] In the embodiments, the MNIST dataset and the Mushroom dataset are selected to test the image classification performance. These two datasets respectively represent the classic problems of image recognition and classification tasks in machine learning, corresponding to typical applications in the fields of computer vision and data mining respectively.

[0086] Multi-type image classification: The multi-type image classification task is evaluated on the MNIST dataset. The MNIST dataset contains grayscale images of handwritten digits with a pixel size of 28×28, and these digits range from 0 to 9. In the training set, there are 6,000 training images for model training. In the test set, there are 10,000 test images for evaluating the performance of the model. Each image has a corresponding label indicating the handwritten digit in the image, and the label is an integer between 0 and 9, corresponding to 10 classifications.

[0087] Image binary classification: The image binary classification task is evaluated on the Mushroom dataset using a Linear Support Vector Machine (Linear SVM). The Mushroom dataset is a classic binary classification problem dataset, which contains 8,124 samples. Each sample has 22 feature descriptions of the mushroom's attributes, such as the shape, color, smell, etc. of the mushroom, and 1 label indicating whether the mushroom is edible. The dataset is split into a training set and a test set for evaluating the training effect and generalization ability of the model.

[0088] The MNIST and Mushroom datasets are imported respectively, and useful information is read from the datasets, including the feature matrices of the training data and test data: x_train and x_test, the labels of the training data and test data: y_train and y_test, the optimal weight vector w_opt, and the number of classifications l.

[0089] After the initialization and preprocessing in step S1 above, the training of the model begins.

[0090] S2. On each node, update the temporary variable using the estimated solution and the Nesterov momentum variable, and calculate the current gradient using the temporary variable;

[0091] More specifically, step S2 includes:

[0092] S21. Update the temporary variable; use the estimated solution and the Nesterov momentum variable to update the temporary variable of node i at time t Its formula expression is:

[0093]

[0094] where θ t represents the decreasing step size, is the estimated solution of node i at time t, is the estimated Nesterov momentum variable of node i at time t;

[0095] S22. Calculate the current gradient; Use the temporary variable obtained in step S21 to calculate the gradient of node i at time t Its formula is expressed as:

[0096]

[0097] where is the gradient calculation function.

[0098] S3. On each node, update the gradient estimator based on the gradient tracking method:

[0099] The gradient estimator of node i at time t is denoted as Its update expression is:

[0100]

[0101] where N i represents the neighborhood of node i, and j is the neighbor node of node i in the neighborhood N i ; are the gradients of node i at time t and t - 1 respectively, w ij is an element in the weight matrix W of the time-varying network, reflecting the weight of node i on the information of its neighbor node j.

[0102] The gradient estimator is used to represent the update descent direction of the Nesterov momentum variable, which is beneficial to accelerating convergence. It also directly determines the update directions of the first-order moment estimate, the second-order moment estimate, and the Nesterov momentum variable, and indirectly affects the learning rate and the update of the estimated solution.

[0103] S4. On each node, update the first-order moment estimate and the second-order matrix estimate using exponential moving average;

[0104] More specifically, step S4 includes:

[0105] S41. Update the first-order moment estimate; Node i updates the first-order moment estimate using exponential moving average. The first-order moment estimate of node i at time t + 1 is denoted as Its update expression is:

[0106]

[0107] where β 1 represents the exponential decay rate of the first-order moment estimate, is the first-order moment estimate of node i at time t, is the gradient estimator of node i at time t;

[0108] S42. Update the second-moment estimate; Node i updates the second-moment estimate using exponential moving average. The second-moment estimate of node i at time t + 1 is denoted as Its update expression is:

[0109]

[0110] where β 2 represents the exponential decay rate of the second-moment estimate, the symbol ⊙ represents the Hadamard product of matrices, and V m > 0 is the saturation parameter to ensure the upper bound of the estimate; is the second-moment estimate of node i at time t.

[0111] S5. On each node, adaptively update the gradient descent direction of the estimated solution using the first-moment estimate and the second-moment estimate updated in step S4:

[0112] Use the first-moment estimate and the second-moment estimate updated in step S4 to adaptively update the descent direction of the estimated solution. The gradient descent direction of the estimated solution of node i at time t + 1 is denoted as Its update expression is:

[0113]

[0114] where the parameter ∈ > 0 is used to ensure the stability of numerical operations and avoid gradient explosion; are the first-moment estimate and the second-moment estimate of node i at time t + 1, respectively.

[0115] S6. Combine the Nesterov momentum variables of neighbor nodes and update the Nesterov momentum variable of this node based on the gradient descent method:

[0116] Node i combines the Nesterov momentum variables of neighbor nodes and updates its own Nesterov momentum variable based on the gradient descent method. The Nesterov momentum variable of node i at time t + 1 is denoted as Its update expression is:

[0117]

[0118] where N i is the neighborhood of node i, j is the neighbor node of node i in the neighborhood N i and is the Nesterov momentum variable of neighbor node j of node i at time t; α is the fixed step size, θ t is the decreasing step size, and w ijis an element in the weight matrix W of the time-varying network, reflecting the weight of node i on the information of its neighbor node j; the gradient estimator represents the update descent direction.

[0119] S7. Combine the temporary variables of neighbor nodes and update the estimated solution of this node based on the gradient descent method:

[0120] Node i combines the temporary variables of neighbor node j and updates the estimated solution based on the gradient descent method; the estimated solution of node i at time t + 1 is denoted as Its update expression is:

[0121]

[0122] where N i is the neighborhood of node i, j is the neighbor node of node i in the neighborhood N i , w ij is an element in the weight matrix W of the time-varying network, reflecting the weight of node i on the information of its neighbor node j; is the temporary variable of neighbor node j of i at time t, α is a fixed step size, is the gradient descent direction of the estimated solution of node i at time t + 1.

[0123] S8. Repeat S2 - S7 for multiple iterations until the algorithm termination condition is satisfied.

[0124] More specifically, in this example, the process from step S2 to step S7 is denoted as one iteration process of the NGTAdam algorithm. All nodes from 1 to N need to complete one iteration. After completion, calculate the error value and compare it with the set convergence threshold tol_optgap. The error value can be selected as needed, such as the optimal solution error or the optimal loss function error, etc.; when the error value > tol_optgap and the number of iterations < the maximum number of iterations, repeat the iteration until the set conditions for iteration are no longer satisfied.

[0125] In addition, in this embodiment, the specific process of using the above NGTAdam algorithm for fast image classification is as follows:

[0126] (1) Establish an image data set. Use the collected real image data to establish the data set, mark the training samples and test samples in the image data set, and form the training set and test set;

[0127] (2) Import the data set to be distributed and perform data preprocessing. On the decentralized network, distribute the data to all nodes;

[0128] (3) Initialize the NGTAdam algorithm model parameters on each node;

[0129] (4) Construct a distributed optimization problem based on the NGTAdam algorithm. Define the objective function of the optimization problem as the combination of the average hinge loss sum of all data points and the regularization term, i.e.,

[0130]

[0131] where w represents the weight vector to be optimized, corresponding to the estimated solution finally output by the NGTAdam algorithm; c i represents the class or classification target to which the data point b i belongs. N represents the total number of nodes in the distributed system, and λ is the regularization parameter; the loss of the i-th node is calculated through the function f i (w) to evaluate the optimization performance.

[0132] (5) Import the image dataset and use the NGTAdam algorithm for training under the framework of the distributed optimization problem;

[0133] (6) Output the classification result and evaluate the accuracy of image classification;

[0134] (7) Deploy the trained model to the actual environment for actual image classification tasks.

[0135] In this embodiment, NGTAdam is compared with DAdam and Acc-DNGD algorithms, focusing on the convergence speed and convergence error of the algorithms; DAdam is a distributed Adam algorithm, commonly used in image classification tasks, showing the advantages of the adaptive moment estimation algorithm; the Acc-DNGD algorithm is a Nesterov acceleration algorithm based on the gradient descent method, showing the acceleration advantages of the Nesterov momentum and gradient tracking mechanisms; the proposed NGTAdam algorithm of the present invention combines the Nesterov momentum, gradient tracking and adaptive moment estimation methods, having the advantages of both.

[0136] In this embodiment, the optimal value error is further calculated, and the error curve and classification result are output to evaluate the classification accuracy.

[0137] Select the optimal loss function error to evaluate the algorithm performance, and the calculation expression is as follows:

[0138]

[0139] where represents the average value of the estimated solutions of all nodes at time t. is the optimal solution of the centralized solution of the optimization problem at time t.

[0140] Figure 3It is a convergence curve graph for performing multi-type image classification tasks, where the vertical axis represents the optimal loss function error and the horizontal axis represents the total number of gradient calculations. From Figure 3 It can be seen that compared with Acc-DNGD, DAdam has a faster convergence speed, which reflects the advantage of adaptive momentum estimation. The convergence speed of NGTAdam is significantly better than that of DAdam and Acc-DNGD, and it can achieve a smaller convergence error in a shorter time.

[0141] Figure 4 It is a convergence curve graph for performing binary image classification tasks. Similarly, the vertical axis represents the optimal loss function error and the horizontal axis represents the total number of gradient calculations. From Figure 4 It can be seen that the convergence speeds of Acc-DNGD and DAdam are similar. The convergence speed of NGTAdam is significantly better than the other two algorithms, and it can obtain a smaller optimization error in a shorter time. The classification results show that NGTAdam can achieve an accuracy rate of 100% in binary image classification tasks and 99.897% in multi-type image classification tasks.

[0142] These two embodiments show that the image classification method proposed by the present invention has a faster classification speed while ensuring accurate classification, and has advantages in both image classification efficiency and accuracy.

[0143] In addition, the present invention gives the dynamic regret upper bound of NGTAdam for online convex optimization problems, indicating the theoretical feasibility of the algorithm when dealing with online convex problems.

[0144] Through theoretical analysis, the dynamic upper bound of the convergence rate of NGTAdam when solving line convex optimization problems is:

[0145]

[0146] Among them, the matrix is related to the initial conditions of the linear inequality system, Q T and W T are related to the time-varying nature of the online optimization problem. L is the Lipschitz constant of the optimization objective function, is the second largest eigenvalue of the communication network weight matrix W, and the constant λ > 0. Q T and W T both have explicit upper bounds. In the application of this invention, we do not need to pay attention to the specific upper bound expressions, so the upper bound expressions are omitted here.

[0147] In summary, the method proposed by the present invention focuses on real-time decision-making and dynamic optimization, can efficiently solve large-scale online optimization problems, is applicable to various scenarios, and solves the problems of slow speed and poor accuracy of existing image classification algorithms when training large-scale image data.

[0148] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

[0149] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A distributed model training algorithm for fast image classification problem, characterized in that: The distributed model training algorithm is an adaptive moment estimation algorithm based on Nesterov acceleration, referred to as the NGTAdam algorithm, which specifically includes the following steps: S1. Initialize the parameters of the NGT Adam algorithm; import the image data set and perform data preprocessing, and distribute the training data to all nodes on the decentralized network; S2. At each node, the temporary variables are updated using the estimated solution and the Nesterov momentum variable, and the current gradient is calculated using the temporary variables. S3. At each node, update the gradient estimator based on the gradient tracking method; S4, at each node, using exponential moving average to update the first-order moment estimate and the second-order matrix estimate; step S4 specifically includes: S41, update the first-order moment estimate; node i uses exponential moving average to update the first-order moment estimate, and the first-order moment estimate of node i at time t+1 is recorded as Its update expression is: Where β1 represents the exponential decay rate of the first-order moment estimate, is the first-order moment estimate of node i at time t, is the gradient estimator of node i at time t; S42, update the second-order moment estimate; node i uses exponential moving average to update the second-order moment estimate, and the second-order moment estimate of node i at time t+1 is recorded as Its update expression is: Where β2 represents the exponential decay rate of the second-order moment estimate, the symbol ⊙ represents the Hadamard product of the matrix, and V m >>0 is the saturation parameter, which guarantees the upper bound of the estimated value; is the second-order moment estimate of node i at time t, is the gradient estimator of node i at time t; S5. At each node, adaptively update the gradient descent direction of the estimated solution using the first-order moment estimate and the second-order moment estimate updated in step S4; S6. Combine the Nesterov momentum variables of neighboring nodes and update the Nesterov momentum variables of this node based on the gradient descent method; S7, combining the temporary variables of neighboring nodes, and updating the estimated solution of this node based on the gradient descent method; S8. Repeat S2-S7 for multiple iterations until the algorithm termination condition is met.

2. A distributed model training algorithm for fast image classification problem according to claim 1 The method is characterized in that Step S2 is specifically as follows: S21, update temporary variables; use the estimated solution and Nesterov momentum variables to update the temporary variables of node i at time t Its formula is: Among them, θ t Indicates the decreasing step size, is the estimated solution of node i at time t, is the estimated Nesterov momentum variable of node i at time t; S22, calculate the current gradient; use the temporary variable obtained in step S21 Calculate the gradient of node i at time t Its formula is: in, is the gradient calculation function.

3. The distributed model training algorithm for fast image classification problem according to claim 1, characterized in that: Step S3 is specifically as follows: The gradient estimator of node i at time t is recorded as It is used to indicate the update descent direction of the Nesterov momentum variable, and its update expression is: Among them, N i represents the neighborhood of node i, j is the neighborhood of node i in N i Neighbor nodes in are the gradient w of node i at time t and time t-1 respectively. ij It is an element in the weight matrix W of the time-varying network, reflecting the weight of node i on the information of its neighbor node j.

4. The distributed model training algorithm for fast image classification problem according to claim 1, characterized in that: Step S5 is specifically as follows: The descent direction of the estimated solution is adaptively updated using the first-order moment estimate and the second-order moment estimate updated in step S4. The gradient descent direction of the estimated solution of node i at time t+1 is recorded as Its update expression is: Among them, the parameter ∈>0 is used to ensure the stability of numerical operations and avoid gradient explosion; are the first-order moment estimate and the second-order moment estimate of node i at time t+1, respectively.

5. The distributed model training algorithm for fast image classification problem according to claim 1, characterized in that: Step S6 is specifically as follows: Node i combines the Nesterov momentum variables of neighboring nodes and updates its own Nesterov momentum variables based on the gradient descent method. The Nesterov momentum variable of node i at time t+1 is recorded as Its update expression is: Among them, N i is the neighborhood of node i, j is the neighborhood of node i in domain N i Neighbor nodes within is the Nesterov momentum variable of node i’s neighbor node j at time t; α is a fixed step size, θ t is the decreasing step size, w ij It is an element in the weight matrix W of the time-varying network, reflecting the weight of node i on the information of its neighbor node j.

6. A distributed model training algorithm for fast image classification problem according to claim 1, characterized in that: Step S7 is specifically as follows: Node i combines the temporary variables of neighbor node j and updates the estimated solution based on the gradient descent method; the estimated solution of node i at time t+1 is recorded as Its update expression is: Among them, N i is the neighborhood of node i, j is the neighborhood of node i in domain N i Neighbor nodes within, w ij It is an element in the weight matrix W of the time-varying network, reflecting the weight of the information of node i on its neighbor node j; is the temporary variable of i’s neighbor node j at time t, α is a fixed step size, is the gradient descent direction of the estimated solution of node i at time t+1.

7. A distributed model training algorithm for fast image classification problem according to claim 2, 5 or 6, characterized in that: The decrement step length θ in step S2 and step S6 t Specifically: Wherein, t represents time t; The fixed step length α in step S6 and step S7 selects different specific values ​​for different classification problems. The specific selection range is: α∈{0.001, 0.01, 0.1}.

8. The distributed model training algorithm for fast image classification problem according to claim 1, characterized in that: Step S8 is specifically as follows: Each repetition of steps S2-S7 is recorded as an iteration. After each iteration, the optimal error of the loss function corresponding to the estimated solution of the current output needs to be evaluated. If it does not reach the set error range, the iteration is repeated until the set error range or the set maximum number of iterations is reached, and the algorithm training is terminated.

9. The distributed model training algorithm for fast image classification problem according to claim 1, characterized in that: The NGTAdam algorithm is used to achieve rapid classification of images, which specifically includes the following steps: (1) Establish an image dataset by using the collected real image data to establish the dataset, label the training samples and test samples in the image dataset, and form a training set and a test set; (2) Import the data set to be distributed and perform data preprocessing, and distribute the data to all nodes on the decentralized network; (3) Initialize the NGT Adam algorithm model parameters at each node; (4) Construct a distributed optimization problem based on the NGTAdam algorithm and define the objective function of the optimization problem as the average hinge loss of all data points combined with the regularization term, that is: in: w represents the weight vector to be optimized and corresponds to the estimated solution finally output by the NGTAdam algorithm; c i Represents data point b i The category or classification target, N represents the total number of nodes in the distributed system, and λ is the regularization parameter; through the function f i (w) Calculate the loss of the i-th node to evaluate the optimization performance; (5) Import the image dataset and use the NGT Adam algorithm for training in the framework of distributed optimization problem; (6) Output classification results and evaluate the accuracy of image classification; (7) Deploy the trained model to the actual environment to perform actual image classification tasks.

Images