Self-adaptive machine learning algorithm based on dynamic weight adjustment

Abstract

The invention provides a self-adaptive machine learning algorithm based on dynamic weight adjustment. The self-adaptive machine learning algorithm comprises the following steps: preprocessing data; initializing model parameters; the dynamic weight adjusting module is used for calculating statistical characteristics of the gradient, adjusting a learning rate, adjusting a weight updating strategy and updating the weight; model training; evaluating the model; and iteratively updating. According to the method, the statistical characteristics of the gradient are calculated, the learning rate and the weight updating strategy are dynamically adjusted, and momentum and second-order moment correction are introduced, so that the model can adaptively update parameters in the training process, and the convergence speed and the stability are improved. Experiments show that the method has high accuracy and generalization ability when being used for processing non-stationary data and noise data, so that the method has a wide application prospect.

Description

Technical Field

[0001] The present invention relates to the field of computer software algorithms, and in particular to optimization algorithms in machine learning and deep learning. Specifically, the present invention relates to an adaptive machine learning algorithm based on dynamic weight adjustment, which is used to improve the training efficiency and generalization ability of a model. Background Art

[0002] In the field of machine learning, the choice of optimization algorithm has a crucial impact on the performance of the model. Traditional optimization algorithms, such as Gradient Descent and Stochastic Gradient Descent (SGD), usually use a fixed learning rate or a preset learning rate decay strategy. However, in practical applications, the distribution of data may be non-stationary, and the data set may contain noise, outliers, or complex feature relationships, all of which will affect the convergence speed and final performance of the model.

[0003] Adaptive optimization algorithms, such as AdaGrad, RMSProp, and Adam, improve the training effect of models on complex data sets by introducing adaptive learning rate adjustment strategies. However, these algorithms still have some shortcomings, such as poor performance on sparse data or non-convex optimization problems.

[0004] Therefore, designing an optimization algorithm that can dynamically adjust weights and learning rates according to data characteristics and gradient information is of great significance to improving the training efficiency and generalization ability of the model. Summary of the invention

[0005] The present invention aims to overcome the shortcomings of the prior art and provide an adaptive machine learning algorithm based on dynamic weight adjustment. By real-time monitoring of gradient changes, the learning rate and weight update strategy are dynamically adjusted, so that the model can converge to the global optimal solution faster, while improving the robustness to non-stationary data and noisy data.

[0006] To achieve the above object, the present invention proposes the following technical solutions:

[0007] An adaptive machine learning algorithm based on dynamic weight adjustment includes the following steps:

[0008] Step 1, data preprocessing: normalize, standardize, reduce dimension, extract and select features of input data to improve data quality;

[0009] Step 2: Initialize model parameters: Initialize the model weights and biases randomly or through pre-training according to the model structure;

[0010] Step 3, dynamic weight adjustment module: In each iteration, the statistical properties of the gradient are calculated, and the learning rate and weight update strategy are dynamically adjusted according to these properties;

[0011] Step 4, model training: Use the adjusted parameters to train the model using forward propagation and back propagation algorithms;

[0012] Step 5, model evaluation: Evaluate the performance of the model on the validation set or test set;

[0013] Step 6, iterative update: Based on the evaluation results of the model and the changing trend of the loss function, determine whether it is necessary to continue training or adjust the parameters until the preset stopping conditions are met.

[0014] In step 3, the statistical properties of the gradient include but are not limited to mean, variance, skewness and kurtosis.

[0015] The dynamic weight adjustment method described in step 3 includes the following four steps:

[0016] Step 1: Compute the statistical properties of the gradient

[0017] Gradient Mean

[0018] Gradient Variance

[0019] Gradient skewness

[0020] Gradient Kurtosis

[0021] Step 2: Adjust the learning rate

[0022] Learning rate adjustment based on gradient variance: When the gradient variance is large, it means that the gradient fluctuates greatly, and the learning rate should be reduced to avoid excessive update steps. When the gradient variance is small, it means that the gradient is relatively stable, and the learning rate should be increased to speed up the convergence.

[0023] Step 3: Adjust weight update strategy

[0024] Introduction of momentum term: v t+1 =βv t +1-β)g t , β is the momentum coefficient, which is generally set to 0.9. Using momentum can smooth the gradient update and prevent falling into the local minimum.

[0025] Second-order moment correction: γ is a smoothing parameter, which is generally taken as 0.999.

[0026] Step 3 combines momentum and second-order moment to further improve the stability of the algorithm.

[0027] Step 4: Update weights

[0028] Comprehensive update formula: ∈ is a small constant to prevent division by zero, usually 10 -8 . This formula combines momentum and an adaptive learning rate adjustment strategy.

[0029] In step 5, the performance of the model includes but is not limited to accuracy, precision, recall, F1 score and AUC indicators.

[0030] In step 6, the loss function is denoted as Lθ, which is a model optimization objective function that measures the difference between the model prediction result and the true value; Lθ≥0, which is used to guide the gradient calculation and optimize the target.

[0031] In step 6, the preset stop condition includes a preset threshold and a maximum number of iterations; the preset threshold is the target value reached by the performance indicator of the model, and when the preset threshold is reached, the training stops; the maximum number of iterations is the maximum number of iterations allowed during the training process to prevent the model from overfitting training.

[0032] The present invention overcomes the shortcomings of traditional optimization algorithms by dynamically adjusting the learning rate and weight update strategy, improves the training efficiency and generalization ability of the model, and has the following beneficial effects:

[0033] 1. Improve convergence speed

[0034] (1) Adaptive learning rate adjustment: The learning rate is dynamically adjusted according to the variance of the gradient. When the gradient fluctuates greatly, the learning rate is reduced to avoid oscillation caused by excessive update step size. When the gradient fluctuates slightly, the learning rate is increased to speed up the convergence of the model.

[0035] (2) Momentum and second-order moment correction: The introduction of momentum terms and smoothed second-order moments makes the parameter update direction more accurate and further accelerates the convergence process.

[0036] 2. Enhance the stability of the model

[0037] (1) Preventing overfitting and underfitting: By dynamically adjusting the weight update strategy and adaptively adjusting the parameter update according to the statistical characteristics of the gradient, the complexity of the model is effectively balanced and the risk of overfitting and underfitting is reduced.

[0038] (2) Improved robustness: The algorithm has a higher tolerance to noisy data and outliers and can maintain good performance in non-stationary data environments.

[0039] 3. Improve generalization ability

[0040] (1) Adaptive weight update: Dynamic weight adjustment enables the model to better adapt to the characteristics of the training data, thereby improving the prediction accuracy on unknown data.

[0041] (2) Preventing the local optimal trap: The introduction of the momentum term helps the model escape from the local minimum and move towards the global optimal solution.

[0042] 4. Wide range of applications

[0043] (1) Strong versatility: The algorithm does not depend on a specific model structure or data type and is applicable to a variety of machine learning models such as deep neural networks, support vector machines, and decision trees.

[0044] (2) Easy to integrate: The algorithm’s dynamic adjustment strategy can be combined with existing optimization algorithms (such as SGD, Adam, etc.) to enhance their performance.

[0045] 5. Reduce computing costs

[0046] (1) Efficient calculation: The statistical properties of the gradient are simple to calculate, and the additional computational overhead is small, which does not significantly increase the training time.

[0047] (2) Reduce parameter adjustment work: Adaptive learning rate and weight adjustment strategies reduce the sensitivity to hyperparameters and reduce the complexity of manual parameter adjustment.

[0048] 6. Improve model performance

[0049] (1) Improved accuracy: Experimental results show that the model using the algorithm of the present invention achieves higher accuracy and F1 score in tasks such as classification and regression.

[0050] (2) Loss function converges faster: Compared with traditional optimization algorithms, the loss function decreases faster and reaches convergence earlier.

[0051] 7. Theoretical innovation

[0052] (1) Novel algorithmic framework: By combining the high-order statistical properties of gradients, a new optimization algorithm is proposed, which provides new ideas for the field of machine learning optimization.

[0053] (2) Scalability: The algorithm framework can be further expanded by introducing more statistical features or combining other optimization techniques, which has great research and application value.

[0054] 8. Significant practical application effects

[0055] (1) Verification on real datasets: The algorithm has demonstrated excellent performance in practical tasks such as image classification, natural language processing, and time series prediction.

[0056] (2) Improved user experience: For applications that require real-time response, such as online recommendation systems and real-time risk assessment, the efficiency and accuracy of the algorithm can significantly improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is an overall flow chart of the adaptive machine learning algorithm based on dynamic weight adjustment described in the present invention.

[0058] Figure 2 Flowchart of the dynamic weight adjustment according to the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] 1. Specific embodiments of the present invention

[0061] like Figure 1 As shown, the adaptive machine learning algorithm based on dynamic weight adjustment of the present invention comprises the following steps:

[0062] 1. Data preprocessing: Perform preprocessing operations such as normalization, standardization, dimensionality reduction and feature extraction on the input data to improve the quality of the data.

[0063] Data cleaning: dealing with missing values, outliers, and noisy data.

[0064] Data normalization: Mapping data to a specific range, such as [0,1] or a standard normal distribution.

[0065] Feature extraction and selection: Use dimensionality reduction algorithms such as PCA and LDA, or perform feature selection based on feature importance.

[0066] 2. Initialize model parameters: Initialize the model weights and biases randomly or through pre-training according to the model structure.

[0067] Model selection: Select the appropriate model structure according to the task, such as the number of neural network layers, the number of neurons in each layer, the activation function, etc.

[0068] Parameter initialization: Use He initialization, Xavier initialization, and other methods to randomly initialize weights and biases, or load the parameters of a pre-trained model.

[0069] 3. Dynamic weight adjustment module: In each iteration, the statistical properties of the gradient are calculated, and the learning rate and weight update strategy are dynamically adjusted based on these properties.

[0070] Calculate the statistical characteristics of the gradient: In each iteration, record the historical data of the gradient and calculate the mean, variance, skewness, kurtosis, etc. of the gradient.

[0071] Dynamic adjustment strategy:

[0072] Learning rate adjustment: Dynamically adjust the learning rate based on the variance or second-order moment information of the gradient.

[0073] Weight update strategy: According to the skewness and kurtosis of the gradient, determine the degree of deviation of the gradient distribution and adjust the update direction and amplitude of the weight.

[0074] 4. Model training: Use the adjusted parameters to train the model using forward propagation and back propagation algorithms.

[0075] Forward propagation: Calculate the output and loss function value of the model under the current parameters.

[0076] Back propagation: Calculate the gradient of the loss function with respect to each parameter.

[0077] Parameter update: Update the model parameters using the dynamically adjusted learning rate and weight update strategy.

[0078] 5. Model evaluation: Evaluate the performance of the model on the validation set or test set, including but not limited to accuracy, precision, recall, F1 score, AUC and other indicators.

[0079] Performance indicator calculation: Calculate various performance indicators of the model on the validation set.

[0080] Overfitting detection: Monitor the loss function values ​​of the training set and validation set to determine whether overfitting occurs.

[0081] 6. Iterative update: Based on the evaluation results of the model and the changing trend of the loss function, determine whether it is necessary to continue training or adjust the parameters until the preset stopping conditions are met.

[0082] Stop condition judgment: If the model performance reaches the preset threshold, or the loss function converges, or the number of iterations reaches the upper limit, the training is stopped.

[0083] Adjustment strategy: If the stopping criteria are not met, you may need to adjust the model structure, reinitialize parameters, or modify the dynamic adjustment strategy.

[0084] like Figure 2 As shown, the above-mentioned dynamic weight adjustment method includes the following 4 steps:

[0085] Step 1: Compute the statistical properties of the gradient

[0086] Gradient Mean

[0087] Gradient Variance

[0088] Gradient skewness

[0089] Gradient Kurtosis

[0090] Step 2: Adjust the learning rate

[0091] Learning rate adjustment based on gradient variance: When the gradient variance is large, it means that the gradient fluctuates greatly, and the learning rate should be reduced to avoid excessive update steps. When the gradient variance is small, it means that the gradient is relatively stable, and the learning rate should be increased to speed up the convergence.

[0092] Step 3: Adjust weight update strategy

[0093] Introduction of momentum term: v t+1 =βv t +1-β)g t , β is the momentum coefficient, which is generally set to 0.9. Using momentum can smooth the gradient update and prevent falling into the local minimum.

[0094] Second-order moment correction: γ is a smoothing parameter, which is generally taken as 0.999.

[0095] Step 3 combines momentum and second-order moment to further improve the stability of the algorithm.

[0096] Step 4: Update weights

[0097] Comprehensive update formula: ε is a small constant to prevent division by zero, usually set to 10 -8 . This formula combines momentum and an adaptive learning rate adjustment strategy.

[0098] 2. Description of parameters involved in the present invention:

[0099] The adaptive machine learning algorithm based on dynamic weight adjustment described in the present invention involves multiple key parameters, which play a vital role in the execution of the algorithm and affect the training efficiency, convergence speed and final performance of the model. The following will describe each parameter in detail, including its meaning, value range, typical value and impact on the algorithm.

[0100] 1. Learning rate related parameters

[0101] 1.1 Initial learning rate η0

[0102] Meaning: The initial learning rate is the basic learning rate used when the algorithm starts training, which is used to control the step size of each parameter update.

[0103] Value range: Usually a positive real number, η0>0.

[0104] Typical values: 0.001, 0.01, 0.1, depending on the specific task and model.

[0105] Impact: If the value is too large, the loss function may oscillate and fail to converge. If the value is too small, the convergence speed will be slow and the training time will be too long.

[0106] Selection method: This can be done through cross-validation or dynamically adjusted using a learning rate scheduler. Usually the search is done on a logarithmic scale, for example, from 10 -5 to 10 -1 .

[0107] 1.2 Learning rate adjustment parameter λ

[0108] Meaning: Used to control the sensitivity of the learning rate to the gradient variance V(g), and plays a regulatory role in the dynamic learning rate adjustment formula.

[0109] Value range: non-negative real number, λ≥0.

[0110] Typical values: 0.1, 1, 10, depending on the gradient fluctuation and data characteristics.

[0111] Impact: (1) Increasing λ makes the learning rate more sensitive to the gradient variance and the learning rate changes more significantly. (2) Decreasing λ makes the learning rate change more slowly and closer to a fixed learning rate.

[0112] Selection method: Set according to the noise level and gradient volatility of the dataset. You can find the best value by adjusting λ on the validation set.

[0113] 1.3 Learning rate η t

[0114] Meaning: The actual learning rate used in the tth iteration is dynamically adjusted according to the initial learning rate and the gradient variance.

[0115] Calculation formula:

[0116] Impact: (1) Adaptive adjustment enables the learning rate to change dynamically according to the volatility of the gradient, improving the stability and efficiency of training. (2) Preventing excessive updates. When the gradient fluctuates greatly, the learning rate is automatically reduced to prevent convergence problems caused by excessive parameter updates.

[0117] 2. Gradient related parameters

[0118] 2.1 Gradient g t

[0119] Meaning: The derivative of the loss function with respect to the parameters, representing the gradient vector at the tth iteration.

[0120] Calculation method: Calculated by back propagation algorithm.

[0121] Impact: Guiding the direction and magnitude of parameter updates. The size and direction of the gradient directly affect the convergence of the model.

[0122] 2.2 Gradient variance V(g)

[0123] Meaning: The variance of the gradient indicates the volatility of the gradient and measures the degree of change of the gradient in each iteration.

[0124] Calculation formula: V(g) = E[g 2 ]-E([g]) 2

[0125] Where E[g] is the expected value of the gradient (gradient mean).

[0126] Impact: Used to dynamically adjust the learning rate. When the gradient variance is large, the learning rate decreases, and vice versa.

[0127] Calculation method: It can be estimated using a sliding average or an exponentially weighted moving average (EWMA).

[0128] 2.3 Gradient momentum term v t

[0129] Meaning: The exponentially weighted moving average of the gradient accumulates information about past gradients and smoothes the update direction of the gradient.

[0130] Update formula: v t+1 =βv t +(1-β)g t

[0131] Value range: vector, and gradient g t have the same dimensions.

[0132] Impact: (1) Smooth gradient updates, prevent drastic changes in gradient direction, and help the model escape from local minima. (2) Accelerate convergence. When near the bottom of the valley, accelerate the gradient descent along a consistent direction.

[0133] 3. Momentum and second-order moment related parameters

[0134] 3.1 Momentum coefficient β

[0135] Meaning: Controls the importance of historical gradients in the momentum term, determining the extent to which past gradients affect the current momentum.

[0136] Value range: 0≤β<1.

[0137] Typical values: 0.9, 0.99.

[0138] Impact: (1) Increasing β places more emphasis on historical gradients and reduces sensitivity to current gradients. Decreasing β (2) places more emphasis on current gradients and weakens the momentum term’s memory of historical information.

[0139] Selection method: Usually set to a value close to 1, such as 0.9, to obtain a better smoothing effect.

[0140] 3.2 Second-order moment smoothing parameter γ

[0141] Meaning: The smoothing parameter used to update the second-order moment of the gradient (the average value of the square of the gradient), which controls the degree to which the second-order moment remembers historical information.

[0142] Value range: 0≤γ<1.

[0143] Typical value: 0.999.

[0144] Impact: (1) Increasing γ emphasizes the influence of the square of the historical gradient, making the second-order moment update smoother. (2) Reducing γ emphasizes the square of the current gradient, making it more sensitive to changes in the second-order moment.

[0145] Selection method: Usually set to a value close to 1, such as 0.999, to ensure the stability of the second-order moment estimate.

[0146] 3.3 Second-order moment estimation t

[0147] Meaning: The exponentially weighted moving average of the squared gradient, representing the second-order moment estimate of the gradient.

[0148] Update formula:

[0149] Impact: (1) Adaptive adjustment of the update step size. Combined with the second-order moment estimation, the update step size of each parameter dimension can be adjusted to make the update more stable and accurate. (2) Prevent the gradient from vanishing or exploding. Through the second-order moment correction, the adverse effects of excessive or too small gradients on training are avoided.

[0150] 4. Small constant ∈ to prevent division by zero

[0151] Meaning: A small constant added to the denominator of the parameter update formula to prevent division by zero.

[0152] Value range: positive decimal, generally ∈>0.

[0153] Typical value: 10 -8 , 10 -7 .

[0154] Impact: Numerical stability, ensuring that no division by zero errors occur during the calculation process and maintaining the numerical stability of the calculation.

[0155] Selection method: Generally, a small enough positive number is selected so as not to have a significant impact on the calculation results.

[0156] 5. Parameter vector θ t

[0157] Meaning: The set of model parameters, including weights and biases, at the tth iteration.

[0158] Value range: real vector, the dimension depends on the complexity of the model.

[0159] Impact: Determines the predictive power and performance of the model. During the training process, it is continuously updated to minimize the loss function.

[0160] 6. Sample size N

[0161] Meaning: The number of samples used to calculate the statistical properties of the gradient, usually the size of a batch.

[0162] Value range: positive integer, N>0.

[0163] Typical values: 32, 64, 128, 256, depending on hardware resources and data set size.

[0164] Impact: (1) Batch size, which affects the accuracy of gradient estimation and the frequency of updates. (2) Computational efficiency, larger batches can improve hardware utilization but may lead to insufficient memory.

[0165] 7. Gradient skewness Skew(g) and kurtosis Kurt(g)

[0166] Meaning: Gradient skewness Skew(g) measures the symmetry of the gradient distribution and reflects the degree of skewness of the gradient distribution. Gradient kurtosis Kurt(g) measures the sharpness of the gradient distribution and reflects the peak height and tail thickness of the gradient distribution.

[0167] Calculation formula:

[0168]

[0169]

[0170] Impact: (1) Adjust the weight update strategy. According to the skewness and kurtosis, the shape of the gradient distribution can be judged and the parameter update method can be further optimized. (2) Risk assessment. High skewness and kurtosis may indicate that the gradient distribution is abnormal and needs to be updated with caution.

[0171] 8. Loss function L(θ)

[0172] Meaning: A function that measures the difference between the model prediction result and the true value, which is the objective function of model optimization.

[0173] Value range: non-negative real number, Lθ≥0.

[0174] Common forms: (1) Mean squared error (MSE), used for regression problems. (2) Cross entropy loss, used for classification problems.

[0175] Impact: (1) Guiding gradient calculation. The form of the loss function directly affects the calculation of the gradient. (2) Optimization goal. During the training process, the goal of parameter update is to minimize the loss function.

[0176] 9. Stop condition related parameters

[0177] 9.1 Preset Threshold

[0178] Meaning: The target value reached by the model performance indicators (such as accuracy and loss value), which is used to determine whether to stop training.

[0179] Value range: set according to specific tasks, for example, the accuracy rate reaches 95%.

[0180] Impact: (1) Training termination: When the preset threshold is reached, the training process ends. (2) Model performance: The setting of the threshold affects the performance and generalization ability of the final model.

[0181] 9.2 Maximum number of iterations t max

[0182] Meaning: The maximum number of iterations allowed during training to prevent unlimited training.

[0183] Value range: positive integer, t max >0.

[0184] Typical values: 1000, 10000, 100000, set according to model complexity and dataset size.

[0185] Impact: (1) Prevent overfitting. Limiting the number of iterations can prevent the model from overfitting the training data. (2) Control training costs. Control training time and consumption of computing resources.

[0186] 10. Hyperparameter Adjustment Method

[0187] Grid Search: Within the preset parameter range, traverse each parameter combination with a fixed step size to select the parameter set with the best performance.

[0188] Random Search: Randomly sample in the parameter space and search for parameter combinations with better performance.

[0189] Bayesian optimization: Using Bayesian theory, a probability model between parameters and model performance is established to iteratively optimize parameters.

[0190] Automatic parameter tuning tools: Use existing automatic parameter tuning libraries, such as Hyperopt and Optuna, to automatically optimize hyperparameters.

[0191] 3. The present invention has fully considered the following factors when selecting parameters:

[0192] 1. Mutual influence between parameters: There is a mutual dependence between some parameters, such as the learning rate η0 and the momentum coefficient β.

[0193] 2. Model and data characteristics: According to the specific model structure and dataset characteristics, adjust the parameters to obtain the best performance.

[0194] 3. Validation set evaluation: Evaluate the effects of different parameter combinations on the validation set to avoid overfitting caused by adjusting parameters on the test set.

[0195] 4. Repeated experiments: Due to the randomness of random initialization and data set partitioning, the experiments were repeated multiple times to ensure the stability of parameter selection.

Claims

1. An adaptive machine learning algorithm based on dynamic weight adjustment, characterized in that: The following steps are involved: Step 1, data preprocessing: preprocess the input data to improve the data quality; Step 2: Initialize model parameters: Initialize the model weights and biases randomly or through pre-training according to the model structure; Step 3, dynamic weight adjustment module: In each iteration, the statistical properties of the gradient are calculated, and the learning rate and weight update strategy are dynamically adjusted according to these properties; Step 4, model training: Use the adjusted parameters to train the model using forward propagation and back propagation algorithms; Step 5, model evaluation: Evaluate the performance indicators of the model on the validation set or test set; Step 6, iterative update: Based on the evaluation results of the model and the changing trend of the loss function, determine whether it is necessary to continue training or adjust the parameters until the preset stopping conditions are met.

2. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: The preprocessing described in step 1 includes data cleaning, data normalization, feature extraction and selection.

3. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: The statistical properties of the gradient in step 3 include mean, variance, skewness and kurtosis.

4. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: The dynamic weight adjustment method described in step 3 comprises the following steps: Step 1, calculate the statistical properties of the gradient: (1) Gradient mean Where N represents the number of samples, g i represents the gradient vector in the i-th iteration; (2) Gradient variance (3) Gradient skewness Skew(g) is a measure of the symmetry of the gradient distribution, reflecting the degree of skewness of the gradient distribution; (4) Gradient kurtosis Kurt (g) is a measure of the sharpness of the gradient distribution, reflecting the peak height and tail thickness of the gradient distribution; Step 2, adjust the learning rate: Adjust the learning rate based on the gradient variance: learning rate Among them, η0 is the initial learning rate, λ is the learning rate adjustment parameter; Step 3: Adjust the weight update strategy: Introducing the momentum term: Updated gradient momentum term v t+1 =βv t +(1-β)g t , where β is the momentum coefficient, v t is the gradient momentum term in the tth iteration, g t represents the gradient vector at the tth iteration; momentum is used to smooth the gradient update to prevent falling into the local minimum; Second-order moment correction: Updated second-order moment estimates Among them, Y is the smoothing parameter; s t is the second-order moment estimate; Step 4, update the weights: Comprehensive update formula: Updated parameter vector Among them, θ t is a parameter vector, representing the set of model parameters, including weights and biases, θ t That is, the value of the parameter at the tth iteration; ∈ is a small constant to prevent division by zero.

5. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: The specific method of step 4 is: use the forward propagation algorithm to calculate the output and loss function value of the model under the current parameters, and use the back propagation algorithm to calculate the gradient of the loss function with respect to each parameter; use the dynamically adjusted learning rate and weight update strategy to update the model parameters.

6. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: The performance indicators of the model described in step 5 include accuracy, precision, recall, F1 score and AUC indicators.

7. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: Step 5 also includes overfitting monitoring, which monitors the loss function values ​​of the training set and the validation set to determine whether overfitting occurs.

8. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: The loss function described in step 6, denoted as L(θ), is a model optimization objective function that measures the difference between the model prediction result and the true value; L(θ) ≥ 0, which is used to guide the gradient calculation and optimize the target.

9. The adaptive machine learning algorithm based on dynamic weight adjustment according to claim 1, characterized in that: The preset stop condition in step 6 includes a preset threshold and a maximum number of iterations; The preset threshold is the target value reached by the model's performance indicator. When the preset threshold is reached, the training stops; The maximum number of iterations is the maximum number of iterations allowed during training to prevent the model from overfitting.

Images