Linear regression missing value interpolation and parameter depolarization method based on momentum acceleration

Abstract

The invention relates to a linear regression missing value interpolation and parameter depolarization method based on momentum acceleration. The method comprises the following steps: S1, obtaining a data set D1; s2, a missing value in the data set D1 is interpolated to be 0, and a data set D2 after interpolation is obtained; s3, setting a method parameter, initializing a model parameter beta0, and obtaining an initial linear regression model M; s4, executing outer layer circulation of the model, introducing a momentum acceleration mechanism, and updating outer layer circulation model parameters; s5, executing preset S rounds of model outer layer circulation on the basis of the number S of the outer layer circulation times, and then outputting the updated outer layer circulation model parameter # imgabs0 # as a final model parameter # imgabs1 # S6, obtaining a model # imgabs3 # after final model parameter depolarization on the basis of the final model parameter # imgabs2 #, and outputting a data set # imgabs4 # after interpolation. The integrity of the data set is ensured, and information loss is avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of missing value interpolation, and relates to a linear regression missing value interpolation and parameter debiasing method, in particular to a linear regression missing value interpolation and parameter debiasing method based on momentum acceleration. Background Art

[0002] As a basic and important machine learning model, linear regression is widely used in scenarios where there is an approximate linear relationship between the dependent variable and the independent variable or where the impact of the variable on the target needs to be quantitatively analyzed. It has important value in medical diagnosis (disease risk factor analysis), industrial manufacturing (process parameter optimization), financial forecasting (market indicator correlation analysis) and other fields.

[0003] Currently, the most commonly used method for estimating linear regression model parameters is least squares estimation, which typically involves iteratively solving for model parameters using gradient descent-based optimization algorithms. However, with the advent of the big data era and the continuous expansion of data volumes, the problem of missing data has become increasingly prominent. The presence of missing values ​​can cause the optimization algorithm to deviate from the correct descent direction, resulting in significant errors in the final estimated model parameters. This makes it difficult to accurately capture the linear relationship between variables, affecting the model's predictive performance and generalization ability.

[0004] There are three common methods for dealing with missing values ​​in data:

[0005] 1. Data interpolation: Before applying the gradient descent algorithm, missing values ​​are first filled with a specific value (such as 0 or the column mean). However, this method ignores the original distribution characteristics of the data, resulting in large deviations in the final estimated model parameters and a generally slow parameter convergence process.

[0006] 2. Expectation-Maximization (EM) algorithm: This method relies on idealized assumptions about the data distribution (e.g., normal distribution), which limits its applicability in practical applications. Furthermore, missing data inherently disrupt the original distribution of the dataset, making the algorithm's theoretical premise difficult to meet.

[0007] 3. After imputing missing values ​​to 0, a gradient descent algorithm is applied. In 2022, Aude et al. proposed an average stochastic gradient descent algorithm (AvSGD) based on unbiased gradient estimation. The core of this method is to construct an unbiased gradient estimator, effectively eliminating the estimation bias introduced by missing values, and introducing an average gradient framework. However, AvSGD still has limitations such as insufficient convergence performance (slow convergence and low accuracy under large-scale data sets) and unstable convergence process (parameter oscillation).

[0008] Currently, commonly used missing value interpolation methods have significant flaws: interpolation methods based on statistics (such as mean and median) can easily cause the interpolated values ​​to deviate from the true data distribution, resulting in systematic deviations in subsequent modeling; and complex methods such as multiple interpolation and K-nearest neighbor (KNN) interpolation have high computational complexity and significant resource overhead when processing large-scale data sets. The implementation process is complex and requires parameter tuning and cross-validation.

[0009] In summary, existing technologies have the defects of slow parameter convergence, unstable convergence process and low convergence accuracy when solving the parameters of linear regression models with missing values, which leads to the decline of prediction performance and generalization performance of linear regression models. The currently commonly used missing value interpolation methods can easily cause the interpolated values ​​to deviate from the real data distribution or have high computational complexity and significant resource overhead when processing large-scale data sets, and the implementation process is complicated.

[0010] After searching, no public documents of the prior art that are identical or similar to the present invention were found. Summary of the Invention

[0011] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a linear regression missing value interpolation and parameter debiasing method based on momentum acceleration, which ensures the integrity of the data set and avoids information loss while reducing the computational complexity.

[0012] The present invention solves the practical problem by adopting the following technical solutions:

[0013] A method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration includes the following steps:

[0014] S1, obtain the data set D1, and construct the feature missing ratio diagonal matrix P based on the data missing ratio of each feature in the data set D1;

[0015] S2, based on the data set D1 obtained in S1, interpolate the missing values ​​in the data set D1 to 0 to obtain the interpolated data set D2;

[0016] S3, based on the interpolated data set D2, set the method parameters, initialize the model parameters β0, and obtain the initial linear regression model M;

[0017] S4. Based on the initial linear regression model M, execute the outer loop of the model, introduce the momentum acceleration mechanism, and update the outer loop model parameters;

[0018] S5: Based on the number of outer loops S, after executing the preset S-wheel model outer loop, the outer loop model parameters after the update are output. As the final model parameters

[0019] S6. Based on the final model parameters Get the final model parameter after debiasing Output the imputed dataset

[0020] Furthermore, the specific steps of step S1 include:

[0021] S101, obtaining a data set D1;

[0022] S102. Based on the data set, obtain the feature dimension d of the data set and the data missing ratio p of each feature in the data set j ;

[0023] S103, based on the data missing ratio p of each feature in the dataset j , construct the feature missing ratio diagonal matrix P as follows:

[0024] P=diag((p j ) j∈{1,...,d}

[0025] Among them, d is the feature dimension of dataset D1, p j is the data missing ratio of each feature j in the dataset D1, diag(·) will (p j ) j∈{1,..,d} Arrange it into a diagonal matrix of dimension d×d;

[0026] Moreover, the specific steps of step S3 include:

[0027] S301, based on the interpolated data set D2, obtain the number of samples n and the feature dimension d of the data set D2, and calculate the Lipschitz constant L of the data set D2;

[0028] S302. Based on the number of samples n, feature dimension d, and Lipschitz constant L of the dataset D2, set the method-related parameters, including the number of inner loops m set to 2n, the number of outer loops S set to n, the learning rate set to 1 / 2L, and the momentum coefficient θ set to 0.9;

[0029] S303, based on the feature dimension d of the data set D2, the model parameter β0 is initialized to a zero vector with the same length as the feature dimension d of the data set D2, thereby obtaining the initial model parameter β0;

[0030] S304, based on the initial model parameter β0, the initial model parameter of the outer loop and the inner loop initial model parameters The initial model parameters β0 are assigned as values;

[0031] S305, based on the method parameters set in S302, the initial model parameters β0 obtained in S303 and the outer loop initial model parameters obtained in S304 and the inner loop initial model parameters Get the initial linear regression model M;

[0032] Moreover, the specific steps of step S4 include:

[0033] Based on the initial linear regression model M, the outer cycle of the kth (k=1,2,...,S) round is as follows:

[0034] S401. Calculate the parameters of the outer cycle model for k-1 rounds Full gradient

[0035] S402, based on full gradient Execute m rounds of inner loops, update the inner loop model parameters, and obtain the updated inner loop model parameters

[0036] S403, based on the updated internal loop model parameters The updated inner loop model parameters Assign values ​​to the initial model parameters of the inner loop of the outer loop of k+1 rounds Get the initial model parameters of the k+1 outer and inner cycles Right now

[0037] S404: Based on the inner cycle model parameters of each round in the m inner cycles in S402 and the momentum parameter θ set in S302, a momentum acceleration mechanism is introduced to construct the average value of the inner cycle model parameters of each round in the m inner cycles and the outer cycle model parameters of k-1 rounds. The momentum-weighted combination of the momentum parameter θ obtains the updated outer loop model parameters The specific update method is as follows:

[0038]

[0039] Furthermore, the specific steps of step S402 include:

[0040] S402.1. Based on the interpolated dataset D2 obtained in S2, randomly sample sample i from dataset D2 and obtain the input vector of sample i

[0041] S402.2, based on the momentum parameter θ set in S302, introduce the momentum acceleration mechanism and construct the parameters of the t-1 inner cycle model and k-1 outer cycle model parameters The momentum-weighted combination of the momentum parameter θ forms the temporary model parameter The specific construction method is as follows:

[0042]

[0043] S402.3. Based on temporary model parameters S402.1 Input vector obtained Full gradient obtained by S401 The feature missing ratio diagonal matrix P constructed by S1 is used to construct an unbiased gradient estimation formula and calculate the current gradient The specific calculation method is as follows:

[0044]

[0045] in, is the randomly sampled sample i input vector The transpose of , I represents the d-dimensional identity matrix, P -2 The inverse matrix of the square of the diagonal matrix representing the proportion of missing features;

[0046] S402.4, based on the current gradient Calculate the parameters of the inner cycle model for round t Enter the t+1th round of inner loop;

[0047] S402.5. Based on the inner loop number m set in S302, after completing the preset m rounds of inner loops, obtain the updated inner loop model parameters.

[0048] Advantages and beneficial effects of the present invention:

[0049] The present invention proposes a linear regression missing value interpolation and parameter debiasing method based on momentum acceleration, which obtains a data set D1, constructs a feature missing ratio diagonal matrix P based on the data missing ratio of each feature in the data set D1; based on the data set D1, the missing values ​​in the data set D1 are interpolated to 0 to obtain the interpolated data set D2; based on the interpolated data set D2, the method parameters are set, the model parameters β0 are initialized, and the initial linear regression model M is obtained; based on the initial linear regression model M, the outer loop of the model is executed, the momentum acceleration mechanism is introduced, and the outer loop model parameters are updated; after executing the outer loop of the model for a preset number of rounds, the outer loop model parameters after the update are output. As the final model parameters Based on the final model parameters Get the final model parameter after debiasing Output the imputed dataset When dealing with linear regression models with missing values, existing solutions, especially in large-scale data set scenarios, have the problem of significantly reduced model parameter convergence speed, and oscillation is prone to occur in the convergence process. At the same time, missing values ​​will also affect the accuracy of parameter estimation, thereby causing a decrease in the prediction performance and generalization ability of the linear regression model. The present invention obtains linear regression model parameters, which can eliminate the deviation caused by missing values, accelerate the parameter convergence speed, suppress parameter convergence oscillation, improve the parameter convergence accuracy, and ultimately improve the prediction accuracy and generalization performance of the linear regression model. At the same time, the missing values ​​in the data set are interpolated to 0, and the interpolated data set is output. This missing value interpolation method reduces the computational complexity while ensuring the integrity of the data set and avoiding information loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is a process flow chart of the present invention;

[0051] Figure 2 Comparison of experimental results of convergence performance of the present invention and other methods on the data set;

[0052] Figure 3 This is a comparison chart of the experimental results of the model parameter accuracy of the present invention and other methods on the data set. DETAILED DESCRIPTION

[0053] The embodiments of the present invention are further described below in conjunction with the accompanying drawings:

[0054] A method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration, such as Figure 1 As shown, the following steps are included:

[0055] S1, obtain the data set D1, and construct the feature missing ratio diagonal matrix P based on the data missing ratio of each feature in the data set D1;

[0056] The specific steps of step S1 include:

[0057] S101, obtaining a data set D1;

[0058] S102. Based on the data set, obtain the feature dimension d of the data set and the data missing ratio p of each feature in the data set j ;

[0059] S103, based on the missing data ratio p of each feature in the dataset j , construct the feature missing ratio diagonal matrix P as follows:

[0060] P=diag((p j ) j∈{1,...,d}

[0061] Among them, d is the feature dimension of dataset D1, p j is the data missing ratio of each feature j in the dataset D1, diag(·) will (p j ) j∈{1,..,d} Arrange it into a diagonal matrix of dimension d×d;

[0062] S2, based on the data set D1 obtained in S1, interpolate the missing values ​​in the data set D1 to 0 to obtain the interpolated data set D2;

[0063] S3, based on the interpolated data set D2, set the method parameters, initialize the model parameters β0, and obtain the initial linear regression model M;

[0064] The specific steps of step S3 include:

[0065] S301, based on the interpolated data set D2, obtain the number of samples n and the feature dimension d of the data set D2, and calculate the Lipschitz constant L of the data set D2;

[0066] S302. Based on the number of samples n, feature dimension d, and Lipschitz constant L of the dataset D2, set the method-related parameters, including the number of inner loops m set to 2n, the number of outer loops S set to n, the learning rate set to 1 / 2L, and the momentum coefficient θ set to 0.9;

[0067] S303, based on the feature dimension d of the data set D2, the model parameter β0 is initialized to a zero vector with the same length as the feature dimension d of the data set D2, thereby obtaining the initial model parameter β0;

[0068] S304, based on the initial model parameter β0, the initial model parameter of the outer loop and the inner loop initial model parameters The initial model parameters β0 are assigned as values;

[0069] S305, based on the method parameters set in S302, the initial model parameters β0 obtained in S303 and the outer loop initial model parameters obtained in S304 and the inner loop initial model parameters Get the initial linear regression model M;

[0070] S4. Based on the initial linear regression model M, execute the outer loop of the model, introduce the momentum acceleration mechanism, and update the outer loop model parameters;

[0071] The specific steps of step S4 include:

[0072] Based on the initial linear regression model M, the outer cycle of the kth (k=1,2,...,S) round is as follows:

[0073] S401. Calculate the parameters of the outer cycle model for k-1 rounds Full gradient

[0074] S402, based on full gradient Execute m rounds of inner loops, update the inner loop model parameters, and obtain the updated inner loop model parameters

[0075] The specific steps of step S402 include:

[0076] S402.1. Based on the interpolated dataset D2 obtained in S2, randomly sample sample i from dataset D2 and obtain the input vector of sample i

[0077] S402.2, based on the momentum parameter θ set in S302, introduce the momentum acceleration mechanism and construct the parameters of the t-1 inner cycle model and k-1 outer cycle model parameters The momentum-weighted combination of the momentum parameter θ forms the temporary model parameter The specific construction method is as follows:

[0078]

[0079] S402.3. Based on temporary model parameters S402.1 Input vector obtained Full gradient obtained by S401 The feature missing ratio diagonal matrix P constructed by S1 is used to construct an unbiased gradient estimation formula and calculate the current gradient The specific calculation method is as follows:

[0080]

[0081] in, is the randomly sampled sample i input vector The transpose of , I represents the d-dimensional identity matrix, P -2 The inverse matrix of the square of the diagonal matrix representing the proportion of missing features;

[0082] S402.4, based on the current gradient Calculate the parameters of the inner cycle model in round t Enter the t+1th round of inner loop;

[0083] S402.5. Based on the inner loop number m set in S302, after completing the preset m rounds of inner loops, obtain the updated inner loop model parameters.

[0084] S403, based on the updated internal loop model parameters The updated inner loop model parameters Assign values ​​to the initial model parameters of the inner loop of the outer loop of k+1 rounds Get the initial model parameters of the k+1 outer and inner cycles Right now

[0085] S404: Based on the inner cycle model parameters of each round in the m inner cycles in S402 and the momentum parameter θ set in S302, a momentum acceleration mechanism is introduced to construct the average value of the inner cycle model parameters of each round in the m inner cycles and the outer cycle model parameters of k-1 rounds. The momentum-weighted combination of the momentum parameter θ obtains the updated outer loop model parameters The specific update method is as follows:

[0086]

[0087] S5, based on the outer cycle number S set in S302, after executing the preset S round model outer cycle, output the outer cycle model parameters after the update is completed As the final model parameters

[0088] S6. Based on the final model parameters Get the final model parameter after debiasing Output the imputed dataset

[0089] The working principle of the present invention is:

[0090] The present invention proposes a linear regression missing value interpolation and parameter debiasing method based on momentum acceleration, which obtains a data set D1, constructs a feature missing ratio diagonal matrix P based on the data missing ratio of each feature in the data set D1; based on the data set D1, the missing values ​​in the data set D1 are interpolated to 0 to obtain the interpolated data set D2; based on the interpolated data set D2, the method parameters are set, the model parameters β0 are initialized, and the initial linear regression model M is obtained; based on the initial linear regression model M, the outer loop of the model is executed, the momentum acceleration mechanism is introduced, and the outer loop model parameters are updated; after executing the outer loop of the model for a preset number of rounds, the outer loop model parameters after the update are output. As the final model parameters Based on the final model parameters Get the final model parameter after debiasing Output the imputed dataset The introduction of momentum acceleration mechanism and the construction of unbiased gradient estimator can effectively eliminate the bias caused by missing values, thereby significantly improving the speed and stability of the parameter convergence process, and improving the prediction performance and generalization ability of the linear regression model in the presence of missing data sets.

[0091] Existing solutions for linear regression models with missing values, particularly in large datasets, suffer from a significant slowdown in model parameter convergence and are prone to oscillation during the convergence process. Furthermore, missing values ​​can affect the accuracy of parameter estimation, leading to a decline in the predictive performance and generalization capabilities of linear regression models.

[0092] The present invention introduces a momentum acceleration mechanism to realize momentum acceleration fusion of historical parameter information, utilizes its smooth transition characteristics to suppress oscillation in the parameter convergence process, and enhances the stability of the model parameter convergence process.

[0093] The present invention introduces a momentum acceleration mechanism to retain historical parameter information and accelerate the model parameter convergence process.

[0094] The present invention interpolates missing values ​​in a data set to 0 and outputs the interpolated data set. This missing value interpolation method reduces computational complexity while ensuring the integrity of the data set and avoiding information loss.

[0095] Furthermore, by constructing an unbiased gradient estimator, the gradient system deviation caused by missing values ​​can be effectively eliminated, ensuring that the gradient descent process evolves in the desired direction, thereby accelerating the convergence of model parameters and improving the convergence accuracy of model parameters.

[0096] In summary, the linear regression missing value interpolation and parameter debiasing method based on momentum acceleration proposed in the present invention can effectively eliminate the gradient deviation caused by missing values ​​in the process of solving the parameters of the linear regression model containing missing values, accelerate and stabilize the model parameter convergence process, and improve the model parameter prediction accuracy, thereby improving the prediction ability and generalization performance of the linear regression model.

[0097] The present invention will be further described below through specific experiments:

[0098] The present invention proposes a method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration. In order to illustrate the effectiveness and adaptability of the present invention in improving the parameter convergence speed, an experiment was conducted on a public flood monitoring dataset (Superconductivity Dataset). After the linear regression model is trained according to the method proposed in the present invention, the method of the present invention ( Figure 2 The DMiG method in

[15] was compared with the other four benchmark methods, and the results are shown in Figure 2 shown.

[0099] pass Figure 2 The experimental results show that the linear regression missing value interpolation and parameter debiasing method based on momentum acceleration of the present invention has a parameter convergence process that is significantly faster than that of the other four benchmark methods, and the final convergence accuracy is also higher than that of the other four benchmark methods.

[0100] The present invention proposes a method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration. In order to illustrate the effectiveness and adaptability of the present invention in improving the linear regression prediction performance, an experiment was conducted on a public superconductivity data set (Superconductivity Dataset). After the linear regression model was trained according to the method proposed in the present invention, the coefficient of determination (R) of linear regression, a common evaluation index, was used to evaluate the performance of the linear regression. 2 ) to evaluate the test results. To fully compare the performance, the method of the present invention was compared with the following three benchmark methods:

[0101] Baseline method 1: linear regression is applied on the full dataset;

[0102] Baseline method 2: missing values ​​are imputed using column means and then linear regression is applied;

[0103] Baseline method 3: Missing values ​​are imputed with zero values ​​and then the AvSGD algorithm is applied.

[0104] Detailed performance comparison results are as follows Figure 3 shown.

[0105] Table 1 Comparison of experimental results between this method and other benchmark methods

[0106] method This method Complete data Mean imputation AvSGD <![CDATA[R 2 Coefficient]]> 0.702 0.725 0.657 0.473

[0107] From the experimental results in Table 1, we can see that the R of the model constructed by this method (linear regression missing value interpolation and parameter debiasing based on momentum acceleration) is 2 The coefficient is close to the model performance under the full dataset (0.702 vs. 0.725), and significantly outperforms baseline methods such as mean imputation (0.657) and AvSGD (0.473).

[0108] In summary, the present invention constructs an unbiased gradient estimator and combines it with the momentum acceleration technique to accelerate and stabilize the parameter convergence process, improve the parameter convergence accuracy, and enhance the prediction accuracy and generalization performance of the linear regression model.

[0109] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.

Claims

1. A method for missing value interpolation and parameter debiasing in linear regression based on momentum acceleration, characterized by: The following steps are involved: S1, obtain the data set D1, and construct the feature missing ratio diagonal matrix P based on the data missing ratio of each feature in the data set D1; S2, based on the data set D1 obtained in S1, interpolate the missing values ​​in the data set D1 to 0 to obtain the interpolated data set D2; S3, based on the interpolated data set D2, set the method parameters, initialize the model parameters β0, and obtain the initial linear regression model M; S4. Based on the initial linear regression model M, execute the outer loop of the model, introduce the momentum acceleration mechanism, and update the outer loop model parameters; S5: Based on the number of outer loops S, after executing the preset S-wheel model outer loop, the outer loop model parameters after the update are output. As the final model parameters S6. Based on the final model parameters Get the final model parameter after debiasing Output the imputed dataset 2. The method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration according to claim 1, characterized in that: The specific steps of step S1 include: S101, obtaining a data set D1; S102. Based on the data set, obtain the feature dimension d of the data set and the data missing ratio p of each feature in the data set j ; S103, based on the data missing ratio p of each feature in the dataset j , construct the feature missing ratio diagonal matrix P as follows: P1day((p j ) j∈{1,...,d} Among them, d is the feature dimension of dataset D1, p j is the data missing ratio of each feature j in the dataset D1, diag(·) will (p j ) j∈{1,..,d} Arrange it into a diagonal matrix of dimension d×d.

3. The method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration according to claim 1, characterized in that: The specific steps of step S3 include: S301, based on the interpolated data set D2, obtain the number of samples n and the feature dimension d of the data set D2, and calculate the Lipschitz constant L of the data set D2; S302. Based on the number of samples n, feature dimension d, and Lipschitz constant L of the dataset D2, set the method-related parameters, including the number of inner loops m set to 2n, the number of outer loops S set to n, the learning rate set to 1 / 2L, and the momentum coefficient θ set to 0.9; S303, based on the feature dimension d of the data set D2, the model parameter β0 is initialized to a zero vector with the same length as the feature dimension d of the data set D2, thereby obtaining the initial model parameter β0; S304, based on the initial model parameter β0, the initial model parameter of the outer loop and the inner loop initial model parameters The initial model parameter β0 is assigned as the value; S305, based on the method parameters set in S302, the initial model parameters β0 obtained in S303 and the outer loop initial model parameters obtained in S304 and the inner loop initial model parameters The initial linear regression model M is obtained.

4. The method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration according to claim 1, characterized in that: The specific steps of step S4 include: Based on the initial linear regression model M, the outer cycle of the kth (k=1,2,...,S) round is as follows: S401. Calculate the parameters of the k-1 outer cycle model. Full gradient S402, based on full gradient Execute m rounds of inner loops, update the inner loop model parameters, and obtain the updated inner loop model parameters S403, based on the updated internal loop model parameters The updated inner loop model parameters Assign values ​​to the initial model parameters of the inner loop of the outer loop of k+1 rounds Get the initial model parameters of the k+1 outer and inner cycles Right now S404: Based on the inner cycle model parameters of each round in the m inner cycles in S402 and the momentum parameter θ set in S302, a momentum acceleration mechanism is introduced to construct the average value of the inner cycle model parameters of each round in the m inner cycles and the outer cycle model parameters of k-1 rounds. The momentum-weighted combination of the momentum parameter θ obtains the updated outer loop model parameters The specific update method is as follows:

5. The method for linear regression missing value interpolation and parameter debiasing based on momentum acceleration according to claim 4, characterized in that: The specific steps of step S402 include: S402.

1. Based on the interpolated dataset D2 obtained in S2, randomly sample sample i from dataset D2 and obtain the input vector of sample i S402.2, based on the momentum parameter θ set in S302, introduce the momentum acceleration mechanism and construct the parameters of the t-1 inner cycle model and k-1 outer cycle model parameters The momentum-weighted combination of the momentum parameter θ forms the temporary model parameter The specific construction method is as follows: S402.

3. Based on temporary model parameters S402.1 Input vector obtained Full gradient obtained by S401 The feature missing ratio diagonal matrix P constructed by S1 is used to construct an unbiased gradient estimation formula and calculate the current gradient The specific calculation method is as follows: in, is the randomly sampled sample i input vector The transpose of , I represents the d-dimensional identity matrix, P -2 The inverse matrix of the square of the diagonal matrix representing the proportion of missing features; S402.4, based on the current gradient Calculate the parameters of the inner cycle model in round t Enter the t+1th round of inner loop; S402.

5. Based on the inner loop number m set in S302, after completing the preset m rounds of inner loops, obtain the updated inner loop model parameters.

Images