A gradient optimization method and parameterized model based on distributed computing

Abstract

The present invention relates to a gradient optimization method based on distributed computing and a parameterized model. The gradient optimization method based on distributed computing described in the present invention comprises: performing feature selection, statistics, preprocessing and smooth compensation on the current target defect data in sequence, and dividing the defect feature vector after smooth compensation into workflows according to the streaming distribution based on the hash space, performing local gradient calculation and global gradient calculation on the parameterized model according to the corresponding workflow to obtain the global gradient of the parameterized model; performing approximate loss estimation and parameter iteration based on the global gradient, and judging whether the parameters meet an iteration condition after iteration: if not, continuing the local gradient calculation; if so, completing the gradient optimization of the parameterized model. The gradient optimization method based on distributed computing described in the present invention significantly improves the training efficiency of the parameterized model and effectively avoids the accumulation of model errors caused by missing data.

Description

Technical Field

[0001] The present invention relates to the field of distributed machine learning system optimization, and in particular to a gradient optimization method, device, parameterized model and electronic equipment based on distributed computing. Background Art

[0002] In industrial manufacturing, industrial defect analysis is a key component of quality control. Its core goal is to identify product defects through data analysis, trace their sources, and locate the process or equipment issues that cause them, thereby optimizing production efficiency and improving product quality. In actual production, industrial defects often stem from abnormal product characteristics (such as dimensional deviations and surface cracks) or production equipment anomalies (such as sensor failures and equipment operational deviations). Therefore, industrial defect analysis requires processing both product and equipment data and analyzing the potential correlations between the two.

[0003] To this end, traditional technologies use parametric models to implement industrial defect analysis. The parametric models establish a mapping relationship between data features and output targets through optimizable parameters, such as weight matrices and bias matrices, thereby capturing the global patterns and nonlinear relationships of the data, and then realizing defect classification, process parameter prediction, and multi-device feature association modeling. For example, the data type can be determined based on the input data, the numerical characteristics of the defect can be predicted, and the equipment anomalies or process problems that cause the defect can be tracked based on process parameter predictions, or joint modeling can be performed through parametric models to analyze the potential correlation between products and equipment.

[0004] However, with the rapid increase in the types and number of industrial production equipment and the increasingly complex process flows, the scale of product and equipment data has shown exponential growth. The parametric models of traditional technologies are usually unable to efficiently process data streams from multiple devices, multiple processes, and multi-dimensional features, resulting in a significant decrease in the gradient convergence speed and training efficiency during the training of parametric models. At the same time, due to the potential missing values ​​and outliers in the process of industrial data collection, the training of parametric models is easily unstable, and even the parameter space dimensional disaster may occur, leading to the phenomenon of gradient disappearance.

[0005] Based on this, the existing technology uses distributed computing to perform gradient optimization on parameterized models. By decomposing data and tasks into multiple subtasks and distributing them to different computing nodes to run local gradients in parallel, the computational burden of a single node or process in traditional technology is significantly reduced. At the same time, by completing local gradient calculations in multiple computing nodes and performing global gradient aggregation and parameter updates, the training efficiency of the parameterized model is significantly improved.

[0006] However, industrial data is usually scattered across multiple independent devices and different process flows, making it easy for data silos to appear. The global loss calculation in existing technologies is difficult due to the large number of parameters and the difficulty in interacting with high-dimensional data. The combination of multiple factors makes it difficult to achieve efficient integration and analysis of multi-source heterogeneous data, ultimately leading to the distributed computing of existing technologies and the problem of difficulty in convergence of parameterized models. At the same time, production equipment anomalies, sensor anomalies, and other reasons can easily lead to missing data, further affecting the integrity and accuracy of distributed training. Once a training task on a certain node becomes abnormal, it can easily affect the calculation of the global gradient, leading to global parameter inconsistency, and ultimately causing parameter updates to fail or training to oscillate, resulting in interruptions in the training process and difficulty in converging parameterized models. Therefore, in the distributed gradient optimization process of existing technologies, there is a problem of difficulty in convergence of parameterized models. Summary of the Invention

[0007] Based on this, the purpose of the present invention is to provide a gradient optimization method based on distributed computing.

[0008] S1: Perform feature selection and statistics on the current target defect data to obtain a defect feature set;

[0009] S2: Preprocess the defect feature set and perform smooth compensation on the missing features to generate a defect feature vector;

[0010] S3: Perform workflow division on several defect feature vectors based on the streaming distribution based on the hash space to obtain grouped defect feature datasets;

[0011] Inputting the grouped defect feature data set into a parameterized model for regression prediction, the parameterized model outputting a target value that matches the grouped defect feature data set;

[0012] S4: performing local gradient calculation based on the target value output by the current parameterized model to obtain local gradients of several parameterized models;

[0013] S5: performing global gradient calculation based on the local gradients of several parameterized models to obtain the global gradient of the parameterized model;

[0014] S6: Estimate the approximate loss of the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model;

[0015] S7: Perform parameter iterative calculation based on the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model;

[0016] S8: Determine whether the updated parameters of the current parameterized model meet an iteration condition: if not, iterate the current parameterized model according to the updated parameters of the current parameterized model and execute step S4; if so, use the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model.

[0017] Compared with the existing technology, the gradient optimization method based on distributed computing described in the present invention ensures load balancing of computing nodes through streaming distribution based on hash space, local gradient and global gradient aggregation, thereby improving the robustness and adaptability of the system; at the same time, the present invention approximates the global loss based on the global gradient, significantly reducing the computational complexity of the global loss, thereby greatly improving the efficiency of gradient optimization, and thus solving the problem of difficult convergence of parameterized models.

[0018] Furthermore, the specific calculation of the approximate loss estimation is expressed as follows:

[0019]

[0020] Where, is the approximate global loss of the current parameterized model, m represents the main computing node of the aggregation, where m∈[0,K], K represents the total number of computing nodes; L m (U, V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the t-th iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows:

[0021]

[0022] Where, is the local gradient of the kernel weight parameter U of the kth computation node. The specific calculation is as follows:

[0023]

[0024] Where, Kernel k is the kernel function; k A vector representing the target value of the kth group of defect feature data sets;

[0025] Represents the global gradient of the regularized weight parameter, and the specific calculation is as follows:

[0026]

[0027] Where, is the local gradient of the regularized weight parameter V of the kth computation node. The specific calculation is as follows:

[0028]

[0029] Where ∈ represents the penalty coefficient of the regularization.

[0030] The present invention decomposes the original global loss function based on the functional variation principle, and greatly reduces the complexity of global loss function calculation by approximately replacing the global high-order derivatives with the high-order derivatives of any computing node, thereby significantly reducing the occupation of computing resources and improving the optimization efficiency and convergence speed of high-dimensional parameterized models in a distributed environment; at the same time, by decomposing the global loss function, distributed computing nodes can interact efficiently based on low-order derivatives, avoiding the communication overhead and data island problems caused by high-order computing dependence, and further improving the collaborative efficiency of distributed computing; finally, the present invention effectively solves the problems of high computational cost in distributed gradient optimization and difficulty in directly calculating global loss, resulting in gradient convergence difficulties.

[0031] Furthermore, the parameterized model is a linear regression model or a nonlinear regression model. If the parameterized model is a linear regression model, the parameterized model is specifically expressed as follows:

[0032] y=WX+b

[0033] Where y represents the predicted data of the regression task, which is used to represent the target value corresponding to the current input defect feature data X, W represents the regression weight, and WX is used to indicate that the current regression model is a linear regression model; b represents the bias term; at the same time, the kernel function Kernel k It is expressed as follows:

[0034]

[0035] Where x ki and x kj They represent the input features of the i-th dimension and the j-th dimension in the k-th group of defect feature datasets, respectively. is the inner product operation;

[0036] If the parameterized model is a nonlinear regression model, the parameterized model is specifically expressed as follows:

[0037] y=f(X,θ)+b

[0038] Where f(X,θ) represents a nonlinear function, which is used to indicate that the current regression model is a nonlinear regression model; θ represents the input parameter of the nonlinear function; and the kernel function Kernel k It is expressed as follows:

[0039] Kernel k (x ki ,x kj )=exp(-γ||x ki -x kj || 2 )

[0040] Where ||x ki -x kj || 2 Represents the input feature x ki with x kj The square of the Euclidean distance between them; γ is the width parameter, which indicates the radial range of the kernel function.

[0041] The present invention further limits the kernel functions of linear regression tasks and nonlinear regression tasks. In particular, when the current regression task is nonlinear, the Gaussian radial basis kernel function is used to capture the nonlinear correlation features between input features, thereby ensuring that the parameterized model can adapt to the relationship between linear and nonlinear features.

[0042] Furthermore, the step S1 includes the following steps:

[0043] S11, performing horizontal and vertical feature selection on the current target defect feature to obtain the horizontal feature and vertical feature of the target defect;

[0044] The lateral characteristics of the target defect are expressed as:

[0045] X feature ={x1,x2,…,x C}

[0046] Where x C It is expressed as the Cth transverse feature, where C represents the total number of transverse features; the longitudinal feature of the target defect is expressed as:

[0047] T process ={t1,t2,…,t P}

[0048] Where, t P represents the Pth vertical feature, where P represents the total number of vertical features;

[0049] S12. Using principal component analysis to reduce the dimension and screen the horizontal and vertical features of the target defect to obtain a defect feature set;

[0050] The specific representation of the defect feature set is as follows:

[0051] X grouped ={X′ feature ,T′ process}

[0052] Where X′ feature represents the set of lateral features after principal component analysis and dimensionality reduction, X′ feature ={x′1,x′2,…,x′ c}, its x′ c represents the cth lateral feature after dimensionality reduction by principal component analysis, c represents the total number of lateral features after dimensionality reduction and screening, where c≤C; T′ process represents the set of longitudinal features after principal component analysis and dimensionality reduction, T′ process ={t′1,t′2,…,t′ p}, its t′ p represents the p-th longitudinal feature after dimensionality reduction by principal component analysis, and p represents the total number of longitudinal features after dimensionality reduction and screening, where p≤P;

[0053] The step S2 comprises the following steps:

[0054] S21. Normalize all numerical features in the horizontal features of the defect feature set to obtain normalized numerical features, and combine all normalized numerical features to form a defect numerical feature vector, which is specifically expressed as follows:

[0055]

[0056] Where, The normalized value of the input feature of the i-th dimension representing the numerical lateral feature of the n-th dimension; The original value of the input feature of the i-th dimension represented as the numerical lateral feature of the n-th dimension; x′ n Represents the numerical lateral feature of the nth dimension;

[0057] S22. Constructing a B-spline-based spatial autoregressive model based on the defect numerical feature vector and the longitudinal features after dimensionality reduction, and performing parameter estimation and missing value filling on the B-spline-based spatial autoregressive model using an expectation-maximization algorithm to generate a defect feature vector;

[0058] The B-spline-based spatial autoregressive model is specifically expressed as follows:

[0059] y=A -1 (ρ)X*η+A -1 (ρ)e

[0060] Where y is the response variable vector corresponding to all defect numerical feature vectors, which contains the observed response variable y s and missing response variable y u ; A -1 (ρ)e is the error term, let e *=A -1 (ρ), then e * Satisfy N(0,σ 2 M -1 (ρ)) multivariate normal distribution; σ 2 is the variance coefficient of the error; M(ρ) is the covariance matrix, M(ρ)=A T (ρ)A(ρ),M -1 (ρ) represents the inverse covariance matrix; A(ρ) is the spatial lag coefficient matrix, which is specifically expressed as follows:

[0061] A(ρ)=I-ρW

[0062] Where I represents the identity matrix; W is the spatial weight matrix; ρ is the spatial autoregressive parameter, |ρ|<1; X * is the design matrix, which is expressed as follows:

[0063] X * =(X,S)

[0064] Where X represents the defect numerical feature vector of the current dimension, and S represents the matrix composed of B-spline basis functions based on longitudinal characteristics, which is used to approximate a smooth unknown function α(T′ process ), the specific form is S=(B1(Z),…,B q (Z)) T , B i (Z) represents the i-th matrix composed of B spline functions; Z represents the longitudinal feature T′ process The specific sampling point set of , the total number of sampling points is point; then its q = g + point, g represents the order of the B-spline basis function;

[0065] η represents the regression coefficient vector, which is expressed as η=(β T ,γ T ) T , β is the regression coefficient of the horizontal feature; γ is the regression coefficient corresponding to the B-spline basis function, γ=(γ1,…,γ q ) T .

[0066] The present invention constructs a spatial autoregressive model based on B-splines, combines the multidimensional information of horizontal and vertical features, and adopts spatial correlation modeling and smooth function approximation technology to compensate for missing values ​​with high precision, providing a reliable data foundation for the subsequent parameterized model optimization in distributed computing, while ensuring the integrity and consistency of feature data and the spatial correlation modeling effect.

[0067] Furthermore, the specific steps of using the expectation-maximization algorithm to estimate parameters of the B-spline-based spatial autoregressive model include:

[0068] S221. Divide and jointly distribute the B-spline-based spatial autoregressive model according to the observed response variables and the missing response variables to obtain the mean vector and covariance matrix of the blocks; wherein the specific expression of the mean vector μ is as follows:

[0069]

[0070] Where μ s is the mean vector of the observed response variable; μ u is the mean vector of the missing response variable; the specific expression of the covariance matrix M(ρ) is as follows:

[0071]

[0072] Where M ss (ρ) is the covariance matrix of the observed response variable; M uu (ρ) is the covariance matrix of the missing response variable; M su (ρ) and M us (ρ) is the covariance matrix between the observed response variables and the missing response variables;

[0073] S222, estimate the missing data based on the block mean vector and covariance matrix to obtain the conditional expected value of the missing response variable; where, given the observed response variable y s The conditional expectation of the missing response variable for the i-th iteration of The specific calculation is as follows:

[0074]

[0075] In the formula, (y s -μ s ) represents the observed variable y s Deviation;

[0076] S223. Maximize the log-likelihood expression objective function according to the conditional expected value to obtain an iterative value of the current parameter; wherein the log-likelihood expression objective function is specifically expressed as follows:

[0077]

[0078] Where, Q1(ρ,θ ()) ) is the approximate log-likelihood expression objective function, θ (i) Represents the input parameters of the i-th iteration, which are specifically: θ (i) =(η T ,ρ,ω) T , where ω=σ 2 ; represents the weighted sum of squares of the residuals between the expected conditional value and the model mean vector, is the residual, that is represents the degree of correction of the covariance matrix of the missing response variable, tr(.) is the trace calculation of the matrix; log|M(ρ)| represents the penalty term for the data correlation of the observed response variable;

[0079] The specific calculation process of the maximization is expressed as follows:

[0080] ρ (i+1) =arg max Q1(ρ,θ (i) )

[0081]

[0082]

[0083] Where, and are the partial derivatives of the regression coefficient vector and the error term variance, respectively, which are specifically expressed as follows:

[0084]

[0085]

[0086] Among them, θ (i+1) It is the iteration value of the current parameter;

[0087] S224. Determine whether the iterative value of the current parameter converges: if so, substitute the current parameter into the B-spline-based spatial autoregressive model to complete the supplement of the missing value and generate the defect feature vector; if not, recalculate the conditional expected value, continue the iterative calculation, and execute step S222.

[0088] The present invention combines a B-spline-based spatial autoregressive model with an expectation-maximization algorithm, from the calculation of block means and covariance matrices, to conditional expected value estimation and parameter optimization, and finally to missing value filling. This achieves dynamic estimation and updating of missing values, enhances the flexibility and adaptability of missing value filling, and ensures that the spatial autoregressive model can still accurately describe the target value when the data is incomplete.

[0089] Furthermore, the streaming distribution based on the hash space calculates the group identifier corresponding to any feature dimension i in a number of defect feature vectors, and distributes the corresponding defect feature vector to the corresponding computing node according to the group identifier to obtain a grouped defect feature data set. For the defect feature data set of the kth group, it is specifically expressed as:

[0090]

[0091] Where, Represents the defect feature vector of the i-th dimension divided into the k-th group and the corresponding target value y k,i ; Represents a group identifier, which is used to convert the corresponding defect feature vector into Assigned to the corresponding computing nodes, the specific calculation is as follows:

[0092]

[0093] Where, Represents the current defect feature vector A unique identifier for ; mod indicates a modulo operation.

[0094] The present invention maps each defect feature vector to the corresponding computing node through streaming distribution based on hash space, and utilizes the uniform distribution characteristics of the hash function to evenly distribute the feature vector to all computing nodes, thereby avoiding uneven load between nodes and ensuring full utilization of computing resources.

[0095] A gradient optimization device based on distributed computing, comprising a feature selection unit, a data compensation unit, a workflow division unit, a local gradient calculation unit, a global gradient calculation unit, an approximate loss estimation unit, a parameter iterative calculation unit, an iterative judgment unit, and a parameter updating unit;

[0096] The feature selection unit is used to perform feature selection and statistics on the current target defect data to obtain a defect feature set;

[0097] The data compensation unit is used to pre-process the defect feature set and smoothly compensate for the missing features to generate a defect feature vector;

[0098] The workflow division unit is configured to perform workflow division on a plurality of defect feature vectors according to a streaming distribution based on a hash space to obtain a grouped defect feature data set; input the grouped defect feature data set into a parameterized model for regression prediction, and the parameterized model outputs a target value that matches the grouped defect feature data set;

[0099] The local gradient calculation unit is used to perform local gradient calculation according to the target value output by the current parameterized model to obtain the local gradients of several parameterized models;

[0100] The global gradient calculation unit is used to perform global gradient calculation based on the local gradients of the plurality of parameterized models to obtain the global gradient of the parameterized model;

[0101] The approximate loss estimation unit is used to perform approximate loss estimation on the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model;

[0102] The parameter iterative calculation unit is used to perform parameter iterative calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model;

[0103] The iteration judgment unit is used to judge whether the updated parameters of the current parameterized model meet an iteration condition: if not, calling the parameter updating unit; if so, taking the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model;

[0104] The parameter updating unit is used to iterate the current parameterized model according to the updated parameters of the current parameterized model and call the local gradient calculation unit.

[0105] Furthermore, the specific calculation of the approximate loss estimation is expressed as follows:

[0106]

[0107] Where, is the approximate global loss of the current parameterized model, m represents the main computing node of the aggregation, where m∈[0,K], K represents the total number of computing nodes; L m (U, V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the t-th iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows:

[0108]

[0109] Where, is the local gradient of the kernel weight parameter U of the kth computation node. The specific calculation is as follows:

[0110]

[0111] Where, Kernel k is the kernel function; k is the target value, obtained through the defect feature dataset;

[0112] Represents the global gradient of the regularized weight parameter, and the specific calculation is as follows:

[0113]

[0114] Where, is the local gradient of the regularized weight parameter V of the kth computation node. The specific calculation is as follows:

[0115]

[0116] Where ∈ represents the penalty coefficient of the regularization.

[0117] A parameterized model for gradient optimization based on distributed computing, characterized in that the parameterized model is used to perform the following steps:

[0118] Feature extraction is performed on the defect image to be detected to extract multidimensional features that represent the features to be detected, so as to obtain a feature vector of the defect to be detected; wherein the defect features include but are not limited to the defect area and aspect ratio.

[0119] Linear regression or nonlinear regression is used to model and calculate the characteristic vector of the defect to be detected to obtain the target value; wherein, when the linear regression is used for modeling, it is specifically expressed as:

[0120] y=WX+b

[0121] Where y represents the target value, X represents the defect feature vector, W represents the regression weight, and b represents the bias term.

[0122] When the nonlinear regression is used for modeling, it is specifically expressed as follows:

[0123] y=f(X,θ)+b

[0124] Where f(X,θ) represents a nonlinear function; θ represents the input parameter of the nonlinear function; b represents the bias term;

[0125] The target value is converted according to preset rules to obtain defect information in the current defect image to be detected; wherein, the preset rules can be selected as threshold classification, probability distribution calculation or hierarchical distribution, and different preset rules are selected according to different detection tasks.

[0126] The parameterized model uses the above-mentioned gradient optimization method based on distributed computing to optimize the parameters of the parameterized model.

[0127] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] Figure 1 This is a simplified structural diagram of the gradient optimization device for distributed computing according to the present invention;

[0129] Figure 2The figure is a simplified flowchart of the gradient optimization method based on distributed computing described in the present invention. DETAILED DESCRIPTION

[0130] In order to solve the problem of parameterized model convergence difficulties in the distributed gradient optimization process of the prior art, the present invention constructs a defect feature set by performing feature selection and statistics on the current target defect data; then, the defect feature set is preprocessed and missing feature compensation is performed in sequence to generate several complete defect feature vectors. Next, the defect feature vectors are divided into multiple workflows based on the streaming distribution of the hash space to obtain grouped defect feature vectors; in each workflow, the parameterized model outputs the corresponding target value according to the grouped defect feature vector, performs local gradient calculation according to the corresponding target value, generates several local gradients of the parameterized model, and aggregates the local gradients to calculate the global gradient of the parameterized model. According to the global gradient, an approximate loss estimate is performed on the parameterized model, and the parameters of the parameterized model are iterated based on the loss estimation result. Finally, it is judged whether the parameters after the iteration meet the preset convergence conditions: if they meet, the training is completed and the optimized parameterized model is output; if not, the local gradient calculation is performed again and the iteration continues.

[0131] Based on this, the present invention significantly improves the training efficiency of parameterized models by implementing a workflow partitioning strategy based on a stream-based distribution of hash spaces. This strategy performs local gradient calculations in parallel on multiple computing nodes, aggregates them into global gradients, and performs approximate loss estimation based on the global gradients. Furthermore, this method compensates for missing data through nonparametric interpolation methods, ensuring the integrity of defective feature vectors and effectively preventing the accumulation of model errors caused by missing data.

[0132] Based on the above design, the present invention proposes a gradient optimization method based on distributed computing, and based on this method proposes a gradient optimization device based on distributed computing.

[0133] Please also see Figure 1 and Figure 2 , Figure 1 This is a simplified structural diagram of the gradient optimization device based on distributed computing according to the present invention. Figure 2 The figure is a simplified flowchart of the gradient optimization method based on distributed computing described in the present invention.

[0134] The gradient optimization device based on distributed computing includes a feature selection unit 1, a data compensation unit 2, a workflow division unit 3, a local gradient calculation unit 4, a global gradient calculation unit 5, an approximate loss estimation unit 6, a parameter iterative calculation unit 7, an iterative judgment unit 8 and a parameter updating unit 9.

[0135] The feature selection unit 1 is used to execute step S1: perform feature selection and statistics on current target defect data to obtain a defect feature set.

[0136] Specifically, horizontal and vertical feature selection is performed on the current target defect feature to obtain the horizontal and vertical features of the target defect;

[0137] The feature selection may adopt correlation analysis, which uses a correlation measurement method such as the Pearson correlation coefficient or the partial correlation coefficient to calculate the standard deviation and covariance between the current data and the target value, and calculate the corresponding information gain, thereby screening out the corresponding features using a threshold according to the information gain, and filtering out redundant features, thereby obtaining the horizontal and vertical features of the target defect;

[0138] The lateral feature X feature Expressed as:

[0139] X feature ={x1,x2,…,x C}

[0140] Where x C It is represented as the Cth lateral feature, where C represents the total number of lateral features. In this embodiment, the lateral features include geometric features and statistical features. The geometric features are used to describe the shape, size or spatial distribution of the defect, and can be specifically selected as defect area, aspect ratio, curvature radius, skeleton length or center of gravity position, etc. Since the characteristics of defects corresponding to different products are also different, the present invention does not specifically limit the selection of geometric features. The statistical features are used to characterize the distribution characteristics of defects in the detection area, and can be specifically selected as overall levelness, number of voids, defect distribution density, etc. The present invention does not specifically limit the selection of statistical features.

[0141] The longitudinal feature T process It is used to reflect the corresponding changes in the distribution of target defects over time or process flow, and is specifically expressed as:

[0142] T process ={t1,t2,…,t P}

[0143] Where, t P Represents the Pth longitudinal feature, where P represents the total number of longitudinal features. In this embodiment, the longitudinal feature may be selected from detection time, detection task, product batch, product identification or image serial number, etc. The present invention does not specifically limit the selection of its longitudinal features.

[0144] Next, principal component analysis is used to reduce the dimension and filter the horizontal and vertical features of the target defect to reduce redundant information and highlight the main features, thereby obtaining a defect feature set. The specific representation of the defect feature set is as follows:

[0145] X grouped ={X′ feature ,T′ process}

[0146] Where X′ feature Represents the set of lateral features after principal component analysis and dimensionality reduction, which is specifically expressed as: X′ feature ={x′1,x′2,…,x′ c}, its x′ c represents the cth lateral feature after dimensionality reduction by principal component analysis, c represents the total number of lateral features after dimensionality reduction and screening, where c≤C;

[0147] T′ process Represents the set of longitudinal features after principal component analysis and dimensionality reduction, which is specifically expressed as T′ process ={t′1,t′2,…,t′ p}, its t′ p It represents the p-th longitudinal feature after dimensionality reduction by principal component analysis, and p represents the total number of longitudinal features after dimensionality reduction and screening, where p≤P.

[0148] The principal component analysis is performed by calculating the mean of the horizontal and vertical features for centering, and the covariance matrix is ​​calculated based on the characteristic matrix obtained by the centering, and the eigenvalue decomposition of the covariance matrix is ​​performed to obtain the eigenvector corresponding to the eigenvalue, thereby accumulating the variance contribution rate according to the size of the eigenvalue, and thus screening out the features corresponding to the corresponding main components, that is, X′ feature and T′ process .

[0149] Since principal component analysis is not the core innovation of the present invention, and the corresponding selected features are different according to different task objectives or input data, and feature selection can also be combined with causal analysis, the present invention does not specifically limit the means of feature selection.

[0150] The data compensation unit 2 is used to execute step S2: preprocessing the defect feature set and performing smooth compensation on the missing features to generate a defect feature vector.

[0151] The specific steps include:

[0152] S21. Normalize all numerical features in the horizontal features of the defect feature set to obtain normalized numerical features, and combine all normalized numerical features to form a defect numerical feature vector. The specific expression of normalizing the original value of the input feature of the i-th dimension of the n-th dimension numerical horizontal feature is as follows:

[0153]

[0154] Where, The normalized value of the input feature of the i-th dimension representing the numerical lateral feature of the n-th dimension; The original value of the input feature of the i-th dimension represented as the numerical lateral feature of the n-th dimension; x′ n Represents the numerical lateral feature of the nth dimension. Max(x′ n ) and min(x′ n ) represent the minimum and maximum values ​​of the n-th dimension numerical lateral feature respectively.

[0155] When all numerical lateral features are normalized, all normalized numerical features are combined to form a defect numerical feature vector, which is specifically expressed as follows:

[0156]

[0157] Where, X numeric Represents the numerical feature vector of the defect; n represents the total number of dimensions of the numerical lateral features, where n≤c.

[0158] S22. Construct a B-spline-based spatial autoregressive model based on the defect numerical feature vector and the longitudinal feature after dimensionality reduction, and use the expectation maximization algorithm to perform parameter estimation and missing value filling on the B-spline-based spatial autoregressive model to ensure the continuity and consistency of the data in the feature dimension, thereby generating a defect feature vector. The specific expression of the B-spline-based spatial autoregressive model is:

[0159] y=A -1 (ρ)X*η+A -1 (ρ)e

[0160] Where y is the response variable vector corresponding to all defect numerical feature vectors, which contains the observed response variable y s and missing response variable y u ; A -1 (η)e is the error term, let e * =A -1 (η) is used to represent the noise caused by random factors in the model, which satisfies e * ~N(0,σ 2 M -1 (ρ)) multivariate normal distribution; σ 2 is the variance coefficient of the error, which is used to control the scale of the error term; M(ρ) is the covariance matrix, that is, M(ρ)=A T (ρ)A(ρ), then M -1(ρ) represents the inverse matrix of the covariance, which is used to describe the correlation between the observed variables to reflect the error dependence between different observed variables;

[0161] A(ρ) is the spatial lag coefficient matrix, that is, the spatial autoregressive correlation coefficient between the response variable and the adjacent values, which is specifically expressed as:

[0162] A(ρ)=I-ρW

[0163] Where I is the unit matrix; W is the spatial weight matrix, which is used to describe the spatial adjacency relationship between input features under the same feature dimension. Its matrix element w is ij represents the spatial relationship between input feature i and input feature j; ρ is the spatial autoregressive parameter, which is used to control the spatial dependence between response variables and is constrained to be |ρ|<1;

[0164] X * Represents the design matrix, which contains the B-spline basis functions of the horizontal and vertical features. The specific form is:

[0165] X * =(X,S)

[0166] Where X represents the defect numerical feature vector of the current dimension, and S represents the matrix composed of B-spline basis functions based on longitudinal characteristics, which is used to approximate a smooth unknown function α(T′ process ), the specific form is S=(B1(Z),…,B q (Z)) T , B i (Z) represents the i-th matrix composed of B spline functions, which is used to represent the vertical feature T′ process Mapped to the basis function space, and Z represents the longitudinal feature T′ process The specific sampling point set of the sampling point is point, that is, point points are selected from the interval [0,1], (for example: 0 = sample1 <sample2<…<sample point-1 <sample point =1, sample i represents the i-th sampling point), which is used to define the control point of the B-spline basis function; where q = g + point, g represents the order of the B-spline basis function;

[0167] η represents the regression coefficient vector, which is expressed as η=(β T ,γ T ) T , β is the regression coefficient of the horizontal feature; γ is the regression coefficient corresponding to the B-spline basis function, which is used to express the influence of the approximation of the longitudinal feature on the response variable through the B-spline basis function, γ=(γ1,…,γ q )T .

[0168] Among them, X*η represents the approximation of the longitudinal feature T′ by the B-spline basis function process The corresponding smooth unknown function α(T′ process ) is a non-parametric term used to represent the influence of longitudinal characteristics on the response variable, and the smooth unknown function can be expressed as

[0169] α(T′ process )=(α(z1),…,a(z p )) T

[0170] In the formula, α(z i ) represents the smooth function corresponding to the i-th feature dimension in the longitudinal feature, which is used to reflect the nonlinear effect of the i-th dimension longitudinal feature on the response variable. The smooth function α(z i ) is constructed by performing data approximation on the longitudinal features of the i-th dimension through the B-spline basis function to capture the changing trend of the feature in the continuous space.

[0171] According to this, α(T′ process ) can be expressed as:

[0172] α(T′ process )=Sγ+∈

[0173] Where ∈ represents the approximation error term, that is, the error introduced by the B-spline approximation process. However, as the number of sampling points point→∞, the approximation error term ∈ can be considered as an asymptotic constant, that is, its error magnitude approaches O p (1), which means that the error term tends to a finite constant and can usually be ignored in practice, making the model theoretically very smooth and stable.

[0174] The specific steps of using the expectation-maximization algorithm to estimate the parameters of the B-spline-based spatial autoregressive model include:

[0175] S221. Divide and jointly distribute the B-spline-based spatial autoregressive model according to the observed response variables and the missing response variables to obtain the mean vector and covariance matrix of the blocks. The specific expression of the mean vector μ is as follows:

[0176]

[0177] Where μ s is the mean vector of the observed response variable; μ u is the mean vector of the missing response variables.

[0178] The specific expression of its covariance matrix M(ρ) is as follows:

[0179]

[0180] Where M ss (ρ) is the covariance matrix of the observed response variable; M uu (ρ) is the covariance matrix of the missing response variable; M su (ρ) and M us (ρ) is the covariance matrix between the observed response variables and the missing response variables.

[0181] S222, based on the block mean vector and covariance matrix, the missing data are estimated to estimate the potential latent variables, so as to help fill the missing data and obtain the conditional expected value of the missing response variable, given the observed response variable y s The conditional expectation of the missing response variable for the i-th iteration of The specific calculation is as follows:

[0182]

[0183] In the formula, (y s -∈ s ) represents the observed variable y s Deviations from the mean are used to adjust predicted values ​​for missing response variables.

[0184] S223. Maximize the log-likelihood expression objective function according to the conditional expected value to obtain the iterative value of the current parameter. The log-likelihood expression objective function is specifically expressed as follows:

[0185]

[0186] Where θ (i) Represents the input parameters of the i-th iteration, which are specifically: θ (i) =(η T ,ρ,ω) T , where ω=σ 2 ; is a constant term, used as a standardization factor for normal distribution; Represents the variance of the error term, which is used to adjust the degree of adaptation of the data and optimize the variance parameter by maximizing the log-likelihood; The penalty term representing the data correlation of the observed response variables is used to reflect the size of the covariance matrix through the natural logarithm to characterize the correlation between the observed response variables; It represents the difference between the expected conditional value and the mean vector of the model, which is used to measure the weighted deviation between the estimated value and the theoretical mean. make Make the predicted values ​​as close as possible to the observed values, thereby maximizing the fitness of the filled data; Indicates the degree of correction of the covariance matrix of the missing response variable. Tr(.) is calculated by the trace of the matrix to improve the estimation accuracy of the missing data. (i) represents the variance of the error term in the i-th iteration.

[0187] Furthermore, in order to simplify the iterative process, the iterative value θ of the current parameter is obtained. (i+1) , the present invention expresses the objective function Q(θ,θ (i) ) and the error term variance ω in the regression coefficient vector η and the error term variance ω are partially derived and solved to obtain the log-likelihood function with only the spatial autoregressive parameter ρ. The specific calculation of its partial derivative is expressed as follows:

[0188]

[0189] Where, Represents the partial derivative of the regression coefficient vector, which is used to update the regression coefficients to reflect the correction of the model based on the new data; The partial derivative of the error term variance is used to update the error variance based on the covariance matrix of the current error term and the model. Represents the weighted sum of squares of the residuals between the expected conditional value and the model mean vector, which is used to quantify the size of the error. At the same time, combined with the covariance matrix update, it reflects the model's estimate of the variance of the error;

[0190] The approximate log-likelihood expression objective function of the spatial autoregressive parameter ρ can be expressed as:

[0191]

[0192] Where, Q1(ρ,θ (i) ) represents the log-likelihood function of the spatial autoregressive parameter ρ. By finding the optimal spatial autoregressive parameter ρ, the current model can fit the missing values ​​in the current defect numerical feature vector.

[0193] Based on this, the iterative value of the current parameter is obtained by maximizing the log-likelihood function of the spatial autoregressive parameter ρ, which is specifically expressed as follows:

[0194] ρ (i+1) =argmaxQ1(ρ,θ (i) )

[0195]

[0196] According to this, the iterative value of the current parameter θ (i+1) =(η (i+1)T ,ρ (i+1) ,ω (i+1) ) T .

[0197] Among them, regarding the initial value of the regression coefficient vector, the preliminary regression coefficient can be estimated by least squares (OLS); the variance of the error term can be obtained by using the variance of the existing observed response variable or the mean as the initial value; and the spatial autoregressive parameter can be initialized by a priori estimation, or initialized using a random number [0,1]. Since the initialization does not essentially affect the final fitting effect, but only affects the efficiency of the fitting, and at the same time, the corresponding initial value can be set differently according to different input features, the present invention does not specifically limit the selection of the initial value.

[0198] S224, determine whether the iterative value of the current parameter converges: if so, substitute the current parameter into the spatial autoregressive model based on B-spline to complete the supplement of the missing value and generate the defect feature vector If not, recalculate the conditional expected value, continue iterative calculation, and execute step S222.

[0199] The workflow division unit 3 is used to execute step S3: performing workflow division on a plurality of defect feature vectors according to the streaming distribution based on the hash space to obtain grouped defect feature data sets.

[0200] Specifically, the streaming distribution based on the hash space calculates the group identifier corresponding to any feature dimension i in several defect feature vectors, and distributes the corresponding defect feature vector to the corresponding computing node according to the group identifier to obtain the grouped defect feature data set. For the defect feature data set of the kth group, it is specifically expressed as:

[0201]

[0202] Where, Represents the defect feature vector of the i-th dimension divided into the k-th group and the corresponding target value y k,i , its target value y k,i According to the corresponding task definition or feature engineering, several defect feature vectors are analyzed and obtained. The present invention does not specifically limit the target value y k,i choice; Represents a group identifier, which is used to convert the corresponding defect feature vector into Assigned to the corresponding computing nodes, the specific calculation is as follows:

[0203]

[0204] Where, Represents the current defect feature vector A unique identifier, which can be obtained by a custom incrementing function or other hash algorithms such as MD5. The present invention does not specifically limit the calculation method of the unique identifier; mod represents a modulo operation, where K represents the total number of computing nodes. The total number of nodes is divided or allocated according to computing resources, which is not specifically limited in the present invention.

[0205] Furthermore, when the workload of the computing node reaches the upper limit or the group identifier When the corresponding computing node and its nearest neighbor nodes are both busy, a new computing node is created and the identifier of the new computing node is equal to the hash value of the current defect feature vector. And update the total number of nodes K, K←K+1, and redistribute the current defect feature vector;

[0206] Based on this, when computing resources are limited or node loads are uneven, dynamically expanding computing nodes can effectively avoid computing bottlenecks and ensure task allocation efficiency in a distributed environment.

[0207] After the workflow is divided, the grouped defect feature data sets are input into the parameterized models of each computing node for regression prediction. The parameterized model of each computing node generates a matching target value based on its assigned data set.

[0208] The parameterized model is a regression model, which is used to output the corresponding target value y according to the input defect feature data X. To parameterize the model, to obtain y k,i ), its specific model can be expressed as:

[0209] y=WX+b

[0210] or,

[0211] y=f(X,θ)+b

[0212] Wherein, y represents the predicted data of the regression task, which is used to represent the target value corresponding to the currently input defect feature data X, and is specifically set according to different application scenarios, which is not specifically limited in the present invention; W represents the regression weight, and WX is used to indicate that the current regression model is a linear regression model; f(X, θ) represents a nonlinear function, which is used to indicate that the current regression model is a nonlinear regression model, where θ represents the input parameter of the nonlinear function; and b represents a bias term.

[0213] Since all computing nodes share consistent parameterized model parameters but process different data sets, distributed collaborative computing can fully utilize the computing resources of each node while ensuring the consistency of global computing results, improve task execution efficiency and reduce the computing load of a single node.

[0214] Based on this, local loss calculation and local gradient calculation are performed according to the target value matched by the grouped defect feature data set, and the local gradient corresponding to each computing node can be obtained.

[0215] The local gradient calculation unit 4 is used to execute step S4: performing local gradient calculation according to the target value output by the current parameterized model to obtain local gradients of several parameterized models.

[0216] Specifically, according to the corresponding computing node k and the corresponding k-th group of defect feature data sets, the local loss of the parameterized model is calculated to obtain the local loss L of the current computing node k. k (U, V), its specific calculation is as follows:

[0217]

[0218] Where U is the kernel weight parameter, which is used to represent the regression weight of the kernel function. ki The contribution degree of the model prediction value to fit the target value y ki In this embodiment, the input features and the corresponding target values ​​adopt the defect feature vector after smooth compensation, that is, y ki =y k,i ; V is the regularization weight parameter, which is used to limit the complexity of the parameterized model and avoid overfitting to ensure the robustness and generalization ability of the model; ∈ represents the penalty term coefficient of the regularization, which is used to measure the impact of regularization on the overall loss; Kernel k It is a kernel function that describes the relationship and similarity between input features in high-dimensional space, guides the model to regress in the kernel space, and thus guides the parameterized model to regress in the kernel space. When the regression task shows linear correlation, the specific expression of its kernel function is:

[0219]

[0220] Where x ki and x kj They represent the input features of the i-th dimension and the j-th dimension in the k-th group of defect feature datasets, respectively. is the inner product operation, used to calculate x ki and x kj The similarity between them.

[0221] When the regression task represents nonlinear correlation, the specific expression of its kernel function is:

[0222] Kernel k (x ki ,x kj )=exp(-γ‖x ki-x kj ‖ 2 )

[0223] Where ‖x ki -x kj ‖ 2 Represents the input feature x ki with x kj The square of the Euclidean distance between them is used to represent the distance between the two in the feature space; γ is the width parameter, which represents the radial range of the kernel function. The larger its value, the wider the kernel function. k The faster it approaches 0.

[0224] is a quadratic regularization term, representing the kernel weight parameter U of the current node k k In the kernel function Kernel k The quadratic sum on the kth computing node is used to measure the kernel weight parameter U k The degree of fitting of the similarity between input features in the high-dimensional kernel space; ∈V k Represents the regularization penalty term, which is used to limit the weight parameter V of the regularization of the kth computing node k The size of , thereby controlling the degree of fit of the model on local data; is the fitting term, which is used to represent the kernel weight parameter U of the kth computing node k For the target value y ki The fitting effect of the parameterized model reflects the prediction ability of the target value.

[0225] Furthermore, in order to improve the efficiency of gradient descent, the initial value U of the kth computing node is k and V k The calculation is expressed as:

[0226]

[0227] Where, and Used to represent the initial value of the k-th workflow, y k The vector representing the target value of the kth group of defect feature data sets, i.e. y k ={y k1 ,…,y ki ,…,y kn}.

[0228] Next, the local loss L of the current calculation node k is k (U, V) performs partial derivative calculation to obtain the local gradient of the parameterized model of the current computing node. The specific calculation is as follows:

[0229]

[0230] Where, is the local gradient of the kernel weight parameter U of the kth computational node, that is, the partial derivative of the kernel weight parameter U, which is used to guide the model to adjust the kernel weight parameter U to reduce the fitting error; is the local gradient of the regularized weight parameter V of the kth computing node, that is, the partial derivative of the regularized weight parameter, which is used to limit the complexity of the model and prevent overfitting; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the t-th iteration respectively.

[0231] Finally, the local gradients of the parameterized models of all computing nodes in the current iteration are summarized to obtain the local gradients of several parameterized models.

[0232] The summary will go through the Mapper stage, mapping the local gradient of each computing node into a key-value pair (e.g., <node ID, local gradient>). Then, in the Reducer stage, all mapped local gradients are grouped and summarized according to the specific rules of the key-value pairs, and the global gradient is calculated by weighted average or accumulation.

[0233] The global gradient calculation unit 5 is used to execute step S5: performing global gradient calculation according to the local gradients of the plurality of parameterized models to obtain the global gradient of the parameterized model.

[0234] Specifically, the specific calculation of its global gradient is expressed as follows:

[0235]

[0236] Where, Represents the global gradient of the kernel weight parameter U, that is, the normalized mean of the local gradients on all computing nodes. It is used to summarize the local gradients of all nodes and balance the impact of data distribution on different nodes on the kernel weight parameter U. K represents the total number of computing nodes. It represents the global gradient of the regularized weight parameter V, that is, the normalized mean of the local gradients on all computing nodes. It is used to summarize the regularized gradients of all nodes and balance the impact of the regularization term on the global loss, thereby ensuring that the complexity of the model is controlled within a reasonable range. t represents the iteration round of the current global gradient.

[0237] Since the global gradient can only provide directional information about the current parameter update, it is impossible to directly quantify the changing trend of the global loss. Especially in the optimization of high-dimensional parameterized models, simply relying on the global gradient may lead to improper selection of the update step size, thereby affecting the convergence speed and optimization stability of the model. Therefore, the present invention combines the calculation of approximate global loss and comprehensively considers the relationship between the global gradient and the current parameters to effectively estimate the decrease in the global loss, thereby providing a more accurate optimization basis for parameter updates.

[0238] The approximate loss estimation unit 6 is used to execute step S6: performing an approximate loss estimation on the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model.

[0239] Specifically, the specific calculation of the approximate loss estimation is expressed as follows:

[0240]

[0241] Where, is the approximate global loss of the current parameterized model, which is used to estimate the global loss in the current iteration by using the global gradient and the loss information of the main computing node m that is summarized. m represents the main computing node that is summarized, where m∈[0,K], and m defaults to 1; L m (U, V) represents the local loss of the mth computational node; <·,·> is the inner product operation, which is used to represent the inner product calculation between two vectors, thereby indicating the directional influence of the gradient information on the parameter U or V of the current parameterized model.

[0242] Since the shape of the parameters U and V under the calculation of the global loss is extremely large and the corresponding calculation cost is complex, the present invention derives the expression of the approximate loss estimation through a series of derivations, and the corresponding specific derivation process is as follows:

[0243] It is known that the loss of any computing node k is calculated as:

[0244]

[0245] ∴The global loss calculation can be expressed as:

[0246]

[0247] Therefore, according to the functional variation principle, the function can be approximately decomposed into a finite-order differential operator, which is specifically expressed as:

[0248]

[0249] in, represents the initial model parameters of the parameterized model, so is a constant; at the same time, and The order is low and can interact in the workflow of each computing node, while Q(U,V) is a high-order derivative, so the interaction in the workflow of the computing node is difficult. However, since the present invention divides the workflow through streaming distribution based on hash space, all computing nodes are in a load-balanced state, and the difference in all losses on each computing node is small. Therefore, any node can be selected as the main computing node, and the present invention uses the first node as the main computing node by default. That is, the high-order derivative Q1(U,V) of the workflow of the first computing node can be directly used instead of Q(U,V). Based on this, the following expression of the high-order derivative is obtained:

[0250]

[0251] ∴Substitute Q1(U,V) into L N (U, V), the expression for the approximate loss estimation is obtained, which is specifically expressed as:

[0252]

[0253] In order to solve the problem that the global loss in distributed computing will not lead to excessively high gradient calculation complexity due to the large number of parameters, thereby causing difficulty in interaction of high-order derivative terms in the global loss calculation and difficulty in gradient convergence of the parameterized model, the present invention utilizes the functional variation principle to decompose the global loss function into a low-order derivative part and a high-order derivative part; among them, the low-order derivative part can be efficiently completed directly through the gradient interaction of each computing node; and although the high-order derivative part has the problems of high complexity and high computational overhead, the present invention selects any node as the main computing node (main workflow) when all computing nodes are load-balanced to replace the high-order derivative terms, thereby effectively solving the problem of difficulty in interaction of high-order derivative terms in the global loss through approximation, and thereby reducing the computational complexity of the global loss to the computational complexity of a single node.

[0254] Accordingly, the present invention significantly reduces computational complexity and improves computational efficiency in a distributed environment through approximate loss estimation, and effectively solves the optimization bottleneck phenomenon of high-dimensional parameterized models.

[0255] The parameter iterative calculation unit 7 is used to execute step S7: performing parameter iterative calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model.

[0256] Specifically, the parameter iterative calculation is specifically expressed as follows:

[0257]

[0258] Where U (t+1) and V(t+1) They represent the updated kernel weight parameters and regularized weight parameters, that is, the updated parameters of the current parameterized model; U and V represent the unupdated kernel weight parameters and regularized weight parameters, that is, the original parameters of the current parameterized model.

[0259] The iteration judgment unit 8 is used to execute step S8: judging whether the updated parameters of the current parameterized model meet an iteration condition; if not, calling the parameter updating unit 9; if so, taking the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model.

[0260] The specific expression of the iteration condition is as follows:

[0261]

[0262] Where, ||U (t+1) -U (t) || represents the distance between the kernel weight parameters of the current iteration round and the previous iteration round; ||V (t+1) -V (t) || represents the distance between the regularized weight parameters of the current iteration round and the previous iteration round; ε represents the error limit, and the default value is 0.00001.

[0263] When the distance between the updated parameters and the original parameters is less than the preset error limit, it can be considered that the current parameterized model has reached the convergence condition. At this time, there is no need to continue gradient optimization, thus avoiding unnecessary computational overhead.

[0264] The parameter updating unit 9 is used to execute step S9: iterate the current parameterized model according to the updated parameters of the current parameterized model, and call the local gradient calculation unit 4.

[0265] Specifically, according to the updated parameter U of the current parameterized model (t+1) ,V (t+1) , replace the original parameters U, V of the current parameterized model, so as to adjust the kernel weight parameters and regularization weight parameters of the model to make it more consistent with the current optimization goal.

[0266] Accordingly, the updated parameterized model will serve as the initial parameterized model for the next round of iteration to generate target values ​​that match the grouped defect feature datasets and continue to participate in the gradient calculation and optimization process until the iteration conditions are met.

[0267] Compared with the existing technology, the present invention constructs a spatial autoregressive model based on B-splines and combines it with the expectation-maximization algorithm (EM algorithm) to smoothly compensate for the missing feature data, thereby achieving high-precision filling of missing features and ensuring the continuity and consistency of feature dimensions; at the same time, the present invention adopts a streaming distribution method based on hash space to evenly distribute defective feature vectors to computing nodes through group identifiers, thereby ensuring load balancing of computing tasks and preventing the phenomenon of gradient convergence difficulties when aggregating and calculating global gradients due to uneven gradient calculation efficiency.

[0268] In addition, the present invention decomposes the global loss function based on the functional variation principle and replaces the global high-order derivatives with the high-order derivatives of any computing node through approximate calculation, thereby effectively reducing the computational complexity of the global loss function. At the same time, the present invention combines the global gradient to approximate the global loss, significantly reducing the optimization cost of high-dimensional parameterized models in distributed computing and significantly improving the convergence speed and accuracy of the model; ultimately, the present invention effectively solves the problems of high computational cost and convergence difficulty in the prior art.

[0269] Based on the same inventive concept, the present application also provides an electronic device, which can be a terminal device such as a server, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the gradient optimization method based on distributed computing according to an embodiment of the present invention; and the memory is used to store a computer program executable by the processor.

[0270] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to the aforementioned embodiment of the gradient optimization method based on distributed computing, wherein the computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, the steps of the gradient optimization method based on distributed computing recorded in any of the aforementioned embodiments are implemented.

[0271] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. The information can be computer-readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0272] The above-described embodiments merely represent several implementations of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, and the present invention is intended to encompass such modifications and variations.

Claims

1. A gradient optimization method based on distributed computing, characterized in that: The following steps are involved: S1: Perform feature selection and statistics on the current target defect data to obtain a defect feature set; S2: Preprocess the defect feature set and perform smooth compensation on the missing features to generate a defect feature vector; S3: performing workflow division on the plurality of defect feature vectors according to a streaming distribution based on a hash space to obtain a grouped defect feature data set. Specifically, the streaming distribution based on the hash space calculates a group identifier corresponding to any feature dimension i in the plurality of defect feature vectors, and distributes the corresponding defect feature vector to a corresponding computing node according to the group identifier to obtain the grouped defect feature data set. Inputting the grouped defect feature data set into a parameterized model for regression prediction, the parameterized model outputting a target value that matches the grouped defect feature data set; S4: performing local gradient calculation based on the target value output by the current parameterized model to obtain local gradients of several parameterized models; S5: performing global gradient calculation based on the local gradients of several parameterized models to obtain the global gradient of the parameterized model; S6: Estimating the approximate loss of the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model; wherein the specific calculation of the approximate loss estimation is expressed as follows: Where, is the approximate global loss of the current parameterized model, m represents the main computing node of the summary, where m∈[0,K], K represents the total number of computing nodes; K m (U, V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the t-th iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows: Where, is the local gradient of the kernel weight parameter U of the kth computation node. The specific calculation is as follows: Where, Kernel k is the kernel function; k A vector representing the target value of the kth group of defect feature data sets; Represents the global gradient of the regularized weight parameter, and the specific calculation is as follows: Where, is the local gradient of the regularized weight parameter V of the kth computation node. The specific calculation is as follows: Where, ∈ represents the penalty coefficient of regularization; S7: Perform parameter iterative calculation based on the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model; S8: Determine whether the updated parameters of the current parameterized model meet an iteration condition: if not, iterate the current parameterized model according to the updated parameters of the current parameterized model and execute step S4; if so, use the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model.

2. The gradient optimization method based on distributed computing according to claim 1, characterized in that: The parameterized model is a linear regression model or a nonlinear regression model. If the parameterized model is a linear regression model, the parameterized model is specifically expressed as follows: y=WX+b Where y represents the predicted data of the regression task, which is used to represent the target value corresponding to the current input defect feature data X, W represents the regression weight, and WX is used to indicate that the current regression model is a linear regression model; b represents the bias term; at the same time, the kernel function Kernel k It is expressed as follows: Where x ki and x kj They represent the input features of the i-th dimension and the j-th dimension in the k-th group of defect feature datasets, respectively. is the inner product operation; If the parameterized model is a nonlinear regression model, the parameterized model is specifically expressed as follows: y=f(X,θ)+b Where f(X,θ) represents a nonlinear function, which is used to indicate that the current regression model is a nonlinear regression model; θ represents the input parameter of the nonlinear function; and the kernel function Kernel k It is expressed as follows: Kernel k (x ki ,x kj )=exp(-γ||x ki -x kj || 2 ) Where ||x ki -x kj ‖ 2 Represents the input feature x ki with x kj The square of the Euclidean distance between them; γ is the width parameter, which indicates the radial range of the kernel function.

3. The gradient optimization method based on distributed computing according to claim 2, characterized in that: The step S1 comprises the following steps: S11, performing horizontal and vertical feature selection on the current target defect feature to obtain the horizontal feature and vertical feature of the target defect; The lateral characteristics of the target defect are expressed as: X feature ={x1,x2,…,x C } Where x C It is expressed as the Cth transverse feature, where C represents the total number of transverse features; the longitudinal feature of the target defect is expressed as: T process ={t1,t2,…,t P } Where, t P represents the Pth vertical feature, where P represents the total number of vertical features; S12. Using principal component analysis to reduce the dimension and screen the horizontal and vertical features of the target defect to obtain a defect feature set; The specific representation of the defect feature set is as follows: X grouped ={X ′ feature ,T ′ process } Where, X ′ feature represents the set of lateral features after principal component analysis and dimensionality reduction, X ′ feature ={x ′ 1,x ′ 2,…,x ′ c }, its x ′ c represents the cth lateral feature after dimensionality reduction by principal component analysis, c represents the total number of lateral features after dimensionality reduction and screening, where c≤C; T ′ process represents the set of longitudinal features after principal component analysis and dimensionality reduction, T ′ process ={t ′ 1,t ′ 2,…,t ′ p }, its ′ p represents the p-th longitudinal feature after dimensionality reduction by principal component analysis, and p represents the total number of longitudinal features after dimensionality reduction and screening, where p≤P; The step S2 comprises the following steps: S21. Normalize all numerical features in the horizontal features of the defect feature set to obtain normalized numerical features, and combine all normalized numerical features to form a defect numerical feature vector, which is specifically expressed as follows: Where, The normalized value of the input feature of the i-th dimension representing the numerical lateral feature of the n-th dimension; The original value of the input feature of the i-th dimension represented as the numerical lateral feature of the n-th dimension; x ′ n Represents the numerical lateral feature of the nth dimension; S22. Constructing a B-spline-based spatial autoregressive model based on the defect numerical feature vector and the longitudinal features after dimensionality reduction, and performing parameter estimation and missing value filling on the B-spline-based spatial autoregressive model using an expectation-maximization algorithm to generate a defect feature vector; The B-spline-based spatial autoregressive model is specifically expressed as follows: y=A -1 (p)X * the+A -1 (p)e Where y is the response variable vector corresponding to all defect numerical feature vectors, which contains the observed response variable y s and missing response variable y u ; A -1 (ρ)e is the error term, let e * =A -1 (ρ), then e * Satisfy N(0,σ 2 M -1 (ρ)) multivariate normal distribution; σ 2 is the variance coefficient of the error; M(ρ) is the covariance matrix, M(ρ)=A T (ρ)A(ρ),M -1 (ρ) represents the inverse covariance matrix; A(ρ) is the spatial lag coefficient matrix, which is specifically expressed as follows: A(ρ)=I-ρW Where I represents the identity matrix; W is the spatial weight matrix; ρ is the spatial autoregressive parameter, |ρ|<1; X * is the design matrix, which is expressed as follows: X * =(X,S) Where X represents the defect numerical feature vector of the current dimension, and S represents the matrix composed of B-spline basis functions based on longitudinal characteristics, which is used to approximate a smooth unknown function α(T′ process ), the specific form is S=(B1(Z),…,B q (Z)) T , B i (Z) represents the i-th matrix composed of B spline functions; Z represents the longitudinal feature T′ process The specific sampling point set of , the total number of sampling points is point; then its q = g + point, g represents the order of the B-spline basis function; η represents the regression coefficient vector, which is expressed as η=(β T ,γ T ) T , β is the regression coefficient of the horizontal feature; γ is the regression coefficient corresponding to the B-spline basis function, γ=(γ1,…,γ q ) T .

4. The gradient optimization method based on distributed computing according to claim 3, characterized in that: The specific steps of using the expectation-maximization algorithm to estimate the parameters of the B-spline-based spatial autoregressive model include: S221. Divide and jointly distribute the B-spline-based spatial autoregressive model according to the observed response variables and the missing response variables to obtain the mean vector and covariance matrix of the blocks; wherein the specific expression of the mean vector μ is as follows: Where μ s is the mean vector of the observed response variable; μ u is the mean vector of the missing response variable; the specific expression of the covariance matrix M(ρ) is as follows: Where M ss (ρ) is the covariance matrix of the observed response variable; M uu (ρ) is the covariance matrix of the missing response variable; M su (ρ) and M us (ρ) is the covariance matrix between the observed response variables and the missing response variables; S222, estimate the missing data based on the block mean vector and covariance matrix to obtain the conditional expected value of the missing response variable; where, given the observed response variable y s The conditional expectation of the missing response variable for the i-th iteration of The specific calculation is as follows: In the formula, (y s -μ s ) represents the observed variable y s Deviation; S223. Maximize the log-likelihood expression objective function according to the conditional expected value to obtain an iterative value of the current parameter; wherein the log-likelihood expression objective function is specifically expressed as follows: Where, Q1(ρ,θ (i) ) is the approximate log-likelihood expression objective function, θ (i) Represents the input parameters of the i-th iteration, which are specifically: θ (i) =(ρ T ,ρ,ω) T , where ω=σ 2 ; represents the weighted sum of squares of the residuals between the expected conditional value and the model mean vector, is the residual, that is represents the degree of correction of the covariance matrix of the missing response variable, tr(.) is the trace calculation of the matrix; log|M(ρ)| represents the penalty term for the data correlation of the observed response variable; The specific calculation process of the maximization is expressed as follows: r (i+1) =arg max Q1(ρ,θ (i) ) Where, and are the partial derivatives of the regression coefficient vector and the error term variance, respectively, which are specifically expressed as follows: Among them, θ (i+1) It is the iteration value of the current parameter; S224, determine whether the iterative value of the current parameter converges: if so, substitute the current parameter into the spatial autoregressive model based on B-spline to complete the supplement of the missing value and generate the defect feature vector If not, recalculate the conditional expected value, continue iterative calculation, and execute step S222.

5. The gradient optimization method based on distributed computing according to claim 4, characterized in that: For the defect feature data set of group k, it is specifically expressed as: Where, Represents the defect feature vector of the i-th dimension divided into the k-th group and the corresponding target value y k,i ; Represents a group identifier, which is used to convert the corresponding defect feature vector into Assigned to the corresponding computing nodes, the specific calculation is as follows: Where, Represents the current defect feature vector A unique identifier for ; mod indicates a modulo operation.

6. A gradient optimization device based on distributed computing, characterized in that: It includes a feature selection unit, a data compensation unit, a workflow division unit, a local gradient calculation unit, a global gradient calculation unit, an approximate loss estimation unit, a parameter iterative calculation unit, an iterative judgment unit and a parameter updating unit; The feature selection unit is used to perform feature selection and statistics on the current target defect data to obtain a defect feature set; The data compensation unit is used to pre-process the defect feature set and smoothly compensate for the missing features to generate a defect feature vector; The workflow partitioning unit is configured to perform workflow partitioning on a plurality of defect feature vectors according to a streaming distribution based on a hash space to obtain a grouped defect feature data set. Specifically, the streaming distribution based on the hash space calculates a group identifier corresponding to any feature dimension i in the plurality of defect feature vectors and distributes the corresponding defect feature vector to a corresponding computing node according to the group identifier to obtain the grouped defect feature data set. Inputting the grouped defect feature data set into a parameterized model for regression prediction, the parameterized model outputting a target value that matches the grouped defect feature data set; The local gradient calculation unit is used to perform local gradient calculation according to the target value output by the current parameterized model to obtain the local gradients of several parameterized models; The global gradient calculation unit is used to perform global gradient calculation based on the local gradients of the plurality of parameterized models to obtain the global gradient of the parameterized model; The approximate loss estimation unit is used to perform an approximate loss estimation on the parameterized model according to the global gradient of the parameterized model to obtain an approximate global loss of the current parameterized model; wherein the specific calculation of the approximate loss estimation is expressed as follows: Where, is the approximate global loss of the current parameterized model, m represents the main computing node of the summary, where m∈[0,K], K represents the total number of computing nodes; K m (U, V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the t-th iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows: Where, is the local gradient of the kernel weight parameter U of the kth computation node. The specific calculation is as follows: Where, Kernel k is the kernel function; k A vector representing the target value of the kth group of defect feature data sets; Represents the global gradient of the regularized weight parameter, and the specific calculation is as follows: Where, is the local gradient of the regularized weight parameter V of the kth computation node. The specific calculation is as follows: Where, ∈ represents the penalty coefficient of regularization; The parameter iterative calculation unit is used to perform parameter iterative calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model; The iteration judgment unit is used to judge whether the updated parameters of the current parameterized model meet an iteration condition: if not, calling the parameter updating unit; if so, taking the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model; The parameter updating unit is used to iterate the current parameterized model according to the updated parameters of the current parameterized model and call the local gradient calculation unit.

7. A parameterized model for gradient optimization based on distributed computing, characterized in that: The parameterized model is used to perform the following steps: Extract features from the image to be inspected to obtain a feature vector of the defect to be inspected; Linear regression or nonlinear regression is used to model and calculate the characteristic vector of the defect to be detected to obtain the target value; wherein, when the linear regression is used for modeling, it is specifically expressed as: y=WX+b Where y represents the target value, x represents the defect feature vector, W represents the regression weight, and b represents the bias term. When the nonlinear regression is used for modeling, it is specifically expressed as follows: y=f(X,θ)+b Where f(X,θ) represents a nonlinear function; θ represents the input parameter of the nonlinear function; b represents the bias term; Convert the target value according to the preset rules to obtain the defect information in the current defect to be detected image; Wherein, the parameterized model adopts the gradient optimization method based on distributed computing described in any one of claims 1-5 to optimize the parameters of the parameterized model.

8. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the parameterized model of gradient optimization based on distributed computing as claimed in claim 7 is implemented.

Images