Distributed calculation-based gradient optimization method and parameterized model

Abstract

The invention relates to a gradient optimization method based on distributed calculation and a parameterized model. The gradient optimization method based on distributed calculation comprises the following steps: sequentially carrying out feature selection, statistics, preprocessing and smooth compensation on current target defect data, carrying out workflow division on defect feature vectors after smooth compensation according to streaming distribution based on Hash space, and according to the corresponding workflow, carrying out gradient optimization on the defect feature vectors after smooth compensation; performing local gradient calculation and global gradient calculation on the parameterized model to obtain a global gradient of the parameterized model; carrying out approximate loss estimation and parameter iteration according to the global gradient, and judging whether the iterated parameter reaches an iteration condition or not: if not, continuing to carry out local gradient calculation; if yes, gradient optimization of the parameterized model is completed. According to the gradient optimization method based on distributed calculation, the training efficiency of the parameterized model is remarkably improved, and model error accumulation caused by data missing is effectively avoided.

Description

Technical Field

[0001] The present invention relates to the field of distributed machine learning system optimization, and in particular to a gradient optimization method, device, parameterized model and electronic device based on distributed computing. Background Art

[0002] In the field of industrial manufacturing, industrial defect analysis is one of the key links in quality control. Its core goal is to discover product defects through data analysis, trace the source of defects, and locate the process or equipment problems that cause defects, so as to optimize production efficiency and improve product quality. In actual production, industrial defects usually come from abnormal product characteristics (such as dimensional deviation, surface cracks) or abnormal production equipment (such as sensor failure, equipment operation deviation). Therefore, industrial defect analysis needs to process both product and equipment data at the same time and analyze the potential correlation between the two.

[0003] To this end, parametric models are used in traditional technologies to implement industrial defect analysis. The parametric models establish a mapping relationship between data features and output targets through optimizable parameters, such as weight matrices and bias matrices, so as to capture the global patterns and nonlinear relationships of the data, and then realize defect classification, process parameter prediction, and multi-device feature association modeling. For example, the data type is determined based on the input data, the numerical characteristics of the defect are predicted, and the equipment anomalies or process problems that cause the defect are tracked based on process parameter predictions, or joint modeling is performed through parametric models to analyze the potential correlation between products and equipment through parametric models.

[0004] However, with the rapid increase in the types and numbers of industrial production equipment and the increasing complexity of process flows, the scale of product and equipment data has shown an exponential growth. The parametric models of traditional technologies are usually unable to efficiently process data streams from multiple devices, multiple processes, and multi-dimensional features, resulting in a significant decrease in the gradient convergence speed and training efficiency during the training of parameterized models. At the same time, due to the potential missing values ​​and outliers in the process of industrial data collection, the training of parameterized models is easily unstable, and even parameter space dimensionality disasters may occur, leading to the phenomenon of gradient disappearance.

[0005] Based on this, the existing technology uses distributed computing to perform gradient optimization on parameterized models. By decomposing data and tasks into multiple subtasks and distributing them to different computing nodes to run local gradients in parallel, the computing burden of a single node or process in traditional technology is significantly reduced. At the same time, by completing local gradient calculations in multiple computing nodes and through global gradient aggregation and parameter updates, the training efficiency of the parameterized model is significantly improved.

[0006] However, industrial data is usually scattered among multiple independent devices and different process flows, so data islands are prone to occur. The global loss calculation of the existing technology is difficult to achieve efficient integration and analysis of multi-source heterogeneous data due to the large number of parameters and the difficulty of interaction between high-dimensional data. The combination of multiple factors ultimately leads to the problem of difficulty in convergence of parameterized models in the distributed computing of the existing technology. At the same time, production equipment anomalies, sensor anomalies and other reasons are prone to missing data, further affecting the integrity and accuracy of distributed training, and once the training task of a certain node is abnormal, it is easy to affect the calculation of the global gradient, causing global parameter inconsistency, and ultimately causing parameter update failure or training oscillation, resulting in interruption of the training process, making it difficult for the parameterized model to converge. Based on this, in the distributed gradient optimization process of the existing technology, there is a problem of difficulty in convergence of parameterized models. Summary of the invention

[0007] Based on this, the object of the present invention is to provide a gradient optimization method based on distributed computing.

[0008] S1: Perform feature selection and statistics on the current target defect data to obtain a defect feature set;

[0009] S2: Preprocess the defect feature set and perform smooth compensation on the missing features to generate a defect feature vector;

[0010] S3: Perform workflow division on several defect feature vectors according to the streaming distribution based on the hash space to obtain a grouped defect feature data set;

[0011] Inputting the grouped defect feature data set into a parameterized model for regression prediction, the parameterized model outputting a target value matching the grouped defect feature data set;

[0012] S4: performing local gradient calculation according to the target value output by the current parameterized model to obtain local gradients of several parameterized models;

[0013] S5: performing global gradient calculation according to local gradients of several parameterized models to obtain the global gradient of the parameterized model;

[0014] S6: Estimate the approximate loss of the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model;

[0015] S7: performing parameter iterative calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model;

[0016] S8: Determine whether the update parameters of the current parameterized model meet an iteration condition: if not, iterate the current parameterized model according to the update parameters of the current parameterized model and execute step S4; if so, take the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model.

[0017] Compared with the prior art, the gradient optimization method based on distributed computing described in the present invention ensures load balancing of computing nodes through streaming distribution based on hash space, local gradient and global gradient aggregation, thereby improving the robustness and adaptability of the system; at the same time, the present invention calculates the global loss based on the global gradient approximation, significantly reducing the computational complexity of the global loss, thereby greatly improving the efficiency of gradient optimization, and further solving the problem of difficult convergence of parameterized models.

[0018] Furthermore, the specific calculation of the approximate loss estimation is expressed as follows:

[0019]

[0020] In the formula, is the approximate global loss of the current parameterized model, m represents the main computing node of the aggregation, where m∈[0,K], K represents the total number of computing nodes; L m (U,V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the tth iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows:

[0021]

[0022] In the formula, is the local gradient of the kernel weight parameter U of the kth computing node. The specific calculation is as follows:

[0023]

[0024] In the formula, Kernel k is the kernel function; k A vector representing the target value of the kth group of defect feature data sets;

[0025] Represents the global gradient of the regularized weight parameter. The specific calculation is as follows:

[0026]

[0027] In the formula, is the local gradient of the regularized weight parameter V of the kth computing node. The specific calculation is as follows:

[0028]

[0029] Where ∈ represents the regularized penalty coefficient.

[0030] The present invention decomposes the original global loss function based on the functional variation principle, and greatly reduces the complexity of global loss function calculation by approximately replacing the global high-order derivatives with the high-order derivatives of any computing node, thereby significantly reducing the occupation of computing resources and improving the optimization efficiency and convergence speed of high-dimensional parameterized models in a distributed environment; at the same time, by decomposing the global loss function, distributed computing nodes can interact efficiently based on low-order derivatives, avoiding communication overhead and data island problems caused by high-order computing dependence, and further improving the collaborative efficiency of distributed computing; finally, the present invention effectively solves the problems of high computational cost in distributed gradient optimization and difficulty in directly calculating global loss, resulting in difficult gradient convergence.

[0031] Furthermore, the parameterized model is a linear regression model or a nonlinear regression model. If the parameterized model is a linear regression model, the parameterized model is specifically expressed as follows:

[0032] y=WX+b

[0033] In the formula, y represents the predicted data of the regression task, which is used to represent the target value corresponding to the current input defect feature data X, W represents the regression weight, and WX is used to indicate that the current regression model is a linear regression model; b represents the bias term; at the same time, the kernel function Kernel k It is expressed as follows:

[0034]

[0035] In the formula, x ki and x kj They represent the input features of the i-th dimension and the j-th dimension in the k-th group of defect feature data sets, respectively. is the inner product operation;

[0036] If the parameterized model is a nonlinear regression model, the parameterized model is specifically expressed as follows:

[0037] y=f(X,θ)+b

[0038] In the formula, f(X,θ) represents a nonlinear function, which is used to indicate that the current regression model is a nonlinear regression model; θ represents the input parameter of the nonlinear function; at the same time, the kernel function Kernel k It is expressed as follows:

[0039] Kernel k (x ki ,x kj ) = exp(-γ||x ki -x kj || 2 )

[0040] In the formula, ||x ki -x kj || 2 Represents the input feature x ki With x kj The square of the Euclidean distance between them; γ is the width parameter, which indicates the radial range of the kernel function.

[0041] The present invention further limits the kernel functions of linear regression tasks and nonlinear regression tasks. In particular, when the current regression task is nonlinear, the nonlinear correlation features between input features are captured by the Gaussian radial basis kernel function, thereby ensuring that the parameterized model can adapt to the relationship between linear and nonlinear features.

[0042] Furthermore, the step S1 includes the following steps:

[0043] S11, performing horizontal and vertical feature selection on the current target defect feature to obtain the horizontal feature and the vertical feature of the target defect;

[0044] Wherein, the lateral characteristics of the target defect are expressed as:

[0045] X feature ={x 1 ,x 2 ,…,x C}

[0046] In the formula, x C It is expressed as the Cth transverse feature, where C represents the total number of transverse features; the longitudinal feature of the target defect is expressed as:

[0047] T process ={t 1 ,t 2 ,…,t P}

[0048] Where, t P represents the Pth vertical feature, where P represents the total number of vertical features;

[0049] S12, using principal component analysis to reduce the dimension and screen the horizontal and vertical features of the target defect to obtain a defect feature set;

[0050] The specific representation of the defect feature set is as follows:

[0051] X grouped ={X′ feature ,T′ process}

[0052] In the formula, X′ feature represents the set of lateral features after dimensionality reduction by principal component analysis, X′ feature ={x′ 1 ,x′ 2 ,…,x′ c}, its x′ c represents the cth lateral feature after dimension reduction by principal component analysis, c represents the total number of lateral features after dimension reduction and screening, where c≤C; T′ process represents the set of longitudinal features after dimensionality reduction by principal component analysis, T′ process ={t′ 1 ,t′ 2 ,…,t′ p}, its t′ p represents the pth longitudinal feature after dimension reduction by principal component analysis, p represents the total number of longitudinal features after dimension reduction and screening, where p≤P;

[0053] The step S2 comprises the following steps:

[0054] S21, normalize all numerical features in the lateral features of the defect feature set to obtain normalized numerical features, and combine all normalized numerical features to form a defect numerical feature vector, which is specifically expressed as follows:

[0055]

[0056] In the formula, The normalized value of the input feature of the i-th dimension representing the numerical lateral feature of the n-th dimension; The original value of the input feature of the i-th dimension represented as the numerical lateral feature of the n-th dimension; x′ n Represents the numerical lateral feature of the nth dimension;

[0057] S22. Constructing a B-spline-based spatial autoregressive model according to the defect numerical feature vector and the longitudinal feature after dimensionality reduction, and using the expectation-maximization algorithm to perform parameter estimation and missing value filling on the B-spline-based spatial autoregressive model to generate a defect feature vector;

[0058] The B-spline-based spatial autoregressive model is specifically expressed as follows:

[0059] y=A -1 (ρ)X*η+A -1 (ρ)e

[0060] Where y is the response variable vector corresponding to all defect numerical feature vectors, which contains the observed response variable y s and missing response variable y u ; A -1 (ρ)e is the error term, let e * =A -1 (ρ), then e * Satisfy N(0,σ 2 M -1 (ρ)) multivariate normal distribution; σ 2 is the variance coefficient of the error; M(ρ) is the covariance matrix, M(ρ)=A T (ρ)A(ρ),M -1 (ρ) represents the inverse matrix of covariance; A(ρ) is the spatial lag coefficient matrix, which is specifically expressed as follows:

[0061] A(ρ)=I-ρW

[0062] Where I is the unit matrix; W is the spatial weight matrix; ρ is the spatial autoregressive parameter, |ρ|<1; X * is the design matrix, which is expressed as follows:

[0063] X * =(X,S)

[0064] Where X represents the defect numerical feature vector of the current dimension, and S represents the matrix composed of B-spline basis functions based on longitudinal characteristics, which is used to approximate a smooth unknown function α(T′ process ), the specific form is S = (B 1 (Z),…,B q (Z) T , B i (Z) represents the i-th matrix composed of B spline functions; Z represents the longitudinal feature T′ process The specific sampling point set of , the total number of sampling points is point; then q = g + point, g represents the order of the B-spline basis function;

[0065] η represents the regression coefficient vector, which is expressed as η = (β T ,γ T ) T , β is the regression coefficient of the lateral feature; γ is the regression coefficient corresponding to the B-spline basis function, γ=(γ 1 ,…,γ q ) T .

[0066] The present invention constructs a spatial autoregressive model based on B-splines, combines the multidimensional information of horizontal and vertical features, adopts spatial correlation modeling and smooth function approximation technology, and compensates for missing values ​​with high precision, providing a reliable data basis for the optimization of parameterized models in subsequent distributed computing, while ensuring the integrity and consistency of feature data and the spatial correlation modeling effect.

[0067] Furthermore, the specific steps of using the expectation-maximization algorithm to estimate the parameters of the B-spline-based spatial autoregressive model include:

[0068] S221. According to the observed response variables and the missing response variables, the B-spline-based spatial autoregressive model is divided into blocks and jointly distributed to obtain the mean vector and covariance matrix of the blocks; wherein the specific expression of the mean vector μ is as follows:

[0069]

[0070] In the formula, μ s is the mean vector of the observed response variable; μ u is the mean vector of the missing response variable; the specific expression of the covariance matrix M(ρ) is as follows:

[0071]

[0072] Where M ss (ρ) is the covariance matrix of the observed response variable; M uu (ρ) is the covariance matrix of the missing response variable; M su (ρ) and M us (ρ) is the covariance matrix between observed response variables and missing response variables;

[0073] S222, estimate the missing data according to the mean vector and covariance matrix of the blocks to obtain the conditional expected value of the missing response variable; where, given the observed response variable y s The conditional expectation of the missing response variable for the i-th iteration of The specific calculation is as follows:

[0074]

[0075] In the formula, (y s -μ s ) represents the observed variable y s Deviation;

[0076] S223. According to the conditional expected value, the log-likelihood expression objective function is maximized to obtain the iterative value of the current parameter; wherein the log-likelihood expression objective function is specifically expressed as follows:

[0077]

[0078] In the formula, Q 1 (ρ,θ ()) ) is the approximate log-likelihood expression objective function, θ (i) Represents the input parameters of the i-th iteration, which are: θ (i) =(η T ,ρ,ω) T , where ω=σ 2 ; represents the weighted sum of squares of the residuals between the expected conditional value and the model mean vector, is the residual, that is represents the degree of correction of the covariance matrix of the missing response variable, tr(.) is the trace calculation of the matrix; log|M(ρ)| represents the penalty term for the data correlation of the observed response variable;

[0079] The specific calculation process of the maximization is expressed as follows:

[0080] ρ (i+1) = arg max Q 1 (ρ,θ (i) )

[0081]

[0082]

[0083] In the formula, and They are the partial derivatives of the regression coefficient vector and the partial derivatives of the error term variance, respectively, which are specifically expressed as follows:

[0084]

[0085]

[0086] Among them, θ (i+1) It is the iteration value of the current parameter;

[0087] S224. Determine whether the iterative value of the current parameter converges: if so, substitute the current parameter into the B-spline-based spatial autoregressive model to complete the supplement of the missing values ​​and generate a defect feature vector; if not, recalculate the conditional expected value, continue the iterative calculation, and execute step S222.

[0088] The present invention combines the B-spline-based spatial autoregressive model with the expectation-maximization algorithm, from the calculation of block means and covariance matrices, to conditional expected value estimation and parameter optimization, and finally to missing value filling, thereby realizing dynamic estimation and updating of missing values, enhancing the flexibility and adaptability of missing value filling, and ensuring that the spatial autoregressive model can still accurately describe the target value when the data is incomplete.

[0089] Furthermore, the streaming distribution based on the hash space calculates the group identifier corresponding to any feature dimension i in a number of defect feature vectors, and distributes the corresponding defect feature vector to the corresponding computing node according to the group identifier to obtain a grouped defect feature data set. For the defect feature data set of the kth group, it is specifically expressed as:

[0090]

[0091] In the formula, Represents the defect feature vector of the i-th dimension divided into the k-th group and the corresponding target value y k,i ; Represents a group identifier, which is used to convert the corresponding defect feature vector into Assigned to the corresponding computing nodes, the specific calculation is as follows:

[0092]

[0093] In the formula, Represents the current defect feature vector A unique identifier for a ; mod indicates a modulo operation.

[0094] The present invention maps each defect feature vector to the corresponding computing node through streaming distribution based on hash space, and utilizes the uniform distribution characteristics of hash function to evenly distribute the feature vector to all computing nodes, thereby avoiding uneven load between nodes and ensuring full utilization of computing resources.

[0095] A gradient optimization device based on distributed computing, comprising a feature selection unit, a data compensation unit, a workflow division unit, a local gradient calculation unit, a global gradient calculation unit, an approximate loss estimation unit, a parameter iteration calculation unit, an iteration judgment unit and a parameter update unit;

[0096] The feature selection unit is used to perform feature selection and statistics on the current target defect data to obtain a defect feature set;

[0097] The data compensation unit is used to preprocess the defect feature set and smoothly compensate the missing features to generate a defect feature vector;

[0098] The workflow division unit is used to perform workflow division on a plurality of defect feature vectors according to a streaming distribution based on a hash space to obtain a grouped defect feature data set; input the grouped defect feature data set into a parameterized model for regression prediction, and the parameterized model outputs a target value matching the grouped defect feature data set;

[0099] The local gradient calculation unit is used to perform local gradient calculation according to the target value output by the current parameterized model to obtain local gradients of several parameterized models;

[0100] The global gradient calculation unit is used to perform global gradient calculation according to the local gradients of the plurality of parameterized models to obtain the global gradient of the parameterized model;

[0101] The approximate loss estimation unit is used to perform approximate loss estimation on the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model;

[0102] The parameter iteration calculation unit is used to perform parameter iteration calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model;

[0103] The iteration judgment unit is used to judge whether the update parameters of the current parameterized model meet an iteration condition: if not, the parameter update unit is called; if so, the current parameterized model is used as the final parameterized model to complete the gradient optimization of the parameterized model;

[0104] The parameter updating unit is used to iterate the current parameterized model according to the updated parameters of the current parameterized model and call the local gradient calculation unit.

[0105] Furthermore, the specific calculation of the approximate loss estimation is expressed as follows:

[0106]

[0107] In the formula, is the approximate global loss of the current parameterized model, m represents the main computing node of the aggregation, where m∈[0,K], K represents the total number of computing nodes; L m (U,V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the tth iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows:

[0108]

[0109] In the formula, is the local gradient of the kernel weight parameter U of the kth computing node. The specific calculation is as follows:

[0110]

[0111] In the formula, Kernel k is the kernel function; k is a target value, obtained through the defect feature data set;

[0112] Represents the global gradient of the regularized weight parameter. The specific calculation is as follows:

[0113]

[0114] In the formula, is the local gradient of the regularized weight parameter V of the kth computing node. The specific calculation is as follows:

[0115]

[0116] Where ∈ represents the regularized penalty coefficient.

[0117] A parameterized model for gradient optimization based on distributed computing, characterized in that the parameterized model is used to perform the following steps:

[0118] Feature extraction is performed on the defect image to be detected to extract multi-dimensional features that characterize the features to be detected, so as to obtain a feature vector of the defect to be detected; wherein the defect features include but are not limited to defect area and aspect ratio.

[0119] Linear regression or nonlinear regression is used to model and calculate the feature vector of the defect to be detected to obtain the target value; wherein, when the linear regression is used for modeling, it is specifically expressed as:

[0120] y=WX+b

[0121] In the formula, y represents the target value, X represents the defect feature vector, W represents the regression weight, and b represents the bias term;

[0122] When the nonlinear regression is used for modeling, it is specifically expressed as:

[0123] y=f(X,θ)+b

[0124] Where f(X,θ) represents a nonlinear function; θ represents the input parameter of the nonlinear function; b represents the bias term;

[0125] The target value is converted according to preset rules to obtain defect information in the current defect image to be detected; wherein the preset rules may be selected as threshold classification, probability distribution calculation or graded distribution, and different preset rules are selected according to different detection tasks.

[0126] Wherein, the parameterized model uses the above-mentioned gradient optimization method based on distributed computing to optimize the parameters of the parameterized model.

[0127] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] Figure 1 It is a simplified structural diagram of the gradient optimization device for distributed computing according to the present invention;

[0129] Figure 2 The figure is a simplified flowchart of the gradient optimization method based on distributed computing described in the present invention. DETAILED DESCRIPTION

[0130] In order to solve the problem of difficulty in convergence of parameterized models in the distributed gradient optimization process of the prior art, the present invention constructs a defect feature set by performing feature selection and statistics on the current target defect data; then, the defect feature set is preprocessed and missing feature compensation is performed in turn to generate several complete defect feature vectors. Next, the defect feature vector is divided into multiple workflows based on the streaming distribution of the hash space to obtain grouped defect feature vectors; in each workflow, the parameterized model outputs the corresponding target value according to the grouped defect feature vector, performs local gradient calculation according to the corresponding target value, generates local gradients of several parameterized models, and aggregates the local gradients to calculate the global gradient of the parameterized model. According to the global gradient, the parameterized model is approximated for loss estimation, and the parameter of the parameterized model is iterated based on the loss estimation result. Finally, it is judged whether the parameters after iteration meet the preset convergence conditions: if they meet, the training is completed and the optimized parameterized model is output; if they do not meet, the local gradient calculation is performed again and the iteration continues.

[0131] Based on this, the present invention uses a workflow partitioning strategy based on the streaming distribution of hash space to perform local gradient calculations in parallel on multiple computing nodes and summarize the global gradient, and performs approximate loss estimation based on the global gradient, which significantly improves the training efficiency of the parameterized model. At the same time, the missing data is compensated by the non-parametric interpolation method to ensure the integrity of the defect feature vector and effectively avoid the accumulation of model errors caused by missing data.

[0132] Based on the above design, the present invention proposes a gradient optimization method based on distributed computing, and based on this method proposes a gradient optimization device based on distributed computing.

[0133] Please also see Figure 1 and Figure 2 , Figure 1 is a simplified structural diagram of the gradient optimization device based on distributed computing according to the present invention, Figure 2 The figure is a simplified flowchart of the gradient optimization method based on distributed computing described in the present invention.

[0134] The gradient optimization device based on distributed computing includes a feature selection unit 1, a data compensation unit 2, a workflow division unit 3, a local gradient calculation unit 4, a global gradient calculation unit 5, an approximate loss estimation unit 6, a parameter iterative calculation unit 7, an iterative judgment unit 8 and a parameter updating unit 9.

[0135] The feature selection unit 1 is used to execute step S1: perform feature selection and statistics on current target defect data to obtain a defect feature set.

[0136] Specifically, horizontal and vertical feature selection is performed on the current target defect feature to obtain the horizontal and vertical features of the target defect;

[0137] The feature selection may adopt correlation analysis, which uses correlation measurement methods such as Pearson correlation coefficient or partial correlation coefficient to calculate the standard deviation and covariance between the current data and the target value, and calculate the corresponding information gain, so as to screen out the corresponding features according to the information gain using the threshold, and filter out the redundant features, so as to obtain the horizontal and vertical features of the target defect;

[0138] The lateral feature X feature It is expressed as:

[0139] X feature ={x 1 ,x 2 ,…,x C}

[0140] In the formula, x CIt is represented as the Cth lateral feature, where C represents the total number of lateral features. In this embodiment, the lateral features include geometric features and statistical features. The geometric features are used to describe the shape, size or spatial distribution of defects, and can be specifically selected as defect area, aspect ratio, radius of curvature, skeleton length or center of gravity position, etc. Since the characteristics of defects corresponding to different products are also different, the present invention does not specifically limit the selection of geometric features. The statistical features are used to characterize the distribution characteristics of defects in the detection area, and can be specifically selected as overall horizontality, number of voids, defect distribution density, etc. The present invention does not specifically limit the selection of statistical features.

[0141] The longitudinal characteristic T process It is used to reflect the corresponding changes in the distribution of target defects over time or process flow, which is specifically expressed as:

[0142] T process ={t 1 ,t 2 ,…,t P}

[0143] Where, t P It represents the Pth longitudinal feature, where P represents the total number of longitudinal features. In this embodiment, the longitudinal feature may be selected from detection time, detection task, product batch, product identification or image serial number, etc. The present invention does not specifically limit the selection of the longitudinal feature.

[0144] Next, principal component analysis is used to reduce the dimension and screen the horizontal and vertical features of the target defect to reduce redundant information and highlight the main features, and obtain the defect feature set. The specific representation of the defect feature set is as follows:

[0145] X grouped ={X′ feature ,T′ process}

[0146] In the formula, X′ feature It represents the set of lateral features after the principal component analysis and dimensionality reduction, which is specifically expressed as: X′ feature ={x′ 1 ,x′ 2 ,…,x′ c}, its x′ c represents the cth lateral feature after dimension reduction by principal component analysis, c represents the total number of lateral features after dimension reduction and screening, where c≤C;

[0147] T′ process Represents the set of longitudinal features after dimensionality reduction by principal component analysis, which is specifically expressed as T′ process ={t′ 1 ,t′ 2 ,…,t′p}, its t′ p represents the pth longitudinal feature after dimensionality reduction by principal component analysis, p represents the total number of longitudinal features after dimensionality reduction and screening, and p≤P.

[0148] The principal component analysis is performed by calculating the mean of the horizontal and vertical features for centralization, and the covariance matrix is ​​calculated based on the characteristic matrix obtained by centralization, and the eigenvalue decomposition of the covariance matrix is ​​performed to obtain the eigenvector corresponding to the eigenvalue, so as to accumulate the variance contribution rate according to the eigenvalue size, so as to screen out the features corresponding to the corresponding main components, that is, X′ feature and T′ process .

[0149] Since principal component analysis is not the core innovation of the present invention, and the corresponding selected features are different according to different task objectives or input data, and feature selection can also be combined with causal analysis, the present invention does not specifically limit the means of feature selection.

[0150] The data compensation unit 2 is used to execute step S2: preprocessing the defect feature set and performing smooth compensation on the missing features to generate a defect feature vector.

[0151] The specific steps include:

[0152] S21, normalize all numerical features in the lateral features of the defect feature set to obtain normalized numerical features, and combine all normalized numerical features to form a defect numerical feature vector, which is specifically expressed as normalizing the original value of the input feature of the i-th dimension of the n-th dimension numerical lateral feature as follows:

[0153]

[0154] In the formula, The normalized value of the input feature of the i-th dimension representing the numerical lateral feature of the n-th dimension; The original value of the input feature of the i-th dimension represented as the numerical lateral feature of the n-th dimension; x′ n Represents the numerical lateral feature of the nth dimension. Max(x′ n ) and min(x′ n ) represent the minimum and maximum values ​​of the n-th dimension numerical lateral feature respectively.

[0155] When all numerical lateral features are normalized, all normalized numerical features are combined to form a defect numerical feature vector, which is specifically expressed as follows:

[0156]

[0157] In the formula, X numeric Represents the numerical feature vector of the defect; n represents the total number of dimensions of the numerical lateral features, where n≤c.

[0158] S22. A spatial autoregressive model based on B-splines is constructed according to the defect numerical feature vector and the longitudinal feature after dimensionality reduction, and an expectation-maximization algorithm is used to perform parameter estimation and missing value filling on the spatial autoregressive model based on B-splines to ensure the continuity and consistency of the data in the feature dimension, thereby generating a defect feature vector. The specific representation of the spatial autoregressive model based on B-splines is as follows:

[0159] y=A -1 (ρ)X*η+A -1 (ρ)e

[0160] Where y is the response variable vector corresponding to all defect numerical feature vectors, which contains the observed response variable y s and missing response variable y u ; A -1 (η)e is the error term, let e * =A -1 (η) is used to represent the noise caused by random factors in the model, which satisfies e * ~N(0,σ 2 M -1 (ρ)) multivariate normal distribution; σ 2 is the variance coefficient of the error, which is used to control the scale of the error term; M(ρ) is the covariance matrix, that is, M(ρ) = A T (ρ)A(ρ), then M -1 (ρ) represents the inverse matrix of the covariance, which is used to describe the correlation between the observed variables to reflect the error dependence between different observed variables;

[0161] A(ρ) is the spatial lag coefficient matrix, that is, the spatial autoregressive correlation coefficient between the response variable and the neighboring values, which is specifically expressed as:

[0162] A(ρ)=I-ρW

[0163] In the formula, I represents the unit matrix; W is the spatial weight matrix, which is used to describe the spatial adjacency relationship between input features under the same feature dimension. Its matrix element w ij represents the spatial relationship between input feature i and input feature j; ρ is the spatial autoregression parameter, which is used to control the spatial dependence between response variables and is constrained to be |ρ|<1;

[0164] X *Represents the design matrix, which contains the B-spline basis functions of the horizontal and vertical features. The specific form is:

[0165] X * =(X,S)

[0166] Where X represents the defect numerical feature vector of the current dimension, and S represents the matrix composed of B-spline basis functions based on longitudinal characteristics, which is used to approximate a smooth unknown function α(T′ process ), the specific form is S = (B 1 (Z),…,B q (Z) T , B i (Z) represents the i-th matrix composed of B spline functions, which is used to represent the vertical feature T′ process Mapped to the basis function space, and Z represents the longitudinal feature T′ process The specific sampling point set of the sampling point is point, that is, point points are selected from the interval [0,1], (for example: 0 = sample 1 <sample 2 <… <sample point-1 <sample point =1, sample i represents the i-th sampling point), which is used to define the control point of the B-spline basis function; its q = g + point, g represents the order of the B-spline basis function;

[0167] η represents the regression coefficient vector, which is expressed as η = (β T ,γ T ) T , β is the regression coefficient of the horizontal feature; γ is the regression coefficient corresponding to the B-spline basis function, which is used to represent the influence of the approximation of the longitudinal feature on the response variable through the B-spline basis function, γ = (γ 1 ,…,γ q ) T .

[0168] Among them, X*η represents the approximation of the longitudinal feature T′ by the B-spline basis function process The corresponding smooth unknown function α(T′ process ) is a non-parametric term used to represent the impact of longitudinal characteristics on the response variable, and the smooth unknown function can be expressed as

[0169] α(T′ process )=(α(z 1 ),…,a(z p )) T

[0170] In the formula, α(z i) represents the smooth function corresponding to the i-th feature dimension in the longitudinal feature, which is used to reflect the nonlinear effect of the i-th dimension of the longitudinal feature on the response variable. The smooth function α(z i ) is constructed by performing data approximation on the longitudinal features of the i-th dimension through the B-spline basis function to capture the changing trend of the feature in the continuous space.

[0171] According to this, α(T′ process ) can be expressed as:

[0172] α(T′ process )=Sγ+∈

[0173] Where ∈ represents the approximation error term, that is, the error introduced by the B-spline approximation process. However, as the number of sampling points point→∞, the approximation error term ∈ can be considered as an asymptotic constant, that is, its error magnitude approaches O p (1), which means that the error term tends to a finite constant and can usually be ignored in practice, making the model very smooth and stable in theory.

[0174] The specific steps of using the expectation-maximization algorithm to estimate the parameters of the B-spline-based spatial autoregressive model include:

[0175] S221. According to the observed response variables and the missing response variables, the B-spline-based spatial autoregressive model is divided into blocks and jointly distributed to obtain the mean vector and covariance matrix of the blocks, wherein the specific expression of the mean vector μ is as follows:

[0176]

[0177] In the formula, μ s is the mean vector of the observed response variable; μ u is the mean vector of the missing response variables.

[0178] The specific expression of its covariance matrix M(ρ) is as follows:

[0179]

[0180] Where M ss (ρ) is the covariance matrix of the observed response variable; M uu (ρ) is the covariance matrix of the missing response variable; M su (ρ) and M us (ρ) is the covariance matrix between the observed response variables and the missing response variables.

[0181] S222, perform missing data estimation based on the block mean vector and covariance matrix to estimate potential latent variables, so as to help fill the missing data and obtain the conditional expected value of the missing response variable, given the observed response variable y s The conditional expectation of the missing response variable for the i-th iteration of The specific calculation is as follows:

[0182]

[0183] In the formula, (y s -∈ s ) represents the observed variable y s Deviations from the mean are used to adjust the predicted values ​​of the missing response variable.

[0184] S223. According to the conditional expected value, the log-likelihood expression objective function is maximized to obtain the iterative value of the current parameter. The specific expression of the log-likelihood expression objective function is as follows:

[0185]

[0186] In the formula, θ (i) Represents the input parameters of the i-th iteration, which are: θ (i) =(η T ,ρ,ω) T , where ω=σ 2 ; is a constant term, used as a standardization factor for normal distribution; Represents the variance of the error term, which is used to adjust the degree of data adaptation and optimize the variance parameter by maximizing the log-likelihood; The penalty term representing the data correlation of the observed response variables is used to reflect the size of the covariance matrix through the natural logarithm to characterize the correlation between the observed response variables; Represents the difference between the expected conditional value and the mean vector of the model, which is used to measure the weighted deviation between the estimated value and the theoretical mean, so by minimizing the residual make Make the predicted values ​​as close as possible to the observed values, thereby maximizing the fitness of the filled data; Indicates the degree of correction of the covariance matrix of the missing response variable. Tr(.) is calculated by the trace of the matrix to improve the estimation accuracy of the missing data. (i) represents the variance of the error term in the i-th iteration.

[0187] Furthermore, in order to simplify the iteration process, the iteration value θ of the current parameter is obtained. (i+1) The present invention expresses the objective function Q(θ,θ (i)) is partially derived and solved to obtain the log-likelihood function with only the spatial autoregressive parameter ρ. The specific calculation of its partial derivative is expressed as follows:

[0188]

[0189] In the formula, The partial derivative of the regression coefficient vector is used to update the regression coefficients to reflect the modification of the model based on the new data; The partial derivative of the error term variance is used to update the error variance based on the covariance matrix of the current error term and the model. The weighted sum of squares of the residuals between the expected conditional value and the model mean vector is used to quantify the size of the error. It is combined with the covariance matrix update to reflect the model's estimate of the error variance.

[0190] The approximate log-likelihood expression objective function of the spatial autoregressive parameter ρ can be expressed as:

[0191]

[0192] In the formula, Q 1 (ρ,θ (i) ) represents the log-likelihood function of the spatial autoregressive parameter ρ. By finding the optimal spatial autoregressive parameter ρ, the current model can fit the missing values ​​in the current defect numerical feature vector.

[0193] Accordingly, the iterative value of the current parameter is obtained by maximizing the log-likelihood function of the spatial autoregressive parameter ρ, which is specifically expressed as follows:

[0194] ρ (i+1) = argmaxQ 1 (ρ,θ (i) )

[0195]

[0196] Based on this, the iteration value θ of the current parameter (i+1) =(η (i+1)T ,ρ (i+1) ,ω (i+1) ) T .

[0197] Among them, regarding the initial value of the regression coefficient vector, the preliminary regression coefficient can be estimated by least squares (OLS); the variance of the error term can be the variance of the existing observed response variable or the mean as the initial value; and the spatial autoregression parameter can be initialized by a priori estimation, or initialized by a random number [0,1]. Since the initialization does not essentially affect the final fitting effect, but only affects the efficiency of the fitting, and at the same time, the corresponding initial value can be set differently according to different input features, so the present invention does not specifically limit the selection of the initial value.

[0198] S224, determine whether the iterative value of the current parameter converges: if so, substitute the current parameter into the B-spline-based spatial autoregressive model to complete the supplement of the missing value and generate the defect feature vector If not, recalculate the conditional expected value, continue iterative calculation, and execute step S222.

[0199] The workflow division unit 3 is used to execute step S3: performing workflow division on a plurality of defect feature vectors according to the streaming distribution based on the hash space to obtain a grouped defect feature data set.

[0200] Specifically, the streaming distribution based on hash space calculates the group identifier corresponding to any feature dimension i in a number of defect feature vectors, and distributes the corresponding defect feature vector to the corresponding computing node according to the group identifier to obtain a grouped defect feature data set. For the defect feature data set of the kth group, it is specifically expressed as:

[0201]

[0202] In the formula, Represents the defect feature vector of the i-th dimension divided into the k-th group and the corresponding target value y k,i , its target value y k,i According to the corresponding task definition or feature engineering, several defect feature vectors are analyzed and obtained. The present invention does not specifically limit its target value y k,i Choice; Represents a group identifier, which is used to convert the corresponding defect feature vector into Assigned to the corresponding computing nodes, the specific calculation is as follows:

[0203]

[0204] In the formula, Represents the current defect feature vector A unique identifier, which can be obtained by a custom increment function or other hash algorithms such as MD5. The present invention does not specifically limit the calculation method of the unique identifier; mod represents a modulus operation, where K represents the total number of computing nodes, and the total number of nodes is divided or allocated according to computing resources, which is not specifically limited by the present invention.

[0205] Furthermore, when the workload of the computing node reaches the upper limit or the group identifier When the corresponding computing node and its nearest neighbor nodes are both busy, a new computing node is created, and the identifier of the new computing node is equal to the hash value of the current defect feature vector. And update the total number of nodes K, K←K+1, and redistribute the current defect feature vector;

[0206] Based on this, when computing resources are limited or node loads are uneven, dynamically expanding computing nodes can effectively avoid computing bottlenecks and ensure task allocation efficiency in a distributed environment.

[0207] After the workflow is divided, the grouped defect feature data sets are input into the parameterized model of each computing node for regression prediction. The parameterized model of each computing node generates a matching target value according to its assigned data set.

[0208] The parameterized model is a regression model, which is used to output the corresponding target value y according to the input defect feature data X. To parameterize the model, to obtain y k,i ), its specific model can be expressed as:

[0209] y=WX+b

[0210] or,

[0211] y=f(X,θ)+b

[0212] Wherein, y represents the predicted data of the regression task, which is used to represent the target value corresponding to the currently input defect feature data X, and is specifically set according to different application scenarios, which is not specifically limited in the present invention; W represents the regression weight, and WX is used to indicate that the current regression model is a linear regression model; f(X, θ) represents a nonlinear function, which is used to indicate that the current regression model is a nonlinear regression model, and θ represents the input parameter of the nonlinear function; and b represents a bias term.

[0213] Since all computing nodes share consistent parameterized model parameters but process different data sets, distributed collaborative computing can fully utilize the computing resources of each node while ensuring the consistency of global computing results, thereby improving task execution efficiency and reducing the computing load of a single node.

[0214] Based on this, local loss calculation and local gradient calculation are performed according to the target value matched by the grouped defect feature data set, and the local gradient corresponding to each computing node can be obtained.

[0215] The local gradient calculation unit 4 is used to execute step S4: perform local gradient calculation according to the target value output by the current parameterized model to obtain local gradients of several parameterized models.

[0216] Specifically, according to the corresponding computing node k and the corresponding defect feature data set of the kth group, the local loss of the parameterized model is calculated to obtain the local loss L of the current computing node k. k (U,V), the specific calculation is as follows:

[0217]

[0218] Where U is the kernel weight parameter, which is used to represent the regression weight of the kernel function. ki The contribution degree of the model prediction value to fit the target value y ki In this embodiment, the input features and the corresponding target values ​​adopt the defect feature vector after smooth compensation, that is, y ki =y k,i ; V is the regularized weight parameter, which is used to limit the complexity of the parameterized model and avoid overfitting to ensure the robustness and generalization ability of the model; ∈ represents the penalty coefficient of the regularization, which is used to measure the impact of regularization on the overall loss; Kernel k It is a kernel function that describes the relationship and similarity between input features in high-dimensional space, guides the model to regress in the kernel space, and thus guides the parameterized model to regress in the kernel space. When the regression task shows linear correlation, the specific expression of its kernel function is:

[0219]

[0220] In the formula, x ki and x kj They represent the input features of the i-th dimension and the j-th dimension in the k-th group of defect feature data sets, respectively. is the inner product operation, used to calculate x ki and x kj The similarity between .

[0221] When the regression task represents nonlinear correlation, the specific expression of its kernel function is:

[0222] Kernel k (x ki ,x kj )=exp(-γ‖x ki-x kj ‖ 2 )

[0223] In the formula, ‖x ki -x kj ‖ 2 Represents the input feature x ki With x kj The square of the Euclidean distance between them is used to represent the distance between them in the feature space; γ is the width parameter, which represents the radial range of the kernel function. The larger its value, the larger the kernel function. k The faster it approaches 0.

[0224] is a quadratic regularization term, representing the kernel weight parameter U of the current node k k In the kernel function Kernel k The quadratic sum on is used to measure the kernel weight parameter U of the kth computing node k The degree of fit between the similarities of input features in the high-dimensional kernel space; ∈V k Represents the regularization penalty term, which is used to limit the weight parameter V of the regularization of the kth computing node k The size of , thereby controlling the degree of fit of the model on local data; is the fitting term, which is used to represent the kernel weight parameter U of the kth computing node k For the target value y ki The fitting effect of the model can be used to reflect the prediction ability of the current parameterized model for the target value.

[0225] Furthermore, in order to improve the efficiency of gradient descent, the initial value U of the kth computation node is k and V k The calculation is expressed as:

[0226]

[0227] In the formula, and Used to represent the initial value of the kth workflow, y k The vector representing the target value of the kth defect feature data set, i.e., y k ={y k1 ,…,y ki ,…,y kn}.

[0228] Next, the local loss L of the current computing node k is k (U, V) performs partial derivative calculation to obtain the local gradient of the parameterized model of the current computing node. The specific calculation is expressed as follows:

[0229]

[0230] In the formula, is the local gradient of the kernel weight parameter U of the kth computing node, that is, the partial derivative of the kernel weight parameter U, which is used to guide the model to adjust the kernel weight parameter U to reduce the fitting error; is the local gradient of the regularized weight parameter V of the kth computing node, that is, the partial derivative of the regularized weight parameter, which is used to limit the complexity of the model and prevent overfitting; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the tth iteration respectively.

[0231] Finally, the local gradients of the parameterized models of all computing nodes in the current iteration are summarized to obtain the local gradients of several parameterized models.

[0232] Among them, the summary will pass through the Mapper stage, mapping the local gradient of each computing node into a key-value pair (such as <node ID, local gradient>); then, through the Reducer stage, all mapped local gradients are grouped and summarized according to the specific rules of the key-value pair, and the global gradient is calculated by weighted average or accumulation.

[0233] The global gradient calculation unit 5 is used to execute step S5: perform global gradient calculation according to the local gradients of several parameterized models to obtain the global gradient of the parameterized model.

[0234] Specifically, the specific calculation of its global gradient is expressed as follows:

[0235]

[0236] In the formula, Represents the global gradient of the kernel weight parameter U, that is, the normalized mean of the local gradients on all computing nodes, which is used to summarize the local gradients of all nodes and balance the impact of data distribution on different nodes on the kernel weight parameter U. K represents the total number of computing nodes; It represents the global gradient of the regularized weight parameter V, that is, the normalized mean of the local gradients on all computing nodes, which is used to summarize the regularized gradients of all nodes and balance the impact of the regularization term on the global loss, thereby ensuring that the complexity of the model is controlled within a reasonable range; t represents the iteration round of the current global gradient.

[0237] Since the global gradient can only provide directional information of the current parameter update, it is impossible to directly quantify the changing trend of the global loss. Especially in the optimization of high-dimensional parameterized models, simple reliance on the global gradient may lead to improper selection of the update step size, thereby affecting the convergence speed and optimization stability of the model. Therefore, the present invention combines the calculation of approximate global loss, comprehensively considers the relationship between the global gradient and the current parameters, and effectively estimates the decrease in the global loss, thereby providing a more accurate optimization basis for parameter updating.

[0238] The approximate loss estimation unit 6 is used to execute step S6: perform approximate loss estimation on the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model.

[0239] Specifically, the specific calculation of the approximate loss estimation is expressed as follows:

[0240]

[0241] In the formula, is the approximate global loss of the current parameterized model, which is used to estimate the global loss in the current iteration by using the global gradient and the loss information of the main computing node m that is summarized. m represents the main computing node that is summarized, where m∈[0,K], and m defaults to 1; L m (U,V) represents the local loss of the mth computing node; <·,·> is the inner product operation, which is used to represent the inner product calculation between two vectors, thereby indicating the directional influence of the gradient information on the parameter U or V of the current parameterized model.

[0242] Since the shape of the parameters U and V under the calculation of the global loss is extremely large, the corresponding calculation cost is complex. Therefore, the present invention obtains the expression of the approximate loss estimation through a series of derivations, and the corresponding specific derivation process is as follows:

[0243] It is known that the loss of any computing node k is calculated as:

[0244]

[0245] ∴The global loss calculation can be expressed as:

[0246]

[0247] Therefore, according to the functional variation principle, the function can be approximately decomposed into a finite-order differential operator, which is specifically expressed as:

[0248]

[0249] in, represents the initial model parameters of the parameterized model, so is a constant; at the same time, and The order of is low, and it can interact in the workflow of each computing node, while Q(U, V) is a high-order derivative, so it is difficult to interact in the workflow of the computing node. However, since the present invention divides the workflow through streaming distribution based on hash space, all computing nodes are in a load-balanced state, and the difference in all losses on each computing node is small, so any node can be selected as the main computing node, and the present invention uses the first node as the main computing node by default, that is, the high-order derivative Q of the workflow of the first computing node can be directly used. 1 (U,V) replaces Q(U,V), and thus the following expression of higher-order derivatives is obtained:

[0250]

[0251] ∴ Q 1 Substitute (U,V) into L N (U, V), the expression of the approximate loss estimation is obtained, which is specifically expressed as:

[0252]

[0253] In order to solve the problem that the global loss in distributed computing will not be too complex in gradient calculation due to too large a number of parameters, which will lead to difficulty in interaction of high-order derivative terms in global loss calculation and difficulty in gradient convergence of parameterized models, the present invention utilizes the functional variation principle to decompose the global loss function into a low-order derivative part and a high-order derivative part; wherein, the low-order derivative part can be efficiently completed directly through the gradient interaction of each computing node; and although the high-order derivative part has the problems of high complexity and high computational overhead, the present invention selects any node as the main computing node (main workflow) to replace the high-order derivative terms when all computing nodes are load-balanced, thereby effectively solving the problem of difficulty in interaction of high-order derivative terms in global loss through approximation, and further reducing the computational complexity of global loss to the computational complexity of a single node.

[0254] Accordingly, the present invention significantly reduces computational complexity and improves computational efficiency in a distributed environment through approximate loss estimation, and effectively solves the optimization bottleneck phenomenon of high-dimensional parameterized models.

[0255] The parameter iteration calculation unit 7 is used to execute step S7: perform parameter iteration calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model.

[0256] Specifically, the specific expression of the parameter iterative calculation is as follows:

[0257]

[0258] Where U (t+1) and V (t+1) They represent the updated kernel weight parameters and regularized weight parameters respectively, that is, the updated parameters of the current parameterized model; U and V represent the unupdated kernel weight parameters and regularized weight parameters, that is, the original parameters of the current parameterized model.

[0259] The iteration judgment unit 8 is used to execute step S8: judging whether the update parameters of the current parameterized model meet an iteration condition: if not, calling the parameter updating unit 9; if so, taking the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model.

[0260] The specific expression of the iteration condition is as follows:

[0261]

[0262] In the formula, ||U (t+1) -U (t) || represents the distance between the kernel weight parameters of the current iteration round and the previous iteration round; ||V (t+1) -V (t) || represents the distance between the regularized weight parameters of the current iteration round and the previous iteration round; ε represents the error limit, and the default value is 0.00001.

[0263] When the distance between the updated parameters and the original parameters is less than the preset error limit, it can be considered that the current parameterized model has reached the convergence condition. At this time, there is no need to continue gradient optimization, thus avoiding unnecessary computational overhead.

[0264] The parameter updating unit 9 is used to execute step S9: iterate the current parameterized model according to the updated parameters of the current parameterized model, and call the local gradient calculation unit 4.

[0265] Specifically, according to the update parameter U of the current parameterized model (t+1) ,V (t+1) , replace the original parameters U and V of the current parameterized model, so as to adjust the kernel weight parameters and regularization weight parameters of the model to make it more in line with the current optimization goal.

[0266] Accordingly, the updated parameterized model will be used as the initial parameterized model for the next round of iteration to generate target values ​​that match the grouped defect feature data set and continue to participate in the gradient calculation and optimization process until the iteration conditions are met.

[0267] Compared with the prior art, the present invention achieves high-precision filling of missing features and ensures the continuity and consistency of feature dimensions by constructing a spatial autoregressive model based on B-splines and combining the expectation-maximization algorithm (EM algorithm) to smoothly compensate for missing feature data; at the same time, the present invention adopts a streaming distribution method based on hash space to evenly distribute defective feature vectors to computing nodes through grouping identifiers, thereby ensuring load balancing of computing tasks and preventing the phenomenon of gradient convergence difficulties when aggregating and calculating global gradients due to different gradient calculation efficiencies.

[0268] In addition, the present invention decomposes the global loss function based on the functional variation principle, and replaces the global high-order derivatives with the high-order derivatives of any computing node through approximate calculation, thereby effectively reducing the computational complexity of the global loss function. At the same time, the present invention combines the global gradient to approximate the global loss, significantly reducing the optimization cost of the high-dimensional parameterized model in distributed computing, and significantly improving the convergence speed and accuracy of the model; finally, the present invention effectively solves the problems of high computational cost and difficulty in convergence in the prior art.

[0269] Based on the same inventive concept, the present application also provides an electronic device, which may be a terminal device such as a server, a desktop computing device or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). The device includes one or more processors and a memory, wherein the processor is used to execute a program to implement the gradient optimization method based on distributed computing in an embodiment of the present invention; and the memory is used to store a computer program that can be executed by the processor.

[0270] Based on the same inventive concept, the present application also provides a computer-readable storage medium, corresponding to the aforementioned embodiment of a gradient optimization method based on distributed computing, wherein the computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, the steps of the gradient optimization method based on distributed computing recorded in any of the above embodiments are implemented.

[0271] The present application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0272] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, and the present invention is also intended to include these modifications and modifications.

Claims

1. A gradient optimization method based on distributed computing, characterized in that: The following steps are involved: S1: Perform feature selection and statistics on the current target defect data to obtain a defect feature set; S2: Preprocess the defect feature set and perform smooth compensation on the missing features to generate a defect feature vector; S3: Perform workflow division on several defect feature vectors according to the streaming distribution based on the hash space to obtain a grouped defect feature data set; Inputting the grouped defect feature data set into a parameterized model for regression prediction, the parameterized model outputting a target value matching the grouped defect feature data set; S4: performing local gradient calculation according to the target value output by the current parameterized model to obtain local gradients of several parameterized models; S5: performing global gradient calculation according to local gradients of several parameterized models to obtain the global gradient of the parameterized model; S6: Estimate the approximate loss of the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model; S7: performing parameter iterative calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model; S8: Determine whether the update parameters of the current parameterized model meet an iteration condition: if not, iterate the current parameterized model according to the update parameters of the current parameterized model and execute step S4; if so, take the current parameterized model as the final parameterized model to complete the gradient optimization of the parameterized model.

2. The gradient optimization method based on distributed computing according to claim 1, characterized in that: The specific calculation of the approximate loss estimate is as follows: In the formula, is the approximate global loss of the current parameterized model, m represents the main computing node of the aggregation, where m∈[0,K], K represents the total number of computing nodes; L m (U,V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the tth iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows: In the formula, is the local gradient of the kernel weight parameter U of the kth computing node. The specific calculation is as follows: In the formula, Kernel k is the kernel function; k A vector representing the target value of the kth group of defect feature data sets; Represents the global gradient of the regularized weight parameter. The specific calculation is as follows: In the formula, is the local gradient of the regularized weight parameter V of the kth computing node. The specific calculation is as follows: Where ∈ represents the regularized penalty coefficient.

3. The gradient optimization method based on distributed computing according to claim 2, characterized in that: The parameterized model is a linear regression model or a nonlinear regression model. If the parameterized model is a linear regression model, the parameterized model is specifically expressed as follows: y=WX+b In the formula, y represents the predicted data of the regression task, which is used to represent the target value corresponding to the current input defect feature data X, W represents the regression weight, and WX is used to indicate that the current regression model is a linear regression model; b represents the bias term; at the same time, the kernel function Kernel k It is expressed as follows: In the formula, x ki and x kj They represent the input features of the i-th dimension and the j-th dimension in the k-th group of defect feature data sets, respectively. is the inner product operation; If the parameterized model is a nonlinear regression model, the parameterized model is specifically expressed as follows: y=f(X,θ)+b In the formula, f(X,θ) represents a nonlinear function, which is used to indicate that the current regression model is a nonlinear regression model; θ represents the input parameter of the nonlinear function; at the same time, the kernel function Kernel k It is expressed as follows: Kernel k (x ki ,x kj )=exp(-γ||x ki -x kj || 2 ) In the formula, ||x ki -x kj || 2 Represents the input feature x ki With x kj The square of the Euclidean distance between them; γ is the width parameter, which indicates the radial range of the kernel function.

4. The gradient optimization method based on distributed computing according to claim 3 is characterized in that: The step S1 comprises the following steps: S11, performing horizontal and vertical feature selection on the current target defect feature to obtain the horizontal feature and the vertical feature of the target defect; Wherein, the lateral characteristics of the target defect are expressed as: X feature ={x1,x2,…,x C } In the formula, x C It is expressed as the Cth transverse feature, where C represents the total number of transverse features; the longitudinal feature of the target defect is expressed as: T process ={t1,t2,…,t P } Where, t P represents the Pth vertical feature, where P represents the total number of vertical features; S12, using principal component analysis to reduce the dimension and screen the horizontal and vertical features of the target defect to obtain a defect feature set; The specific representation of the defect feature set is as follows: X grouped ={X ′ feature ,T ′ process } Where, X ′ feature represents the set of lateral features after dimensionality reduction by principal component analysis, X ′ feature ={x ′ 1,x ′ 2,…,x ′ c }, its x ′ c represents the cth lateral feature after dimension reduction by principal component analysis, c represents the total number of lateral features after dimension reduction and screening, where c≤C; T ′ process represents the set of longitudinal features after dimensionality reduction by principal component analysis, T ′ process ={t ′ 1,t ′ 2,…,t ′ p }, its ′ p represents the pth longitudinal feature after dimension reduction by principal component analysis, p represents the total number of longitudinal features after dimension reduction and screening, where p≤P; The step S2 comprises the following steps: S21, normalize all numerical features in the lateral features of the defect feature set to obtain normalized numerical features, and combine all normalized numerical features to form a defect numerical feature vector, which is specifically expressed as follows: In the formula, The normalized value of the input feature of the i-th dimension representing the numerical lateral feature of the n-th dimension; The original value of the input feature of the i-th dimension represented as the numerical lateral feature of the n-th dimension; x ′ n Represents the numerical lateral feature of the nth dimension; S22. Constructing a B-spline-based spatial autoregressive model according to the defect numerical feature vector and the longitudinal feature after dimensionality reduction, and using the expectation-maximization algorithm to perform parameter estimation and missing value filling on the B-spline-based spatial autoregressive model to generate a defect feature vector; The B-spline-based spatial autoregressive model is specifically expressed as follows: y=A -1 (p)X * the+A -1 (p)e Where y is the response variable vector corresponding to all defect numerical feature vectors, which contains the observed response variable y s and missing response variable y u ; A -1 (ρ)e is the error term, let e * =A -1 (ρ), then e * Satisfy N(0,σ 2 M -1 (ρ)) multivariate normal distribution; σ 2 is the variance coefficient of the error; M(ρ) is the covariance matrix, M(ρ)=A T (ρ)A(ρ),M -1 (ρ) represents the inverse matrix of covariance; A(ρ) is the spatial lag coefficient matrix, which is specifically expressed as follows: A(ρ)=I-ρW Where I is the unit matrix; W is the spatial weight matrix; ρ is the spatial autoregressive parameter, |ρ|<1; X * is the design matrix, which is specifically expressed as follows: X * =(X,S) Where X represents the defect numerical feature vector of the current dimension, and S represents the matrix composed of B-spline basis functions based on longitudinal characteristics, which is used to approximate a smooth unknown function α(T ′ process ), the specific form is S=(B1(Z),…,B q (Z) T , B i (Z) represents the i-th matrix composed of B spline functions; Z represents the longitudinal feature T ′ process The specific sampling point set of , the total number of sampling points is point; then q = g + point, g represents the order of the B-spline basis function; η represents the regression coefficient vector, which is expressed as η = (β T ,γ T ) T , β is the regression coefficient of the lateral feature; γ is the regression coefficient corresponding to the B-spline basis function, γ=(γ1,…,γ q ) T .

5. The gradient optimization method based on distributed computing according to claim 4, characterized in that: The specific steps of using the expectation-maximization algorithm to estimate the parameters of the B-spline-based spatial autoregressive model include: S221. According to the observed response variables and the missing response variables, the B-spline-based spatial autoregressive model is divided into blocks and jointly distributed to obtain the mean vector and covariance matrix of the blocks; wherein the specific expression of the mean vector μ is as follows: In the formula, μ s is the mean vector of the observed response variable; μ u is the mean vector of the missing response variable; the specific expression of the covariance matrix M(ρ) is as follows: Where M ss (ρ) is the covariance matrix of the observed response variable; M uu (ρ) is the covariance matrix of the missing response variable; M su (ρ) and M us (ρ) is the covariance matrix between observed response variables and missing response variables; S222, estimate the missing data according to the mean vector and covariance matrix of the blocks to obtain the conditional expected value of the missing response variable; where, given the observed response variable y s The conditional expectation of the missing response variable for the i-th iteration of The specific calculation is as follows: In the formula, (y s -μ s ) represents the observed variable y s Deviation; S223. According to the conditional expected value, the log-likelihood expression objective function is maximized to obtain the iterative value of the current parameter; wherein the log-likelihood expression objective function is specifically expressed as follows: In the formula, Q1(ρ,θ (i) ) is the approximate log-likelihood expression objective function, θ (i) Represents the input parameters of the i-th iteration, which are: θ (i) =(η T ,ρ,ω) T , where ω=σ 2 ; represents the weighted sum of squares of the residuals between the expected conditional value and the model mean vector, is the residual, that is represents the degree of correction of the covariance matrix of the missing response variable, tr(.) is the trace calculation of the matrix; log|M(ρ)| represents the penalty term for the data correlation of the observed response variable; The specific calculation process of the maximization is expressed as follows: r (i+1) =arg max Q1(ρ,θ (i) ) i (i+1) =(the (i+1)T ,r (i+1) ,oh (i+1) ) T In the formula, and They are the partial derivatives of the regression coefficient vector and the partial derivatives of the error term variance, respectively, and their specific expressions are as follows: Among them, θ (i+1) It is the iteration value of the current parameter; S224, determine whether the iterative value of the current parameter converges: if so, substitute the current parameter into the B-spline-based spatial autoregressive model to complete the supplement of the missing value and generate the defect feature vector If not, recalculate the conditional expected value, continue iterative calculation, and execute step S222.

6. The gradient optimization method based on distributed computing according to claim 5, characterized in that: The streaming distribution based on hash space calculates the group identifier corresponding to any feature dimension i in a number of defect feature vectors, and distributes the corresponding defect feature vector to the corresponding computing node according to the group identifier to obtain the grouped defect feature data set. For the defect feature data set of the kth group, it is specifically expressed as: In the formula, Represents the defect feature vector of the i-th dimension divided into the k-th group and the corresponding target value y k,i ; Represents a group identifier, which is used to convert the corresponding defect feature vector into Assigned to the corresponding computing nodes, the specific calculation is as follows: In the formula, Represents the current defect feature vector A unique identifier for a ; mod indicates a modulo operation.

7. A gradient optimization device based on distributed computing, characterized in that: It includes a feature selection unit, a data compensation unit, a workflow division unit, a local gradient calculation unit, a global gradient calculation unit, an approximate loss estimation unit, a parameter iteration calculation unit, an iteration judgment unit and a parameter update unit; The feature selection unit is used to perform feature selection and statistics on the current target defect data to obtain a defect feature set; The data compensation unit is used to preprocess the defect feature set and smoothly compensate the missing features to generate a defect feature vector; The workflow division unit is used to perform workflow division on a plurality of defect feature vectors according to a streaming distribution based on a hash space to obtain a grouped defect feature data set; input the grouped defect feature data set into a parameterized model for regression prediction, and the parameterized model outputs a target value matching the grouped defect feature data set; The local gradient calculation unit is used to perform local gradient calculation according to the target value output by the current parameterized model to obtain local gradients of several parameterized models; The global gradient calculation unit is used to perform global gradient calculation according to the local gradients of the plurality of parameterized models to obtain the global gradient of the parameterized model; The approximate loss estimation unit is used to perform approximate loss estimation on the parameterized model according to the global gradient of the parameterized model to obtain the approximate global loss of the current parameterized model; The parameter iteration calculation unit is used to perform parameter iteration calculation according to the approximate global loss of the current parameterized model to obtain updated parameters of the current parameterized model; The iteration judgment unit is used to judge whether the update parameters of the current parameterized model meet an iteration condition: if not, the parameter update unit is called; if so, the current parameterized model is used as the final parameterized model to complete the gradient optimization of the parameterized model; The parameter updating unit is used to iterate the current parameterized model according to the updated parameters of the current parameterized model and call the local gradient calculation unit.

8. The gradient optimization device based on distributed computing according to claim 7, characterized in that: The specific calculation of the approximate loss estimate is as follows: In the formula, is the approximate global loss of the current parameterized model, m represents the main computing node of the aggregation, where m∈[0,K], K represents the total number of computing nodes; L m (U,V) represents the local loss of the mth computing node; <·,·> is the inner product operation; U (t) and V (t) They represent the kernel weight parameter U and the regularization weight parameter V of the tth iteration respectively; Represents the global gradient of the kernel weight parameter U. The specific calculation is as follows: In the formula, is the local gradient of the kernel weight parameter U of the kth computing node. The specific calculation is as follows: In the formula, Kernel k is the kernel function; k A vector representing the target value of the kth group of defect feature data sets; Represents the global gradient of the regularized weight parameter. The specific calculation is as follows: In the formula, is the local gradient of the regularized weight parameter V of the kth computing node. The specific calculation is as follows: Where ∈ represents the regularized penalty coefficient.

9. A parameterized model for gradient optimization based on distributed computing, characterized in that: The parameterized model is used to perform the following steps: Extract features of the defect image to be detected to obtain a feature vector of the defect to be detected; Linear regression or nonlinear regression is used to model and calculate the feature vector of the defect to be detected to obtain the target value; wherein, when the linear regression is used for modeling, it is specifically expressed as: y=WX+b In the formula, y represents the target value, X represents the defect feature vector, W represents the regression weight, and b represents the bias term; When the nonlinear regression is used for modeling, it is specifically expressed as: y=f(X,θ)+b In the formula, f(X,θ) represents a nonlinear function; θ represents the input parameter of the nonlinear function; b represents the bias term; The target value is converted according to the preset rules to obtain the defect information in the current defect to be detected image; Wherein, the parameterized model optimizes the parameters of the parameterized model by using the gradient optimization method based on distributed computing described in any one of claims 1 to 6.

10. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the parameterized model of gradient optimization based on distributed computing as described in claim 9 when executing the computer program.

Images