An online prediction method for assembly errors of electromechanical products

Abstract

The present invention proposes an online prediction method for assembly error of electromechanical products based on autoencoder and Boosting-OSKELM algorithm. First, for a certain assembly process, the corresponding sample data set is constructed by combining its assembly process and assembly error historical assembly data; secondly, the input process data of each sample point is reduced in dimension based on the autoencoder with Fine-Tuning technique, and the reduced-dimensional process data and the original assembly error data form new reduced-order representation data; then the reduced-dimensional process data is input into the Boosting-KELM model, and the boosting order of the kernel function and the corresponding hyperparameters are determined in combination with the prediction performance on the test set; finally, the Boosting-OSKELM model is formed according to the incremental learning recursive formula of the online sequential kernel extreme learning machine, thereby realizing the online prediction of assembly error. This method can perform online prediction of assembly error of electromechanical products.

Description

Technical Field

[0001] The invention belongs to the field of electromechanical product assembly quality, and in particular is an online prediction method for electromechanical product assembly error based on an autoencoder and a Boosting-OSKELM algorithm. Background Art

[0002] Intelligent manufacturing is an important direction for the manufacture of electromechanical products, and is of great significance to the development of industries such as machinery, information, and electronics. Electromechanical products are of various types and functions, and have been widely used in many fields such as civil life, equipment production, ocean exploration, aerospace, etc., and play an important role in ensuring people's livelihood and national strategic level.

[0003] Electromechanical products, especially high-precision electromechanical products, often have a very complex assembly process. Most of the assembly activities are performed by assembly operators in a specific environment, either manually or by operating machines and equipment, in accordance with the prescribed assembly process. Due to the uncertainty of assembly, the assembly quality of electromechanical products will be affected by many different factors, making it difficult to reasonably and accurately predict assembly errors, which is not conducive to the development of assembly activities and product quality control of enterprises, and increases the production burden of enterprises.

[0004] In the assembly of electromechanical products, a large amount of industrial data is generated, such as assembly sequence, assembly force, solder joint stress, etc. These data implicitly affect product assembly errors from multiple levels and angles. However, due to the limitations of data analysis capabilities and assembly data characteristics, companies have not been able to make good use of this part of the data to establish a mapping relationship between process parameters and assembly errors. Therefore, how to achieve scientific prediction of assembly errors based on this part of industrial data is of great significance for improving assembly processes, improving enterprise assembly efficiency, and reducing enterprise assembly costs. Summary of the invention

[0005] The purpose of the present invention is to provide an online prediction method for assembly errors of electromechanical products based on autoencoders and Boosting-OSKELM algorithms, which is conducive to promoting quality control and assembly process improvement of electromechanical products and promoting the implementation of intelligent manufacturing in enterprises.

[0006] The technical solution to achieve the purpose of the present invention is:

[0007] An online prediction method for assembly error of electromechanical products based on autoencoder and Boosting-OSKELM algorithm includes the following steps:

[0008] Step 1, construct a training sample data set: determine the assembly process parameters and assembly errors of a certain process in the electromechanical product assembly line; according to the historical assembly process data, form a sample set, the input of each sample point is the assembly process parameter of the process, and the output is the corresponding assembly error; finally, perform dimensionless processing on the input and output of the sample set; finally, divide the sample data into a training set and a test set;

[0009] Step 2, perform reduced-order representation of sample data based on the autoencoder with Fine-Tuning technique: build an autoencoder neural network structure according to the number of input dimensions of the sample set; the neural network structure is divided into an encoder and a decoder part, the encoder is composed of a stack of network layers with a gradually decreasing number of neurons, and the decoder is composed of a stack of network layers with a gradually increasing number of neurons; then, a multi-layer perceptron is spliced ​​in the output layer of the encoder, and its output result is the assembly error to be predicted; then, combined with the corrected overall network loss function, the gradient descent method is used to update the overall network connection weights and bias; after the training is completed, the autoencoder at the front of the overall network is stripped off, and the loss function is reconstructed in combination with the autoencoder, and retrained; then, the encoder part of the front of the autoencoder is stripped off, and the process input data in the original sample data is input into the structure, and the output result is the dimensionality reduction result of the original sample input data; finally, the dimensionality reduction result and the original sample output data are combined into a new training data set, thereby realizing the reduced-order representation of the sample data;

[0010] Step 3, online prediction of assembly error based on Boosting-OSKELM: determine the kernel functions that can be selected and the hyperparameter values ​​that can be selected in the kernel function; use the Boosting strategy to determine the boosting order of different kernels; under the given hyperparameters and boosting order, use the kernel extreme learning machine based on the first kernel to make predictions on the reduced-order training set, and replace the output in the sample set with the residual of the prediction result; similarly, use the kernel extreme learning machine based on the second kernel to make predictions on the sample set, and replace the residual of the prediction result with the residual of the previous time, and so on, until all kernel extreme learning machines have completed the prediction; then, accumulate the prediction results of different kernel extreme learning machines to obtain the prediction performance of Boosting-KELM; by comparing the prediction performance, screen out the Boosting-KELM model with the highest prediction accuracy and its corresponding kernel function hyperparameters and boosting order; finally, based on the optimal hyperparameters and the optimal boosting order, according to the incremental learning recursive formula of the online sequential extreme learning machine, realize the online prediction of assembly error. Compared with the prior art, the present invention has the following significant advantages:

[0011] (1) In the assembly of complex electromechanical products, an assembly process may involve a large number of assembly processes, so the sample data points are in a higher-dimensional space, which makes the assembly error prediction susceptible to the influence of multiple different-dimensional data. The reduced-order representation proposed in the present invention can reduce the distance between samples in the high-dimensional space and retain the valuable information of the samples as much as possible, which is beneficial to improving the inference speed of the prediction algorithm and the prediction accuracy of the prediction algorithm.

[0012] (2) For some complex electromechanical products, enterprises' production is mostly in small batch mode, which makes the amount of sample data available for machine learning insufficient. In engineering, the Kriging algorithm is often used to solve the small sample problem. Although this method does not rely too much on the amount of data, it relies more on the distribution of data. If the data set is poorly distributed, Kriging may be difficult to converge. In addition, statistical learning methods such as k-nearest neighbor algorithm and support vector regression can also solve the small sample problem, but these algorithms often adopt offline learning methods, and there is no good online learning method for new sample data generated during the assembly process. The prediction method based on the Boosting-OSKELM algorithm proposed in the present invention not only absorbs the advantages of the extreme learning machine: extremely fast learning speed and global optimality of network weights, but also introduces incremental learning, which can better adapt to the small sample environment, and can also deal with the "data poison" problem in traditional online learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a flow chart of an online prediction method for assembly error based on autoencoder and Boosting-OSKELM algorithm.

[0014] Figure 2 The training process of the autoencoder with Fine-Tuning technique.

[0015] Figure 3 This is the principle of Boosting. DETAILED DESCRIPTION

[0016] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0017] Attached Figure 1 The present invention is a flow chart of an assembly error online prediction method based on an autoencoder and a Boosting-OSKELM algorithm. The present invention comprises the following steps:

[0018] Step 1: Build a training sample dataset:

[0019] According to the historical data of a certain process, the corresponding data set is constructed, and 80% of the data set is divided into the training set D, and the remaining 20% ​​is divided into the test set E. The two only differ in the amount of sample data, while they are consistent in feature structure and output structure.

[0020] Remember the training set X is the process input, Y is the error output, and the training set matrix is ​​as follows:

[0021]

[0022] In the training set matrix, each row is a sample point, N is the number of samples, M1 and M2 are the number of dimensions of process parameters and errors respectively; for the i-th sample point, its input and output are recorded as x i =[x i1 ,x i2 ,x i3 ,...] and y i =[y i1 ,y i2 ,y i3 ,...]; According to the normalized data preprocessing method, each column of the training set matrix D is scaled to the range of [0,1]. The normalization formula is as follows:

[0023]

[0024] Where, d (j) and d′ (j) are the original training set matrix and the j-th column after preprocessing, and are the minimum and maximum values ​​of the j-th column of the training set matrix, respectively.

[0025] Similarly, the test set E also needs to be normalized, and the normalization formula is as follows:

[0026]

[0027] In the formula, e (j) and e′ (j) They are the original test set matrix and the jth column after preprocessing. Finally, the preprocessed training set matrix and test set matrix are recorded as D′ and E′ respectively.

[0028] Step 2: Reduce the order of sample data representation based on the autoencoder with Fine-Tuning technique:

[0029] Reduced-order representation is a feature extraction method that can effectively remove the redundancy and complex relationships between features, which is conducive to the implementation of the next prediction algorithm.

[0030] Attached Figure 2 The training process of the autoencoder with Fine-Tuning technique.

[0031] (1) Build the prediction network structure: The number of neurons in the input layer and output layer of the autoencoder is consistent with the input dimension M1 in the training set. The encoder and decoder parts are symmetrical. The neurons in the middle hidden layer follow the principle of gradually decreasing first and then gradually increasing. The activation function selects the Tanh function, which is as follows:

[0032]

[0033] The number of neurons in the input layer of the multilayer perceptron is M1, and the number of output layers is consistent with the number of output dimensions M2 in the training set. The number of neurons in the intermediate hidden layer follows the principle of increasing first and then decreasing. The activation function selects the ReLU function, which is as follows:

[0034]

[0035] After determining the network structure of the autoencoder and the multi-layer perceptron, the input layer of the multi-layer perceptron is spliced ​​to the output layer of the autoencoder to form the overall network.

[0036] (2) Initialization: Randomly initialize the overall network connection weight W and bias b, set the optimizer learning rate η and the total number of termination condition iterations Epoch.

[0037] (3) Gradient descent method to update network connection weights: The loss function of the overall network structure is modified according to the following formula:

[0038] L=α1L1+α2L2

[0039] In the formula, α1 and α2 are balance coefficients, L1 is the reconstruction loss of the autoencoder, and L2 is the prediction loss of the entire network. The calculation formulas for the two are as follows:

[0040]

[0041]

[0042] Where f(·) and g(·) represent the encoder and decoder respectively, W (1) , b (1) With W (2) , b (2) are the weights and biases corresponding to the encoder and decoder respectively, and H(·) represents the overall network.

[0043] According to the network parameters initialized in (2), the normalized X′ is input into the overall network structure to obtain the corresponding loss L(X′). Then, the gradient descent method is used to update the network connection weights and biases. The update iteration formula is as follows:

[0044]

[0045] According to the above formula, the network parameters are updated within the specified total number of termination condition iterations Epoch, and the change of loss L on the training set and test set with iteration is plotted respectively. The Early-Stopping strategy is used to select the optimal network parameters.

[0046] (4) Fine-tune the connection weights of the autoencoder network: After the overall network training is completed, the front autoencoder is stripped off and its internal network weights are retained; based on the reconstruction loss function, the gradient descent method is used to continue training its internal parameters; similarly, it is also necessary to plot the change of the loss L1 on the training set and the test set with iterations, and use the Early-Stopping strategy to select the optimal internal parameters of the autoencoder network.

[0047] (5) Remove the encoder part from the autoencoder and retain the internal parameters of the autoencoder network determined in (3); the input of the encoder is X′ in the dataset D′, and its output is is the dimension reduction result of the input features; Together with the normalized training set output Y′, a new training set is formed That is, the reduced-order representation of the training data set is completed.

[0048] Step 3: Online prediction of assembly error based on Boosting-OSKELM:

[0049] (1) Determine the kernel function type: The kernel function is a commonly used method for solving nonlinear problems. It maps the data in the original feature space to a new high-dimensional feature space. Learning is performed implicitly in the new feature space, and there is no need to explicitly define the feature space kernel mapping function. The kernel matrix Ω is defined according to the Mercer condition, as shown in the following formula:

[0050]

[0051] In the formula, K(x i ,x j ) represents the kernel function relationship between the i-th sample and the j-th sample in the training set, and its output is a scalar. Common kernel functions are:

[0052] 1) Polynomial kernel function:

[0053] K(x i ,x j)=(a·x i ·x j +b) p

[0054] 2) Gaussian kernel function:

[0055]

[0056] 3) Linear kernel function (linear kernel means no kernel):

[0057] K(x i ,x j )=x i ·x j

[0058] In the above kernel functions, a, b, p, and σ are all constants.

[0059] In some processes with obvious linear relationships, linear kernels can be given priority; for processes without obvious linear relationships, nonlinear kernels such as Gaussian kernels are given priority.

[0060] (2) Determine the kernel function hyperparameters: The principle of boosting is as follows Figure 3 As shown in Figure 1, various boosting orders are enumerated based on the kernel function determined in (1), such as the boosting order of Gaussian kernel-linear kernel-polynomial kernel. The boosting order does not necessarily require different kernel functions. The same kernel can also be boosted multiple times, such as Gaussian kernel-Gaussian kernel-Gaussian kernel. Based on experience, optional kernel function hyperparameter values ​​are given. Common hyperparameter values ​​are: a = 1, 2, 3; b = 1, 2, 3; p = 1, 2, 3; σ = 10, 50, 100. Subsequently, a grid search is performed on each boosting order, using different hyperparameter combinations to learn on the training set and test on the test set, so as to select the boosting order and hyperparameter combination corresponding to the optimal test performance.

[0061] (3) Online Sequential Kernel Extreme Learning Machine Incremental Learning: Given a given kernel function and hyperparameters, assume that the sample set {(x1, t1), ..., (x t ,t t )}, for the new sample x generated at time t+1 t+1 , can be predicted according to the following formula:

[0062]

[0063] In the formula, I represents the unit matrix, C takes a larger number, usually 1000, and T t A column in the output of the dataset.

[0064] remember:

[0065] k t (x) = [K(x) t+1 ,x1),K(x t+1 ,x2),...,K(x t+1 ,x N )]

[0066]

[0067] but Therefore, A t Iteration can be performed according to the following formula:

[0068]

[0069] In the formula, v t =C -1 +K(x t+1 ,x t+1 ).

[0070] According to the block matrix inversion formula, we get matrix A t+1 The inverse matrix of is:

[0071]

[0072] In the formula When the assembly error corresponding to the t+1th sample is measured, the θ corresponding to the t+1th time can be obtained according to the following formula: t+1 , thereby completing the incremental learning of the online sequential kernel extreme learning machine.

[0073]

[0074] Similarly, the assembly error prediction result at time t+2 is:

[0075]

[0076] The above process is only for the assembly error prediction result of a single core. According to the optimal boosting order and optimal hyperparameters in (2), the online sequential kernel extreme learning machine corresponding to different cores is used to predict one by one, and the prediction residual results are added together to obtain the final assembly error prediction result.

Claims

1. An online prediction method for assembly error of electromechanical products based on autoencoder and Boosting-OSKELM algorithm, characterized in that: The following steps are involved: Step 1: Construct a training sample data set: determine the assembly process parameters and assembly errors of a certain process in the electromechanical product assembly line; construct a sample set based on historical assembly process data, where the input of each sample point is the assembly process parameter of the process, and the output is the corresponding assembly error; then, perform dimensionless processing on the input and output of the sample set; finally, divide the sample data into a training set and a test set; Step 2: Perform reduced-order representation of sample data based on an autoencoder with Fine-Tuning techniques: Build an autoencoder neural network structure based on the number of input dimensions of the sample set; the neural network structure is divided into an encoder and a decoder. The encoder is composed of a stack of network layers with a gradually decreasing number of neurons, while the decoder is composed of a stack of network layers with a gradually increasing number of neurons; then, a multilayer perceptron is spliced ​​on the output layer of the encoder, and its output result is the assembly error to be predicted; Then, combined with the modified overall network loss function, the gradient descent method is used to update the overall network connection weights and biases; After the training is completed, the autoencoder at the front of the whole network is stripped off, the loss function is reconstructed with the autoencoder, and retrained; Then, the encoder part of the front of the autoencoder is stripped off, and the process input data in the original sample data is input into the structure, and the output result is the dimensionality reduction result of the original sample input data; finally, the dimensionality reduction result and the original sample output data are combined into a new training data set, thereby realizing the reduced-order representation of the sample data; Step 3, online prediction of assembly error based on Boosting-OSKELM: determine the available kernel functions and the hyperparameter values ​​that can be selected in the kernel function; use the Boosting strategy to determine the boosting order of different kernels; under the given hyperparameters and boosting order, use the kernel extreme learning machine based on the first kernel to make predictions on the reduced-order training set, and replace the output in the sample set with the residual of the prediction result; similarly, use the kernel extreme learning machine based on the second kernel to make predictions on the sample set, and replace the residual of the prediction result with the residual of the previous time, and so on, until all kernel extreme learning machines have completed the prediction; then, accumulate the prediction results of different kernel extreme learning machines to obtain the prediction performance of Boosting-KELM; by comparing the prediction performance, select the Boosting-KELM model with the highest prediction accuracy and its corresponding kernel function hyperparameters and boosting order; finally, based on the optimal hyperparameters and the optimal boosting order, the incremental learning recursive formula of the online sequential extreme learning machine is used to realize the online prediction of assembly error.

2. The online prediction method for assembly error of electromechanical products based on autoencoder and Boosting-OSKELM algorithm according to claim 1 is characterized in that: Step 1: Construct a training sample data set, which includes the following steps: According to the historical data of a process, the corresponding data set is constructed, and 80% of the data set is divided into the training set D, and the remaining 20% ​​is divided into the test set E. The two sets differ only in the amount of sample data, but remain consistent in feature structure and output structure; Remember the training set X is the process input, Y is the error output, and the training set matrix is ​​as follows: In the training set matrix, each row is a sample point, N is the number of samples, M1 and M2 are the number of dimensions of process parameters and errors respectively; for the i-th sample point, its process input and assembly error output are recorded as x i =[x i1 ,x i2 ,x i3 ,...] and y i =[y i1 ,y i2 ,y i3 ,...], D is simplified as According to the normalized data preprocessing method, each column of the training set matrix D is scaled to the range of [0,1]. The normalization formula is as follows: Where, d (j) and d′ (j) are the original training set matrix and the j-th column after preprocessing, and are the minimum and maximum values ​​of the jth column of the training set matrix respectively; Similarly, the test set E also needs to be normalized, and the normalization formula is as follows: In the formula, e (j) and e′ (j) are the original test set matrix and the jth column after preprocessing; finally, the preprocessed training set matrix and test set matrix are recorded as and Where N1 and N2 represent the number of samples in the training set and test set respectively, and N1+N2=N.

3. The online prediction method for assembly error of electromechanical products based on autoencoder and Boosting-OSKELM algorithm according to claim 1 is characterized in that: Step 2: Perform a reduced-order representation of sample data based on the autoencoder, specifically including the following steps: (1) The prediction network structure is built based on the Fine-Tuning technique: the number of neurons in the input layer and output layer of the autoencoder is consistent with the number of input dimensions M1 in the training set. The encoder part and the decoder part are symmetrical in structure. The neurons in the middle hidden layer follow the principle of gradually decreasing first and then gradually increasing. The activation function selects the Tanh function. The Tanh function is as follows, where e is the base of the natural logarithm: The number of neurons in the input layer of the multilayer perceptron is M1, and the number of output layers is consistent with the number of output dimensions M2 in the training set. The number of neurons in the intermediate hidden layer follows the principle of increasing first and then decreasing. The activation function selects the ReLU function, which is as follows: After determining the network structure of the autoencoder and the multi-layer perceptron, the input layer of the multi-layer perceptron is spliced ​​to the output layer of the autoencoder to form the overall network; (2) Initialization: Randomly initialize the overall network connection weight W and bias b, set the optimizer learning rate η and the total number of termination condition iterations Epoch; (3) Gradient descent method to update network connection weights: The loss function of the overall network structure is modified according to the following formula: L=α1L1+α2L2 In the formula, α1 and α2 are balance coefficients, L1 is the reconstruction loss of the autoencoder, and L2 is the prediction loss of the entire network. The calculation formulas for the two are as follows: z i =f(W (1) x i +b (1) ),x′ i =g(W (2) z i + b(2) ) Where N is the number of samples, f(·) and g(·) represent the encoder and decoder respectively, and W (1) , b (1) With W (2) , b (2) are the weights and biases corresponding to the encoder and decoder respectively, x i With y i Represent the input and output of the i-th sample, z i and x′ i They represent the encoding result of the i-th sample and the result returned by the autoencoder, respectively, and H(·) represents the overall network; According to the network parameters initialized in (2), the normalized X′ is input into the overall network structure to obtain the corresponding loss L(X′). Then, the gradient descent method is used to update the network connection weights and biases. The update iteration formula is as follows: In the formula, η represents the learning rate, and They represent the partial derivatives of the loss function with respect to the network weight W and the network bias b respectively. According to the above formula, the network parameters are updated within the specified total number of termination condition iterations Epoch, and the changes of the loss L on the training set and the test set with iterations are plotted respectively. The Early-Stopping strategy is used to select the optimal network parameters. (4) Fine-tune the autoencoder network connection weights: After the overall network training is completed, the front autoencoder is stripped off and its internal network weights are retained; according to the reconstruction loss function, the gradient descent method is used to continue training its internal parameters; similarly, it is also necessary to plot the change of loss L1 on the training set and the test set with iterations, and use the Early-Stopping strategy to select the best autoencoder network internal parameters; (5) Remove the encoder part from the autoencoder and retain the internal parameters of the autoencoder network determined in (3); the input of the encoder is X′ in the dataset D′, and its output is is the dimension reduction result of the input features; Together with the normalized training set output Y′, a new training set is formed Similarly, we get the test set after the reduced-order representation At this point, the reduced-order representation of the entire data set is completed.

Images