Industrial abnormal data detection method based on weighted multilayer width learning system

Abstract

The invention discloses an industrial abnormal data detection method based on a weighted multilayer width learning system in the technical field of industrial data detection, which comprises the following three stages: firstly, performing n times of feature mapping on input data X to obtain mapping features; secondly, inputting the mapping features into an enhanced node layer, and performing random transformation activation to obtain enhanced features; and finally, the mapping feature nodes and the enhanced feature nodes are all connected to output, a model output weight W is calculated by using a least square method, and a weighted loss function is introduced. According to the method in the scheme, the feature learning ability is improved, and the negative influence of uncertain data such as training sample abnormal values and noise values on the training result is relieved.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial data detection, and in particular to an industrial abnormal data detection method based on a weighted multi-layer width learning system. Background Art

[0002] In recent years, deep learning models such as Convolutional Neural Networks (CNNs) have been widely used in the industrial field, improving the classification and prediction accuracy of industrial process data analysis. However, traditional deep learning models use gradient-based optimization algorithms to update network model parameters, which have defects such as high computational complexity, heavy computational burden, and long training time. They are not suitable for the construction and operation of industrial systems, affecting the real-time performance of industrial data analysis.

[0003] The Broad Learning System (BLS) was proposed by Professor Chen Junlong in 2018. It is a neural network model based on a randomized learning method. Compared with the back-propagation method used by deep models such as CNNs to update model parameters, BLS based on a randomized learning method can greatly improve the training efficiency of image data analysis while ensuring learning accuracy, and has been favored by domestic and foreign research and learning. Subsequently, Chen Junlong et al. proved the universal approximation ability of BLS and proposed a variety of model variants, providing effective references for domestic and foreign scholars. To improve the interpretability of BLS, Shuang et al. combined the TS (Takagi-Sugeno) fuzzy model with BLS and proposed a fuzzy broad learning system (Fuzzy BLS), which further improved the learning ability of the BLS model. For time series data analysis, Xu et al. proposed a recurrent broad learning system (Recurrent BLS), which improved the prediction accuracy of time series data.

[0004] However, in the industrial production process, the data collected by sensors are affected by the complex industrial environment, which is prone to abnormal values ​​and noise values, affecting the modeling accuracy of the BLS model. In order to process chaotic time series data with noise, Feng et al. proposed a robust manifold broad learning system (RMBLS) and analyzed its effectiveness. Chu et al. proposed a weighted broad learning system framework (WBLS) to process industrial noise data by introducing different weighting functions. Subsequently, Zheng et al. designed a broad learning system based on the maximum correlation entropy criterion (BLS-MCC) and used the maximum correlation entropy criterion to calculate the weights of training samples. In addition, Liu et al. used the Cauchy loss function to process noisy data. However, although the above-mentioned robust BLS model based on the weighted loss function can effectively overcome the influence of noise data on the model, compared with the multi-layer network architecture, its feature learning ability is still seriously insufficient, which affects the accuracy of large-scale industrial data analysis.

[0005] Therefore, in order to predict the angular acceleration of the robot arm link, this paper proposes a weighted cascade broad learning system (WCBLS). Compared with WBLS, WCBLS introduces a multi-layer network structure to achieve multi-layer feature mapping and feature enhancement, in which the feature nodes and enhancement nodes of each layer are used as the input of the next layer, and the feature nodes and enhancement nodes of the last layer are fully connected to the output; and the M-estimation function based on the interquartile range (IQR) is used to assign different penalty weights to the training samples to calculate the WCBLS model weights to reduce the negative impact of noise data on the model prediction results. Finally, under 10%, 20% and 30% noise ratio data, WCBLS can effectively overcome the impact of outliers on regression results compared with BLS. Summary of the invention

[0006] The present invention aims to provide an industrial abnormal data detection method based on a weighted multi-layer width learning system to solve the problem that the feature learning ability of the existing technology is still seriously insufficient compared with the multi-layer network architecture, which affects the accuracy of large-scale industrial data analysis.

[0007] The industrial abnormal data detection method based on the weighted multi-layer width learning system in this scheme includes three stages. First, the input data X is feature mapped n times to obtain the mapping features; second, the mapping features are input to the enhancement node layer and randomly transformed and activated to obtain the enhanced features; finally, the mapping feature nodes and the enhancement feature nodes are fully connected to the output, and the model output weight W is calculated using the least squares method.

[0008] Specifically, the following steps are included: Assume that the input data is represented by X = {x1, x2, ..., x N}, the data label is represented by Y = {y1,y2,…,y N}, N represents the number of data samples;

[0009] Step 1: Perform n feature mappings on the input data X according to formula (1). Each feature mapping contains p nodes, where represents the activation function, W ei and β ei represents random weights and biases, and the feature combination is performed through formula (2) to obtain the mapping feature node Zn,

[0010]

[0011] Z n =[Z1,Z2,...,Z n ] (2)

[0012] Step 2: The mapping feature node Zn is enhanced m times by using equations (3) and (4), each feature enhancement includes q nodes, and feature combination is performed, where φ represents the activation function, W hj and represent random weights and biases,

[0013] H j =φ(Z n W hj +β hj ),j=1,2,...,m (3)

[0014] H m =[H1,H2,…,H m ] (4)

[0015] Step 3: Merge the mapping feature nodes and the enhanced feature nodes into formula (5), and calculate the BLS output weight W using the least squares method according to formula (6), where C represents the regularization parameter,

[0016] A=[Z n |H m ] (5)

[0017] β=(A T A+CI)-1 A T Y (6)

[0018] Step 4: Calculate the model output Y according to formula (7),

[0019] Y=Aβ (7),

[0020] Introduce the weighted loss function, see formula (8),

[0021] β=(A T θ 2 A+CI) -1 A T θ 2 Y (8)

[0022] Among them, θ=diag{θ1, θ2,…,θ N} represents the penalty weight of sample data.

[0023] The working principle of this scheme and its beneficial effects: Although BLS can perform feature mapping and feature enhancement based on training data, when there are many outliers and noise in the data, the prediction accuracy of the model will be reduced. Therefore, it is necessary to introduce a weighted loss function to assign different penalty thresholds to the training samples and correct the model output weights.

[0024] Further, the WCBLS algorithm process is as follows:

[0025] 1) Feature mapping stage:

[0026] WCBLS uses the same feature mapping process as BLS. It maps the input feature X using formula (1) and combines the mapping results using formula (2).

[0027] 2) Feature enhancement stage:

[0028] WCBLS uses m enhancement node layers to enhance the mapping feature Fn layer by layer through equations (8) and (9), where m represents the number of feature enhancement layers and q represents the dimension of each feature enhancement. In order to comprehensively learn the bearing fault features, WCBLS combines the enhanced features at all levels through equation (10).

[0029] H k =φ(F n W hk +β hk ),k=1 (8)

[0030] H k =φ(H k-1 W hk +β hk ),k=2,3,…,m (9)

[0031] H m =[H1,H2,…,H m ] (10)

[0032] Among them, φ represents the activation function, W hk and b hk The points represent random weights and biases.

[0033] 3) Output weight calculation:

[0034] The mapping features of formula (2) and the enhanced features of each layer of formula (10) are combined using formula (11), and the output weight β of the BLS model is obtained according to the ridge regression calculation method of formula (6).

[0035] A=[F n |H m ] (11)

[0036] β=(A T A+CI) -1 A T Y (12)

[0037] Among them, C is the regularization parameter.

[0038] 4) Calculate model output:

[0039] The output result Y of each layer is calculated by formula (13), and the results are fused according to formula (8).

[0040] Y=Aβ(13). As a random neural network model, BLS adopts a feedforward network architecture to reduce the amount of model calculation and improve learning accuracy. Unlike the shallow network structure, the BLS model with a multi-layer structure can extract effective representation information and improve the accuracy of large-scale industrial data modeling. At the same time, the noise and outliers in the training data seriously affect the accuracy and generalization performance of BLS. Compared with the traditional BLS, the deep width learning system adopts multiple enhancement layers, which can cascade the features of each layer. However, after WCBLS adopts a randomized parameter allocation process, it has a higher learning efficiency. In order to avoid model overfitting, it is necessary to effectively select its regularization parameter C to improve the classification accuracy of the model. Unlike the traditional BLS, WCBLS has a multi-layer feature node and enhancement node structure. The feature nodes and enhancement nodes of each layer serve as the input of the next layer, and the feature nodes and enhancement nodes of the last layer are fully connected to the output. In order to reduce the impact of outliers on the generalization performance of the model, the M-estimation function based on the interquartile range (IQR) is used to assign different penalty weights to the training samples and update the WCBLS model weights.

[0041] Furthermore, the corrected model output weight is formula (15):

[0042]

[0043] Among them, θ=diag{θ1, θ2,…,θ N} is calculated by equation (16) and equation (17), which represents the penalty weight of the sample data. represents the standard deviation between error variables, the IQR represents the difference between the 75th percentile and the 25th percentile, and the constants c1 and c2 are usually set to 2.5 and 3.

[0044] Finally, the WCBLS output is shown in formula (18):

[0045] Y=S U β(18). Although WCBLS can perform multiple feature mapping and feature enhancement based on the training data, when there are many outliers and noise in the data, the prediction accuracy of the model will be reduced. Therefore, the M-estimation function based on the Interquartile Range (IQR) is used to assign different penalty thresholds to the training samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is the structure diagram of the width learning system;

[0047] Figure 2 This is the WCBLS network architecture. DETAILED DESCRIPTION

[0048] The following is a further detailed description through specific implementation methods:

[0049] Example

[0050] 1.1 Width Learning System

[0051] BLS was proposed in 2018 and evolved from the Random Vector Functional Link Neural Network (RVFL). It is a neural network model based on a randomized learning method. Compared with deep learning models such as CNN that update model parameters through the back-propagation algorithm, its network parameters use a random allocation strategy and calculate the output weights through the least squares method, which improves the learning efficiency of the model, reduces the computational burden, and avoids the risk of gradient explosion.

[0052] The BLS network structure is as follows Figure 1As shown in the figure, the modeling process is divided into two stages. First, the input data X is feature mapped n times to obtain the mapping features; secondly, the mapping features are input to the enhancement node layer and randomly transformed and activated to obtain the enhanced features; finally, the mapping feature nodes and the enhancement feature nodes are fully connected to the output, and the model output weight W is calculated using the least squares method.

[0053] Assume that the input data is represented by X = {x1, x2, ..., x N}, the data label is represented by Y = {y1,y2,…,y N}, N represents the number of data samples.

[0054] Step 1: Perform n feature mappings on the input data X according to formula (1). Each feature mapping contains p nodes, where represents the activation function, W ei and β ei Represents random weights and biases. And the mapping feature node Zn is obtained by combining features through formula (2).

[0055]

[0056] Z n =[Z1,Z2,...,Z n ] (2)

[0057] Step 2: Use equations (3) and (4) to perform m feature enhancements on the mapping feature node Zn, each feature enhancement includes q nodes, and perform feature combination. Where φ represents the activation function, W hj and represent random weights and biases.

[0058] H j =φ(Z n W hj +β hj ),j=1,2,...,m (3)

[0059] H m =[H1,H2,…,H m ] (4)

[0060] Step 3: Merge the mapping feature nodes and the enhanced feature nodes into formula (5), and calculate the BLS output weight W using the least squares method according to formula (6), where C represents the regularization parameter.

[0061] A=[Z n |H m ] (5)

[0062] β=(A T A+CI) -1 A T Y (6)

[0063] Step 4: Calculate the model output Y according to formula (7).

[0064] Y=Aβ (7)

[0065] 1.2 Weighted Width Learning System Framework

[0066] Although BLS can perform feature mapping and feature enhancement based on training data, when there are many outliers and noise in the data, the prediction accuracy of the model will be reduced. Therefore, it is necessary to introduce a weighted loss function, assign different penalty thresholds to the training samples, and correct the model output weights, as shown in formula (8):

[0067] β=(A T θ 2 A+CI) -1 A T θ 2 Y (8)

[0068] Among them, θ=diag{θ1, θ2,…,θ N} represents the penalty weight of sample data.

[0069] 2 Weighted Multi-layer Width Learning System

[0070] As a random neural network model, BLS uses a feedforward network architecture to reduce model calculations and improve learning accuracy. Unlike shallow network structures, the BLS model with a multi-layer structure can extract effective representation information and improve the accuracy of large-scale industrial data modeling. At the same time, the noise and outliers in the training data seriously affect the accuracy and generalization performance of BLS. Therefore, this section proposes a new WCBLS to solve the problem of industrial abnormal data detection. The WCBLS network architecture is as follows: Figure 1 shown.

[0071] Unlike traditional BLS, WCBLS has a multi-layer feature node and enhancement node structure. The feature nodes and enhancement nodes of each layer serve as the input of the next layer, and the feature nodes and enhancement nodes of the last layer are fully connected to the output. In order to reduce the impact of outliers on the generalization performance of the model, the M-estimation function based on the Interquartile Range (IQR) is used to assign different penalty weights to the training samples and update the WCBLS model weights. The WCBLS network architecture is as follows: Figure 2

[0072] 2.1 Multi-layer Width Learning System Architecture

[0073] WCBLS parameter initialization

[0074] The WCBLS algorithm process is as follows:

[0075] 1) Feature mapping stage:

[0076] WCBLS uses the same feature mapping process as BLS. It maps the input feature X using formula (1) and combines the mapping results using formula (2).

[0077] 2) Feature enhancement stage:

[0078] WCBLS uses m enhancement node layers to enhance the mapping feature Fn layer by layer through equations (8) and (9), where m represents the number of feature enhancement layers and q represents the dimension of each feature enhancement. In order to comprehensively learn the bearing fault features, WCBLS combines the enhanced features at all levels through equation (10).

[0079] H k =φ(F n W hk +β hk ),k=1 (8)

[0080] H k =φ(H k-1 W hk +β hk ), k=2,3,…,m (9)

[0081] H m =[H1,H2,...,H m ] (10)

[0082] Among them, φ represents the activation function, W hk and b hk The points represent random weights and biases.

[0083] 3) Output weight calculation:

[0084] The mapping features of formula (2) and the enhanced features of each layer of formula (10) are combined using formula (11), and the output weight β of the BLS model is obtained according to the ridge regression calculation method of formula (6).

[0085] A=[F n |H m ] (11)

[0086] β=(A T A+CI) -1 A T Y (12)

[0087] Among them, C is the regularization parameter.

[0088] 4) Calculate model output:

[0089] The output result Y of each layer is calculated by formula (13), and the results are fused according to formula (8).

[0090] Y=Aβ (13)

[0091] Compared with traditional BLS, the deep width learning system uses multiple enhancement layers to cascade features of each layer.

[0092] However, after WCBLS adopts a randomized parameter allocation process with high learning efficiency, in order to avoid model overfitting, it is necessary to effectively select its regularization parameter C to improve the classification accuracy of the model.

[0093] 2.2 Multi-layer width learning system weight calculation

[0094] Although WCBLS can perform multiple feature mapping and feature enhancement based on training data, when there are many outliers and noise in the data, the prediction accuracy of the model will be reduced. Therefore, this paper uses the M-estimation function based on the Interquartile Range (IQR) to assign different penalty thresholds to the training samples, and the modified model output weight is formula (15):

[0095]

[0096]

[0097] Among them, θ=diag{θ1, θ2,…,θ N} is calculated by equation (16) and equation (17), which represents the penalty weight of the sample data. represents the standard deviation between error variables, the IQR represents the difference between the 75th percentile and the 25th percentile, and the constants c1 and c2 are usually set to 2.5 and 3.

[0098] Finally, the WCBLS output is shown in formula (18):

[0099] Y=S U β (18)

[0100] 3 Experimental analysis

[0101] 3.1 Experimental Environment

[0102] In order to demonstrate the effectiveness of the WCBLS model, this section designs a simulation experiment based on the Windows10 / MATLAB R2019b experimental platform.

[0103] 3.2 Experimental Dataset

[0104] The Puma32h dataset of the UCI machine learning library was selected as the experimental data. The Puma32h dataset is generated by the dynamic synthesis of the Unimation Puma560 robot arm. Its task is to predict the angular acceleration of the robot arm link. It contains 32-dimensional input features such as the robot arm angular position, velocity, and torque, and 8192 sample data.

[0105] In order to reduce the impact of different variable ranges on the model prediction results, the input data and label values ​​are normalized using formula (19), where a represents the actual value of the input variable, a′ represents the normalized result, and max{a} and min{a} represent the maximum and minimum values ​​of the input variable, respectively.

[0106]

[0107] 3.3 Experimental Results Analysis

[0108] To verify the effectiveness of the WCBLS model, BLS, WBLS-Huber, RVFL and Support Vector Regression (SVR) were selected as comparison algorithms. The number of hidden layer nodes / enhancement layer nodes of the four algorithm models were all 100. The mapping feature nodes of BLS, WBLS-Huber and WCBLS were automatically determined by grid search, and WCBLS adopted a multi-layer network structure. All models were run independently 20 times, and 75% of the sample data were randomly selected as training data each time, and the remaining 25% were used as test data. The noise in the range of [-0.5, 0.5] was increased by 10%, 20% and 30% respectively to the training data labels. The average values ​​of mean absolute error (MAE) (Equation (20)), root mean square error (RMSE) (Equation (21)) and correlation coefficient R2 (Equation (22)) were used as experimental results.

[0109]

[0110] Among them, y i represents the actual label value of sample point i, represents the predicted value of sample point i, and N represents the number of sample points.

[0111] First of all, it should be pointed out that in order to add noise data to the training samples, this paper normalizes the sample label values ​​to [0,1]. The sample labels containing noise data range from [-0.5,1.5], so the statistical results in Tables 1 to 3 are relatively small. Tables 1 to 3 list the average prediction errors of the four models, including WCBLS, BLS, RVFL, and SVR, at 10%, 20%, and 30% noise ratios. The smaller the MAE and RMSE values, the better. R2 represents the correlation between the prediction result and the prediction label. The closer it is to 1, the better the prediction result.

[0112]

[0113] Table 1 Prediction results under 10% noise

[0114]

[0115] Table 2 Prediction results under 20% noise

[0116]

[0117] Table 3 Prediction results under 30% noise

[0118] It can be seen from Tables 1 to 3 that, under 10%, 20% and 30% noise ratios, as the noise data increases, the angular acceleration prediction accuracy of the WCBLS, BLS, RVFL and SVR manipulator link decreases. However, WCBLS, WBLS-Huber and BLS are better than RVFL and SVR models in all three cases, which shows the effectiveness of the BLS model. WCBLS has higher regression performance than WBLS-Huber with a single-layer structure. Secondly, WCBLS can alleviate the impact of noise value on the prediction results and improve the prediction accuracy as the noise increases.

[0119] To sum up, the WCBLS model has better prediction results for abnormal data, which is beneficial to improve the prediction accuracy of industrial abnormal data.

[0120] 4 Conclusion

[0121] In order to detect abnormal data in industrial processes, this paper proposes an industrial abnormal data prediction method based on WCBLS. First, in order to improve the feature learning ability and alleviate the negative impact of uncertain data such as training sample outliers and noise values ​​on the training results, this paper uses a multi-layer feature mapping and feature enhancement structure, and the IQR-based M-estimation function is used to calculate the penalty weights of training samples, giving smaller weights to outliers and larger weights to normal samples; secondly, the obtained training sample penalty weights are used to update the output weights of the WCBLS model; finally, based on the robot arm link angular acceleration prediction data set with 10%, 20% and 30% noise ratios, the effectiveness of the WCBLS model is illustrated.

[0122] The above is only an embodiment of the present invention, and the common knowledge such as the known specific structure and characteristics in the scheme is not described in detail here. It should be pointed out that for those skilled in the art, several deformations and improvements can be made without departing from the structure of the present invention, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. An industrial abnormal data detection method based on a weighted multi-layer width learning system, characterized by: It includes three stages. First, feature mapping is performed n times on the input data X to obtain mapping features. Secondly, the mapping features are input to the enhancement node layer and randomly transformed and activated to obtain the enhanced features; finally, the mapping feature nodes and the enhancement feature nodes are fully connected to the output, and the model output weight W is calculated using the least squares method. Specifically, the following steps are included: Assume that the input data is represented by X = {x1, x2, ..., x N }, the data label is represented by Y = {y1,y2,…,y N }, N represents the number of data samples; Step 1: Perform n feature mappings on the input data X according to formula (1). Each feature mapping contains p nodes, where represents the activation function, W ei and β ei represents random weights and biases, and the feature combination is performed through formula (2) to obtain the mapping feature node Zn, WITH n =[Z1,Z2,...,Z n ] (2) Step 2: The mapping feature node Zn is enhanced m times by using equations (3) and (4), each feature enhancement includes q nodes, and feature combination is performed, where φ represents the activation function, W hj and represent random weights and biases, H j =φ(Z n W hj +β hj ), j=1,2,...,m (3) H m =[H1,H2,…,H m ] (4) Step 3: Merge the mapping feature nodes and the enhanced feature nodes into formula (5), and calculate the BLS output weight W using the least squares method according to formula (6), where C represents the regularization parameter, A=[Z n |H m ] (5) β=(A T A+CI) -1 AND T Y (6) Step 4: Calculate the model output Y according to formula (7), Y=Aβ (7), Introduce the weighted loss function, see formula (8), β=(A T θ 2 A+CI) -1 AND T θ 2 Y (8) Among them, θ=diag{θ1, θ2,…,θ N } represents the penalty weight of sample data.

2. The industrial abnormal data detection method based on the weighted multi-layer width learning system according to claim 1 is characterized in that: The WCBLS algorithm process is as follows: 1) Feature mapping stage: WCBLS uses the same feature mapping process as BLS. It maps the input feature X using formula (1) and combines the mapping results using formula (2). 2) Feature enhancement stage: WCBLS uses m enhancement node layers and enhances the mapping feature Fn layer by layer through equations (8) and (9), where m represents the number of feature enhancement layers and q represents the dimension of each feature enhancement. In order to comprehensively learn the bearing fault features, WCBLS combines the enhanced features at all levels through equation (10): H k =φ(F n W hk +b hk ),k=1 (8) H k =φ(H k-1 W hk +b hk ), k=2,3,…,m (9) H m =[H1,H2,...,H m ] (10) Among them, φ represents the activation function, W hk and b hk represents random weights and biases, 3) Output weight calculation: Formula (11) is used to combine the mapping features of formula (2) and the enhanced features of each layer of formula (10), and the output weight β of the BLS model is obtained according to the ridge regression calculation method of formula (6). A=[F n |H m ] (11) β=(A T A+CI) -1 AND T Y (12) Where C is the regularization parameter, 4) Calculate model output: The output result Y of each layer is calculated by formula (13), and the results are fused according to formula (8). Y = Aβ (13).

3. The industrial abnormal data detection method based on the weighted multi-layer width learning system according to claim 2 is characterized in that: The corrected model output weight is formula (15): Among them, θ=diag{θ1, θ2,…,θ N } is calculated through equations (16) and (17), which represents the penalty weight of the sample data. represents the standard deviation between error variables, the IQR represents the difference between the 75th percentile and the 25th percentile, and the constants c1 and c2 are usually set to 2.5 and 3. Finally, the WCBLS output is shown in formula (18): Y=S U β (18)。

Images