A fault diagnosis method for heavy equipment data preprocessing
By optimizing the hyperparameters of the neural network using the improved UMAP algorithm and Bayesian optimization algorithm, a DNN model was constructed, which solved the difficulty of fault diagnosis caused by the high-dimensional coupling of heavy equipment and improved the diagnostic accuracy and precision.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-08-25
- Publication Date
- 2026-03-27
AI Technical Summary
The detection data of heavy equipment is highly dimensional and tightly coupled, resulting in long fault diagnosis time and low accuracy.
An improved UMAP algorithm (MS-UMAP) is used for dimensionality reduction, and a Bayesian optimization algorithm is combined to optimize the hyperparameters of the fully connected neural network, thereby constructing a DNN model for fault diagnosis.
It improves the accuracy and precision of fault diagnosis for heavy equipment and solves the diagnostic difficulties caused by high-dimensional coupling.
Smart Images
Figure CN117216651B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of heavy equipment control, and particularly relates to a heavy equipment data preprocessing fault diagnosis method. BACKGROUND
[0002] The working state of heavy equipment directly affects the running state of the entire mine grinding system. Heavy equipment control has the characteristics of large response lag in process, large inertia in control system, strong coupling of multiple variables, and complex working mechanism. Therefore, the detection data of heavy equipment presents the characteristics of multi-dimension and strong coupling. The multi-dimension is reflected in the complexity of heavy equipment working condition, the multiple parts to be monitored by sensors, and the high dimension of detection data. With the increase of detection data dimension, the distribution of detection data in space becomes sparse, and if it is not processed, it will cause "dimension disaster". The strong coupling is reflected in the fact that the data distribution generated by heavy equipment presents the characteristics of "strong coupling and strong dispersion", that is, the detection temperature data has strong coupling, while the flow, pressure, current and vibration data have strong dispersion, and strong coupling and strong dispersion data appear at the same time, which also increases the difficulty of fault diagnosis. Due to the high dimension of heavy equipment industrial data, direct fault diagnosis leads to long diagnosis time and is easy to fall into "dimension disaster". Due to the strong correlation of heavy equipment industrial data and the strong coupling between data, direct fault diagnosis leads to low fault diagnosis accuracy. Therefore, how to correctly process heavy equipment industrial data and ensure the complete structure of data has become a big difficulty in heavy equipment fault diagnosis. SUMMARY
[0003] In view of the above shortcomings and deficiencies of the prior art, the present application provides a heavy equipment data preprocessing fault diagnosis method to solve the problem of low diagnosis accuracy caused by high dimension and strong coupling of heavy equipment data.
[0004] In order to achieve the above purpose, the main technical scheme adopted by the present application includes:
[0005] A heavy equipment data preprocessing fault diagnosis method includes the following steps:
[0006] Step S01: obtaining heavy equipment fault diagnosis data set;
[0007] Step S02: cleaning the collected heavy equipment fault diagnosis data set, checking data outliers and missing values, filling the missing data by taking the average value of the information of the nearest ten groups of data, directly deleting abnormal data for the outliers, and inputting the cleaned data into the MS-UMAP dimension reduction, and using the dimension-reduced data for the input of the DNN model;
[0008] Step S03: The dimension-reduced heavy equipment fault diagnosis data set is divided into a training set and a test set, the hyperparameters of the full connection neural network are optimized and solved by using the Bayesian optimization algorithm through the training set, and an optimal fault diagnosis model is obtained;
[0009] Step S04: An activation function of the DNN model is set, the DNN model is constructed according to the optimal hyperparameter combination after the Bayesian optimization, and the trained optimal DNN fault diagnosis model is tested for fault diagnosis through the test set.
[0010] Further, in the step S01, the heavy equipment fault diagnosis data set is acquired on site through the PLC.
[0011] Further, in the step S02, the MS-UMAP dimension reduction specifically comprises the following steps:
[0012] An improved UMAP algorithm is adopted to obtain the improved direction similarity measure of the original data space, and the original high-dimensional space measure is constructed, and the formula of the improved UMAP algorithm is as follows:
[0013]
[0014] In the formula, p j|i represents the probability of the correlation of data points x i and x j , d1(x i , x j ) represents the Mahalanobis distance between x i and x j , x i , x j represent the i-th and j-th sample data, ρ i is the distance from x i to the nearest neighbor, and σ i is a scale parameter.
[0015] The low-dimensional embedding space is constructed by using formula (8):
[0016]
[0017] In the formula, q ij is a symmetric measure of the similarity between points y i and y j in the embedding space, and variables a and b are hyperparameters.
[0018] Formula (9) is used to link the two spaces of formula (6) and (8) to make the difference between them minimized, that is, the two spaces are most similar, and the dimension reduction is completed.
[0019]
[0020] In equation (9), C1(p) ji ,q ji ) represents p ij and q ij The cross-entropy loss function, p ij Represents a point x in the input space i and x j A symmetric measure of similarity between them, q ij Embedded space midpoint y i and y j A symmetric measure of similarity between them.
[0021] Furthermore, step S03 specifically includes the following steps:
[0022] Using 70% of the dimensionality-reduced heavy equipment data as the training set, the optimization of DNN hyperparameters is treated as a black-box model problem, as shown in the following formula:
[0023]
[0024] in For the optimal hyperparameter combination of the DNN model, X is the search space defined by the hyperparameters, and f DNN (·) represents a DNN model;
[0025] Gaussian processes were chosen as the surrogate model for Bayesian optimization.
[0026]
[0027] In the formula, m(x) is the mean function, and k(X,X′) is the covariance function. For the noise term, the posterior distribution of the model is obtained based on the sampling function EI:
[0028]
[0029] In the formula, E[·] represents the expectation, and f * Represented as the current minimum sampled function value, φ(·) and Let μ(x) represent the cumulative distribution function (CDF) and probability density function (PDF), respectively, and let σ(x) represent the prediction mean and prediction variance. The model iteratively updates the probabilistic surrogate model by sampling new observations from the training set using a sampling function, gradually improving the accuracy of the surrogate model until the optimal combination of hyperparameters is found. Obtain the optimal fault diagnosis model.
[0030] The heavy equipment data preprocessing fault diagnosis method has the advantages that the heavy equipment data preprocessing fault diagnosis method is a data preprocessing-based fault diagnosis model composed of an improved UMAP Mahalanobis distance MS-UMAP dimension reduction model and a Bayesian optimization full connection neural network model, the fault diagnosis precision and accuracy of heavy equipment are improved through improvement of the original model. BRIEF DESCRIPTION OF DRAWINGS
[0031] Figure 1 The heavy equipment data preprocessing fault diagnosis method. DETAILED DESCRIPTION
[0032] In order to better explain the present application, the present application is described in detail in combination with the drawings and specific embodiments.
[0033] The present application provides a heavy equipment data preprocessing fault diagnosis method, as shown in Figure 1 The method comprises the following steps:
[0034] Step S01: obtaining a heavy equipment fault diagnosis data set; specifically, obtaining a heavy equipment fault diagnosis data set on site through a PLC.
[0035] Step S02: data cleaning of the collected heavy equipment fault diagnosis data set, checking data outliers and missing values, filling in missing data by taking the average value of the information of the nearest ten groups of data, directly deleting abnormal data for abnormal values, and inputting the cleaned data into MS-UMAP dimension reduction, and using the dimension-reduced data for input of a DNN model.
[0036] The collected data is cleaned, the data set is reviewed, and the meaning of each data variable in the data set is understood. The data outliers and missing values are checked, the corresponding data is directly deleted for data outliers, and the missing values are filled in by taking the average value of the corresponding data of the nearest ten groups. The cleaned data is put into MS-UMAP for data dimension reduction. Ms-umap dimension reduction is to construct a low-dimensional embedding space through the original input data space, and then use cross entropy to reduce the difference between the two distributions so that the similarity of the points in the low-dimensional embedding space is approximately the same as the points in the original input data space, so as to realize dimension reduction. Let the heavy equipment data sample be n data samples in the original space, and p is the sample dimension. x ij represents the jth neighbor of the ith data sample of the heavy equipment monitoring data sample. Given the input hyperparameter k, for each x i , the nearest neighbor descent algorithm is used to calculate the set of k nearest neighbors of x i under the metric. i1 ik }, using a Gaussian or Radial Basis Function (RBF) kernel to measure the similarity between points in the input space. Heavy equipment data samples x i and x j The probability that x
[0037]
[0038] where ||·||2denotes the l2norm, x i , x j denote the i-th and j-th sample data, p i is the distance from x i to the nearest neighbor. σ i is a scale parameter.
[0039] p i ensures that x i is connected to at least one other data point with an edge of weight 1, which is equivalent to locally connected simplicial sets at x i The mathematical definition is expressed as:
[0040] p i = min{d(x i , x ij ) | 1≤j≤k, d(x i , x ij )>0} (2)
[0041] σ i is chosen such that the total similarity of a point x i to its k nearest neighbors is normalized. By binary search, σ i satisfies:
[0042]
[0043] The UMAP original model uses Euclidean distance as the distance metric of the original input data space, but the heavy equipment industry data presents high dimension and strong coupling. The Euclidean distance can only simply calculate the distance between two data points without considering the correlation. Therefore, using Euclidean distance as the distance metric of the original input data space cannot guarantee the quality of the heavy equipment data after dimension reduction. Therefore, Mahalanobis distance with correlation measurement is used as the distance metric of the original input data space. Mahalanobis distance considers unequal variances and the correlation between features, and fully evaluates the distance by assigning different weights or importance factors to the features of the data points. Therefore, an improved MS-UMAP algorithm is proposed. Assuming x m and x h are heavy equipment sample data sets with covariance matrix ∑ (>0) The two samples are extracted, there is an orthogonal matrix T and a diagonal matrix Λ = diag(λ1, λ2, …, λ p ), where the diagonal matrix elements are positive numbers, so that ∑ = TΛT'. Let y = T'x = (y1, y2, …, y p )'that is, the orthogonal rotation is made on the data axis in geometry, so that y1 = T'x1 = (y 11 , y 12 , …, y 1p )'and y2 = T'x2 = (y 21 , y 22 , …, y 2p )'are the new coordinates of the samples, since V(y) = T'∑T = Λ, the new coordinates are not correlated with each other after orthogonal rotation, in order to further eliminate the influence of variance difference between the new data, the standardized squared Euclidean distance between y1 and y2 is calculated:
[0044]
[0045] The Mahalanobis distance can be obtained:
[0046]
[0047] In formula (5), Σ -1 represents the inverse covariance matrix of the data points, and represents the matrix transpose.
[0048] Only when the characteristics are not correlated (at this time the covariance matrix is equal to 1), the distance under the Mahalanobis distance measurement is equal to the distance under the Euclidean distance measurement.
[0049] The improved UMAP algorithm uses the Mahalanobis distance instead of the Euclidean distance when measuring the similarity between samples in the original sample space. The improved similarity expression is:
[0050]
[0051] Formula (6) is the improved direction similarity measurement of the original data space. In order to have a symmetric measurement about i and j, it needs to be symmetrized as:
[0052]
[0053] In formula (7), p ij represents the symmetric measurement of the similarity between points x i and x j in the input space.
[0054] After constructing the high-dimensional space measurement, the construction of the low-dimensional embedding space is carried out. Let the embedding of the low-dimensional point be where p is the dimension of the embedding space, and the dimension of the embedding space is required to be less than the input dimension. In the embedding space, the point yi with points y j The probability of being a neighbor of y can be computed by the similarity of these points:
[0055]
[0056] q in equation (8) ij The similarity between points y in the embedding space i and y j The symmetric measure of similarity between points y in the embedding space and y in the original input data space, the variables a and b are hyperparameters. Finally, the cross-entropy is adopted to reduce the difference between the two space distributions of the original input data space and the low-dimensional embedding space so that the similarity of points in the low-dimensional embedding space is similar to the similarity of points in the original input data space:
[0057]
[0058] The first term in equation (9) is the attractive force that attracts the embeddings of neighboring points to each other. The second term is the repulsive force that repels the embeddings of non-adjacent points from each other. Convergence to a local minimum is guaranteed by slowly reducing the attractive and repulsive forces. By disassembling equation (9), equation (10) is obtained:
[0059]
[0060] It can be seen from equation (10) that the first and only depends on p ij Since p ij is a fixed value, it can be omitted during optimization, and the cost function after optimization is revised:
[0061]
[0062] The first term in equation (11) is optimized based on the sampling method, by sampling p ij , and updating according to the value of q ij . The second term is optimized based on the negative sampling method, by randomly sampling, and assuming to be a negative sample, and then updating according to the value of (1-ln q ij ). For negative samples, for a large enough data set, it can be reasonably approximated by a uniform distribution. When the cost function is minimized, the points in the low-dimensional embedding space and the points in the original input data space are most similar, that is, the data after dimensionality reduction.
[0063] Step S03: Divide the heavy equipment fault diagnosis data set after dimensionality reduction into a training set and a test set, and optimize the hyperparameters of the full connection neural network through the training set using the Bayesian optimization algorithm to obtain the optimal fault diagnosis model.
[0064] The heavy equipment data after dimensionality reduction by the improved MS-UMAP algorithm of UMAP is imported into a fully connected neural network model (DNN) for fault diagnosis. The DNN is a fully connected neural network with multiple hidden layers, including an input layer, a hidden layer and an output layer. Each layer of the network contains multiple neurons connected in a fully connected manner. The deep neural network takes the output features of the previous layer as the input of the next layer to realize feature mapping and learn the rules of data features. The number of neurons in the input layer of the DNN model is set to the dimensionality of the heavy equipment fault sample after dimensionality reduction, the number of neurons in the output layer is set to the number of fault types to be diagnosed, and the activation function is set to ReLU function. Given the heavy equipment fault data set (x, y) x∈R l , y∈R after dimensionality reduction, where x is the input data of the heavy equipment fault after dimensionality reduction, and y represents the corresponding fault type. Through parameter learning, the relationship between the input data and the label is established. The calculation formula of a single neuron node of the DNN is as follows:
[0065] y=σ(ω1x1+ω2x2+…+ω p-1 x p-1 +ω p x p )+b (12)
[0066] In formula (12), x1, x2, …, x p-1 , x p represent the input data, ω1, ω2, …, ω p-1 , ω p represent the weights, and b represents the bias; σ(·) represents the activation function. The fully connected neural network has many hyperparameters, which directly affect the accuracy of fault diagnosis. Therefore, before fault diagnosis, the hyperparameters of the DNN model need to be optimized to find the best diagnosis model. The optimization of DNN hyperparameters is generally regarded as a black box model problem, which can be expressed as follows:
[0067]
[0068] where is the optimal hyperparameter set of the DNN model, x is the search space defined by the hyperparameters, and f(·) is a high-cost black box target function.
[0069] Because the evaluation cost of the target function to be evaluated is high and complex, a probabilistic surrogate model is used to approximately represent the current target function. The probabilistic surrogate model starts from the initial prior knowledge and improves the model by continuously increasing the data observation points. The probabilistic surrogate model used in this paper is Gaussian process.
[0070] Gaussian Process (GP) is defined as the extension of multivariate Gaussian distribution on continuous domain, which can obtain accurate prediction value from a small amount of prior knowledge and provide uncertainty. Any linear combination of random variables obeys normal distribution, which is defined by mean function m(x) and covariance function k(x, x'):
[0071] m(x) = E[f(x)] (14)
[0072] k(x, x') = E[(f(x)-m(x))(f(x')-m(x'))] (15)
[0073] In formula (14) and formula (15), f(·) is the target function, and E[·] represents expectation.
[0074] The Gaussian process is expressed as:
[0075] f(x) ~ GP(m(x), k(x, x')) (16)
[0076] When the heavy equipment training set D = {(x i , y i )i = 1, 2, …, n} is known, the Gaussian process learning is performed, and there is a Gaussian distribution f ~ N(0, K(X, X)), the covariance matrix K(X, X) and the covariance function k(x, x') are expressed as:
[0077]
[0078] Considering the case that the DNN model y DNN = f(x) + ε exists noise ε, a noise term is introduced, and
[0079]
[0080] In the formula, x is the input, y is the output, I is the unit matrix, n is the sample number, and T is the transpose. After adding a new observation sample X * , the trained model can predict the output value y * , and the joint distribution is
[0081]
[0082]
[0083] In the formula, T is the matrix transpose, K(X, X) is the covariance matrix, K(X, X * ) = [k(x1, x * ), k(x2, x * ), …, k(xn , x * ) T , I is identity matrix, is noise variance. The conditional probability distribution p(y|x * , X, y) ~ N(μ(x * ), σ 2 (x * )) obeys multivariate Gaussian distribution. The mathematical expectation and variance of the predicted output y * are expressed as:
[0084] μ(x * ) = k T (K + σ 2 I) -1 k (20)
[0085] σ 2 (x * ) = k(x * , x * ) - K(X, x * ) T (K(X, X) + σ 2 I) -1 K(X, x * ) (21)
[0086] The expectation μ(x * ) can be used as the predicted value of y * , and the variance can be used as the uncertainty of the prediction at this point. From the above formula, it can be seen that the Gaussian random process calculates the posterior distribution through the kernel function and the observed value. After setting the probability agent model, the next step is to set the acquisition function.
[0087] The acquisition function determines where to sample next to guide the search to the optimum of the objective function. The next sampling point is selected by querying the samples in the probability agent model that are known to be in a highly suitable region and a region with high confidence. The mathematical expression is as follows:
[0088] x t+1 = argmaxα t (x; D) (22)
[0089] When selecting the next sampling point, the region with a higher mean and a wider confidence interval is generally selected. The EI algorithm is applied as the acquisition function for model sampling.
[0090] EI (Expected Improvement) represents finding a point that is expected to be better than the optimal point, and the expression is as follows
[0091]
[0092] where E[·] denotes expectation, f * is denoted as the current minimum function value of the sampling, φ(·) and denote the cumulative distribution function CDF and the probability density function PDF, respectively, μ(x) is the predicted mean, and σ(x) is the predicted variance.
[0093] By comparing with the optimal point x + , find a point greater than the optimal point expectation. x + The expression is as follows
[0094]
[0095] Formula (24) is converted into formula (23):
[0096]
[0097] Through the probability agent model and the acquisition function, the DNN fault diagnosis model is optimized by Bayes, and the optimal model parameter is found. The MS-UMAP reduced dimension parameter is imported into the DNN model, and the heavy equipment fault diagnosis is completed.
[0098] Step S04: set the activation function of the DNN model as the ReLU function, construct the DNN model according to the optimal hyperparameter combination after the Bayes optimization, and perform fault diagnosis test on the trained optimal DNN fault diagnosis model through the test set.
[0099] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application, and the modifications, modifications, replacements and variations of the above embodiments by those skilled in the art all belong to the scope of the present application.
Claims
1. A fault diagnosis method for heavy equipment data preprocessing, characterized in that: Includes the following steps: Step S01: Obtain the heavy equipment fault diagnosis dataset; Step S02: Clean the collected heavy equipment fault diagnosis dataset, check for outliers and missing values. Missing data is filled by averaging the data from the last ten sets. Outliers are deleted directly. The cleaned data is then dimensionality reduced using MS-UMAP and used as input to the DNN model. Step S03: Divide the dimensionality-reduced heavy equipment fault diagnosis dataset into a training set and a test set. Use the Bayesian optimization algorithm on the training set to optimize the hyperparameters of the fully connected neural network and obtain the optimal fault diagnosis model. Step S04: Set the activation function of the DNN model, construct the DNN model based on the optimal hyperparameter combination after Bayesian optimization, and perform fault diagnosis testing on the trained optimal DNN fault diagnosis model through the test set. In step S02, the specific steps of MS-UMAP dimensionality reduction are as follows: An improved UMAP algorithm is used to obtain an improved directional similarity measure of the original data space, and to construct the original high-dimensional space measure. The improved UMAP algorithm formula is as follows: （6）； In the formula Representing data points x i and x j The probability of correlation. d 1 ( x i , x j ) represents x i and x j Mahalanobis distance between them x i , x j Indicates the first i The and the first j One sample data, ρ i yes x i The distance to the nearest neighbor, σ i It is the scale parameter; the low-dimensional embedding space is constructed using formula (8): （8）； In the formula: For embedding the midpoint of space and Symmetric measure of similarity between variables and It's a hyperparameter; By using formula (9) to connect the two spaces of formula (6) and (8), the difference between them is minimized, that is, the two spaces are most similar, thus completing the dimensionality reduction; （9）； In equation (9) express and The cross-entropy loss function, Represents points in the input space x i and x j A symmetric measure of similarity between them; The specific steps of step S03 are as follows: Using 70% of the dimensionality-reduced heavy equipment data as the training set, the optimization of DNN hyperparameters is treated as a black-box model problem, as shown in the following formula: (14)； in, The optimal hyperparameter combination for the DNN model is given by X, which is the search space defined by the hyperparameters. It is a DNN model; Gaussian processes were chosen as the surrogate model for Bayesian optimization. （18） ； In the formula, m ( X ) is the mean function, Let covariance function be used. For the noise term, the posterior distribution of the model is obtained based on the sampling function EI: （23）； In the formula Expressing expectations, f * This represents the minimum function value currently available for sampling. and Let CDF and PDF represent the cumulative distribution function and probability density function, respectively. To predict the mean, To predict variance, a sampling function is used to iterate through new observations from the training set, continuously updating the probabilistic surrogate model and gradually improving its accuracy until the optimal combination of hyperparameters is found. To obtain the optimal fault diagnosis model.
2. The fault diagnosis method for heavy equipment data preprocessing according to claim 1, characterized in that: In step S01, a heavy equipment fault diagnosis dataset is acquired on-site via a PLC.