Intelligent manufacturing production line low quality monitoring big data intelligent cleaning method

Abstract

The application discloses a kind of intelligent manufacturing production line low quality monitoring big data intelligent cleaning method, first additional Gaussian noise is added to original multi-source monitoring data, and a certain proportion of subsequence in each sample is masked;Using the Transform network, the visible sequence in the simulated noise data is mapped into latent representation, and the latent representation is reconstructed into original data;By minimizing the mean square error between the reconstructed data and the original multi-source monitoring data, the model has the ability to reduce the complex noise in the production line monitoring data;Finally, the original test data is added to the big data intelligent cleaning model after noise, masking processing, and the cleaning effect of the model is tested;The application has strong feature extraction and data reconstruction capability, can effectively reduce the noise component in the intelligent manufacturing production line monitoring data, and provides data support for subsequent intelligent diagnosis and other downstream tasks of production line equipment.

Description

Technical Field

[0001] This invention belongs to the field of data cleaning technology, specifically relating to an intelligent cleaning method for low-quality monitoring big data in intelligent manufacturing production lines. Background Technology

[0002] With the rise of the smart manufacturing concept, enterprises have begun to consciously collect data generated during the processing of intelligent equipment on production lines in order to achieve predictive maintenance of production line equipment. However, the processing flow of industrial intelligent equipment is complex, and multiple devices on the production line usually work together, resulting in diverse data record file types and intricate relationships between files, making it difficult to directly extract the required information from massive amounts of monitoring data. In addition, production line equipment usually moves at high speeds and operates in harsh environments during processing, with many random interference factors. External environmental interference such as impacts from foreign objects, as well as the performance degradation of internal data acquisition devices, sensor failures, and abnormal data recordings, can all affect the monitoring data of key components, resulting in noise, outliers, missing values, and other interference data, thus reducing data quality and affecting the performance of subsequent intelligent diagnostic and predictive models. Therefore, researching cleaning methods for low-quality monitoring big data from smart manufacturing production lines is particularly important.

[0003] For low-quality monitoring data, traditional processing methods mainly include manual data removal and signal processing techniques. Manual data removal typically involves equipment maintenance personnel using their professional knowledge and experience to identify and remove abnormal data. This method is only suitable for situations with small data volumes and severe data quality degradation, and it requires a high level of expertise from the maintenance personnel. In practice, human error inevitably comes into play during the cleanup process, affecting the accuracy of subsequent analysis. Signal processing techniques are widely used in data cleaning methods, with common methods including wavelet transform and empirical mode decomposition (EMD). Wavelet transform is relatively complex in theory and implementation, requires significant expert knowledge, and has high computational complexity; its cleaning effect depends on the selected wavelet basis functions. EMD methods are difficult to optimize, as different parameter choices can lead to different decomposition results. Furthermore, the number of mode functions obtained from the decomposition is uncertain, making it difficult to determine how many mode functions should be retained after cleaning.

[0004] In recent years, deep neural networks have demonstrated superior performance in feature extraction and data reconstruction, providing new ideas for low-quality data cleaning methods. Compared with traditional cleaning methods, data-driven intelligent cleaning methods (Qiang Zhang, Yaming Zheng, Qiangqiang Yuan, et al. "Hyperspectral Image Denoising: From Model-Driven, Data-Driven, to Model-Data-Driven", IEEE Transactions on Neural Networks and Learning Systems, 2023.) are highly adaptable, do not require manual feature extraction, and are efficient. However, this method also has some limitations: on the one hand, it often uses only a single type of noise, making it difficult to apply to situations where intelligent manufacturing production line data contains complex noise; on the other hand, it typically uses convolutional neural networks or long short-term memory networks for feature extraction and data reconstruction. However, convolutional neural networks and their variants are difficult to capture long-distance features, and the computation of long short-term memory networks and their variants is serial, resulting in low computational efficiency and poor performance when processing large-scale data; in addition, long short-term memory networks may be affected by the vanishing or exploding gradient problem, leading to difficulties in model training. Summary of the Invention

[0005] To overcome the shortcomings of the existing technology, the present invention aims to propose an intelligent big data cleaning method for low-quality monitoring in intelligent manufacturing production lines. This method uses a Transformer network to extract robust high-level representations from complex noisy data and reconstructs them back into the original multi-source monitoring data, thereby reducing the impact of noise, outliers, and missing values ​​in the data on the accuracy of subsequent fault diagnosis and life prediction.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A smart cleaning method for low-quality monitoring big data in intelligent manufacturing production lines is proposed. This method simulates noisy data by adding various types of noise to the original multi-source monitoring data and masking a certain proportion of subsequences in each sample. The method uses a Transformer network to map the simulated noisy data into a latent representation and reconstructs the latent representation into the original multi-source monitoring data, thereby reducing the impact of complex noise in the original multi-source monitoring data on subsequent analysis.

[0008] A smart cleaning method for monitoring low-quality data in intelligent manufacturing production lines includes the following steps:

[0009] Step 1: Obtain the raw multi-source monitoring dataset of the equipment processing process from the data log files of the production line. Where N represents the number of samples, Let X be the i-th sample, and each sample has a sequence length of L and the number of channels C; then the dataset X is divided into training sets. and test set Where N1 represents the number of samples in the training set;

[0010] Step 2: For the samples x in the training set X1 i Add Gaussian noise x G Poisson noise x P and impulse noise x S The simulated noise data x is obtained. n The specific calculation formula is as follows:

[0011]

[0012] Among them, P s P represents the power of the original vacuum signal. n The power of the Gaussian noise signal is represented by SNR, the signal-to-noise ratio is represented by Var(·), and x represents the variance. rand ∈N(0,1) represents a random sample sequence that follows a standard normal distribution, and its sequence length is related to x. i same;

[0013] Step 3: Convert the simulated noise dataset X n The input is fed into a single convolutional neural network to perform sequence segmentation, dividing a single sample into multiple regular, non-overlapping subsequences. Each subsequence is then mapped to a high-dimensional vector. The noise signal after segmentation and mapping is... Where, N p The number of blocks is represented, D represents the dimension of the mapped vector, and Embed(·) represents the partitioning mapping operation;

[0014] Step 4: Randomly shuffle the subsequences mapped from each sample to obtain the dataset Shuffle(Emed(X)). n Then, a portion of the subsequence is randomly masked (deleted) at a masking ratio of r. Finally, the masked dataset is positionally encoded using sine and cosine functions, calculated as follows:

[0015]

[0016] Where pos represents the position, i represents the index of the encoding dimension, Shuffle(·) represents the random shuffling operation, Mask(·) represents the masking operation, and X′ n This is a noisy dataset that has been denoised, masked, and position-encoded.

[0017] Step 5: Establish a big data intelligent cleaning model based on a masked autoencoder, where both the encoder and decoder use Transformer networks; [The text then abruptly shifts to a different topic:] ...dataset X′ n The feature map obtained by computing the encoder part of the model is: in, Using a single-layer fully connected network to connect z j The dimension is mapped to the decoder dimension to obtain The input Z′ of the decoder is composed of high-level features Fc(z) j and learnable mask vector set The composition, specifically the calculation formula, is as follows:

[0018] Z′=Unshuffle([Fc(z j );Z m ])+E pos (3)

[0019] Where Unshuffle(·) means canceling the random shuffling operation, E pos Represents the position encoding vector;

[0020] Step 6: Input Z′ into the decoder for calculation to obtain the model's prediction result. The noise reduction result of the model is measured by the mean squared error between the predicted result Y1 and the training set X1, and is achieved by minimizing the mean squared error L between the samples in Y1 and X1. mse To improve the model's cleaning ability, the calculation formula is as follows:

[0021]

[0022] Step 7: The reconstruction error L obtained in Step 6 mse As the optimization objective during the training phase, gradient descent is used to update the model parameters θ:

[0023]

[0024] Where α represents the learning rate;

[0025] Step 8: Repeat steps 2-7 to continuously iterate and optimize the big data intelligent cleaning model until the maximum number of iterations is reached;

[0026] Step 9: Testing the Model. First, the big data intelligent cleaning model is tested using simulated noisy data. The test set X2 is processed through steps 2-4 to obtain the test sample dataset X′, which includes noise addition, masking, and location encoding. n2 ; Transfer the dataset X′ n2The input is fed into the trained big data intelligent cleaning model to obtain the denoising result Y2 of the noisy data. The signal-to-noise ratio (SNR) between Y2 and the test set X2 is calculated using the following formula:

[0027]

[0028] Where SNR represents the signal-to-noise ratio, P s2 P represents the power of the test signal. n2 Indicates the power of the noise signal;

[0029] Meanwhile, the big data intelligent cleaning model was tested using real data. The test set X2 was directly input into the trained big data intelligent cleaning model to obtain the denoising result Y2′ of the original multi-source monitoring dataset.

[0030] In step 5, the big data intelligent cleaning model is divided into two parts: an encoder and a decoder, both of which are composed of Transformer networks. The encoder maps the unmasked subsequences in each sample to latent representations. It contains three Transformer modules, each with a hidden layer dimension of 256. Each Transformer module consists of layer normalization, a multi-head self-attention mechanism, and a multilayer perceptron. The decoder reconstructs the latent representations and learnable mask vectors into the original input. It contains two Transformer modules with the same structure, each with a hidden layer dimension of 64. In addition, a fully connected layer is used to map the encoder's hidden layer dimension to the decoder's hidden layer dimension.

[0031] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0032] This invention proposes an intelligent big data cleaning method for monitoring low-quality data in intelligent manufacturing production lines. It simulates noisy data by adding various types of noise to the original multi-source monitoring data and masking a certain proportion of subsequences in each sample. A Transformer network is used instead of a traditional convolutional neural network or long short-term memory network to map the simulated noisy data into a latent representation, which is then reconstructed back into the original multi-source monitoring data. This effectively reduces the impact of complex noise in the monitoring data on subsequent analysis, improving data quality. Furthermore, the constructed intelligent big data cleaning model has an asymmetric structure. Its encoder only operates on the unmasked subsequences, and the decoder is lightweight, narrower, and shallower than the encoder. This design significantly reduces the computational overhead and training time. Moreover, the decoder is only used to perform data reconstruction tasks during training; after only minor adjustments to the encoder, it can be used for downstream diagnostic or prediction tasks, improving the model's generalization ability. Attached Figure Description

[0033] Figure 1 This is a flowchart of the present invention.

[0034] Figure 2 This is a schematic diagram of the structure of the big data intelligent cleaning model of the present invention. Detailed Implementation

[0035] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0036] like Figure 1 As shown, a smart cleaning method for monitoring low-quality data in intelligent manufacturing production lines includes the following steps:

[0037] Step 1: Obtain the raw multi-source monitoring dataset of the equipment processing process from the data log files of the production line. Where N represents the number of samples, Let X be the i-th sample, and each sample has a sequence length of L and the number of channels C; then the dataset X is divided into training sets. and test set Where N1 represents the number of samples in the training set;

[0038] Step 2: For the samples x in the training set X1 i Add Gaussian noise x G Poisson noise x P and impulse noise x S The simulated noise data x is obtained. n The specific calculation formula is as follows:

[0039]

[0040] Among them, P s P represents the power of the original vacuum signal. n The power of the Gaussian noise signal is represented by SNR, the signal-to-noise ratio is represented by Var(·), and x represents the variance. rand ∈N(0,1) represents a random sample sequence that follows a standard normal distribution, and its sequence length is related to x. i same;

[0041] Since the type and intensity of noise during the processing of production line equipment are unknown, common Gaussian white noise, Poisson noise and impulse noise are added to the original multi-source monitoring data to simulate noise data.

[0042] Step 3: Convert the simulated noise dataset X n The input is fed into a single convolutional neural network to perform sequence segmentation, dividing a single sample into multiple regular, non-overlapping subsequences. Each subsequence is then mapped to a high-dimensional vector. The noise signal after segmentation and mapping is... Where, N pThe number of blocks is represented, D represents the dimension of the mapped vector, and Embed(·) represents the partitioning mapping operation;

[0043] Step 4: Randomly shuffle the subsequences mapped from each sample to obtain the dataset Shuffle(Emed(X)). n Then, a portion of the subsequence is randomly masked (deleted) at a masking ratio of r. Finally, the masked dataset is positionally encoded using sine and cosine functions, calculated as follows:

[0044]

[0045] Where pos represents the position, i represents the index of the encoding dimension, Shuffle(·) represents the random shuffling operation, Mask(·) represents the masking operation, and X′ n This is a noisy dataset that has been denoised, masked, and position-encoded.

[0046] By randomly masking a certain proportion of subsequences, the situation of missing values ​​in the original multi-source monitoring data is simulated. Since Transformer does not have memory function like recurrent neural networks or convolutional neural networks when processing sequence data, it cannot capture the position information in the input sequence. Therefore, position encoding is additionally required.

[0047] Step 5: Establish a big data intelligent cleaning model based on a mask autoencoder, such as... Figure 2 As shown, the big data intelligent cleaning model consists of two parts: an encoder and a decoder, both composed of Transformer networks. The encoder maps the unmasked subsequences in each sample to latent representations. It contains three Transformer modules, each with a hidden layer dimension of 256. Each Transformer module consists of layer normalization, a multi-head self-attention mechanism, and a multilayer perceptron. The decoder reconstructs the latent representations and learnable mask vectors into the original input. It contains two Transformer modules with the same structure, each with a hidden layer dimension of 64. In addition, a fully connected layer is used to map the encoder's hidden layer dimension to the decoder's hidden layer dimension.

[0048] Data set X′ n The feature map obtained by computing the encoder part of the model is: in, Using a single-layer fully connected network to connect z j The dimension is mapped to the decoder dimension to obtain The input Z′ of the decoder is composed of high-level features Fc(z) j and mask vector set The composition, specifically the calculation formula, is as follows:

[0049] Z′=Unshuffle([Fc(z j );Z m ])+E pos (3)

[0050] Where Unshuffle(·) means canceling the random shuffling operation, E pos Represents the position encoding vector;

[0051] Step 6: Input Z′ into the decoder for calculation to obtain the model's prediction result. The noise reduction result of the model is measured by the mean squared error between the predicted result Y1 and the training set X1, and is achieved by minimizing the mean squared error L between the samples in Y1 and X1. mse To improve the model's cleaning ability, the calculation formula is as follows:

[0052]

[0053] The mean square error between the output of the big data intelligent cleaning model and the original training data is used as the loss. By minimizing the loss, the model can reconstruct the original data from the noisy data, that is, the model can have a certain data cleaning capability.

[0054] Step 7: The reconstruction error L obtained in Step 6 mse As the optimization objective during the training phase, gradient descent is used to update the model parameters θ:

[0055]

[0056] Where α represents the learning rate;

[0057] Step 8: Repeat steps 2-7 to continuously iterate and optimize the big data intelligent cleaning model until the maximum number of iterations E is reached;

[0058] Step 9: Test the model. First, use simulated noise data to test the big data intelligent cleaning model. Process the test set X2 through steps 2-4 to obtain the test sample dataset X′, which includes noise addition, masking, and location encoding. n2 ; Transfer the dataset X′ n2 The input is fed into the trained big data intelligent cleaning model to obtain the denoising result Y2 of the noisy data. The signal-to-noise ratio (SNR) between Y2 and the test set X2 is calculated using the following formula:

[0059]

[0060] Where SNR represents the signal-to-noise ratio, P s2 P represents the power of the test signal. n2It represents the power of the noise signal; by adding noise and masking the test set data and inputting it into the big data intelligent cleaning model, and calculating the signal-to-noise ratio of the test set data after model cleaning, the cleaning effect and generalization ability of the model can be quantitatively measured.

[0061] Meanwhile, the big data intelligent cleaning model is tested using real data. The test set X2 is directly input into the trained big data intelligent cleaning model to obtain the denoising result Y2′ of the original multi-source monitoring dataset. Since it is impossible to obtain ideal clean data without any noise, the signal-to-noise ratio cannot be directly calculated to quantitatively measure the cleaning effect of the model on real data. However, the cleaning effect can be measured by using the real data and the denoised data to perform downstream tasks such as fault diagnosis or life prediction.

[0062] Example: The effectiveness of the method of the present invention is verified by taking the cleaning of multi-source monitoring data collected during the mass production of a certain machine tool in a certain intelligent manufacturing production base as an example.

[0063] Raw multi-source monitoring data from the machine tool processing process was extracted from the intelligent manufacturing production line data acquisition system. The raw multi-source monitoring data was preprocessed, and a continuous time period was divided into a training set and a test set at a 9:1 ratio. Random sampling was performed on the two sets of data to obtain 2000 training samples and 200 test samples, each containing 300 sample points. Gaussian white noise was added to the training set data with a fixed signal-to-noise ratio of 0. Each sample was divided into 50 subsequences, each containing 5 sample points, with approximately 25% of the subsequences randomly masked. The preprocessed noisy data was input into a big data intelligent cleaning model for training. The encoder of the big data intelligent cleaning model consisted of three stacked Transformer modules, each with a hidden layer dimension of 256. The decoder, which was narrower and shallower than the encoder, consisted of two stacked Transformer modules, each with a hidden layer dimension of 64. The training parameters of the big data intelligent cleaning model are shown in Table 1.

[0064] Table 1 Training parameters of the big data intelligent cleaning model

[0065]

[0066] After the big data intelligent cleaning model was trained, the test set was preprocessed in the same way and then input into the big data intelligent cleaning model to test the cleaning effect. To reduce the randomness of the experiment, the experiment was repeated 5 times and the statistical values ​​of the cleaning results were calculated. The average signal-to-noise ratios of some samples in the test set after cleaning were 4.10, 6.02, 5.16 and 5.94, respectively. It can be seen that the big data intelligent cleaning model can effectively remove the extra noise added to the data and reconstruct the missing values, and has good generalization ability.

[0067] To more fully verify the effectiveness of the invention method, fault diagnosis of the machine was performed using data cleaned by the method of the present invention. The constructed fault diagnosis model includes two parts: feature extraction and state recognition. The feature extraction module is the encoder of the big data intelligent cleaning model in the method of the present invention. The state recognition module contains two fully connected layers. The first fully connected layer uses LeakyReLU activation function and has a Dropout layer with a probability of 0.2. The second fully connected layer outputs the predicted state and uses the Softmax function to normalize the predicted value. The training parameters of the fault diagnosis model are shown in Table 2, and the diagnosis results are shown in Table 3. In Table 3, the model structure of Comparison Method 1 is the same as that of the invention method, but the weights of its feature extraction module are randomly initialized, that is, the input data is not cleaned. The feature extraction module of Comparison Method 2 is composed of a convolutional neural network, and the number of network layers is also three. As can be seen from Table 3, after using the intelligent cleaning method of the present invention, the intelligent fault diagnosis accuracy can reach 92.75%. Compared to the present invention, Method 1, due to the lack of data cleaning, achieves an average diagnostic accuracy of only 86.09%, far lower than the method described in this invention. Furthermore, noise interference leads to unstable training and significant fluctuations in test results. Method 2, employing a convolutional neural network for feature extraction, has a lower feature mining capability than the Transformer network, resulting in an average diagnostic accuracy of only 80.72%, also far lower than the method described in this invention.

[0068] Table 2 Training parameters for the fault diagnosis model

[0069]

[0070] Table 3 Diagnostic results of different methods

[0071]

[0072] By comparing the diagnostic effects of the method of the present invention with those of comparison method 1 and comparison method 2, it is shown that the method of the present invention can effectively reduce noise in the original multi-source monitoring data, improve the data quality, and thus improve the effect of subsequent intelligent diagnosis, verifying the generalization ability of the method of the present invention; in addition, the Transformer network used in the present invention has a strong feature extraction capability, and its performance is better than that of traditional convolutional neural networks.

Claims

1. A smart cleaning method for monitoring low-quality products in intelligent manufacturing production lines using big data, characterized in that: Includes the following steps: Step 1: Obtain the raw multi-source monitoring dataset of the equipment processing process from the data log files of the production line. ,in, Indicates the number of samples. For the first There are samples, and the sequence length of each sample is . The number of channels is Then the dataset Divided into training set and test set ,in, Indicates the number of samples in the training set; Step 2: Process the training set samples Add Gaussian noise Poisson noise and impulse noise To obtain simulated noise data The specific calculation formula is as follows: （1） in, The power of the original vacuum signal. This represents the power of the Gaussian noise signal. Indicates the signal-to-noise ratio. Represents variance. Describes a random sample sequence that follows a standard normal distribution, and its sequence length is equal to the length of the random sample sequence. same; Step 3: Simulate the noise dataset The input is fed into a single convolutional neural network to perform sequence segmentation, dividing a single sample into multiple regular, non-overlapping subsequences. Each subsequence is then mapped to a high-dimensional vector. The noise signal after segmentation and mapping is... ,in, Indicates the number of blocks. This represents the dimension of the mapped vector. This represents a split mapping operation; Step 4: Randomly shuffle the subsequences mapped from each sample to obtain the dataset. Then, a portion of the subsequence is randomly masked, with a masking ratio of . Then, the masked dataset is positionally encoded using sine and cosine functions, with the specific calculation formula as follows: （2） in, Indicates location, Index representing the encoded dimension, This indicates a random shuffling operation. This indicates a masking operation. This is a noisy dataset that has been denoised, masked, and position-encoded. Step 5: Establish a big data intelligent cleaning model based on a masked autoencoder, where both the encoder and decoder use Transformer networks; [The dataset is then processed / decoded]. The feature map obtained by computing the encoder part of the model is: ,in, Using a single-layer fully connected network The dimension is mapped to the decoder dimension to obtain ; Input to the decoder Features of high-level and learnable mask vector set The composition, specifically the calculation formula, is as follows: （3） in, This indicates that the random shuffling operation has been cancelled. Represents the position encoding vector; Step 6: Put The input is fed into the decoder for computation, yielding the model's prediction result. To predict the results With training set The mean squared error between the two values ​​is used to measure the noise reduction result of the model, and the noise reduction result is measured by minimizing the mean squared error between the two values. and Mean square error of medium sample To improve the model's cleaning ability, the calculation formula is as follows: （4） Step 7: Calculate the reconstruction error obtained in Step 6. As the optimization objective during the training phase, gradient descent is used to update the model parameters. : （5） in, Indicates the learning rate; Step 8: Repeat steps 2-7 to continuously iterate and optimize the big data intelligent cleaning model until the maximum number of iterations is reached; Step 9: Test the model. First, use simulated noise data to test the big data intelligent cleaning model. (The test set will be used for this purpose.) The dataset of test samples after processing in steps 2-4 includes the results of noise addition, occlusion, and location encoding. ; The dataset The data is input into a pre-trained big data intelligent cleaning model to obtain the denoising results of the noisy data. ,calculate With test set The signal-to-noise ratio between them is calculated using the following formula: （6） in, Indicates the signal-to-noise ratio. Indicates the power of the test signal. Indicates the power of the noise signal; At the same time, the big data intelligent cleaning model is tested using real data, and the test set is used... The data is directly input into a pre-trained big data intelligent cleaning model to obtain the denoising results of the original multi-source monitoring dataset. .

2. The method according to claim 1, characterized in that: In step 5, the big data intelligent cleaning model is divided into two parts: an encoder and a decoder, both of which are composed of Transformer networks. The encoder maps the unmasked subsequences in each sample to latent representations. It contains three Transformer modules, each with a hidden layer dimension of 256. Each Transformer module consists of layer normalization, a multi-head self-attention mechanism, and a multilayer perceptron. The decoder reconstructs the latent representations and learnable mask vectors into the original input. It contains two Transformer modules with the same structure, each with a hidden layer dimension of 64. In addition, a fully connected layer is used to map the encoder's hidden layer dimension to the decoder's hidden layer dimension.

Images