Expansion method and device of unbalanced data set and storage medium

Abstract

The invention discloses an unbalanced data set expansion method and device and a storage medium, and relates to the technical field of data enhancement, and the method comprises the steps: inputting fault data into an improved network based on VAE-GAN, and obtaining a plurality of newly generated fault data; adding the newly generated fault data into an unbalanced data set to obtain a balanced data set; wherein the improved network based on the VAE-GAN comprises a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a variational auto-encoder similar to Fourier transform. The generator provided by the embodiment of the invention can simultaneously capture the characteristics of the time domain and the frequency domain in the potential space, so that the generated fault data has the actual fault characteristics, and the time domain discriminator and the frequency domain discriminator can perform double identification on the generated data, so that the fault detection efficiency is improved. The generated data is ensured to be matched with the actual data on multiple levels, and the quality of the generated data is further ensured.

Description

Technical Field

[0001] The present invention relates to the field of data enhancement technology, and in particular to a method, device and storage medium for expanding an imbalanced data set. Background Art

[0002] Before performing bearing fault diagnosis on mechanical equipment, it is necessary to collect bearing data to train the fault diagnosis model. However, because bearings operate normally most of the time, the abnormal data in the collected bearing data is far less than the normal data, that is, the collected bearing data set is an unbalanced data set, which will cause the fault diagnosis model obtained by subsequent training to be biased towards the majority class data (normal class data), which in turn leads to low accuracy of the fault diagnosis results. Therefore, it is necessary to perform data enhancement on the collected unbalanced data set.

[0003] Among the existing data enhancement technologies, the most common ones are oversampling and undersampling. Oversampling achieves data enhancement by duplicating minority class samples or through interpolation methods. Undersampling reduces the imbalance of the data set by randomly deleting majority class samples. However, both oversampling and undersampling technologies have obvious defects: oversampling may cause overfitting, causing the fault diagnosis model to learn the specific noise of these repeated or similar samples rather than the real, broad data features, thereby limiting the generalization ability of the fault diagnosis model; while undersampling may remove some key information when deleting majority class samples, resulting in insufficient training of the fault diagnosis model, thereby reducing the accuracy of the fault diagnosis results. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device and storage medium for expanding an unbalanced data set, which can generate more diverse fault data with actual fault characteristics, thereby achieving data enhancement for the unbalanced data set.

[0005] To achieve the above object, an embodiment of the present invention provides a method for expanding an unbalanced data set, comprising: Input the fault data into the improved network based on VAE-GAN to obtain some newly generated fault data; Adding the plurality of newly generated fault data to the unbalanced data set to obtain a balanced data set; The improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a Fourier transform-like variational autoencoder.

[0006] As an improvement of the above scheme, the Fourier transform-like variational autoencoder includes an encoder and a decoder, and the encoder includes a label embedding layer, a first fully connected layer, and a second fully connected layer. Then, the fault data is input into the improved network based on VAE-GAN to obtain a number of newly generated fault data, including: The label embedding layer is used to convert the category label of the fault data to obtain an embedding vector; Merging the embedding vector and the fault data and inputting them into the first fully connected layer and the second fully connected layer respectively, so as to obtain the mean of the real part and the logarithmic variance of the real part through the first fully connected layer, and obtain the mean of the imaginary part and the logarithmic variance of the imaginary part through the second fully connected layer; Reparameterizing the mean of the real part and the standard deviation of the real part to obtain a number of real part latent variables, and reparameterizing the mean of the imaginary part and the standard deviation of the imaginary part to obtain a number of imaginary part latent variables; The decoder reconstructs data according to the cosine variable, the sine variable, the real potential variable and the imaginary potential variable to obtain newly generated fault data.

[0007] As an improvement of the above solution, before inputting the fault data into the improved network based on VAE-GAN, the generator, the time domain discriminator and the frequency domain discriminator are trained in an alternating training manner, specifically including: The generator is fixed, and the time domain discriminator and the frequency domain discriminator are iteratively trained using a time domain discriminator loss function and a frequency domain discriminator loss function to obtain an intermediate time domain discriminator and an intermediate frequency domain discriminator; The intermediate time domain discriminator and the intermediate frequency domain discriminator are fixed, and the generator loss function is used to iteratively train the generator to obtain an intermediate generator, and the above-mentioned step of fixing the generator is returned to repeat the alternating training process until an improved network based on VAE-GAN that meets the requirements is obtained.

[0008] As an improvement of the above scheme, the generator loss function is shown as follows: in, represents the generator loss function; represents the reconstruction loss function; represents the KL divergence function; represents the adversarial loss function; Represents input data; Represents reconstructed data; and represents a hyperparameter; represents the mean; represents standard deviation; The adversarial loss function is shown as follows: in, represents the binary cross entropy loss function; represents the time domain discriminator; represents the frequency domain discriminator; Represents short-time Fourier transform.

[0009] As an improvement of the above scheme, the time domain discriminator loss function and the frequency domain discriminator loss function are respectively shown as follows: in, represents the temporal discriminator loss function; represents the frequency domain discriminator loss function; represents the binary cross entropy loss function; represents the time domain discriminator; Represents real data; Indicates the generation of data; represents the frequency domain discriminator; Represents short-time Fourier transform.

[0010] As an improvement of the above solution, the identification process of the time domain identifier on the data to be identified includes: The data to be identified and its label are fused through a time domain discriminator embedding layer to obtain first fused data; wherein the data to be identified is real data or generated data; Input the first fused data into a three-layer time domain discriminator fully connected layer for step-by-step dimension reduction, and output a time domain discriminant value through a time domain discriminator activation function, wherein the time domain discriminant value is used to indicate whether the data to be identified is real data; The identification process of the frequency domain identifier on the data to be identified includes: Performing Fourier transform on the data to be identified to obtain frequency domain data; fusing the frequency domain data and the label through a frequency domain discriminator embedding layer to obtain second fused data; The second fused data is input into the three-layer frequency domain discriminator fully connected layer for step-by-step dimension reduction, and a frequency domain discriminant value is output through the frequency domain discriminator activation function, and the frequency domain discriminant value is used to characterize whether the data to be identified is real data.

[0011] As an improvement of the above solution, after obtaining the balanced data set, the method further includes using the balanced data set to train a pre-built CNN-LSTM network to obtain a trained bearing fault diagnosis model.

[0012] As an improvement of the above solution, the CNN-LSTM network includes a convolution layer, a maximum pooling layer, an LSTM layer and an output layer. Then, the fault diagnosis process of the CNN-LSTM network includes: The convolutional layer is used to extract local spatiotemporal features of bearing data; Using the maximum pooling layer to perform dimensionality reduction processing on the local spatiotemporal features to obtain dimensionality reduction features; Using the LSTM layer to perform time series information processing on the dimensionality reduction features to obtain high-dimensional features; The output layer is used to map the high-dimensional features into probability distribution and output the bearing fault type.

[0013] To achieve the above object, an embodiment of the present invention further provides a device for expanding an unbalanced data set, comprising: A fault data generation module is used to input the fault data into the improved network based on VAE-GAN to obtain a number of newly generated fault data; A data set construction module is used to add the several newly generated fault data to an unbalanced data set to obtain a balanced data set; wherein the improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a Fourier transform-like variational autoencoder.

[0014] To achieve the above objectives, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method for expanding the unbalanced data set as described in any of the above embodiments.

[0015] Compared with the prior art, the expansion method, device and storage medium of the imbalanced data set provided by the embodiment of the present invention inputs the fault data into the improved network based on VAE-GAN to obtain a number of newly generated fault data; the several newly generated fault data are added to the imbalanced data set to obtain a balanced data set; wherein the improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a variational autoencoder of the Fourier transform class. The generator provided by the embodiment of the present invention can capture the characteristics of the time domain and the frequency domain at the same time in the latent space, so that the generated fault data has the actual fault characteristics, and the time domain discriminator and the frequency domain discriminator of the embodiment of the present invention can perform double identification on the generated data, ensuring that the generated data matches the actual data at multiple levels, further ensuring the quality of the generated data. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a method for expanding an unbalanced data set provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of FVAE provided by one embodiment of the present invention; Figure 3 is a schematic diagram of the structure of FVAE-TFGAN provided in an embodiment of the present invention; Figure 4 1 is a schematic diagram of training and application of FVAE-TFGAN provided by an embodiment of the present invention; Figure 5 is a confusion matrix under an unbalanced data set provided by an embodiment of the present invention; Figure 6 is a confusion matrix under a balanced data set provided by an embodiment of the present invention; Figure 7 It is a schematic diagram of a device for expanding an unbalanced data set provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.

[0019] It is worth noting that in addition to the oversampling technology and undersampling technology mentioned in the background technology, there are other methods for processing imbalanced data in the prior art, such as cost-sensitive learning algorithms, which solve the data imbalance problem by assigning different error weights to different categories of data. However, although this method can theoretically adjust the model's attention to minority class samples, in actual operation, how to set a reasonable weight ratio is a difficult problem, and the scalability of this method is limited when processing complex data sets with multiple categories.

[0020] It is worth noting that in the field of data enhancement technology, existing research often only focuses on the time domain characteristics of samples, while ignoring the frequency domain information, thus failing to fully utilize the multi-dimensional characteristics of the data. In fact, in the fault diagnosis of rotating mechanical equipment, frequency domain characteristics often play an important role. This is because for many types of bearing faults, specific changes and patterns will occur in the spectrum along with the occurrence of the fault. Therefore, an embodiment of the present invention proposes a method for expanding an unbalanced data set, aiming to fully exploit the time domain characteristics and frequency domain characteristics of minority class samples, thereby generating more diversified fault data with actual fault characteristics.

[0021] See also Figure 1 , is a method for expanding an unbalanced data set provided by an embodiment of the present invention, comprising steps S1-S2: S1. Input the fault data into the improved network based on VAE-GAN to obtain some newly generated fault data; S2. Add the several newly generated fault data to the unbalanced data set to obtain a balanced data set; wherein the improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a Fourier transform-like variational autoencoder.

[0022] It is worth noting that the fault data in step S1 can be understood as minority class data in an unbalanced data set, wherein the fault data can be bearing fault data or other fault data, which is not limited here.

[0023] Furthermore, "an improved network based on VAE-GAN (Variational Autoencoder-Generative Adversarial Network)" is a network proposed in an embodiment of the present invention, and its English abbreviation is FVAE-TFGAN (Fourier Variational Autoencoder-Time Frequency Generative Adversarial Network), and its full Chinese name is "Fourier Transform-like Variational Autoencoder Generator-Time Frequency Discrimination Generative Adversarial Network" (for the convenience of description, it will be referred to as "improved network based on VAE-GAN"). In addition, the improved network based on VAE-GAN consists of a generator, a time domain discriminator and a frequency domain discriminator, wherein the generator is a Fourier Transform-like Variational Autoencoder (FVAE, Fourier Variational Autoencoder).

[0024] Furthermore, in actual application, FVAE will generate new fault data based on the input fault data, thereby expanding the unbalanced data set. Compared with other generators, FVAE can capture the characteristics of both time domain and frequency domain in the latent space, so that the generated fault data has actual fault characteristics.

[0025] Furthermore, the improved network based on VAE-GAN introduces a time domain discriminator and a frequency domain discriminator. The time domain discriminator can evaluate the time domain characteristics of the generated data, while the frequency domain discriminator can evaluate the frequency domain characteristics of the generated data, thereby enabling dual identification of the generated data to ensure that the generated data matches the actual data at multiple levels, thereby ensuring the quality of the generated data and improving the practicality and reliability of the model.

[0026] Compared with the prior art, the method for expanding an unbalanced data set provided in the embodiment of the present invention can generate more diverse fault data with actual fault characteristics and has stronger practicality.

[0027] As one optional implementation, the Fourier transform-like variational autoencoder includes an encoder and a decoder, and the encoder includes a label embedding layer, a first fully connected layer, and a second fully connected layer. Then, the fault data is input into the improved network based on VAE-GAN to obtain a number of newly generated fault data, including: The label embedding layer is used to convert the category label of the fault data to obtain an embedding vector; Merging the embedding vector and the fault data and inputting them into the first fully connected layer and the second fully connected layer respectively, so as to obtain the mean of the real part and the logarithmic variance of the real part through the first fully connected layer, and obtain the mean of the imaginary part and the logarithmic variance of the imaginary part through the second fully connected layer; Reparameterizing the mean of the real part and the standard deviation of the real part to obtain a number of real part latent variables, and reparameterizing the mean of the imaginary part and the standard deviation of the imaginary part to obtain a number of imaginary part latent variables; The decoder reconstructs data according to the cosine variable, the sine variable, the real potential variable and the imaginary potential variable to obtain newly generated fault data.

[0028] It is worth noting that in the actual application process, that is, when the improved network based on VAE-GAN is used to generate new fault data, the input data is the fault data, but when the improved network based on VAE-GAN is trained, the input data includes fault data and normal data, so the following description uses "input data". Furthermore, each input data has a corresponding category label to characterize normal data, fault data and the corresponding fault type. For example, the category label can be "normal data", "0.014-inch inner ring fault", "0.021-inch inner ring fault" and "0.014-inch outer ring fault".

[0029] See also Figure 2 , is a schematic diagram of the structure of FVAE provided by an embodiment of the present invention. FVAE includes an encoder and a decoder, wherein the encoder is composed of a series of linear layers. First, the category label of the input data is converted into a continuous vector representation (embedded vector) through a label embedding layer, and the embedding vector is combined with the input data to predict the mean and logarithmic variance in the latent space through two independent fully connected layers (the first fully connected layer and the second fully connected layer). Mathematically, the encoder converts the input data Converted into two latent space representations, namely the mean With log variance , as shown in formula (1) to formula (5): (1) (2) (3) (4) (5) in, represents the encoder network; Represents the label embedding layer; Represents the category label; Indicates a merge operation; represents the mean of the real part; represents the logarithmic variance of the real part; represents the first weight of the first fully connected layer; represents the first bias of the first fully connected layer; represents the second weight of the first fully connected layer; represents the second bias of the first fully connected layer; represents the mean of the imaginary part; represents the logarithmic variance of the imaginary part; represents the third weight of the second fully connected layer; represents the third bias of the second fully connected layer; represents the fourth weight of the second fully connected layer; Represents the fourth bias of the second fully connected layer.

[0030] Furthermore, in order to facilitate back-propagation and allow stochastic gradient descent optimization, a reparameterization trick is used, as shown in Equation (6): (6) That is: (7) In equations (6) and (7), represents latent variables, including real latent variables and the imaginary latent variable ; represents the mean; represents standard deviation; Represents element-wise multiplication; represents the noise vector sampled from a standard normal distribution, that is, ; represents the exponential function; represents the logarithmic variance.

[0031] Further, the decoder utilizes a sequence of transposed convolutional layers, which incrementally upsample the latent representation back to the original data dimension. The output of the decoder is the input data The reconstruction of (When the input data is faulty data, reconstruct the data is the newly generated fault data), which is mathematically expressed as shown in formula (8): (8) in, Represents the reconstructed data (also the generated data of the generator); represents a decoder; represents latent variables; represents the real part of the latent variable; represents the imaginary latent variable; Represents the length of the discrete signal sequence; represents the cosine variable; represents a sinusoidal variable. Furthermore, the sinusoidal variable Represents the matrix multiplication form of the sine components, cosine variables The matrix multiplication forms representing the cosine components are shown in equations (9) and (10) respectively: (9) (10) From the above content, it can be known that the Fourier transform-like variational autoencoder FVAE provided in an embodiment of the present invention includes an encoder and a decoder, wherein the encoder performs a Fourier transform-like and the decoder performs an inverse Fourier transform-like. Specifically, the encoder outputs the input data in the time domain as the real layer and the imaginary layer (real latent variables and imaginary latent variables) of the latent random variables to generate a representation in the latent space; further, the decoder uses the linear combination of the latent variables and the learned sinusoidal variables , cosine variable Multiply together to simulate the process of inverse Fourier transform and finally obtain the reconstructed data.

[0032] Compared with the prior art, the Fourier transform-like variational autoencoder FVAE provided in the embodiment of the present invention By simulating the characteristics of Fourier transform, more complex and comprehensive time domain features and frequency domain features can be captured in the latent space, thereby ensuring that the generated data is highly consistent with the real data in both time series and frequency domain characteristics, improving the data representation ability, and significantly improving the diversity and authenticity of the generated data.

[0033] As an optional implementation manner, before inputting the fault data into the improved network based on VAE-GAN, the generator, the time domain discriminator and the frequency domain discriminator are trained in an alternating training manner, specifically including: The generator is fixed, and the time domain discriminator and the frequency domain discriminator are iteratively trained using a time domain discriminator loss function and a frequency domain discriminator loss function to obtain an intermediate time domain discriminator and an intermediate frequency domain discriminator; The intermediate time domain discriminator and the intermediate frequency domain discriminator are fixed, and the generator loss function is used to iteratively train the generator to obtain an intermediate generator, and the above-mentioned step of fixing the generator is returned to repeat the alternating training process until an improved network based on VAE-GAN that meets the requirements is obtained.

[0034] See also Figure 3 , which is a schematic diagram of the structure of FVAE-TFGAN provided by an embodiment of the present invention. During the training process of FVAE-TFGAN, the time domain discriminator and the frequency domain discriminator learn the difference between the generated data and the real data, and pass the difference to the generator to continuously adjust the generator to perform adversarial training on the generation process until the discriminator can no longer recognize that the generated data is significantly different from the real data.

[0035] It is worth noting that in the training process of FVAE-TFGAN, the strategy of alternating training of the generator and the discriminator is used: the generator is fixed and the discriminator loss function is minimized. To update the parameters of the two discriminators, in order to improve the ability of the two discriminators to distinguish between real data and generated data in the time domain and frequency domain; then, fix the two discriminators and minimize the generator loss function To update the parameters of the generator so that the generator generates more realistic data, the training is repeated until the requirements are met.

[0036] Furthermore, the training process of the discriminator is through the joint feedback of the frequency domain discriminator and the time domain discriminator, using the binary cross-entropy loss function (BCE loss function, Binary Cross-Entropy Loss) To quantify the discriminator's ability to judge generated data With real data When the ability is high, for a data to be identified, record the label of the real data , the label of the generated data is Specifically, the binary cross entropy loss function The expression of is shown in formula (11): (11) in, represents the binary cross entropy loss function; represents the true label; Represents the label prediction value output by the discriminator; Represents a logarithmic function.

[0037] As an optional implementation manner, the time domain discriminator loss function and the frequency domain discriminator loss function are respectively shown as follows: (12) (13) in, represents the temporal discriminator loss function; represents the frequency domain discriminator loss function; represents the binary cross entropy loss function; represents the time domain discriminator; Represents real data; Indicates the generation of data; represents the frequency domain discriminator; Represents short-time Fourier transform.

[0038] Furthermore, the discriminator loss function As shown in formula (14): (14) The training of the generator FVAE adjusts the parameters through back propagation according to the feedback from the discriminator. During the training process of the generator FVAE, the loss of the generator must not only consider the reconstruction error, but also the discrimination result from the discriminator. The dual discrimination mechanism of the two discriminators enables the generator to generate more realistic data to reduce the probability of being identified as "fake data" by the discriminator.

[0039] As an optional implementation, the generator loss function is shown as follows: (15) in, represents the generator loss function; represents the reconstruction loss function; Represents the KL divergence (Kullback-Leibler divergence, relative entropy) function; represents the adversarial loss function; Represents input data; Represents reconstructed data; and Represents a hyperparameter, which is used to balance the weights of different loss terms; represents the mean; represents standard deviation; The adversarial loss function is shown as follows: (16) in, represents the binary cross entropy loss function; represents the adversarial loss function of the temporal discriminator; represents the adversarial loss function of the frequency domain discriminator; represents the time domain discriminator; represents the frequency domain discriminator; Represents short-time Fourier transform.

[0040] For example, the reconstruction loss function is You can choose mean squared error (MSE) as the loss function; It represents the KL divergence between the learned potential distribution and the prior distribution, which is usually assumed to be a standard normal distribution. .

[0041] Furthermore, the frequency domain discriminator (FD) performs a short-time Fourier transform (STFT) on the data to be identified (generated data / real data) to obtain enhanced frequency domain data and analyze the difference in frequency distribution of the data to be identified, and outputs a discrimination result (true or false). The frequency domain discriminator is built using a convolutional neural network (CNN). The time domain discriminator (TD) is used to determine the authenticity of the data to be identified in the time series and output a discrimination result (true or false).

[0042] As an optional implementation manner, the identification process of the time domain identifier on the data to be identified includes: The data to be identified and its label are fused through a time domain discriminator embedding layer to obtain first fused data; wherein the data to be identified is real data or generated data; Input the first fused data into a three-layer time domain discriminator fully connected layer for step-by-step dimension reduction, and output a time domain discriminant value through a time domain discriminator activation function, wherein the time domain discriminant value is used to indicate whether the data to be identified is real data; The identification process of the frequency domain identifier on the data to be identified includes: Performing Fourier transform on the data to be identified to obtain frequency domain data; fusing the frequency domain data and the label through a frequency domain discriminator embedding layer to obtain second fused data; The second fused data is input into the three-layer frequency domain discriminator fully connected layer for step-by-step dimension reduction, and a frequency domain discriminant value is output through the frequency domain discriminator activation function, and the frequency domain discriminant value is used to characterize whether the data to be identified is real data.

[0043] It is understandable that the same data to be identified must be identified by the time domain discriminator and the frequency domain discriminator, and both the time domain discriminator and the frequency domain discriminator will feed back the identification results to the generator. Through the collaborative work of the two discriminators, the ability to distinguish the generated data can be enhanced.

[0044] As an optional implementation, after obtaining the balanced data set, the method further includes using the balanced data set to train a pre-built CNN-LSTM (Convolutional Neural Networks- Long Short-Term Memory) network to obtain a trained bearing fault diagnosis model.

[0045] As can be known from the background technology, using an unbalanced data set to train a fault diagnosis model will cause the fault diagnosis model to be biased towards the majority class data. To avoid the above problem, the balanced data set generated by the embodiment of the present invention can be used as the training data of the fault diagnosis model to improve the diagnostic accuracy and robustness of the fault diagnosis model. Specifically, see Figure 4 , is a schematic diagram of the training and application of FVAE-TFGAN provided by an embodiment of the present invention. Figure 4 It can be seen that, first, the original vibration signal dataset is divided into a training dataset and a test dataset, an unbalanced dataset is constructed according to the training dataset and used to train FVAE-TFGAN to obtain a trained FVAE-TFGAN, and then the fault data in the unbalanced dataset is input into the trained FVAE-TFGAN to obtain newly generated fault data, and the newly generated fault data is added to the unbalanced dataset to obtain a balanced dataset; further, the balanced dataset is input into the CNN-LSTM network for training to obtain a trained CNN-LSTM network; finally, the test dataset is used to verify the bearing fault classification effect of the CNN-LSTM network.

[0046] Exemplarily, the CNN-LSTM network adopts a small batch training method, and the size of each batch is set to 32; further, because the Adam optimizer has the characteristic of adaptively adjusting the learning rate and has strong convergence and stability, the Adam optimizer can be selected to optimize the network parameters; further, the cross entropy loss function can also be used to optimize the accuracy of multi-classification problems, and the Dropout technology can be introduced to prevent network overfitting.

[0047] As an optional implementation, the CNN-LSTM network includes a convolution layer, a maximum pooling layer, an LSTM layer and an output layer. Then, the fault diagnosis process of the CNN-LSTM network includes: The convolutional layer is used to extract local spatiotemporal features of bearing data; Using the maximum pooling layer to perform dimensionality reduction processing on the local spatiotemporal features to obtain dimensionality reduction features; Using the LSTM layer to perform time series information processing on the dimensionality reduction features to obtain high-dimensional features; The output layer is used to map the high-dimensional features into probability distribution and output the bearing fault type.

[0048] Exemplarily, the CNN-LSTM network is composed of multiple convolutional layers, each of which extracts the local spatiotemporal features of the bearing vibration signal (bearing data) through different numbers of convolution kernels, and then is equipped with batch normalization and ReLU activation functions to enhance learning ability and stability, and the spatial dimension of the feature map is reduced through the maximum pooling layer to reduce the amount of calculation. After extracting the features, the CNN-LSTM network introduces two LSTM layers, and the output of each layer is passed to the fully connected layer. The first layer contains 128 units to capture long-term time dependencies, and the second layer contains 64 units to further mine complex patterns in time series. Finally, through an output layer with the same number of diagnostic categories and a softmax activation function, the high-dimensional features are mapped to the probability distribution of various fault types and normal data, realizing accurate classification diagnosis of multiple categories. The CNN-LSTM network of the embodiment of the present invention ensures the high accuracy and robustness of bearing fault diagnosis by combining the feature extraction capability of the convolutional layer with the time series modeling advantage of the LSTM layer.

[0049] Compared with the prior art, the expansion method of the imbalanced data set provided by the embodiment of the present invention inputs the fault data into the improved network based on VAE-GAN to obtain a number of newly generated fault data; the several newly generated fault data are added to the imbalanced data set to obtain a balanced data set; wherein the improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a variational autoencoder of the Fourier transform class. The generator provided by the embodiment of the present invention can capture the characteristics of the time domain and the frequency domain at the same time in the latent space, so that the generated fault data has the actual fault characteristics, and the time domain discriminator and the frequency domain discriminator of the embodiment of the present invention can perform double identification on the generated data, ensuring that the generated data matches the actual data at multiple levels, further ensuring the quality of the generated data.

[0050] In order to verify the effectiveness of the expansion method of the unbalanced data set provided in the embodiment of the present invention, the following experiments are compared with several representative existing data enhancement methods, including: FLT (Fourier-Like Transformer), VAE-GAN, SMOTE (Synthetic Minority Over-sampling Technique) and DCGAN (Deep Convolutional Generative Adversarial Network).

[0051] The variational autoencoder generative adversarial network (VAE-GAN) combines the VAE (Variational Autoencoder) network and generates new data by sampling from its latent space. The network can generate more minority class samples by controlling the sampling process, and finally uses the discriminator to distinguish the data generated by VAE from the real data until the discriminator cannot distinguish them. Therefore, this network is used as the ablation experiment provided by the embodiment of the present invention.

[0052] The synthetic minority oversampling technique (SMOTE) is a widely used classic machine learning method in the field of data enhancement with imbalanced classes. It includes several steps: first, identify the minority class that needs to be oversampled to balance the data set; for each instance in the minority class, SMOTE finds its k-nearest neighbors (k is usually 5); then generate a synthetic instance by selecting one of the k-nearest neighbors and drawing a line between them in the feature space. This method not only creates more diverse examples in the minority class, but also mixes the features of the two instances, thereby providing a richer data set for model training while maintaining the overall information. Therefore, this method is used as a comparative method for the embodiments of the present invention.

[0053] The Deep Convolutional Generative Adversarial Network (DCGAN) is a variant of GAN (Generative Adversarial Networks) and is also a widely used deep neural network algorithm in the field of data augmentation. DCGAN modifies the standard GAN architecture by incorporating convolutional layers, which are commonly found in deep learning models for image processing. Like GAN, DCGAN consists of a generator and a discriminator, and both components use convolutional neural networks (CNNs), where the generator takes a random noise vector as input and generates data that is similar to minority class samples in terms of features and distribution. This is achieved through deconvolution layers, which gradually enlarge the input vector into a complete data instance; the discriminator is trained to distinguish between original data and synthetic data generated by the generator. This model helps alleviate the problem of overfitting to majority class samples, and it also enhances the generalization ability of the model across various fields, but the method still encounters problems such as mode collapse and stability during training.

[0054] Furthermore, the data set of this experiment uses the bearing time domain data set of Case Western Reserve University (CWRU), which is an authoritative representative in the field of bearing fault diagnosis. This data set uses data collected by the bearing experimental platform of Case Western Reserve University, which consists of a 1.5kW (kilowatt) motor, a torque sensor, and a power tester. Each fault type is classified according to its size (diameter ranges from 0.007 inches to 0.040 inches), and under different motor loads from 0 to 3 horsepower, corresponding to motor speeds of 1797 to 1720 RPM (Revolutions Per Minute), various bearing fault signals are artificially obtained using electrical discharge machining (EDM) to obtain the bearing time domain data set. The bearing time domain data set mainly includes normal baseline data (normal data under different loads and speeds), a fault data set with a sampling frequency of 12kHz (Kilohertz), and a fault data set with a sampling frequency of 48kHz. This experiment mainly used the fault data set with a sampling frequency of 12kHz and the normal baseline data.

[0055] According to the bearing structure, the fault categories can be divided into inner ring fault, outer ring fault and ball fault. The above three fault types each include three damage sizes of 0.007 inches, 0.014 inches and 0.021 inches. Therefore, there are a total of 9 fault categories. Together with the normal data, the bearing time domain data set includes a total of ten categories of data.

[0056] The bearing time domain data set includes 20,000 (time domain) data. 6,000 data are used as test data according to the ratio of 3:7, and the rest of the data are used as training data. In the training data, in order to simulate the unbalanced data set, four unbalanced ratios are pre-set, namely "1:2", "1:5", "1:10" and "1:20", which are used to indicate the ratio relationship between each type of fault data and normal data in the unbalanced data set. According to the above four unbalanced ratios, data sets A~D are divided, and finally the redundant data are deleted. The data composition of each data set is shown in Table 1.

[0057] Table 1 Data composition of each data set in the verification experiment It is understandable that although accuracy is a commonly used evaluation indicator in fault diagnosis classification tasks, it cannot fully reflect the classification performance of each independent category. Based on this, this experiment introduced the F1 score and AUC (Area Under Curve) value, which are used together with accuracy (ACC) to evaluate the fault diagnosis model's ability to identify multiple fault types.

[0058] Specifically, FLT, the proposed FVAE generator and VAEGAN are used as ablation experiments of the embodiments of the present invention, and the SMOTE technology, DCGAN and VAEGAN's generative adversarial network are selected as comparative experiments of this embodiment. The experimental results are shown in Tables 2 and 3.

[0059] Table 2 Ablation experiment Table 3 Comparative experiment From Table 2 and Table 3, it can be analyzed that although the latent space is added to the original FLT, the FVAE generator does not significantly improve the effect, but the improved network based on VAE-GAN (FVAE-TFGAN) significantly improves the diagnostic performance by applying dual discriminators in the time domain and frequency domain, and the effect is significantly better than the original VAEGAN and SMOTE technology. In addition, the embodiment of the present invention uses a CNN-LSTM network in the fault diagnosis model, which can perform more accurate diagnosis on balanced data sets, and its performance is also better than a single convolutional neural network.

[0060] Furthermore, by analyzing the experimental results of the four data sets, it can be found that the judgment ability of the fault diagnosis model trained by the extremely unbalanced data set after data enhancement is reduced, resulting in the final diagnosis result being inferior to that of the unbalanced data set with a smaller ratio. This shows that when enhancing the data, the original ratio and structure of the data set should be considered to avoid excessive enhancement that affects the diagnostic effect of the model.

[0061] Furthermore, in order to analyze the effect of the FVAE-TFGAN data augmentation algorithm on fault diagnosis, the CNN-LSTM network was trained with the dataset D, and the confusion matrix was used to further verify the effect of FVAE-TFGAN (the confusion matrix shows the diagnostic accuracy and sensitivity of the classification model for each fault category). Figure 5 and Figure 6 It can be seen that the embodiment of the present invention can improve the recognition effect on minority class samples, and the feature clustering effect is more obvious.

[0062] See also Figure 7 The embodiment of the present invention further provides an expansion device 10 for an unbalanced data set, comprising: A fault data generation module 11 is used to input the fault data into the improved network based on VAE-GAN to obtain a number of newly generated fault data; The data set construction module 12 is used to add the several newly generated fault data to the unbalanced data set to obtain a balanced data set; wherein the improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a variational autoencoder of the Fourier transform type.

[0063] The device for expanding an unbalanced data set provided in an embodiment of the present invention can implement all the process steps of the method for expanding an unbalanced data set described in the above embodiment. The functions of each module and unit in the device and the technical effects achieved are respectively the same as the functions and technical effects achieved by the method for expanding an unbalanced data set described in the above embodiment, and the specific implementation method is not repeated here.

[0064] To achieve the above objectives, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method for expanding the unbalanced data set as described in any of the above embodiments.

[0065] Compared with the prior art, the expansion device and storage medium of the imbalanced data set provided by the embodiment of the present invention inputs the fault data into the improved network based on VAE-GAN to obtain a number of newly generated fault data; the several newly generated fault data are added to the imbalanced data set to obtain a balanced data set; wherein the improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a variational autoencoder of the Fourier transform class. The generator provided by the embodiment of the present invention can capture the characteristics of the time domain and the frequency domain at the same time in the latent space, so that the generated fault data has the actual fault characteristics, and the time domain discriminator and the frequency domain discriminator of the embodiment of the present invention can perform double identification on the generated data, ensuring that the generated data matches the actual data at multiple levels, further ensuring the quality of the generated data.

[0066] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for expanding an imbalanced dataset. It is characterized in that include: Input the fault data into the improved network based on VAE-GAN to obtain some newly generated fault data; Adding the plurality of newly generated fault data to the unbalanced data set to obtain a balanced data set; The improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a Fourier transform-like variational autoencoder.

2. The method for expanding an imbalanced data set according to claim 1, It is characterized in that The Fourier transform-like variational autoencoder includes an encoder and a decoder, and the encoder includes a label embedding layer, a first fully connected layer, and a second fully connected layer. Then, the fault data is input into the improved network based on VAE-GAN to obtain a number of newly generated fault data, including: The label embedding layer is used to convert the category label of the fault data to obtain an embedding vector; Merging the embedding vector and the fault data and inputting them into the first fully connected layer and the second fully connected layer respectively, so as to obtain the mean of the real part and the logarithmic variance of the real part through the first fully connected layer, and obtain the mean of the imaginary part and the logarithmic variance of the imaginary part through the second fully connected layer; Reparameterizing the mean of the real part and the standard deviation of the real part to obtain a number of real part latent variables, and reparameterizing the mean of the imaginary part and the standard deviation of the imaginary part to obtain a number of imaginary part latent variables; The decoder reconstructs data according to the cosine variable, the sine variable, the real potential variable and the imaginary potential variable to obtain newly generated fault data.

3. The method for expanding an imbalanced data set according to claim 1, It is characterized in that Before inputting the fault data into the improved network based on VAE-GAN, the generator, the time domain discriminator and the frequency domain discriminator are trained in an alternating training manner, specifically including: The generator is fixed, and the time domain discriminator and the frequency domain discriminator are iteratively trained using a time domain discriminator loss function and a frequency domain discriminator loss function to obtain an intermediate time domain discriminator and an intermediate frequency domain discriminator; The intermediate time domain discriminator and the intermediate frequency domain discriminator are fixed, and the generator loss function is used to iteratively train the generator to obtain an intermediate generator, and the above-mentioned step of fixing the generator is returned to repeat the alternating training process until an improved network based on VAE-GAN that meets the requirements is obtained.

4. The method for expanding an imbalanced data set according to claim 3, It is characterized in that The generator loss function is as follows: in, represents the generator loss function; represents the reconstruction loss function; represents the KL divergence function; represents the adversarial loss function; Represents input data; Represents reconstructed data; and represents a hyperparameter; represents the mean; represents standard deviation; The adversarial loss function is shown as follows: in, represents the binary cross entropy loss function; represents the time domain discriminator; represents the frequency domain discriminator; Represents short-time Fourier transform.

5. The method for expanding an imbalanced data set as claimed in claim 3, It is characterized in that The time domain discriminator loss function and the frequency domain discriminator loss function are respectively shown as follows: in, represents the temporal discriminator loss function; represents the frequency domain discriminator loss function; represents the binary cross entropy loss function; represents the time domain discriminator; Represents real data; Indicates the generation of data; represents the frequency domain discriminator; Represents short-time Fourier transform.

6. The method for expanding an imbalanced data set according to claim 3, It is characterized in that The identification process of the time domain discriminator for identifying data includes: The data to be identified and its label are fused through a time domain discriminator embedding layer to obtain first fused data; wherein the data to be identified is real data or generated data; Input the first fused data into a three-layer time domain discriminator fully connected layer for step-by-step dimension reduction, and output a time domain discriminant value through a time domain discriminator activation function, wherein the time domain discriminant value is used to indicate whether the data to be identified is real data; The identification process of the frequency domain identifier on the data to be identified includes: Performing Fourier transform on the data to be identified to obtain frequency domain data; fusing the frequency domain data and the label through a frequency domain discriminator embedding layer to obtain second fused data; The second fused data is input into the three-layer frequency domain discriminator fully connected layer for step-by-step dimension reduction, and a frequency domain discriminant value is output through the frequency domain discriminator activation function, and the frequency domain discriminant value is used to characterize whether the data to be identified is real data.

7. The method for expanding an imbalanced data set according to claim 1, It is characterized in that After obtaining the balanced data set, the method further includes using the balanced data set to train a pre-built CNN-LSTM network to obtain a trained bearing fault diagnosis model.

8. The method for expanding an imbalanced data set according to claim 7, It is characterized in that The CNN-LSTM network includes a convolution layer, a maximum pooling layer, an LSTM layer and an output layer. Then, the fault diagnosis process of the CNN-LSTM network includes: The convolutional layer is used to extract local spatiotemporal features of bearing data; Using the maximum pooling layer to perform dimensionality reduction processing on the local spatiotemporal features to obtain dimensionality reduction features; Using the LSTM layer to perform time series information processing on the dimensionality reduction features to obtain high-dimensional features; The output layer is used to map the high-dimensional features into probability distribution and output the bearing fault type.

9. An expansion device for an imbalanced data set, It is characterized in that include: A fault data generation module is used to input the fault data into the improved network based on VAE-GAN to obtain a number of newly generated fault data; A data set construction module is used to add the several newly generated fault data to an unbalanced data set to obtain a balanced data set; wherein the improved network based on VAE-GAN includes a generator, a time domain discriminator and a frequency domain discriminator, and the generator is a Fourier transform-like variational autoencoder.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for expanding an unbalanced data set according to any one of claims 1 to 8.

Images