Elevator vibration signal data enhancement method based on improved deep convolutional generative adversarial network

Abstract

An elevator vibration signal data enhancement method based on an improved deep convolutional generative adversarial network, comprising the following steps: first step: collecting original vibration signals of elevator car running acceleration; second step: obtaining effective vibration signals; third step: optimizing the DCGAN network model, introducing a spectral normalization and space-time attention mechanism module into the discriminator and generator to form an improved data enhancement overall model; fourth step: inputting a two-dimensional time-frequency feature image into the data enhancement overall model to obtain a generated new feature image. The application enriches feature information and reduces the probability of'mode collapse' to a certain extent; according to the characteristics of a small amount of data set, more new data sets are generated to achieve the purpose of data enhancement, and the pixel distribution of the elevator high-resolution image can be well learned.

Description

Technical Field

[0001] This invention relates to the field of elevator data processing technology, specifically to a method for enhancing elevator vibration signal data based on an improved deep convolutional generative adversarial network. Background Technology

[0002] With the development of computer technology and hardware upgrades, deep learning has been widely applied in the field of data preprocessing. In the elevator industry, the elevator car is a crucial component, and its safe operation directly affects the stability of the entire elevator system. Currently, data enhancement of elevator vibration signals has attracted widespread attention, but research on it both domestically and internationally is relatively limited. The main reasons for this are as follows:

[0003] (1) Vibration data during elevator operation is difficult to obtain. Acquiring reliable vibration signal data requires actual operation, which necessitates extensive experimentation and testing. This includes steps such as sensor installation, data acquisition, and algorithm verification. Therefore, limitations in experimental conditions may restrict data acquisition and in-depth research. This can also lead to a severe imbalance in the sample size of related datasets.

[0004] (2) The vibration signal of the elevator is highly complex, and the training of deep learning models based on elevator vibration signal data augmentation is difficult and inefficient.

[0005] In deep learning networks, structures with multiple hidden layers require training a large number of free parameters, which can lead to overfitting. Therefore, having high-quality and sufficient training set data is crucial for achieving good data preprocessing results. However, obtaining large-scale, high-quality real-world sample data is often challenging.

[0006] Traditional signal processing methods include stochastic resonance, synchronous compression transform, time-frequency analysis, and decision methods. The stochastic resonance method adjusts the intensity and frequency of noise in the system to create resonance between the noise and the periodic frequency of the signal. When the energy of the noise and the signal's periodic frequency matches, the resonance effect makes the signal easier to detect and extract in noisy environments. The synchronous compression transform method leverages the sparsity and frequency domain characteristics of the signal to compress data while preserving signal information. Time-frequency analysis is used to analyze non-stationary signals, providing characteristic information in both time and frequency dimensions. Decision methods analyze and process vibration signals to make relevant decisions. While these methods are helpful in solving imbalanced dataset problems, their practical effectiveness is not ideal.

[0007] Generative Adversarial Networks (GANs), proposed by Goodfellow et al. in 2014, have become a mainstream method for addressing data scarcity. They have been applied in many fields, and there has been considerable research on improvements to GANs. However, due to the complexity of elevator vibration signals, data augmentation methods for these signals have not yet achieved satisfactory results. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, this invention proposes an elevator vibration signal data enhancement method based on an improved deep convolutional generative adversarial network (DCGAN). First, a continuous wavelet transform method is used to extract two-dimensional time-frequency image features from the original time-domain vibration signal of the elevator. Then, an improved DCGAN model is proposed, which uses spectral normalization (SN) and a space-time attention mechanism (STA) to optimize the discriminator and generator of the DCGAN to generate more fake image samples, thereby addressing the problem of insufficient training data.

[0009] This invention provides the following technical solution:

[0010] A method for augmenting elevator vibration signal data based on an improved deep convolutional generative adversarial network, the method comprising the following steps:

[0011] Step 1: Collect the raw vibration signal of the elevator car's acceleration;

[0012] Step 2: Obtain valid vibration signals;

[0013] Step 3: Optimize the DCGAN network model by introducing the spectrum normalization and spatiotemporal attention mechanism modules into the discriminator and generator to form an improved overall data augmentation model.

[0014] DCGAN is a network model proposed by Radford. Its architecture mainly modifies Convolutional Neural Networks (CNNs). In the generator, images are generated by transposing convolutional layers. Conversely, in the discriminator, the obtained image is taken as input, operated on using standard convolutions, and the displayed binary classification result is taken as output.

[0015] Convolutional neural networks (CNNs) are deep learning models or multilayer perceptrons similar to artificial neural networks, proposed by Yann LeCun. They are often used to analyze visual images. Convolutional layers are the hierarchical structures in CNNs.

[0016] Step 4: Input the two-dimensional time-frequency feature image into the data to enhance the overall model and obtain the generated new feature image.

[0017] Furthermore, the first step is as follows: an accelerometer is installed in the elevator car to measure the vibration signal during its operation and the real-time operation data of the elevator in the sensor is remotely acquired using the TCP / IP communication protocol;

[0018] The sensors are connected to the elevator and gateway via serial ports and communicate with each other using Modbus and RS485 protocols.

[0019] Furthermore, in the second step, the process of obtaining a valid vibration signal is as follows:

[0020] Step (2.1) Calculate the time-domain characteristic index of the elevator car vibration acceleration signal using the time-domain analysis method;

[0021] Step (2.2) segments the vibration signal processed in step (2.1), samples it using a sliding window containing M data points, and then slides the window backward by more than M data points to extract the original signal, finally obtaining the effective vibration signal;

[0022] Step (2.3) extracts the time-frequency features of the signal through continuous wavelet transform:

[0023]

[0024]

[0025] Where, ψ a,b It is a wavelet basis function with two parameters a and b; W f (a,b) represents the coefficients of the wavelet transform, which are obtained by integrating the wavelet basis function ψ(x*) over the noise signal F(x*). denoted by ψ(x*), where a is the translation factor and b is the scaling factor; ∫ represents the integral operation, commonly used in mathematics and statistics.

[0026] Among them, continuous wavelet transform is the most widely used method for time-domain and frequency-domain feature extraction.

[0027] Step (2.4) combines wavelet coefficients at different frequencies to obtain a two-dimensional time-frequency image.

[0028] Preferably, step (2.1) includes the following sub-steps:

[0029] Step (2.1.1) calculates the peak-to-peak value X3 in the vibration signal sample, using the following formula:

[0030] X1 = max(x i )

[0031] X2=min(x i )

[0032] X3 = X1 - X2

[0033] Where, x i The sampled values ​​of the vibration acceleration signal are given, and max is the value of the data x. i The maximum value, min is the value of the data x. i The minimum value of X1 is the maximum value in the signal sample, and the minimum value of X2 is the minimum value in the signal sample.

[0034] Among them, peak-to-peak value is used to describe the range of vibration signal changes within one cycle, but the car vibration signal is a non-periodic signal, so it is used to describe the difference between the maximum and minimum values ​​in the sample;

[0035] Step (2.1.2) calculates the mean X4 and root mean square X5 of the vibration signal samples, using the following formula:

[0036]

[0037]

[0038] Where N is the number of sampling points, and ∑ is the summation operation, which is usually used in mathematics and statistics to represent the addition of a series of values;

[0039] Among them, the mean value reflects the change of vibration signal due to the change of the shaft position during elevator operation; the root mean square value, also known as the effective value, reflects the energy intensity and stability of the vibration signal.

[0040] Step (2.1.3) calculates the standard deviation X6 in the vibration signal samples, using the following formula:

[0041]

[0042] Where, x i X4 represents the sampled value of the vibration acceleration signal, and X4 is the mean value.

[0043] Step (2.1.4) calculates the kurtosis X7 and skewness X8 in the vibration signal sample, using the following formula:

[0044]

[0045]

[0046] Where, x i X4 represents the sampled value of the vibration acceleration signal, X6 represents the mean, kurtosis reflects the impact characteristics of the vibration signal, and skewness reflects the degree of skewness in the data distribution.

[0047] Step (2.1.5) calculates the margin index X9 and waveform index X in the vibration signal sample. 10 Pulse index X 11 Peak index X 12 The formula is as follows:

[0048]

[0049]

[0050]

[0051]

[0052] Where, |x i |for x i The absolute value, margin index, waveform index, and pulse index all reflect the proportion of the impact signal in the vibration signal.

[0053] Step (2.4) includes the following sub-steps:

[0054] Step (2.4.1) selects Morlet wavelet as the basis function, first determines the initial scaling factor and translation factor, and then analyzes the wavelet from high frequency to low frequency. The first value of the scaling factor corresponds to the wavelet with the highest degree of compression. As the wavelet factor increases, the wavelet will also increase.

[0055] The Morlet wavelet basis function expression is as follows:

[0056]

[0057] The Morlet wavelet basis function is composed of a complex trigonometric function multiplied by an exponential decay function, where j represents an imaginary number, ω0 represents the center frequency, t is the time variable, and e is a mathematical constant.

[0058] Step (2.4.2) follows the continuous wavelet transform formula in step (2.3), multiplying the input signal by the wavelet and integrating to obtain the corresponding wavelet coefficients W. f (a,b);

[0059] Step (2.4.3) Change the translation factor until the end of the signal, and multiply the integral result by a constant to normalize it;

[0060] Step (2.4.4) changes the scaling factor and continuously expands the wavelet. Steps (2.4.2)-(2.4.3) are repeated to finally generate the time-frequency image I of the wavelet.

[0061] The process of the third step is as follows;

[0062] Step (3.1) The discriminator in the DCGAN network introduces spectral normalization;

[0063] The discriminator is the discriminator network in the deep convolutional generative adversarial network. It is mainly responsible for distinguishing whether the input data is real or generated by the generator and feeding it back to the generator.

[0064] Among them, spectral normalization is a function that can make the discriminator more stable;

[0065] Step (3.2) introduces a spatiotemporal attention mechanism into the generator in the DCGAN network;

[0066] The generator is a generative network in deep convolutional generative adversarial networks. It learns the features of the training set data and, under the guidance of the discriminator, tries to fit the distribution of random noise as closely as possible to the true distribution of the training data, thereby generating similar data with the features of the training set.

[0067] Among them, the spatiotemporal attention mechanism is an image processing technique that learns the degree of attention to different spatiotemporal locations, thereby enabling the model to capture key spatiotemporal information more effectively.

[0068] The generator consists of a spatiotemporal attention mechanism and four deconvolutional layers (Deconv1, Deconv2, Deconv3, Deconv4).

[0069] Step (3.1) includes the following sub-steps:

[0070] Step (3.1.1) constructs a discriminator using convolutional layers and a recurrent structure, including 5 convolutional layers with a kernel size of 5×5, a stride of 2, and channel counts of 32, 64, 128, 256, and 512, respectively. A spectral normalization layer is then added outside each convolutional layer of the discriminator to normalize the spectrum of the entire discriminator, thereby providing more stable training performance.

[0071] Normalization, in particular, limits the preprocessed data to a certain range, thereby eliminating the adverse effects caused by outlier data.

[0072] Step (3.1.2) imposes a constraint on the spectral norm of each convolutional layer to satisfy Lipschitz = 1;

[0073] For a real matrix A, its spectral norm is defined as follows:

[0074]

[0075] Among them, A T It is the transpose of matrix A. eig(A) represents a function that computes the eigenvalues ​​of matrix A, and returns the vector [λ1, λ2, ..., λ]. n ] T , where λ k This represents the k-th eigenvalue;

[0076] Among them, Lipschitz is the Lipschitz continuity condition, which is a smoothness condition with stronger smoothness than the usual continuity condition;

[0077] In step (3.1.3), the activation function in the discriminator is a LeakyReLU with slope m that satisfies Lipschitz continuity. The spectral norm of the parameter matrix of each layer is obtained by solving layer by layer using the power iteration method, as shown in the following formula:

[0078]

[0079]

[0080] LeakyReLU is an activation function that can solve the vanishing gradient problem.

[0081] Where, sup represents the least upper bound, h represents the number of nodes in the hidden layer; σ(A) is the largest singular value of matrix A, |||² represents the L2 norm of the matrix, and is the square root of the largest eigenvalue of the matrix; C SN C* is the normalized convolutional layer parameter matrix with a maximum singular value of 1, and σ(C*) is the spectral norm of the convolutional layer parameter matrix C*.

[0082] Among them, the power iteration method is mainly used to approximate the solution of the largest eigenvalue of a matrix;

[0083] Step (3.2) includes the following sub-steps:

[0084] In step (3.2.1), the time-frequency image I in step (2.4.4) is first reshaped in dimension by the Reshape function, and then processed by the Deconv1 and Deconv2 deconvolution layers in sequence.

[0085] The Reshape function is a reshaping function used to reset the dimensions of the array. The Deconv1 and Deconv2 deconvolutional layers are structures in the DCGAN model.

[0086] Step (3.2.2) calculates X from the time-frequency image I in step (2.4.4) using the Reshape function and two deconvolution layers. cam (1×N*×C) is input into the STA module.

[0087] Where N* represents the dimension, and C represents the C-th channel in the local input;

[0088] The output Y after feature fusion in step (3.2.3) tam (1×N*×C) is calculated sequentially through Deconv3 and Deconv4 deconvolution layers to obtain the final result; in addition, ReLU function and BN are added to each layer, and hyperbolic tangent (Tanh) activation function is used as the output layer;

[0089] ReLU is a commonly used linear rectified activation function in neural networks, while Tanh is a type of hyperbolic function in mathematics.

[0090] Step (3.2.2) includes the following sub-steps:

[0091] Step (3.2.2.1) in the time dimension first involves X... cam Using (1×N*×C) as the input signal, X is obtained by transposing it. trans (1×C×N*);

[0092] Here, transpose represents the matrix transpose.

[0093] Step (3.2.2.2)X trans Q is obtained by (1×C×N*) through the convolutional layer CONV1(1×1×N*). cam (1×C×1);

[0094] Convolution is an effective method for extracting image features. It uses a square convolution kernel that slides across the input feature map with a specified stride.

[0095] For each step, the convolution kernel will overlap with the input feature map. The elements corresponding to the overlapping areas are multiplied, summed, and then the bias term is added to obtain a pixel of the output feature.

[0096] Among them, the bias term is an additional parameter outside the initial weight matrix of the network, which is responsible for adjusting the activation function of the neural network to better adapt it to the data features; the pixel is the basic building block of the smallest unit in the image.

[0097] In step (3.2.2.2), the convolution kernel size is 1×1, the number of channels is N*, and the stride is 1.

[0098] Step (3.2.2.3) Q cam(1×C×1) The normalized weight S is obtained through the SoftMax function. cam (1×C×1);

[0099] The SoftMax function can convert the output values ​​of multi-class classification into a probability distribution in the range [0, 1].

[0100] Step (3.2.2.4)S cam (1×C×1) is converted to S by the Transpose function. tras (1×1×C);

[0101] Transpose is a transpose function used to transpose high-dimensional arrays.

[0102] Step (3.2.2.5) processes the input signal X in the spatial dimension through two branches. cam (1×N*×C);

[0103] Step (3.2.2.6) will transfer the feature map S trans and T tam Perform feature fusion to obtain Y tam (1×N*×C);

[0104] Feature fusion is a common operation performed on features extracted from images, which combines multiple features into a single feature that is more discriminative than the input features.

[0105] In step (3.2.2.6), the feature fusion adopts a series of feature fusion methods to connect two features. If the dimensions of the two input features are p* and q*, the dimension of the output feature z is p*+q*.

[0106] Step (3.2.2.5) includes the following sub-steps:

[0107] Step (3.2.2.5.1) Input signal X cam The feature map P is obtained by calculating (1×N*×C) through the convolutional layer CONV2(1×1×1). tam (1×N*×C);

[0108] In step (3.2.2.5.1), the convolution kernel size is 1×1, the channel is 1, and the stride is 1.

[0109] Step (3.2.2.5.2) Input signal X cam The feature map Q is obtained by computing (1×N*×C) through a convolutional layer CONV3 (1×1×1). tam After (1×N*×C), the spatial attention map S is calculated and obtained using the SoftMax function and spatial attention mechanism.tam (1×N*×C);

[0110] The spatial attention map S is calculated and obtained through the SoftMax function. tam The position weights are (1×N*×C), and the formula is as follows:

[0111]

[0112] in, Used to measure the i* position pair feature map P tam and Q tam The impact, It is the feature map Q tam The element at position i*;

[0113] Where exp is an exponential function with the natural number e as its base;

[0114] In step (3.2.2.5.2), the convolution kernel size is 1×1, the channel is 1, and the stride is 1.

[0115] Among them, the spatial attention map S is calculated and obtained through the attention mechanism. tam The weighting coefficient M of (1×N*×C) s (Q tam The formula is as follows:

[0116]

[0117] Among them, Q tam The input features are processed and concatenated by an average pooling layer (Avgpool()) and a max pooling layer (Maxpool()), and then passed through a convolutional layer (f). 7*7 The weighting coefficient M is calculated. s (Q tam );

[0118] Among them, f 7*7 This represents a convolutional layer with a kernel size of 7×7. This is a description obtained after average pooling. The description obtained after max pooling, M s (Q tam ) represents the weight coefficients, and σ* represents the activation function Sigmoid;

[0119] The Sigmoid function is often used as the activation function of neural networks, which maps variables to the range [0, 1].

[0120] Among them, average pooling layer and max pooling layer are two pooling structures in CNN. Average pooling layer takes the average value of the image within the pooling region, while max pooling layer takes the maximum value of the image within the pooling region.

[0121] Step (3.2.2.5.3) uses the Hadamard product ⊙ on the feature map P tam (1×N*×C) and attention map S tam T is obtained by calculating (1×N*×C). tam (1×N*×C), the formula is as follows:

[0122]

[0123] Here, ⊙ represents the Hadamard product, which is a type of matrix operation that performs calculations on the same matrix and produces a third matrix of the same dimensions. It is feature map T tam The element at position i*.

[0124] The process of the fourth step is as follows:

[0125] Step (4.1) uses the two-dimensional time-frequency feature image I as the input to the improved data augmentation overall model.

[0126] The generator deceives the discriminator by generating synthetic data G(z) that resembles real data x until the discriminator can no longer distinguish between real data and synthetic data.

[0127] The generator is a network model in DCGAN. It receives random noise and generates fake data through forward propagation.

[0128] Step (4.2) DCGAN optimizes itself through a continuous struggle between the maximum and minimum probability outputs, with the generator and discriminator alternately updating their model parameters until a Nash equilibrium is reached, as shown in the following formula:

[0129]

[0130] Where D and G are the discriminator network and the generator network, respectively, and E x E represents the mathematical expectation of the real data x**. z P is the mathematical expectation of the noisy data Z. z(Z) It is a prior distribution. It represents the true data distribution, log is the logarithmic function in mathematics, and D(x**) and D(G(z)) represent the probabilities of the true data and the generated data, respectively.

[0131] Compared with existing methods, the beneficial effects of the method described in this invention are:

[0132] (1) Based on the analysis and processing of the collected elevator car acceleration data, the one-dimensional vibration signal is converted into a two-dimensional time-frequency image through continuous wavelet transform, which enriches the feature information;

[0133] (2) Improve the DCGAN network by using spectral normalization to stabilize the training process, which can reduce the probability of "mode collapse" to a certain extent.

[0134] (3) The spatiotemporal attention mechanism is used for weight allocation. It can generate more new datasets based on the features of a small number of datasets to achieve the purpose of data augmentation when the dataset samples are insufficient. It can learn the pixel distribution of high-resolution elevator images very well. Attached Figure Description

[0135] Figure 1 It is an overall model for enhancing elevator vibration signal data based on an improved deep convolutional generative adversarial network.

[0136] Figure 2 This is a comparison of PSNR and SSIM between the data-enhanced image and the original image. Detailed Implementation

[0137] The present invention will now be further described with reference to the accompanying drawings.

[0138] Reference Figure 1 and Figure 2 A method for enhancing elevator vibration signal data based on an improved deep convolutional generative adversarial network includes the following steps:

[0139] Step 1: Collect the raw vibration signal of the elevator car's acceleration. The process is as follows:

[0140] An accelerometer is installed in the elevator car to measure the vibration signal during its operation, and the real-time operation data of the elevator is remotely acquired using the TCP / IP communication protocol.

[0141] The sensors are connected to the elevator and the gateway via serial ports and communicate with each other via Modbus and RS485 protocols.

[0142] Step 2: Obtain a valid vibration signal, the process is as follows:

[0143] Step (2.1) Calculate the time-domain characteristic index of the elevator car vibration acceleration signal using the time-domain analysis method;

[0144] Step (2.1) includes the following sub-steps:

[0145] Step (2.1.1) calculates the peak-to-peak value X3 in the vibration signal sample, using the following formula:

[0146] X1 = max(x i )

[0147] X2=min(x i )

[0148] X3 = X1 - X2

[0149] Where, x i The sampled values ​​of the vibration acceleration signal are given, and max is the value of the data x. i The maximum value, min is the value of the data x. i The minimum value of X1 is the maximum value in the signal sample, and the minimum value of X2 is the minimum value in the signal sample.

[0150] Among them, peak-to-peak value is used to describe the range of vibration signal changes within one cycle, but the car vibration signal is a non-periodic signal, so it is used to describe the difference between the maximum and minimum values ​​in the sample;

[0151] Step (2.1.2) calculates the mean X4 and root mean square X5 of the vibration signal samples, using the following formula:

[0152]

[0153]

[0154] Where N is the number of sampling points, and ∑ is the summation operation, which is usually used in mathematics and statistics to represent the addition of a series of values;

[0155] Among them, the mean value reflects the change of vibration signal due to the change of the shaft position during elevator operation; the root mean square value, also known as the effective value, reflects the energy intensity and stability of the vibration signal.

[0156] Step (2.1.3) calculates the standard deviation X6 in the vibration signal samples, using the following formula:

[0157]

[0158] Where, x i X4 represents the sampled value of the vibration acceleration signal, and X4 is the mean value.

[0159] Step (2.1.4) calculates the kurtosis X7 and skewness X8 in the vibration signal sample, using the following formula:

[0160]

[0161]

[0162] Where, x i X4 represents the sampled value of the vibration acceleration signal, X6 represents the mean, kurtosis reflects the impact characteristics of the vibration signal, and skewness reflects the degree of skewness in the data distribution.

[0163] Step (2.1.5) calculates the margin index X9 and waveform index X in the vibration signal sample. 10 Pulse index X 11 Peak index X 12 The formula is as follows:

[0164]

[0165]

[0166]

[0167]

[0168] Where, |x i |for x i The absolute value, margin index, waveform index, and pulse index all reflect the proportion of the impact signal in the vibration signal.

[0169] Step (2.2) segments the vibration signal processed in step (2.1), samples it using a sliding window containing M data points, and then slides the window backward by more than M data points to extract the original signal, finally obtaining the effective vibration signal;

[0170] Step (2.3) extracts the time-frequency features of the signal through continuous wavelet transform:

[0171]

[0172]

[0173] Where, ψ a,b It is a wavelet basis function with two parameters a and b; W f (a,b) represents the coefficients of the wavelet transform, which are obtained by integrating the wavelet basis function ψ(x*) over the noise signal F(x*). denoted by ψ(x*), where a is the translation factor and b is the scaling factor; ∫ represents the integral operation, commonly used in mathematics and statistics.

[0174] Among them, continuous wavelet transform is the most widely used method for time-domain and frequency-domain feature extraction.

[0175] Step (2.4) combines wavelet coefficients at different frequencies to obtain a two-dimensional time-frequency image, including the following sub-steps:

[0176] Step (2.4.1) selects Morlet wavelet as the basis function, first determines the initial scaling factor and translation factor, and then analyzes the wavelet from high frequency to low frequency. The first value of the scaling factor corresponds to the wavelet with the highest degree of compression. As the wavelet factor increases, the wavelet will also increase.

[0177] The Morlet wavelet basis function expression is as follows:

[0178]

[0179] The Morlet wavelet basis function is composed of a complex trigonometric function multiplied by an exponential decay function, where j represents an imaginary number, ω0 represents the center frequency, t is the time variable, and e is a mathematical constant.

[0180] Step (2.4.2) follows the continuous wavelet transform formula in step (2.3), multiplying the input signal by the wavelet and integrating to obtain the corresponding wavelet coefficients W. f (a,b);

[0181] Step (2.4.3) Change the translation factor until the end of the signal, and multiply the integral result by a constant to normalize it;

[0182] Step (2.4.4) changes the scaling factor, continuously expands the wavelet, and repeats steps (2.4.2)-(2.4.3) to finally generate the time-frequency image I of the wavelet;

[0183] Step 3: Optimize the DCGAN network model by introducing the spectrum normalization and spatiotemporal attention mechanism modules into the discriminator and generator to form an improved overall data augmentation model.

[0184] DCGAN is a network model proposed by Radford. Its architecture mainly modifies Convolutional Neural Networks (CNNs). In the generator, images are generated by transposing convolutional layers. Conversely, in the discriminator, the obtained image is taken as input, operated on using standard convolutions, and the displayed binary classification result is taken as output.

[0185] Convolutional neural networks (CNNs) are deep learning models or multilayer perceptrons similar to artificial neural networks, proposed by Yann LeCun. They are often used to analyze visual images. Convolutional layers are the hierarchical structures in CNNs.

[0186] The process of the third step is as follows:

[0187] Step (3.1) The discriminator in the DCGAN network introduces spectral normalization;

[0188] The discriminator is the discriminator network in the deep convolutional generative adversarial network. It is mainly responsible for distinguishing whether the input data is real or generated by the generator and feeding it back to the generator.

[0189] Among them, spectral normalization is a function that can make the discriminator more stable;

[0190] Step (3.1) includes the following sub-steps:

[0191] Step (3.1.1) constructs a discriminator using convolutional layers and a recurrent structure, including 5 convolutional layers with a kernel size of 5×5, a stride of 2, and channel counts of 32, 64, 128, 256, and 512, respectively. A spectral normalization layer is then added outside each convolutional layer of the discriminator to normalize the spectrum of the entire discriminator, thereby providing more stable training performance.

[0192] Normalization, in particular, limits the preprocessed data to a certain range, thereby eliminating the adverse effects caused by outlier data.

[0193] Step (3.1.2) imposes a constraint on the spectral norm of each convolutional layer to satisfy Lipschitz = 1;

[0194] For a real matrix A, its spectral norm is defined as follows:

[0195]

[0196] Among them, A T It is the transpose of matrix A. eig(A) represents a function that computes the eigenvalues ​​of matrix A, and returns the vector [λ1, λ2, ..., λ]. n ] T , where λ k This represents the k-th eigenvalue;

[0197] Among them, Lipschitz is the Lipschitz continuity condition, which is a smoothness condition with stronger smoothness than the usual continuity condition;

[0198] In step (3.1.3), the activation function in the discriminator is a LeakyReLU with slope m that satisfies Lipschitz continuity. The spectral norm of the parameter matrix of each layer is obtained by solving layer by layer using the power iteration method, as shown in the following formula:

[0199]

[0200]

[0201] LeakyReLU is an activation function that can solve the vanishing gradient problem.

[0202] Where, sup represents the least upper bound, h represents the number of nodes in the hidden layer; σ(A) is the largest singular value of matrix A, |||² represents the L2 norm of the matrix, and is the square root of the largest eigenvalue of the matrix; C SN C* is the normalized convolutional layer parameter matrix with a maximum singular value of 1, and σ(C*) is the spectral norm of the convolutional layer parameter matrix C*.

[0203] Among them, the power iteration method is mainly used to approximate the solution of the largest eigenvalue of a matrix;

[0204] Step (3.2) introduces a spatiotemporal attention mechanism into the generator in the DCGAN network;

[0205] The generator is a generative network in deep convolutional generative adversarial networks. It learns the features of the training set data and, under the guidance of the discriminator, tries to fit the distribution of random noise as closely as possible to the true distribution of the training data, thereby generating similar data with the features of the training set.

[0206] Among them, the spatiotemporal attention mechanism is an image processing technique that learns the degree of attention to different spatiotemporal locations, thereby enabling the model to capture key spatiotemporal information more effectively.

[0207] The generator consists of a spatiotemporal attention mechanism and four deconvolutional layers (Deconv1, Deconv2, Deconv3, Deconv4).

[0208] Step (3.2) includes the following sub-steps:

[0209] In step (3.2.1), the time-frequency image I in step (2.4.4) is first reshaped in dimension by the Reshape function, and then processed by the Deconv1 and Deconv2 deconvolution layers in sequence.

[0210] The Reshape function is a reshaping function used to reset the dimensions of the array. The Deconv1 and Deconv2 deconvolution layers are structures in the DCGAN model. Step (3.2.2) calculates X from the time-frequency image I in step (2.4.4) using the Reshape function and the two deconvolution layers. cam (1×N*×C) is input into the STA module.

[0211] Where N* represents the dimension, and C represents the C-th channel in the local input;

[0212] Step (3.2.2.1) in the time dimension first involves X... cam Using (1×N*×C) as the input signal, X is obtained by transposing it. trans (1×C×N*);

[0213] Here, transpose represents the matrix transpose.

[0214] Step (3.2.2.2)X trans Q is obtained by (1×C×N*) through the convolutional layer CONV1(1×1×N*). cam (1×C×1);

[0215] Convolution is an effective method for extracting image features. It uses a square convolution kernel that slides across the input feature map with a specified stride.

[0216] For each step, the convolution kernel will overlap with the input feature map. The elements corresponding to the overlapping areas are multiplied, summed, and then the bias term is added to obtain a pixel of the output feature.

[0217] Among them, the bias term is an additional parameter outside the initial weight matrix of the network, which is responsible for adjusting the activation function of the neural network to better adapt it to the data features; the pixel is the basic building block of the smallest unit in the image.

[0218] In step (3.2.2.2), the convolution kernel size is 1×1, the number of channels is N*, and the stride is 1.

[0219] Step (3.2.2.3) Q cam (1×C×1) The normalized weight S is obtained through the SoftMax function. cam (1×C×1);

[0220] The SoftMax function can convert the output values ​​of multi-class classification into a probability distribution in the range [0, 1].

[0221] Step (3.2.2.4)S cam (1×C×1) is converted to S by the Transpose function. trans (1×1×C);

[0222] Transpose is a transpose function used to transpose high-dimensional arrays.

[0223] Step (3.2.2.5) processes the input signal X in the spatial dimension through two branches. cam (1×N*×C);

[0224] Step (3.2.2.5.1) Input signal X cam The feature map P is obtained by calculating (1×N*×C) through the convolutional layer CONV2(1×1×1). tam (1×N*×C);

[0225] In step (3.2.2.5.1), the convolution kernel size is 1×1, the channel is 1, and the stride is 1.

[0226] Step (3.2.2.5.2) Input signal X cam The feature map Q is obtained by passing (1×N*×C) through a convolutional layer CONV3(1×1×1). tam After (1×N*×C), the spatial attention map S is calculated and obtained using the SoftMax function and spatial attention mechanism. tam (1×N*×C);

[0227] The spatial attention map S is calculated and obtained through the SoftMax function. tam The position weights are (1×N*×C), and the formula is as follows:

[0228]

[0229] in, Used to measure the i* position pair feature map P tam and Q tam The impact, It is the feature map Q tam The element at position i*;

[0230] Where exp is an exponential function with the natural number e as its base;

[0231] In step (3.2.2.5.2), the convolution kernel size is 1×1, the channel is 1, and the stride is 1.

[0232] Among them, the spatial attention map S is calculated and obtained through the attention mechanism. tam The weighting coefficients (1×N*×C) are calculated using the following formula:

[0233]

[0234] Among them, Q tan The input features are processed and concatenated by an average pooling layer (Avgpool()) and a max pooling layer (Maxpool()), and then passed through a convolutional layer (f). 7*7 The weighting coefficient M is calculated. s (Q tam );

[0235] Among them, f 7*7 This represents a convolutional layer with a kernel size of 7×7. This is a description obtained after average pooling. The description obtained after max pooling, M s (Q tam ) represents the weight coefficients, and σ* represents the activation function Sigmoid;

[0236] The Sigmoid function is often used as the activation function of neural networks, which maps variables to the range [0, 1].

[0237] Among them, average pooling layer and max pooling layer are two pooling structures in CNN. Average pooling layer takes the average value of the image within the pooling region, while max pooling layer takes the maximum value of the image within the pooling region.

[0238] Step (3.2.2.5.3) uses the Hadamard product ⊙ on the feature map P tam (1×N*×C) and attention map S tam T is obtained by calculating (1×N*×C). tam (1×N*×C), the formula is as follows:

[0239]

[0240] Here, ⊙ represents the Hadamard product, which is a type of matrix operation that performs calculations on the same matrix and produces a third matrix of the same dimensions. It is feature map T tam The element at position i*.

[0241] Step (3.2.2.6) will transfer the feature map S trans and T tam Perform feature fusion to obtain Y tam (1×N*×C);

[0242] Feature fusion is a common operation performed on features extracted from images, which combines multiple features into a single feature that is more discriminative than the input features.

[0243] In step (3.2.2.6), the feature fusion adopts a series of feature fusion methods to connect two features. If the dimensions of the two input features are p* and q*, the dimension of the output feature z is p*+q*.

[0244] The output Y after feature fusion in step (3.2.3) tam (1×N*×C) is calculated sequentially through Deconv3 and Deconv4 deconvolution layers to obtain the final result; in addition, ReLU function and BN are added to each layer, and hyperbolic tangent (Tanh) activation function is used as the output layer;

[0245] ReLU is a commonly used linear rectified activation function in neural networks, while Tanh is a type of hyperbolic function in mathematics.

[0246] Step 4: Input the two-dimensional time-frequency feature image into the data to enhance the overall model and obtain the generated new feature image. The process is as follows:

[0247] Step (4.1) uses the two-dimensional time-frequency feature image I as the input to the improved data augmentation overall model.

[0248] The generator deceives the discriminator by generating synthetic data G(z) that resembles real data x until the discriminator can no longer distinguish between real data and synthetic data.

[0249] The generator is a network model in DCGAN. It receives random noise and generates fake data through forward propagation.

[0250] Step (4.2) DCGAN optimizes itself through a continuous struggle between the maximum and minimum probability outputs, with the generator and discriminator alternately updating their model parameters until a Nash equilibrium is reached, as shown in the following formula:

[0251]

[0252] Where D and G are the discriminator network and the generator network, respectively, and E x E represents the mathematical expectation of the real data x**. z P is the mathematical expectation of the noisy data Z. z(Z) It is a prior distribution. It represents the true data distribution, log is the logarithmic function in mathematics, and D(x**) and D(G(z)) represent the probabilities of the true data and the generated data, respectively.

[0253] The implementation scheme of this embodiment is as follows:

[0254] Step 1: Define 4 data augmentation models:

[0255] DCGAN: By modifying CNN, the generator model generates images through transposed convolutional layers, while the discrimination model takes the obtained images as input, performs standard convolution operations, and outputs the displayed binary classification results.

[0256] SN-DCGAN: A data augmentation method that combines DCGAN with SN.

[0257] STA-DCGAN: A data augmentation method that combines DCGAN with STA.

[0258] STA-SN-DCGAN: The data augmentation method of the present invention.

[0259] Step 2: Experimental Dataset

[0260] The experimental data used in this invention comes from real-time elevator operation data obtained remotely from sensors using the TCP / IP communication protocol.

[0261] Among them, the number of samples collected for normal elevator car vibration was 25,037, the number of samples for excessive vertical vibration of the car was 127,359, the number of samples for excessive horizontal vibration of the car was 95,147, and the number of samples for emergency stop of the car was 2,849.

[0262] These data samples, after feature extraction processing, can be used as the experimental dataset for the following experiments.

[0263] Step 3: Define evaluation indicators

[0264] This invention uses peak signal ratio (PSNR) and structural similarity index (SSIM) as important indicators for model evaluation.

[0265]

[0266] SSIM(x^,y)=[k(x^,y)] α ·[o(x^,y)] β ·[s(x^,y)] γ

[0267] Where MSE is the mean squared error, bits is the unit of data transmission, representing the amount of data; x^ and y represent the input images x^ and y, respectively, k is the image brightness; o is the image contrast; s is the structural measurement; α, β, and γ are used to represent the importance of the model.

[0268] Step 4: Analyze and compare the results

[0269] refer to Figure 2 It displays the PSNR and SSIM values ​​for four models (STA-SN-DCGAN, STA-DCGAN, SN-DCGAN, and DCGAN). Figure 2 In the comparison, the STA-SN-DCGAN method achieved the best PSNR and SSIM similarity metrics of 23.44 / 0.507, outperforming other methods. Although STA-DCGAN showed higher image quality and similarity than other models, its average PSNR and SSIM results from ten consecutive images were worse than those of SN-DCGAN. One reason for this is that SN can resolve the instability of DCGAN. Overall, STA-SN-DCGAN showed the best PSNR and SSIM values ​​among all methods. This means that STA-SN-DCGAN fully utilizes the capabilities of the STA module and SN to improve the quality of synthesized images.

[0270] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for enhancing elevator vibration signal data based on an improved deep convolutional generative adversarial network, characterized in that, The method includes the following steps: Step 1: Collect the raw vibration signal of the elevator car's acceleration; Step 2: Obtain effective vibration signals, extract the time-frequency features of the signals through continuous wavelet transform, and combine the wavelet coefficients at different frequencies to obtain a two-dimensional time-frequency image; Step 3: Optimize the DCGAN network model by introducing spectral normalization and spatiotemporal attention mechanism modules into the discriminator and generator to form an improved overall data augmentation model; the process is as follows; Step (3.1) The discriminator in the DCGAN network introduces spectral normalization; Step (3.2) introduces a spatiotemporal attention mechanism into the generator in the DCGAN network, including the following sub-steps: Step (3.2.1) First, the time-frequency image I is reshaped in dimension by the Reshape function, and then it is calculated by the Deconv1 and Deconv2 deconvolution layers in sequence; Step (3.2.2) calculates the time-frequency image I using the Reshape function and two deconvolution layers. The input to the STA module includes the following sub-steps: Step (3.2.2.1) in the time dimension first involves... As the input signal, it is obtained through transpose calculation. ; Step (3.2.2.2) Calculated through convolutional layer CONV1 ; In step (3.2.2.2), the kernel size is 1. The channel is The step size is 1; Step (3.2.2.3) Normalized weights are obtained using the SoftMax function. ; Step (3.2.2.4) Transformed via the Transpose function ; Step (3.2.2.5) processes the input signal in the spatial dimension through two branches. This includes the following sub-steps: Step (3.2.2.5.1) Input signal The corresponding feature map is obtained by calculating the convolutional layer CONV2. ; In step (3.2.2.5.1), the kernel size is 1. The channel is The step size is 1; Step (3.2.2.5.2) Input signal The feature map is calculated using the convolutional layer CONV3. Then, the SoftMax function and spatial attention mechanism are used to calculate and obtain the spatial attention map. ; The spatial attention map is calculated and obtained using the SoftMax function. The position weight is calculated using the following formula: ； in, Used for measurement Location-to-feature map The impact, It is a feature map exist The element at the specified position; In step (3.2.2.5.2), the kernel size is 1. The channel is The step size is 1; Among them, the spatial attention map is calculated and obtained through the attention mechanism. Weighting coefficients The formula is as follows: ； in, The input features are processed through an average pooling layer. and max pooling layer After processing and stitching, it passes through a convolutional layer. Calculate the weighting coefficients ; in, The kernel size represents the convolution kernel size. convolutional layers, This is a description obtained after average pooling. The description obtained after max pooling, These are the weighting coefficients. The activation function is Sigmoid; Step (3.2.2.5.3) utilizes the Hadamard product. For feature maps Attention map Calculations were performed to obtain The formula is as follows: ； in, Represents the Hadamard product, a type of matrix operation that performs calculations on the same matrix and produces a third matrix of the same dimensions. It is a feature map elements Step (3.2.2.6) will convert the feature map and Perform feature fusion to obtain ; Output after feature fusion in step (3.2.3) The final result is obtained by sequentially passing the Deconv3 and Deconv4 deconvolution layers; in addition, ReLU and BN functions are added to each layer, and the hyperbolic tangent activation function is used as the output layer. The generator consists of a spatiotemporal attention mechanism and four deconvolutional layers: Deconv1, Deconv2, Deconv3, and Deconv4. Step 4: Input the two-dimensional time-frequency feature image into the data to enhance the overall model and obtain the generated new feature image.

2. The elevator vibration signal data enhancement method based on an improved deep convolutional generative adversarial network as described in claim 1, characterized in that, The first step is as follows: an accelerometer is installed in the elevator car to measure the vibration signal during its operation and the real-time operation data of the elevator in the sensor is remotely acquired using the TCP / IP communication protocol; The sensors are connected to the elevator and gateway via serial ports and communicate with each other using Modbus and RS485 protocols.

3. The elevator vibration signal data enhancement method based on improved deep convolutional generative adversarial networks as described in claim 1 or 2, characterized in that, The process of obtaining a valid vibration signal in the second step is as follows: Step (2.1) Calculate the time-domain characteristic index of the elevator car vibration acceleration signal using the time-domain analysis method; Step (2.2) segments the vibration signal processed in step (2.1), samples it using a sliding window containing M data points, and then slides the window backward by more than M data points to extract the original signal, finally obtaining the effective vibration signal; Step (2.3) extracts the time-frequency features of the signal through continuous wavelet transform: ； ； in, It is a wavelet basis function with two parameters a and b; This represents the coefficients of the wavelet transform, which are obtained by utilizing wavelet basis functions. noise signal Perform integral transformation; Describing wavelet basis functions The conjugate complex roots, The translation factor is... ∫ represents the scale factor; ∫ represents integration, used in mathematics and statistics. Among them, continuous wavelet transform is the most widely used method for time-domain and frequency-domain feature extraction. Step (2.4) combines wavelet coefficients at different frequencies to obtain a two-dimensional time-frequency image.

4. The elevator vibration signal data enhancement method based on improved deep convolutional generative adversarial networks as described in claim 3, characterized in that, Step (2.1) includes the following sub-steps: Step (2.1.1) Calculate the peak-to-peak value of the vibration signal sample. The formula is as follows: ； ； ； in, The sampled values ​​of the vibration acceleration signal are denoted by 'max', where 'max' represents the data. The maximum value, min is the value of the data. The minimum value, The maximum value in the signal sample. It is the minimum value in the signal sample; Among them, peak-to-peak value is used to describe the range of vibration signal changes within one cycle, but the car vibration signal is a non-periodic signal, so it is used to describe the difference between the maximum and minimum values ​​in the sample; Step (2.1.2) Calculate the mean value in the vibration signal samples. Root Mean Square The formula is as follows: ； ； Where N is the number of sampling points, and ∑ is the summation operation, used in mathematics and statistics, which represents adding a series of values; Among them, the mean value reflects the change of vibration signal due to the change of the shaft position during elevator operation; the root mean square value, also known as the effective value, reflects the energy intensity and stability of the vibration signal. Step (2.1.3) Calculate the standard deviation in the vibration signal samples. The formula is as follows: ； in, These are sampled values ​​of the vibration acceleration signal. The mean; Step (2.1.4) Calculate the kurtosis in the vibration signal sample. and skewness The formula is as follows: ； ； in, These are sampled values ​​of the vibration acceleration signal. The mean, The standard deviation is denoted by kurtosis, which reflects the impact characteristics of the vibration signal, and skewness, which reflects the degree of skewness in the data distribution. Step (2.1.5) Calculate the margin index in the vibration signal sample. Waveform Indicators Pulse Indicators Peak indicators The formula is as follows: ； ； ； ； in, for The absolute value, margin index, waveform index, and pulse index all reflect the proportion of the impact signal in the vibration signal. Step (2.4) includes the following sub-steps: Step (2.4.1) selects Morlet wavelet as the basis function, first determines the initial scaling factor and translation factor, and then analyzes the wavelet from high frequency to low frequency. The first value of the scaling factor corresponds to the wavelet with the highest degree of compression. As the wavelet factor increases, the wavelet will also increase. The Morlet wavelet basis function expression is as follows: ； The Morlet wavelet basis functions are constructed by multiplying a complex trigonometric function by an exponentially decaying function, where j represents an imaginary number. This represents the center frequency; t is a time variable; e is a mathematical constant in mathematics. Step (2.4.2) involves multiplying the input signal by the wavelet and integrating the continuous wavelet transform formula from step (2.3) to obtain the corresponding wavelet coefficients. ; Step (2.4.3) Change the translation factor until the end of the signal, and normalize the integral result by multiplying it by a constant; Step (2.4.4) changes the scaling factor and continuously expands the wavelet. Steps (2.4.2)-(2.4.3) are repeated to finally generate the time-frequency image I of the wavelet.

5. The elevator vibration signal data enhancement method based on an improved deep convolutional generative adversarial network as described in claim 1 or 2, characterized in that, Step (3.1) includes the following sub-steps: Step (3.1.1) constructs a discriminator using convolutional layers and a recurrent structure, comprising 5 convolutional layers with a kernel size of 5.

5. With a stride of 2, the number of channels is 32, 64, 128, 256, and 512 respectively. A spectral normalization layer is added outside each convolutional layer of the discriminator to normalize the spectrum of the entire discriminator in order to provide more stable training performance. Normalization, in particular, limits the preprocessed data to a certain range, thereby eliminating the adverse effects caused by outlier data. Step (3.1.2) imposes a constraint on the spectral norm of each convolutional layer to satisfy Lipschitz=1; For a real matrix A, its spectral norm is defined as follows: ； in, It is the transpose of matrix A. eig(A) represents the function that calculates the eigenvalues ​​of matrix A and returns a vector. ,in This represents the k-th eigenvalue; Wherein, Lipschitz is the Lipschitz continuity condition; In step (3.1.3), the activation function in the discriminator is a LeakyReLU with slope m, satisfying Lipschitz continuity. The spectral norm of the parameter matrix of each layer is obtained by solving layer by layer using the power iteration method, as shown in the following formula: ； ； LeakyReLU is an activation function that can solve the gradient vanishing problem. in, The upper bound is represented by 'h', where 'h' represents the number of nodes in the hidden layer. Let A be the largest singular value of matrix A. The L2 norm of a matrix is ​​the square root of its largest eigenvalue. The normalized convolutional layer parameter matrix has a maximum singular value of 1. Convolutional layer parameter matrix spectral norm; Among them, the power iteration method is used to approximate the solution of the largest eigenvalue of a matrix.

6. The elevator vibration signal data enhancement method based on an improved deep convolutional generative adversarial network as described in claim 1 or 2, characterized in that, The process of the fourth step is as follows: Step (4.1) uses the two-dimensional time-frequency feature image I as the input to the improved data augmentation overall model; The generator generates data that resembles real data. The synthetic data G(z) is used to deceive the discriminator until the discriminator can no longer distinguish between real data and synthetic data; The generator is a network model in DCGAN. It receives random noise and generates fake data through forward propagation. Step (4.2) DCGAN optimizes itself through a continuous struggle between the maximum and minimum probability outputs, with the generator and discriminator alternately updating their model parameters until a Nash equilibrium is reached, as shown in the following formula: ； Where D and G are the discriminator network and the generator network, respectively. Representing real data The mathematical expectation, It is the mathematical expectation of the noisy data Z. It is a prior distribution. It represents the actual data distribution, and log is the logarithmic function in mathematics. and These represent the probabilities of real data and generated data, respectively.

Images