Elevator fault diagnosis method based on audio two-dimensional depth features

Abstract

The invention discloses an elevator fault diagnosis method based on audio two-dimensional depth features. The method comprises the following steps: building an elevator audio signal denoising model; the method comprises the following steps: taking a U-Net architecture as a backbone network, and introducing a Transform encoder layer as a middle layer; constructing a loss function; de-noising processing is carried out; converting the one-dimensional audio data into a two-dimensional image format by adopting an audio data 2D conversion method based on periodic characteristics; performing fault diagnosis; based on an existing 2D-CNN model, a multi-scale convolution kernel is introduced, and a multi-layer T-Block stacking mode is adopted as a core module of the elevator audio fault diagnosis model; training, testing and optimizing the elevator audio fault diagnosis model to obtain an optimized elevator audio fault diagnosis model; and mining the two-dimensional depth features by the optimized elevator audio fault diagnosis model to realize fault diagnosis. According to the method, the two-dimensional depth features of the audio can be efficiently and accurately mined, and the accuracy of fault diagnosis can be effectively improved.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent hoist fault diagnosis, and specifically is a hoist fault diagnosis method based on audio two-dimensional depth features. Background Art

[0002] The hoist is a key transportation equipment in the auxiliary transportation system of coal mines, and plays a very important role in the transportation of personnel and materials in coal mines. As the core component supporting the normal operation of the entire system, the main shaft device of the hoist bears huge load pressure. After long-term high-load operation, the main bearing will fail due to performance degradation, including inner ring rupture, outer ring wear, and rolling element damage. If similar faults are not discovered and handled in time, they will threaten the stability and safety of mine personnel and ore lifting, and may even cause major safety accidents. Therefore, real-time monitoring of the hoist and timely early warning can effectively avoid the chain reaction caused by the fault, which is of great practical significance.

[0003] In recent years, the informatization process in the field of China's machinery industry has accelerated significantly. With the increasing complexity and intelligence of the structure of mechanical equipment, the amount of data generated has exploded. In this context, the traditional fault diagnosis method that relies on experience and simple signal processing can no longer meet the needs of modernization, and modern intelligent fault diagnosis technology that relies on large-scale data analysis has emerged. In particular, with the rapid progress of computer hardware technology, especially the leap in GPU computing power, and the continuous iteration and update of artificial intelligence and Internet of Things technology, deep learning technology has been widely used in application fields such as computer vision, speech recognition, and medical image analysis, showing extremely high recognition accuracy, demonstrating its great potential in data processing. Therefore, intelligent fault diagnosis, especially diagnostic technology based on deep learning, has become a frontier topic of common concern in academia and industry. This type of method has many advantages such as strong adaptability, high flexibility and good diagnostic effect. This trend can not only effectively improve the safety and reliability of equipment operation, but also provide strong technical support for predictive maintenance, optimization of production processes, and cost reduction. It is a key link in promoting the transformation and upgrading of the manufacturing industry and realizing intelligent manufacturing.

[0004] Although deep learning technology has proven its transformative capabilities in multimodal, high-semantic information fields such as image recognition and natural language processing, it faces different challenges when facing application scenarios such as mechanical fault diagnosis. Most mechanical fault data are one-dimensional time series signals, such as audio signals during operation. Time series are recorded continuously, and only some scalars are saved at each time point, while images have rich two-dimensional information and rich contextual information, which hinders the widespread application of intelligent diagnosis technology in actual audio fault diagnosis. For this reason, it is urgent to provide a hoist fault diagnosis method based on audio two-dimensional deep features to help improve the accuracy and efficiency of hoist fault diagnosis. Summary of the invention

[0005] In view of the problems existing in the above-mentioned prior art, the present invention provides a hoist fault diagnosis method based on audio two-dimensional deep features. The method can not only efficiently denoise the audio timing signals generated during the operation of the hoist, and significantly improve the quality of the audio signals, but also use the deep model to efficiently and accurately mine the two-dimensional deep features of the audio, thereby effectively improving the accuracy of fault diagnosis.

[0006] In order to achieve the above object, the present invention provides a method for diagnosing hoist faults based on audio two-dimensional depth features, comprising the following steps: Step 1: Build a denoising model for audio signals of a lifting machine based on U-Net and Transformer; S11: The U-Net architecture is used as the backbone network of the lifting machine audio signal denoising model. The U-Net architecture includes an encoder layer, a decoder layer, and jump connections between the encoder layer and the decoder layer. On this basis, the Transformer encoder layer is introduced as the middle layer of the lifting machine audio signal denoising model to enhance the model's ability to capture audio signals. S12: A loss function is constructed by combining the autoencoder loss and the short-time Fourier transform loss, and the elevator audio signal denoising model is trained using historical data; during the training process of the elevator audio signal denoising model, the model parameters are continuously updated through the back propagation algorithm to reduce the value of the loss function and improve the performance of the model in the time domain and the time-frequency domain, and finally an optimized elevator audio signal denoising model is obtained; Step 2: De-noising; The original audio signal of the hoist bearing is obtained, and the optimized hoist audio signal denoising model is used to perform end-to-end denoising on the original audio signal of the hoist bearing to obtain denoised one-dimensional audio data; Step 3: Convert the one-dimensional audio data into a two-dimensional image format using an audio data 2D conversion method based on periodic features; Firstly, for the denoised one-dimensional audio data, the fast Fourier transform method is used to obtain the periodic features of the original time series, and the original data is reorganized according to the periodic features; then, through convolution feature extraction and aggregation operations, it is ensured that the converted two-dimensional data can effectively reflect the key information of the time series; Step 4: Build an elevator audio fault diagnosis model and diagnose the elevator fault; S41: Based on the existing 2D-CNN model, by integrating the efficient learning path of the residual network and the multi-scale feature extraction idea, a multi-scale convolution kernel is introduced, and a T-Block module is constructed with T-Inception as the core. A multi-layer T-Block stacking method is used as the core module of the elevator audio fault diagnosis model. The T-Block module is organized in a residual connection manner. The output of the T-Block module in the previous layer is used as the input of the T-Block module in the next layer, and a normalization layer and an activation layer are introduced between two adjacent T-Block modules. S42: using historical data to train the elevator audio fault diagnosis model to obtain a trained elevator audio fault diagnosis model, and testing and optimizing the trained elevator audio fault diagnosis model to obtain an optimized elevator audio fault diagnosis model; S43: Input the converted two-dimensional data into the optimized elevator audio fault diagnosis model. The optimized elevator audio fault diagnosis model mines the two-dimensional deep features of the audio data to realize the diagnosis of audio faults under complex working conditions of the elevator.

[0007] As a preferred embodiment, in step 1 S11, the encoder construction process is as follows: An encoder architecture with a depth of 8 is constructed. Each encoder layer includes two key convolutions, Conv1d and Conv1×1. At the same time, a ReLU function is set after Conv1d, and a GLU is set after Conv1×1. Conv1d is used to extract local features on the audio signal time series data. Its convolution kernel size K is set to 4, and the step size S is set to 2, so that the elevator audio signal denoising model can capture local details while gradually expanding the receptive field. The ReLU function is used to introduce nonlinear characteristics and enhance the fitting ability of the elevator audio signal denoising model to the distribution of complex signal data. Conv1×1 is used to adjust the channel data. Without changing the length of the signal sequence, the number of input channels is doubled to increase the representation ability of the elevator audio signal denoising model. GLU is used to halve the number of input channels, and through the gating mechanism, the elevator audio signal denoising model can realize dynamic screening of important features on given data.

[0008] Furthermore, in order to gradually restore the feature representation compressed in the encoder and finally generate a high-quality denoised frequency time series signal, in step 1 S11, the construction process of the decoder is as follows: A structure consisting of D decoder layers is constructed, and each layer is made to correspond to the corresponding layer in the encoder layer to form a pair of paired layers. At the same time, each pair of paired layers can achieve direct communication through jump connections; each decoder layer includes Conv1×1, and GLU and ConvTranspose1d are arranged in sequence after Conv1×1; Conv1×1 is used to adjust the number of channels to prepare for subsequent deconvolution operations; GLU is used to introduce nonlinearity and further filter features; ConvTranspose1d shares the same hyperparameters with the Conv1d of the paired encoder layer, which is used to permute the number of input and output channels.

[0009] Furthermore, in order to effectively process audio time series data and learn the complex temporal dynamics and long-term dependencies in the sequence, in step 1 S11, the construction process of the intermediate layer is as follows: Five Transformer encoder layers are used to form the middle layer of the elevator audio signal denoising model, and each Transformer encoder layer is integrated with a multi-head self-attention mechanism and a position fully connected layer. Through the multi-head self-attention layer, the elevator audio signal denoising model can simultaneously focus on the information of multiple positions in the sequence, significantly improving the ability to capture the long-distance dependencies within the one-dimensional audio time series signal data; the dimension of the elevator audio signal denoising model is set to 512, providing sufficient space for each attention to capture subtle feature differences; Each multi-head self-attention layer in the Transformer encoder layer uses 8 self-attention heads in parallel to enhance the expressiveness and learning ability of the elevator audio signal denoising model, which is used to map the input noisy frequency time series data to multiple subspaces respectively and perform self-attention operations in each subspace; at the same time, a jump connection is followed by each multi-head self-attention layer, and a layer normalization operation is performed after the jump connection; the positional fully connected layer in the Transformer encoder layer is used for further transformation and feature fusion, and the input channel size of the layer is matched with the dimension of the elevator audio signal denoising model, so that the output channel size of the layer is expanded to 2048, and this expansion is used to increase the capacity of the elevator audio signal denoising model, so that the elevator audio signal denoising model can capture more complex features and provide richer information to the decoder layer.

[0010] As a preference, in step 1 S12, the process of constructing the loss function is as follows: A1: Based on the difference between the input data and the reconstructed data, the autoencoder loss is obtained according to formula (1); (1); In the formula, is the denoised signal In the The value of the sample points, It is a pure signal In the The value of the sample points, is the total number of sample points; A2: Based on the Euclidean distance between the two signal spectra, the spectrum convergence loss is obtained according to formula (2); based on the difference between the two signal amplitude spectra, the amplitude loss is obtained according to formula (3); (2); (3); In the formula, and They correspond to pure signals and denoised signal STFT amplitude spectrum of; is the Frobenius norm; and The denoised signals are and pure signal In the The amplitude value of the STFT window; A3: Combining the spectrum convergence loss and amplitude loss, the short-time Fourier transform loss is obtained according to formula (4); (4); In the formula, It is the weight coefficient for balancing SC Loss and MAG Loss, and its value is 0.5; A4: Combining the autoencoder loss and the short-time Fourier transform loss, the loss function of the booster audio signal denoising model is obtained using formula (5); (5).

[0011] As a preferred embodiment, in step three, the process of converting the one-dimensional audio data into a two-dimensional image format is as follows: S31: For a length of Time series data , the frequency domain characteristics of the time series are analyzed by fast Fourier transform, and the components of the time series at different frequencies are obtained according to formula (6); (6); In the formula, is the first frequency components, is the total number of samples in the signal, represents the frequency index, corresponding to The frequency of the time points, is the base of natural logarithms, is an imaginary unit; S32: Identify the main period of the time series based on the frequency components with the largest amplitudes, and obtain the period length corresponding to the frequency with the largest amplitude according to formula (7) ; (7); In the formula, Represents the index of the frequency component with the maximum amplitude; S33: Use modulo operation to calculate the difference between the current data length and the cycle length, and obtain according to formula (8): When When the value of is 0, there is no need to perform zero padding. Otherwise, zero padding is performed on the original time series to ensure that the sequence length can be divided by the period length. Divide integers and calculate the number of zeros to be added according to formula (9) , and the length of the padded sequence is obtained by formula (10): ; (8); (9); (10); In the formula, Indicates rounding up to ensure that the length of the entire sequence is the period length An integer multiple of ; S34: For frequency , the corresponding period is The filled time series , according to formula (11), it is reshaped into multiple two-dimensional tensors ; At the same time, Formula (11) is written as ; (11); In the formula, , , For the corresponding frequency A two-dimensional tensor of size , Indicates Line value.

[0012] As a preferred embodiment, in step 4 S43, the diagnosis process of the audio fault under the complex working condition of the hoist is as follows: B1: Initialization, initialize the weights of each 2D convolutional layer according to formula (12) ; (12); B2: Extract features from the input two-dimensional tensor according to formula (13); (13); In the formula, , The formula is ,in, To obtain the comprehensive output after averaging the results of all convolutional layers, Is the application The result after the convolution kernel, that is, the operation of each convolution layer, , Indicates that the convolution kernel size is Convolution operation; B3: The learned two-dimensional representation Convert back to one-dimensional space and reshape the two-dimensional data into size according to formula (14) Matrix, and using Perform truncation operation; (14); In the formula, One-dimensional signal ; B4: Use the softmax method to normalize the amplitude and obtain the first The one-dimensional representation of a layer represents its weight; (15); B5: The first Layer Output ; (16); B6: According to Layer Output The complex time series dependencies and periodic features in the data are parsed, and then the fault type of the bearing in the elevator is identified and output.

[0013] In the present invention, since the U-Net network can effectively extract the local and detailed features of the signal, and the encoder layer of the Transformer is good at capturing the long-distance dependencies in the sequence data, the combination of the two can enable the elevator audio signal denoising model to capture the local detailed features and accurately understand the global context information of the signal when processing the one-dimensional audio time series signal. Denoising the original audio signal using the optimized elevator audio signal denoising model can greatly improve the accuracy of fault diagnosis. The one-dimensional audio data is converted into a two-dimensional image format using a 2D conversion method of audio data based on periodic features, so that the deep model with strong image analysis ability can also analyze the audio signal of the elevator. In the process of constructing the elevator audio fault diagnosis model, on the basis of the existing 2D-CNN model, a multi-scale convolution kernel is introduced, which can better capture the complex patterns and periodic changes in the time series data, and then more effectively mine the two-dimensional deep features of the audio data, significantly improving the accuracy of audio fault diagnosis under complex working conditions of the elevator. Combining the autoencoder loss and the short-time Fourier transform loss to construct the loss function can simultaneously improve the performance of the machine audio signal denoising model in the time domain and the time-frequency domain, thereby achieving better performance in the denoising task.

[0014] This method can not only efficiently denoise the audio time series signals generated during the operation of the elevator, which can significantly improve the quality of the audio signals, but also use the deep model to efficiently and accurately mine the two-dimensional depth features of the audio, effectively improving the accuracy of fault diagnosis, and thus helping to provide reliable technical support for the preventive maintenance and fault warning of the elevator. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 The figure is a schematic diagram of the audio signal denoising model of the lifting machine based on U-Net and Transformer in the present invention; Figure 2 It is a schematic diagram of two-dimensional conversion of time series based on periodic characteristics in the present invention; Figure 3 The architecture diagram of the T-Block residual structure design in the present invention; Figure 4 It is a structural diagram of the elevator audio fault diagnosis model in the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings.

[0017] like Figure 1 and Figure 2 As shown, the present invention provides a method for diagnosing hoist faults based on audio two-dimensional depth features, comprising the following steps: Step 1: Build a denoising model for audio signals of a lifting machine based on U-Net and Transformer; S11: The U-Net architecture is used as the backbone network of the audio signal denoising model of the lifting machine, such as Figure 1 As shown in the figure, the U-Net architecture includes an encoder layer, a decoder layer, and jump connections between the encoder layer and the decoder layer. On this basis, the Transformer encoder layer is introduced as the middle layer of the elevator audio signal denoising model to enhance the model's ability to capture audio signals, and the encoder-decoder structure is used to achieve end-to-end signal denoising; Since the U-Net network can effectively extract local and detailed features of the signal, and the encoder layer of the Transformer is good at capturing long-distance dependencies in sequence data, the combination of the two enables the elevator audio signal denoising model to capture local detailed features and accurately understand the global context information of the signal when processing one-dimensional audio time series signals.

[0018] S12: A loss function is constructed by combining the autoencoder (AE) loss and the short-time Fourier transform (STFT) loss, and the elevator audio signal denoising model is trained using historical data; during the training process of the elevator audio signal denoising model, the model parameters are continuously updated through the back propagation algorithm to reduce the value of the loss function and improve the performance of the model in the time domain and time-frequency domain, and finally the optimized elevator audio signal denoising model is obtained; The calculation of AE loss is based on the difference between input data and reconstructed data, and its types include mean square error (MSE) loss and L1 norm loss. The present application preferably adopts L1 loss at the waveform level, that is, the average of the absolute value of the difference between input data and reconstructed data. Compared with MSE (mean square error) loss, the advantage of L1 loss lies in its robustness to outliers and its ability to treat differences as linear relationships. STFT (short-time Fourier transform) loss focuses more on the time-frequency representation of the signal, which consists of two key parts: spectral convergence (SC) loss and magnitude (MAG) loss. SC (spectral convergence) loss measures the closeness between two signal spectra by calculating the Euclidean distance between them. The smaller the value, the more similar the spectra of the signals are; MAG (magnitude) loss focuses on the amplitude spectrum of the signal, and evaluates their closeness in amplitude by comparing the difference between the amplitude spectra of the two signals. The smaller the value, the more consistent the amplitude of the signals is. As a preferred method, the SC loss and the MAG loss are set to account for half of the STFT (Short-time Fourier Transform) loss weight to balance the two aspects of time-frequency representation. Through this design, the booster audio signal denoising model can not only learn useful features of the data in the time domain, but also optimize the spectral representation of the signal in the time-frequency domain, thereby achieving better performance in the denoising task.

[0019] Step 2: De-noising; The original audio signal of the hoist bearing is obtained, and the optimized hoist audio signal denoising model is used to perform end-to-end denoising on the original audio signal of the hoist bearing to obtain denoised one-dimensional audio data; Since the audio signals generated by the elevator during operation are often accompanied by complex noises, which seriously affect the accurate identification of bearing fault characteristics, denoising the original audio signals using the optimized elevator audio signal denoising model can greatly improve the accuracy of fault diagnosis.

[0020] Step 3: Convert the one-dimensional audio data into a two-dimensional image format using an audio data 2D conversion method based on periodic features; Firstly, for the denoised one-dimensional audio data, the fast Fourier transform method is used to obtain the periodic features of the original time series, and the original data is reorganized according to the periodic features; then, through convolution feature extraction and aggregation operations, it is ensured that the converted two-dimensional data can effectively reflect the key information of the time series; For time series such as the audio signal of the hoist bearing, the change of each time point is not only affected by its adjacent time region, but also highly correlated with the changes in the previous and next time periods. This kind of change is called intra-cycle change and inter-cycle change: intra-cycle change refers to the short-term time pattern within a single cycle, while inter-cycle change reflects the long-term trend and pattern within several consecutive cycles. However, it is usually difficult for a one-dimensional time series to show these two changes at the same time. In order to overcome this limitation, such as Figure 2 As shown, the present application proposes a 2D conversion method for audio data based on periodic features, which expands the original audio data into a two-dimensional space. The method is mainly divided into three steps: determining the frequency and period length, zero filling, and two-dimensional conversion. Since visual models are usually designed around two-dimensional images, by converting one-dimensional audio data into a two-dimensional image format, deep learning models with strong image analysis capabilities can also be applied to the analysis of audio signals of elevators.

[0021] Step 4: Build an elevator audio fault diagnosis model and diagnose the elevator fault; S41: In order to effectively process the complex time series data in elevator fault diagnosis, we draw on existing high-performance network architectures such as GoogleNet and ResNet to propose T-Inception, the core module of the elevator audio fault diagnosis model. This module uses multi-scale convolution kernels to capture complex patterns and periodic changes in time series data, and mines the two-dimensional deep features of audio data, thereby making full use of the advanced achievements of deep learning technology in the fields of image processing and visual recognition. The T-Inception module processes the input two-dimensional tensor through multiple convolution kernels of different sizes. This design allows the module to capture different scale features from fine to coarse, which is particularly important for identifying complex audio patterns and periodic changes. In the specific implementation, the module includes There are two-dimensional convolutional layers, and the number of kernels in each layer and their size are related to the number of convolutional layers.

[0022] Based on the existing 2D-CNN model, by integrating the efficient learning path of the residual network and the multi-scale feature extraction idea, a multi-scale convolution kernel is introduced. By introducing the multi-scale convolution kernel, the complex patterns and periodic changes in the time series data can be better captured, and the two-dimensional deep features of the audio data can be more effectively mined, which significantly improves the accuracy of audio fault diagnosis under complex working conditions of the elevator.

[0023] like Figure 3As shown in the figure, the T-Block module is built with T-Inception as the core, and the T-Block module is organized in a residual connection manner. This design concept is derived from the residual network. Its core is to introduce jump connections so that the input can be directly passed to the subsequent layers, helping the gradient to flow effectively in the network, thereby supporting the training of deeper network structures and preventing the gradient from disappearing or exploding. Through the residual connection method, T-Block can more effectively process and identify subtle features from the original audio signal of the hoist bearing, which may indicate early failure of the machine.

[0024] like Figure 4 As shown in the figure, a multi-layer T-Block stacking method is used as the core module of the elevator audio fault diagnosis model. The T-Block module is organized in a residual connection manner, and the output of the previous T-Block module is used as the input of the next T-Block module. Through this continuous processing, the model can refine and enhance fault-related features layer by layer. Normalization layers and activation layers are introduced between two adjacent layers of T-Block modules to further optimize the network's expression ability and nonlinear processing capabilities. Through such an overall design, the model not only has a strong data processing capability in theory, but also can achieve the expected fault detection effect in practical applications, greatly improving the accuracy and efficiency of elevator fault diagnosis.

[0025] S42: using historical data to train the elevator audio fault diagnosis model to obtain a trained elevator audio fault diagnosis model, and testing and optimizing the trained elevator audio fault diagnosis model to obtain an optimized elevator audio fault diagnosis model; S43: Input the converted two-dimensional data into the optimized elevator audio fault diagnosis model. The optimized elevator audio fault diagnosis model mines the two-dimensional deep features of the audio data to realize the diagnosis of audio faults under complex working conditions of the elevator.

[0026] As a preferred embodiment, in step 1 S11, the encoder construction process is as follows: Construct an encoder architecture with a depth of 8. Each encoder layer includes two key convolutions: Conv1d (One-Dimensional Convolution) and Conv1×1 (1×1 Convolution). At the same time, a ReLU function is set after Conv1d. In order to further improve the performance of the audio signal denoising model and introduce a sparsity mechanism, a Gated Linear ReLU (GLU) is set after Conv1×1. Unit, gated linear unit); Conv1d is used to extract local features on the audio signal time series data. Its convolution kernel size K is set to 4, and the step size S is set to 2. This setting enables the elevator audio signal denoising model to gradually expand the receptive field while capturing local details; the ReLU function is used to introduce nonlinear characteristics and enhance the fitting ability of the elevator audio signal denoising model to the distribution of complex signal data; Conv1×1 is used to adjust the channel data. Without changing the length of the signal sequence, the number of input channels is doubled to increase the representation ability of the elevator audio signal denoising model; GLU is used to halve the number of input channels, and through the gating mechanism, the elevator audio signal denoising model can learn which features are important on the given data, thereby realizing dynamic screening of important features.

[0027] As a preferred embodiment, in the first layer of the encoder, the number of output channels of Conv1d is determined by H. H, as a hyperparameter for controlling the model capacity, affects the storage and computing requirements of the elevator audio signal denoising model. Further preferred, the H value of the elevator audio signal denoising model is set to 64. As the encoder layer goes deeper, the number of output channels of Conv1d of each layer is doubled. Through such a design, the elevator audio signal denoising model can abstract and extract more advanced feature representations in the audio signal layer by layer, providing in-depth information for the subsequent decoder.

[0028] As a preferred embodiment, in step 1 S11, the construction process of the decoder is as follows: A structure consisting of D decoder layers is constructed, and each layer is made to correspond to the corresponding layer in the encoder layer one by one to form a pair of paired layers, thus forming a good inter-layer association. This pairing mechanism enables the decoder layer to reconstruct and refine the features extracted by the encoder layer in reverse order; at the same time, in order to promote information exchange between the encoder and the decoder, each pair of paired layers is made to communicate directly through jump connections. This design can provide rich contextual information to the decoder while retaining the detailed information in the encoder, so that the decoder can reconstruct and locate features more accurately.

[0029] Inside the decoder layer, each decoder layer includes Conv1×1, and GLU and ConvTranspose1d are arranged in sequence after Conv1×1; Conv1×1 is used to adjust the number of channels to prepare for the subsequent deconvolution operation; GLU is used to introduce nonlinearity and further filter features; ConvTranspose1d (One-Dimensional Transposed Convolution) shares the same hyperparameters with the paired encoder layer's Conv1d, which is used to permute the number of input and output channels. In this way, the consistency between the encoder and the decoder can be maintained, and the spatial expansion and upsampling of the features are effectively achieved through the deconvolution operation, providing a structured information flow for reconstructing the original data. Using this decoder structure, the lifting machine audio signal denoising model can gradually restore the compressed feature representation in the encoder, and finally generate high-quality denoised audio time series signals.

[0030] As a preferred embodiment, in step 1 S11, the construction process of the middle layer is as follows: Five Transformer encoder layers are used to form the middle layer of the elevator audio signal denoising model, and each Transformer encoder layer is integrated with a multi-head self-attention mechanism and a position fully connected layer to form a powerful feature extraction and dependency modeling unit. Through the multi-head self-attention layer, the elevator audio signal denoising model can simultaneously focus on the information of multiple positions in the sequence, significantly improving the ability to capture the long-distance dependency within the one-dimensional audio time series signal data; the dimension of the elevator audio signal denoising model is set to 512, providing sufficient space for each attention to capture subtle feature differences; Each multi-head self-attention layer in the Transformer encoder layer uses 8 self-attention heads in parallel to enhance the expressiveness and learning ability of the lifting machine audio signal denoising model, which is used to map the input noisy audio time series data to multiple subspaces respectively and perform self-attention operations in each subspace; therefore, each subspace has its own attention weight calculation, so that different focus points and semantic information can be captured, thereby enriching the representation ability of the features.

[0031] At the same time, a jump connection is immediately followed by each multi-head self-attention layer. This connection method helps to retain and transmit gradient information in deep networks, thereby alleviating the problem of gradient disappearance. The layer normalization operation is performed after the jump connection, which further stabilizes the training process and improves the generalization ability of the model; the positional fully connected layer in the Transformer encoder layer is used for further transformation and feature fusion, and the input channel size of the layer is matched with the dimension of the elevator audio signal denoising model, so that the output channel size of the layer is expanded to 2048. This expansion is used to increase the capacity of the elevator audio signal denoising model, so that the elevator audio signal denoising model can capture more complex features and provide richer information to the decoder layer. In this way, the elevator audio signal denoising model can not only effectively process audio time series data, but also learn the complex temporal dynamics and long-term dependencies in the sequence, which is extremely critical to improving the performance of the elevator audio signal denoising model in the audio time series signal denoising task.

[0032] As a preference, in step 1 S12, the process of constructing the loss function is as follows: A1: Based on the difference between the input data and the reconstructed data, the autoencoder loss is obtained according to formula (1); (1); In the formula, is the denoised signal In the The value of the sample points, It is a pure signal In the The value of the sample points, is the total number of sample points; A2: Based on the Euclidean distance between the two signal spectra, the spectrum convergence loss is obtained according to formula (2); based on the difference between the two signal amplitude spectra, the amplitude loss is obtained according to formula (3); (2); (3); In the formula, and They correspond to pure signals and denoised signal STFT amplitude spectrum of; is the Frobenius norm; and The denoised signals are and pure signal In the The amplitude value of the STFT window; A3: Combining the spectrum convergence loss and amplitude loss, the short-time Fourier transform loss is obtained according to formula (4); (4); In the formula, It is the weight coefficient for balancing SC Loss and MAG Loss, and its value is 0.5; A4: Combining the autoencoder loss and the short-time Fourier transform loss, the loss function of the booster audio signal denoising model is obtained using formula (5); (5).

[0033] As a preferred embodiment, in step three, the process of converting the one-dimensional audio data into a two-dimensional image format is as follows: S31: In order to accurately capture the intra-periodic changes in time series data, it is necessary to use expert knowledge to obtain the periodic features hidden in the data. It is preferred to use the fast Fourier transform (FFT) to analyze the frequency domain characteristics of the time series, thereby effectively identifying the period. FFT is an efficient algorithm used to calculate the discrete Fourier transform (DFT) to convert the signal in the time domain to the frequency domain, and then use the extracted frequency components to analyze its periodic characteristics.

[0034] Specifically, for a length of Time series data , the frequency domain characteristics of the time series are analyzed by fast Fourier transform, and the components of the time series at different frequencies are obtained according to formula (6); (6); In the formula, is the first frequency components, is the total number of samples in the signal, is the index of the current sample, represents the frequency index, corresponding to The frequency of the time points, is the base of natural logarithms, is an imaginary unit; S32: Identify the main period of the time series based on the frequency components with the largest amplitudes, and obtain the period length corresponding to the frequency with the largest amplitude according to formula (7) ; (7); In the formula, is the total number of samples in the signal, Represents the index of the frequency component with the maximum amplitude; S33: In order to make the data of each period length adapt to the subsequent two-dimensional conversion, the original time series needs to be zero-filled, that is, a sufficient number of zeros are added to the end of the time series to ensure that the sequence length can be divided by the period length. In order to make the sequence length meet the above conditions, we first need to determine whether zero padding is required.

[0035] Use the modulo operation to calculate the difference between the current data length and the cycle length, and obtain according to formula (8): When When the value of is 0, there is no need to perform zero padding. Otherwise, zero padding is performed on the original time series to ensure that the sequence length can be divided by the period length. Divide integers and calculate the number of zeros to be added according to formula (9) , and the length of the padded sequence is obtained by formula (10): ; (8); (9); (10); In the formula, Indicates rounding up to ensure that the length of the entire sequence is the period length An integer multiple of ; Through the above zero padding operation, each period can be evenly represented in the two-dimensional tensor, regardless of whether the length of the original data is completely aligned with the period. Will be reshaped into multiple 2D tensors .

[0036] S34: For frequency , the corresponding period is The filled time series , according to formula (11), it is reshaped into multiple two-dimensional tensors ; At the same time, Formula (11) is written as ; (11); In the formula, , , For the corresponding frequency A two-dimensional tensor of size , Indicates Line value.

[0037] The two-dimensional tensors constructed by the above audio data conversion method can serve as ideal inputs for deep learning models. They are rich in multi-scale dynamic information of time series, which is conducive to the in-depth learning and analysis of complex timing dependencies and periodic characteristics in the data by the audio fault diagnosis model of the elevator, thereby improving the performance of tasks such as prediction or classification.

[0038] As a preferred embodiment, in step 4 S43, the diagnosis process of the audio fault under the complex working condition of the hoist is as follows: B1: In order to optimize learning efficiency and network convergence speed, the T-Inception module uses the initialization method proposed by Kaiming during initialization, which is optimized for the ReLU activation function. Specifically, the weights of each two-dimensional convolutional layer are initialized according to formula (12): ; (12); In the formula, is the number of elements in the weight matrix; this initialization method helps prevent gradients from disappearing or exploding early in training.

[0039] B2: Extract features from the input two-dimensional tensor according to formula (13); (13); In the formula, , The formula is ,in, To obtain the comprehensive output after averaging the results of all convolutional layers, Is the application The result after the convolution kernel, that is, the operation of each convolution layer, , Indicates that the convolution kernel size is Convolution operation; is the total number of convolutional layers; in this way, the T-Inception module can synthesize feature information at different scales and provide a comprehensive feature representation for use in subsequent layers; B3: In order to facilitate subsequent classification operations, the learned two-dimensional representation Convert back to one-dimensional space and reshape the two-dimensional data into a size of Matrix, and using Perform a truncation operation; the purpose of this operation is to restore the time series that has been expanded after previous padding and conversion to its original length ; (14); In the formula, One-dimensional signal ; B4: After 2D conversion and recognition, different 1D representations need to be fused for the next layer In addition, due to the amplitude It can reflect the relative importance of the selected frequency and period, so it is used as the weight of each transformed two-dimensional tensor. In order to emphasize the most critical features of the signal and ensure consistency and stability when processing data of different scales, the amplitude is normalized using the softmax method and converted into a weight representing relative importance, which enables the model to more effectively identify and utilize this key information. Specifically, according to formula (15), the first The one-dimensional representation of a layer represents its weight; (15); B5: The first Layer Output ; (16); In the formula, is the number of cycles selected; B6: According to Layer Output The complex time series dependencies and periodic features in the data are parsed, and then the fault type of the bearing in the elevator is identified and output.

[0040] In the present invention, since the U-Net network can effectively extract the local and detailed features of the signal, and the encoder layer of the Transformer is good at capturing the long-distance dependencies in the sequence data, the combination of the two can enable the elevator audio signal denoising model to capture the local detailed features and accurately understand the global context information of the signal when processing the one-dimensional audio time series signal. Denoising the original audio signal using the optimized elevator audio signal denoising model can greatly improve the accuracy of fault diagnosis. The one-dimensional audio data is converted into a two-dimensional image format using a 2D conversion method of audio data based on periodic features, so that the deep model with strong image analysis ability can also analyze the audio signal of the elevator. In the process of constructing the elevator audio fault diagnosis model, on the basis of the existing 2D-CNN model, a multi-scale convolution kernel is introduced, which can better capture the complex patterns and periodic changes in the time series data, and then more effectively mine the two-dimensional deep features of the audio data, significantly improving the accuracy of audio fault diagnosis under complex working conditions of the elevator. Combining the autoencoder loss and the short-time Fourier transform loss to construct the loss function can simultaneously improve the performance of the machine audio signal denoising model in the time domain and the time-frequency domain, thereby achieving better performance in the denoising task.

[0041] This method can not only efficiently denoise the audio time series signals generated during the operation of the elevator, which can significantly improve the quality of the audio signals, but also use the deep model to efficiently and accurately mine the two-dimensional depth features of the audio, effectively improving the accuracy of fault diagnosis, and thus helping to provide reliable technical support for the preventive maintenance and fault warning of the elevator.

Claims

1. A hoist fault diagnosis method based on audio two-dimensional depth features, characterized in that: The following steps are involved: Step 1: Build a denoising model for audio signals of a lifting machine based on U-Net and Transformer; S11: The U-Net architecture is used as the backbone network of the lifting machine audio signal denoising model. The U-Net architecture includes an encoder layer, a decoder layer, and jump connections between the encoder layer and the decoder layer. On this basis, the Transformer encoder layer is introduced as the middle layer of the lifting machine audio signal denoising model to enhance the model's ability to capture audio signals. S12: A loss function is constructed by combining the autoencoder loss and the short-time Fourier transform loss, and the elevator audio signal denoising model is trained using historical data; during the training process of the elevator audio signal denoising model, the model parameters are continuously updated through the back propagation algorithm to reduce the value of the loss function and improve the performance of the model in the time domain and the time-frequency domain, and finally an optimized elevator audio signal denoising model is obtained; Step 2: De-noising; The original audio signal of the hoist bearing is obtained, and the optimized hoist audio signal denoising model is used to perform end-to-end denoising on the original audio signal of the hoist bearing to obtain denoised one-dimensional audio data; Step 3: Convert the one-dimensional audio data into a two-dimensional image format using an audio data 2D conversion method based on periodic features; Firstly, for the denoised one-dimensional audio data, the fast Fourier transform method is used to obtain the periodic features of the original time series, and the original data is reorganized according to the periodic features; then, through convolution feature extraction and aggregation operations, it is ensured that the converted two-dimensional data can effectively reflect the key information of the time series; Step 4: Build an elevator audio fault diagnosis model and diagnose elevator faults; S41: Based on the existing 2D-CNN model, by integrating the efficient learning path of the residual network and the multi-scale feature extraction idea, a multi-scale convolution kernel is introduced, and a T-Block module is constructed with T-Inception as the core. A multi-layer T-Block stacking method is used as the core module of the elevator audio fault diagnosis model. The T-Block module is organized in a residual connection manner. The output of the T-Block module in the previous layer is used as the input of the T-Block module in the next layer, and a normalization layer and an activation layer are introduced between two adjacent T-Block modules. S42: using historical data to train the elevator audio fault diagnosis model to obtain a trained elevator audio fault diagnosis model, and testing and optimizing the trained elevator audio fault diagnosis model to obtain an optimized elevator audio fault diagnosis model; S43: Input the converted two-dimensional data into the optimized elevator audio fault diagnosis model. The optimized elevator audio fault diagnosis model mines the two-dimensional deep features of the audio data to realize the diagnosis of audio faults under complex working conditions of the elevator.

2. The method for diagnosing hoist faults based on audio two-dimensional depth features according to claim 1 is characterized in that: In step 1 S11, the encoder construction process is as follows: An encoder architecture with a depth of 8 is constructed. Each encoder layer includes two key convolutions, Conv1d and Conv1×1. At the same time, a ReLU function is set after Conv1d, and a GLU is set after Conv1×1. Conv1d is used to extract local features on the audio signal time series data. Its convolution kernel size K is set to 4, and the step size S is set to 2, so that the elevator audio signal denoising model can capture local details while gradually expanding the receptive field. The ReLU function is used to introduce nonlinear characteristics and enhance the fitting ability of the elevator audio signal denoising model to the distribution of complex signal data. Conv1×1 is used to adjust the channel data. Without changing the length of the signal sequence, the number of input channels is doubled to increase the representation ability of the elevator audio signal denoising model. GLU is used to halve the number of input channels, and through the gating mechanism, the elevator audio signal denoising model can realize dynamic screening of important features on given data.

3. A method for diagnosing hoist faults based on audio two-dimensional depth features according to claim 1 or 2, characterized in that: In step 1 S11, the construction process of the decoder is as follows: A structure consisting of D decoder layers is constructed, and each layer is made to correspond to the corresponding layer in the encoder layer to form a pair of paired layers. At the same time, each pair of paired layers can achieve direct communication through jump connections; each decoder layer includes Conv1×1, and GLU and ConvTranspose1d are arranged in sequence after Conv1×1; Conv1×1 is used to adjust the number of channels to prepare for subsequent deconvolution operations; GLU is used to introduce nonlinearity and further filter features; ConvTranspose1d shares the same hyperparameters with the Conv1d of the paired encoder layer, which is used to permute the number of input and output channels.

4. The method for diagnosing hoist faults based on audio two-dimensional depth features according to claim 3 is characterized in that: In step 1 S11, the construction process of the middle layer is as follows: Five Transformer encoder layers are used to form the middle layer of the elevator audio signal denoising model, and each Transformer encoder layer is integrated with a multi-head self-attention mechanism and a position fully connected layer. Through the multi-head self-attention layer, the elevator audio signal denoising model can simultaneously focus on the information of multiple positions in the sequence, significantly improving the ability to capture the long-distance dependencies within the one-dimensional audio time series signal data; the dimension of the elevator audio signal denoising model is set to 512, providing sufficient space for each attention to capture subtle feature differences; Each multi-head self-attention layer in the Transformer encoder layer uses 8 self-attention heads in parallel to enhance the expressiveness and learning ability of the elevator audio signal denoising model, which is used to map the input noisy frequency time series data to multiple subspaces respectively and perform self-attention operations in each subspace; at the same time, a jump connection is followed by each multi-head self-attention layer, and a layer normalization operation is performed after the jump connection; the positional fully connected layer in the Transformer encoder layer is used for further transformation and feature fusion, and the input channel size of the layer is matched with the dimension of the elevator audio signal denoising model, so that the output channel size of the layer is expanded to 2048, and this expansion is used to increase the capacity of the elevator audio signal denoising model, so that the elevator audio signal denoising model can capture more complex features and provide richer information to the decoder layer.

5. The method for diagnosing hoist faults based on audio two-dimensional depth features according to claim 4 is characterized in that: In step 1 S12, the construction process of the loss function is as follows: A1: Based on the difference between the input data and the reconstructed data, the autoencoder loss is obtained according to formula (1); （1）； In the formula, is the denoised signal In the The value of the sample points, It is a pure signal In the The value of the sample points, is the total number of sample points; A2: Based on the Euclidean distance between the two signal spectra, the spectrum convergence loss is obtained according to formula (2); based on the difference between the two signal amplitude spectra, the amplitude loss is obtained according to formula (3); （2）； （3）； In the formula, and They correspond to pure signals and denoised signal STFT amplitude spectrum of; is the Frobenius norm; and The denoised signals are and pure signal In the The amplitude value of the STFT window; A3: Combining the spectrum convergence loss and amplitude loss, the short-time Fourier transform loss is obtained according to formula (4); （4）； In the formula, It is the weight coefficient for balancing SC Loss and MAG Loss, and its value is 0.5; A4: Combining the autoencoder loss and the short-time Fourier transform loss, the loss function of the booster audio signal denoising model is obtained using formula (5); （5）。 6. The method for diagnosing hoist faults based on audio two-dimensional depth features according to claim 5 is characterized in that: In step three, the process of converting the one-dimensional audio data into a two-dimensional image format is as follows: S31: For a length of Time series data , the frequency domain characteristics of the time series are analyzed by fast Fourier transform, and the components of the time series at different frequencies are obtained according to formula (6); （6）； In the formula, is the first frequency components, is the total number of samples in the signal, represents the frequency index, corresponding to The frequency of the time points, is the base of natural logarithms, is an imaginary unit; S32: Identify the main period of the time series based on the frequency components with the largest amplitudes, and obtain the period length corresponding to the frequency with the largest amplitude according to formula (7) ; （7）； In the formula, Represents the index of the frequency component with the maximum amplitude; S33: Use modulo operation to calculate the difference between the current data length and the cycle length, and obtain according to formula (8): When When the value of is 0, there is no need to perform zero padding. Otherwise, zero padding is performed on the original time series to ensure that the sequence length can be divided by the period length. Divide integers and calculate the number of zeros to be added according to formula (9) , and the length of the padded sequence is obtained by formula (10): ; （8）； （9）； （10）； In the formula, Indicates rounding up to ensure that the length of the entire sequence is the period length An integer multiple of ; S34: For frequency , the corresponding period is The filled time series , according to formula (11), it is reshaped into multiple two-dimensional tensors ; At the same time, Formula (11) is written as ; （11）； In the formula, , , For the corresponding frequency A two-dimensional tensor of size , Indicates Line value.

7. The method for diagnosing hoist faults based on audio two-dimensional depth features according to claim 6 is characterized in that: In step 4 S43, the diagnosis process of audio faults under complex working conditions of the elevator is as follows: B1: Initialization, initialize the weights of each 2D convolutional layer according to formula (12) ; （12）； B2: Extract features from the input two-dimensional tensor according to formula (13); （13）； In the formula, , The formula is ,in, To obtain the comprehensive output after averaging the results of all convolutional layers, Is the application The result after the convolution kernel, that is, the operation of each convolution layer, , Indicates that the convolution kernel size is Convolution operation; B3: The learned two-dimensional representation Convert back to one-dimensional space and reshape the two-dimensional data into a size of Matrix, and using Perform truncation operation; （14）； In the formula, One-dimensional signal ; B4: Use the softmax method to normalize the amplitude and obtain the first The one-dimensional representation of a layer represents its weight; （15）； B5: The first Layer Output ; （16）； B6: According to Layer Output The complex time series dependencies and periodic features in the data are parsed, and then the fault type of the bearing in the elevator is identified and output.

Images