Elevator door abnormal sound detection and identification method based on VAE and Transform feature reconstruction

Abstract

The invention provides an elevator door abnormal sound detection and identification method based on a variational auto-encoder (VAE) and a Transform network. The method comprises the steps of data acquisition and preprocessing, model construction and training, abnormal sound detection and identification, method performance evaluation and the like. The method specifically comprises the following steps: acquiring sound signals in the operation process of an elevator door by installing an audio sensor, continuously recording to form time sequence data, and cleaning and normalizing the time sequence data; then, feature reconstruction is carried out through a VAE-Transform network, a reconstruction error is calculated to obtain an abnormal score, abnormal feature vectors are continuously marked, finally, abnormal segments are input into a classifier, and the abnormal sound types are judged according to the confidence score of each abnormal sound type. According to the method, unsupervised network training is introduced to complete abnormal sound detection, and then the detected abnormal sound is further classified and identified, so that the calculation amount of data can be significantly reduced, the efficiency is greatly improved, the specific fault type is rapidly identified, and accurate maintenance suggestions are provided, thereby improving the fault processing efficiency and accuracy, and improving the fault processing efficiency and accuracy. The problem that abnormal sound fault data are scarce is effectively solved, and the method is suitable for abnormal sound fault monitoring of the elevator door in a complex environment.

Description

Technical Field

[0001] The present invention relates to the technical field of elevator fault detection, and in particular to a method for detecting and identifying abnormal noise of elevator doors based on VAE and Transformer feature reconstruction. Background Art

[0002] As an indispensable vertical transportation equipment in modern buildings, the safety and stability of elevator operation are crucial to ensuring personnel safety and improving user experience. In the daily use of elevators, the door system is one of the most frequently operated parts and also the component with a higher failure rate. Common door system failures or abnormalities mainly include: mechanical failure of the door system, people caught in the elevator door, construction waste blocking the guide rails, objects blocking the elevator door, and other improper use. Usually these different failures or abnormal phenomena are accompanied by abnormal sounds with different characteristics. Therefore, by using sound detection and feature recognition, elevator door system failures or abnormalities can be discovered and identified in a timely manner, thereby ensuring the safety of elevator operation and passenger safety.

[0003] Traditional elevator door abnormal noise detection and identification mainly rely on manual experience and regular maintenance inspections, and there are problems such as low detection efficiency, high false alarm rate, and high maintenance cost. In recent years, with the development of artificial intelligence and machine learning technology, data-driven fault detection methods have gradually become a research hotspot. Existing invention methods usually use general detection and identification processes to detect and identify abnormal noises in elevators, and most existing methods rely on labeled data, and obtaining a large amount of high-quality labeled data often requires a lot of manpower and time. Here, by introducing unsupervised model training to complete the detection of abnormal sounds, and then further classifying and identifying the detected abnormal sounds, it can significantly reduce the amount of data calculation and greatly improve efficiency, quickly identify specific fault types and provide accurate maintenance suggestions, thereby improving the efficiency and accuracy of fault handling. Summary of the invention

[0004] In view of this, the present invention aims to provide an elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction, which uses VAE-Transformer to reconstruct the elevator audio signal features and detect abnormal noise by calculating the abnormality score, and combines the abnormal noise detection results to perform efficient category diagnosis of the elevator door abnormal noise, so as to accurately find the root cause of the abnormal noise. The detection and recognition method of the present invention includes the following steps:

[0005] Step 1. Data collection and preprocessing:

[0006] (1) Data collection: An audio sensor is installed in the middle of the elevator door panel to capture the overall vibration and abnormal sound of the door panel when it is opened and closed, and to centrally collect various sound signals when the elevator door is opened and closed. During the process of opening and closing the elevator door, the audio sensor continuously records data to form time series data.

[0007] (2) Data preprocessing: The collected data is preprocessed by data cleaning, missing value filling, normalization, etc.

[0008] (3) Denoising: The least mean square algorithm (LMS) and the recursive least squares algorithm (RLS) are integrated to construct an adaptive filter, adaptively adjust the filter parameters, suppress noise interference, and achieve real-time noise suppression.

[0009] Step 2. Establishment of sample library

[0010] (1) Establishment of abnormal sound detection sample library: The normal and abnormal sound data of elevator doors in different environments and working conditions are simply labeled, and the training set and test set are divided according to a certain ratio to construct the abnormal sound detection sample library.

[0011] (2) Establishment of abnormal sound type library: Four types of abnormal sounds (abnormal sounds) of elevator doors are labeled: door lock failure, foreign objects on the guide rail, human knocking, and obstacles. The training set and test set are divided according to the proportion to build an abnormal sound type library.

[0012] Step 3. Construction and training of abnormal sound detection and recognition model:

[0013] (1) Feature extraction: Extract the time domain, frequency domain and time-frequency domain features of the sound, such as mean, standard deviation, peak value, spectral density, main frequency component, instantaneous frequency and instantaneous amplitude.

[0014] (2) VAE-Transformer network construction: Map the input features to the latent space and extract them as latent variables. Use the sum of reconstruction error and KL divergence as the loss function to optimize the VAE network parameters. Use the multi-head self-attention mechanism and position encoding to capture the global dependencies in time series data, and use the VAE latent variables as the input of the Transformer to further extract deep features.

[0015] (3) Design a classifier: Input the features of various abnormal noise types into the LSTM, learn the features of different types of abnormal noise, model the dependencies on the time series, capture the temporal dynamic characteristics of the data, and finally perform confidence judgment and classification of different abnormal noise types through the fully connected layer.

[0016] (4) Unsupervised training: Use unlabeled data for training and optimize model parameters by minimizing the reconstruction error.

[0017] Step 4. Evaluation of abnormal sound detection and recognition accuracy:

[0018] Calculate the accuracy, precision, recall and other indicators of the model, and calculate the false alarm rate and missed alarm rate to verify the accuracy and reliability of the method. Deploy the detection system in the actual elevator system, conduct field testing and optimization, collect field data, continuously optimize the model, and improve detection accuracy and robustness.

[0019] Step 5. Detection and identification of abnormal noise:

[0020] (1) Reconstruction error calculation: Use the trained VAE-Transformer network to reconstruct the newly collected elevator door data and calculate the reconstruction error of each sample.

[0021] (2) Anomaly score: A detection threshold is set based on the statistical characteristics of the reconstruction error (such as mean, standard deviation, and percentile) to generate an anomaly score. If the reconstruction error exceeds the set threshold, it is judged as an abnormal sound.

[0022] (3) Abnormal sound time mark: When an abnormal sound is detected, the start and end time of the abnormality in the time series are marked to ensure accurate positioning of the time period when the abnormal sound occurs.

[0023] (4) Abnormal sound classification: The abnormal sound detection results are input into the abnormal sound detection network. The local and global features of the abnormal period marked by the abnormal sound detection are quickly extracted, and the result of feature fusion is input into the fully connected layer for classification. Finally, the confidence score of each abnormal sound type is output. If the confidence scores of all abnormal sound types are lower than the preset threshold (for example, 0.75), the abnormality is considered to be an unknown abnormal sound type.

[0024] Compared with the prior art, the elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction described in the present invention has the following advantages:

[0025] 1. Compared with traditional methods, the elevator door abnormal sound detection and recognition method based on VAE and Transformer feature reconstruction uses variational autoencoder (VAE) for unsupervised learning, avoiding the reliance on a large amount of labeled data. Through the reconstruction error of VAE, this method can effectively detect abnormal sounds of elevator doors without labeling, improving the versatility and adaptability of the model.

[0026] 2. The present invention detects abnormal noises by reconstructing audio features, and makes full use of the detection results for efficient classification and identification, accurately determining the type of abnormal noises. The model can accurately distinguish different types of abnormal noises from elevator doors, such as door lock failure, foreign objects on the guide rail, human knocking, and obstacles blocking four types of abnormal noises, greatly improving the efficiency of fault identification in practical applications.

[0027] 3. The present invention combines the Transformer network and the LSTM network. The Transformer extracts global features, and the LSTM is used to model the dependencies on the time series, so as to better capture the characteristics of the abnormal sound of the elevator door in the time dimension. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the design diagram of the method framework of the present invention, which intuitively and specifically demonstrates the design architecture and detection effect of the elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction.

[0029] Figure 2 This is an overall step diagram of the method of the present invention, showing the detailed step flow of the elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction.

[0030] Figure 3 This is a diagram of the detection and recognition training principle of the present invention, which shows the training principle of elevator door abnormal noise detection and recognition based on VAE and Transformer feature reconstruction.

[0031] Figure 4 It is the main network structure diagram of elevator door abnormal noise detection and recognition, which shows how VAE and Transformer reconstruct feature vectors and their specific network structure. DETAILED DESCRIPTION

[0032] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings:

[0033] Reference Figure 2 and Figure 3 The specific implementation method of the elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction includes the following steps:

[0034] Step 1. Data collection and preprocessing:

[0035] (1) Data collection: An audio sensor is installed on the elevator door to collect sound signals during the operation of the elevator door and form time series data.

[0036] (2) Further data cleaning and normalization are performed on the data to ensure data quality. Data cleaning filters out abnormal values ​​in the sound data by setting a threshold (assuming there is a set of elevator sound data X s ={x1,x2,...,x n}):

[0037] X clean ={x i ∈X s abs(xi -μ s )<k·σ s}

[0038] Among them, μ s and σ s are the mean and standard deviation of the sound data, respectively, and k is the set threshold multiple. The data normalization process uses the minimum-maximum normalization method to normalize the sound data X s To process:

[0039]

[0040] Among them, x i is each data point in the sound data, min(X s ) and max(X s ) are the minimum and maximum values ​​of the sound data, respectively, x i ' is the normalized sound data.

[0041] At the same time, for missing data points, interpolation method (average value of adjacent data points) is used to fill them.

[0042] (3) Denoising: The least mean square algorithm (LMS) and the recursive least squares algorithm (RLS) are integrated to construct an adaptive filter, adaptively adjust the filter parameters, suppress noise interference, and achieve real-time noise suppression.

[0043] Step 2. Establishment of sample library:

[0044] (1) Establishment of abnormal sound detection sample library: The normal and abnormal sound data of elevator doors under different loads and various environments are simply labeled, and the training set and test set are divided into a ratio of 9:1 to construct the abnormal sound detection sample library.

[0045] (2) Establishment of abnormal sound type library: Four types of abnormal sounds (abnormal sounds) of elevator doors are labeled: door lock failure, foreign objects on the guide rail, human knocking, and obstacles. The four types of abnormal sounds are divided into training sets and test sets in a 9:1 ratio to build an abnormal sound type library.

[0046] Step 3. Construction and training of abnormal sound detection and recognition model:

[0047] (1) Feature extraction: Extract time domain features, frequency domain features, and time-frequency domain features, including mean, standard deviation, peak value, spectrum density, main frequency component, instantaneous frequency, and instantaneous amplitude. The extracted time domain features, frequency domain features, and time-frequency domain features are concatenated to form a feature vector for model training and detection.

[0048] (2) VAE-Transformer network construction: Figure 4As shown in the figure, the VAE and Transformer networks are designed, and the VAE-Transformer model is trained unsupervised using normal audio data, and the model parameters are optimized by minimizing the reconstruction error. The audio signal x is represented by audio features through feature extraction:

[0049] X=FeatureExtraction(x)

[0050] X is the audio feature matrix. The input feature X is mapped to the latent space through the VAE encoder to obtain the mean μ and logarithmic variance log(σ 2 ):

[0051] μ=f μ (X)

[0052]

[0053] Among them, f μ and is the neural network of the encoder;

[0054] Sampling the latent variable z from the latent space:

[0055]

[0056] in

[0057] Reconstruct the input data X from the latent variables z:

[0058]

[0059] where g is the decoder neural network, is the reconstructed feature;

[0060] The features reconstructed by VAE are input into the Transformer model for processing. Transformer captures the temporal dependency of audio features through self-attention mechanism and position encoding.

[0061] In order to make Transformer aware of the order of the sequence, the position encoding PE is used to first encode the audio sequence features. To process:

[0062]

[0063]

[0064] Among them, pos represents the position, i represents the dimension, and d is the feature dimension.

[0065] Calculate the attention weight through the self-attention mechanism:

[0066]

[0067] Where Q, K, and V are query, key, and value matrices respectively, and d k is the dimension of the key.

[0068] Multi-head self-attention and feed-forward neural networks:

[0069]

[0070]

[0071] After multi-layer encoding, the processed feature sequence Y is:

[0072]

[0073] Input the processed feature sequence Y into VAE for reconstruction:

[0074]

[0075] z Y =μ Y +σ Y ·

[0076]

[0077] (3) Classifier construction: For classification tasks, the cross entropy loss function is used to measure the difference between the output probability distribution of the model and the true label.

[0078]

[0079] Among them, y j is the One-Hot encoding of the true label, and P(y=j|z) is the model's predicted probability for category j.

[0080] The gradient is calculated through the back-propagation algorithm, and the model parameters are updated using the Adam optimizer. The update formula of the Adam optimizer is as follows:

[0081]

[0082] Among them, θ t is the model parameter after the tth iteration, η is the learning rate, m t and v t are the first-order and second-order momentum estimates, representing the expectation of the gradient and the expectation of the squared gradient, respectively.

[0083] (4) Unsupervised training: Set the loss function to optimize the model, including reconstruction error and KL divergence:

[0084]

[0085] Reconstruction error:

[0086]

[0087] KL divergence:

[0088]

[0089] Where N is the dimension of the latent variable z;

[0090] Step 4. Evaluation of abnormal sound detection and recognition accuracy:

[0091] In order to evaluate the performance of the abnormal noise detection and recognition model in the elevator door abnormal noise detection method based on VAE and Transformer feature reconstruction, the following steps are used for verification and evaluation:

[0092] (1) Model testing:

[0093] The trained detection and recognition model was verified using the test data sets of the abnormal sound detection sample library and the abnormal sound type library. The test data was reconstructed through the detection network and the reconstruction error was calculated. The set threshold was used to determine whether there was an abnormality in each time period, and the confidence distribution of each category was calculated through the classifier to determine the type of abnormal sound. The preliminary performance results of the model were verified by calculating the AUC and pAUC indicators.

[0094] (2) Evaluation indicators:

[0095] In order to observe the performance of the detection method, the accuracy, precision, and recall rate indicators of the model are calculated:

[0096] Accuracy:

[0097] Accuracy:

[0098] Recall:

[0099] Among them, TP: abnormal sound data correctly identified; TN: normal data correctly identified; FP: false alarm abnormal sound data; FN: missed alarm abnormal sound data.

[0100] (3) False alarm rate and missed alarm rate:

[0101] The false alarm rate and missed alarm rate can verify the accuracy and reliability of the method. Understanding the system's errors in detecting and identifying abnormal noises can help optimize the decision-making process.

[0102] False Positive Rate:

[0103] False negative rate:

[0104] Step 4. Detection and identification of abnormal noise:

[0105] (1) Reconstruction error calculation:

[0106] Use the trained VAE-Transformer model to extract features and reconstruct the newly collected data:

[0107] X new =FeatureExtraction(x new )

[0108]

[0109] For each feature vector X new,i , calculate its reconstruction error RE i :

[0110]

[0111] The set of reconstruction errors is expressed as: RE = {RE1, RE2, ..., RE N}

[0112] (2) Anomaly score: Generate anomaly score A based on reconstruction error i :

[0113] A i =RE i

[0114] Set a threshold τ to judge anomalies. Here, the threshold is determined based on the reconstruction error distribution of the training data, using the percentile (95 percentile):

[0115] τ=percentile(RE train ,95)

[0116] Determine whether each feature vector is abnormal based on the set threshold:

[0117]

[0118] (3) Abnormal time stamp:

[0119] By analyzing the continuous abnormal marks, we can determine the start and end time of the abnormal sound. Assuming the time step is T, we can get the following time series:

[0120] Anomaly=[Anomaly1,Anomaly2,...,Anomaly N ]

[0121] Find the time point when the first abnormality occurs as the starting time: t start =min{t i ∣Anomaly i =1}

[0122] Find the last abnormal time point as the end time: t end =max{t i ∣Anomaly i =1}

[0123] (4) Abnormal sound classification:

[0124] According to the results of abnormal sound detection, the abnormal sound part in the elevator sound data is feature extracted and input into LSTM to obtain the h of the abnormal sound feature. T Time series modeling, generating the score z for each abnormal sound type through the fully connected layer j :

[0125]

[0126] Finally, the probability distribution (i.e. confidence score) of each category is calculated by the Softmax function:

[0127]

[0128] Among them, P(y=j|z) is the score of category j.

[0129] Set the confidence score threshold to 0.75. If the confidence score of a category exceeds the threshold, it is considered to be an abnormal sound type of that category. If the confidence scores of all categories are lower than the threshold, it is considered to be an abnormal sound:

[0130] max j P(y=j|z)<0.75

[0131] In order to verify the performance of the model in the anomaly detection and recognition task, the method proposed in this description is compared with other typical abnormal sound detection and recognition methods. They are abnormal sound detection and recognition algorithm based on CNN-LSTM, CNN-RNN abnormal detection and recognition algorithm (CRNN), support vector machine abnormal recognition algorithm (SVM), and autoencoder abnormal sound detection and recognition algorithm (AE). In addition, the experimental comparison results are shown in Table 1.

[0132] Table 1 Performance comparison of several commonly used algorithms

[0133]

[0134] The experimental results show that the method of the present invention is better than other typical abnormal sound recognition methods in the detection and recognition of abnormal noise of elevator doors. It also reflects that the reconstruction error combined with VAE-Transformer has excellent effects on the correlation of long time series and abnormal sound monitoring and diagnosis of elevator doors in complex environments.

[0135] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A method for detecting and identifying abnormal noise of elevator doors based on VAE and Transformer feature reconstruction, characterized in that: The detection and identification method comprises the following steps: Step 1: Install an audio sensor in the middle of the elevator door panel to collect audio signals when the elevator is running in different environments and under different working conditions, perform preprocessing and denoising, and retain the main frequency components when the elevator door is running; Step 2: Label the processed data and divide it into training set and test set in proportion to establish a sample library for elevator door abnormal noise detection and recognition; Step 3. Use VAE variable autoencoder to map the normal audio data features in the sample data to the latent space, extract the deep features in the latent variables through Transformer, capture the long-term dependencies, and then re-enter the VAE for feature reconstruction to form a feature vector. The abnormal score is obtained by calculating the reconstruction error. The abnormal segment is input into the classifier for confidence calculation to obtain the probability distribution of each category, so as to judge the abnormal sound category. Continuously use sample data to train the abnormal sound detection and recognition model, and calculate the gradient through the back propagation algorithm to optimize the model parameters; Step 4: Use the test sample data in the sample library to perform performance testing, calculate performance indicators, and verify the accuracy and reliability of the method; Step 5: Detect abnormal noises, calculate the confidence of abnormal segments through a classifier, and finally output the type of abnormal noise.

2. The elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction according to claim 1 is characterized by: In step 1, the audio sensor is installed in the middle of the elevator door panel to capture the overall abnormal sound of the door panel when it is opened and closed, and to more centrally collect various sounds when the elevator door is opened and closed. The data preprocessing is to clean and normalize the data to ensure the data quality. Data cleaning filters out outliers in sound data by setting thresholds, and data normalization uses the minimum-maximum normalization method to process sound data. At the same time, interpolation method (the average value of adjacent data points) is used to fill in missing data points. The denoising method is to integrate the least mean square algorithm (LMS) and the recursive least square algorithm (RLS) to construct an adaptive filter, adaptively adjust the filter parameters, suppress noise interference, and perform real-time noise suppression.

3. The elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction according to claim 1 is characterized by: In step 2, the sample library includes an abnormal sound detection sample library and an abnormal sound type library. The abnormal sound detection sample library is established by annotating the audio data of the elevator door operation, annotating them as normal and abnormal labels according to the abnormal threshold of the audio feature, and dividing them into a training set and a test set at a ratio of 9:1; the abnormal sound type library is established by annotating four abnormal sound types, namely, door lock failure, foreign matter on the guide rail, human knocking, and obstacle blocking, and dividing them into a training set and a test set at a ratio of 9:

1.

4. The elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction according to claim 1 is characterized by: The step 3 further comprises the following steps: (1) Feature extraction: Extract the time domain, frequency domain and time-frequency domain features of the sound, such as mean, standard deviation, peak value, spectral density, main frequency component, instantaneous frequency and instantaneous amplitude, and concatenate them to form a feature vector as the input for detection and recognition. (2) VAE-Transformer network construction: The input feature X is mapped to the latent space through the VAE encoder to obtain the mean μ and logarithmic variance log(σ 2 ), sample the latent variable z from the latent space and reconstruct the input data to obtain the reconstructed data Use position encoding PE to first encode the audio sequence features After processing, the attention weight is calculated through the self-attention mechanism, and the processed feature sequence Y is obtained through multi-layer encoding, and then reconstructed through the VAE network to obtain (3) Design a classifier: Input the features of various abnormal noise types into the LSTM, learn the features of different types of abnormal noise, model the dependencies on the time series, capture the temporal dynamic characteristics of the data, and finally perform confidence judgment and classification of different abnormal noise types through the fully connected layer. (4) Unsupervised training: Use unlabeled data for training, calculate the reconstruction error, and use the sum of the reconstruction error and KL divergence as the loss function for model optimization. For classification tasks, the cross entropy loss function is used to measure the difference between the output probability distribution of the model and the true label, and the gradient is calculated through the back-propagation algorithm, and the Adam optimizer is used to update the model parameters.

5. The elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction according to claim 1 is characterized by: In the step 4, the performance of the model is verified by evaluating the accuracy and reliability of the abnormal noise detection and recognition model in the elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction through model testing, calculation of AUC and pAUC, false alarm rate and missed alarm rate and other evaluation indicators.

6. The elevator door abnormal noise detection and recognition method based on VAE and Transformer feature reconstruction according to claim 1 is characterized by: In step 4, the detection and identification of abnormal sound includes the following steps: (1) Reconstruction error calculation: Use the trained VAE-Transformer model to extract features from the newly collected data and reconstruct them. For each feature vector, calculate its reconstruction error RE i , the set of reconstruction errors is expressed as: RE={RE1,RE2,...,RE N } (2) Anomaly score: Generate anomaly score A based on reconstruction error i : A i =RE i , set a threshold τ to judge the anomaly, determine the threshold based on the reconstruction error distribution of the training data, and judge whether each feature vector has an anomaly according to the set threshold. (3) Abnormal sound time mark: The abnormal sound time mark takes the time point of the first abnormality as the starting time and the time point of the last abnormality as the ending time. The starting time and ending time of the abnormal sound are determined by analyzing the continuous abnormality marks. (4) Abnormal sound classification: Based on the abnormal sound detection results, the abnormal sound part in the elevator sound data is extracted by CNN local features and Transformer global features, and then input into LSTM to obtain the h of the abnormal sound features. T Time series modeling, generating the score z for each abnormal sound type through the fully connected layer j Finally, the probability distribution of each category (i.e., confidence score) is calculated through the Softmax function, and the confidence score threshold is set to 0.

75. If the confidence score of a category exceeds the threshold, it is considered to be the abnormal sound type of that category. If the confidence scores of all categories are lower than the threshold, it is determined to be an abnormal sound, and finally the category of the abnormal sound is output.

Images