Escalator abnormal sound detection method and system based on domain invariant feature transfer and clustering

Abstract

This invention discloses a method and system for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering. The method constructs feature matrices from audio signal samples in the source and target domains, trains a domain adversarial network using these feature matrices to learn its attention weight matrix, applies the attention weight matrix to audio signal samples in the target domain dataset to obtain a weighted feature representation, and determines an anomaly detection and diagnosis threshold using a clustering algorithm. The method then detects the audio signal of the target escalator in real time; if the signal exceeds the anomaly detection and diagnosis threshold, it is determined to be abnormal audio. This invention enables sensitive feature selection and cross-domain adaptation of high-dimensional acoustic features under realistic conditions such as scarce labeled data, significant environmental interference, and domain shift, improving the accuracy, robustness, and interpretability of fault detection. Therefore, it provides a highly adaptable and reliable technical solution for predictive maintenance of escalators.

Description

Technical Field

[0001] This invention relates to the field of fault diagnosis technology for key equipment in rail transit, and in particular to a method and system for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering. Background Technology

[0002] Escalators are critical transportation equipment in densely populated areas of urban rail transit stations, and their operational status directly affects passenger safety and transportation efficiency. Currently, the operation and maintenance of escalators mainly rely on periodic inspections and manual judgment based on experience, which suffers from low detection efficiency, delayed fault response, and high maintenance costs. Although escalator fault diagnosis technology based on sound signals has shown potential, it faces significant challenges in practical engineering deployment.

[0003] The primary challenge is the scarcity of labeled fault data. Due to the high safety standards and low failure rate of escalators, it is difficult to obtain sufficient labeled fault audio data for training reliable supervised learning models.

[0004] Secondly, environmental interference leads to poor audio data quality. Escalator operating environments are subject to various interferences such as passenger noise, background announcements, and noise from other equipment, resulting in frequent abnormal data and significant variations in normal state data in the collected audio signals, severely impacting the accuracy of feature extraction and state recognition.

[0005] Finally, a deeper and often overlooked challenge is the domain shift problem. The operating sound characteristics of escalators are affected by various factors such as ambient noise at their installation location, equipment model, wear and tear, and load conditions. This leads to a significant performance degradation when a diagnostic model trained in a specific scenario is deployed to another scenario with different acoustic environments or equipment (such as another subway station or shopping mall) due to differences in the distribution of sound characteristics. This domain shift phenomenon makes the model difficult to generalize, severely restricting the large-scale, standardized engineering application of sound-based fault diagnosis technology. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide a method and system for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering, which solves the problems of existing technologies relying on labeled fault data or expert experience and having poor generalization ability.

[0007] Technical solution: The escalator abnormal sound detection method based on domain-invariant feature transfer and clustering described in this invention includes the following steps:

[0008] Audio signal samples and fault labels of escalators are obtained to construct a source domain dataset, and audio signal samples of target escalators are obtained to construct a target domain dataset.

[0009] For the source and target domain datasets: the audio signal samples are preprocessed to obtain the audio time-domain signal of each frame, and several time-domain features and frequency-domain features of each audio time-domain signal are calculated. A feature matrix of each audio signal sample is constructed based on the time-domain features and frequency-domain features. A domain adversarial network is trained using the feature matrix. The domain adversarial network includes an attention weight generator, and the attention weight matrix of the feature matrix is ​​learned through a neural network.

[0010] The attention weight matrix is ​​applied to audio signal samples in the target domain dataset to obtain a weighted feature representation, and the anomaly detection and diagnosis threshold is determined by a clustering algorithm.

[0011] The audio signal of the target escalator is detected in real time, a target weighted feature representation of the audio signal is constructed, and the abnormal detection and diagnosis threshold is used to determine whether the audio signal is abnormal.

[0012] Furthermore, the source domain dataset is constructed by obtaining audio signal samples and fault labels from escalators, including:

[0013] Acoustic sensors are installed at key parts of the escalator, including acquiring the operating audio of the motor, drive chain and reducer in the upper pit, acquiring abnormal sound of step offset and compression at two measuring points in the middle of the escalator, and acquiring the operating audio of the step chain in the lower pit.

[0014] Furthermore, preprocessing the audio signal samples to obtain the audio time-domain signal for each frame includes:

[0015] After normalizing the audio signal samples, a high-pass filter is used to boost the high-frequency components. Then, the samples are divided into short time frames and windowed using a Hamming window to obtain the audio time-domain signal of each frame.

[0016] Furthermore, the time-domain features include one or more of the following: mean, RMS value, peak-to-peak value, variance, kurtosis, skewness, peak factor, impulse factor, zero-crossing rate, and energy; and the frequency-domain features include the frequency-domain feature Mel-spectral coefficients.

[0017] Furthermore, the domain adversarial network includes an attention weight generator, a classifier, a domain discriminator, and an inversion layer. Through adversarial training between the attention weight generator, the domain discriminator, and the classifier, the attention weight generator learns attention patterns of domain-invariant features.

[0018] Furthermore, the anomaly detection and diagnostic thresholds are determined using clustering algorithms, including:

[0019] Calculate the unique cluster center of the set of weighted feature representations, and calculate the relative distance from each weighted feature representation to the cluster center;

[0020] Statistical analysis of the set of relative distances based on box plots is used to determine the anomaly diagnosis threshold;

[0021] If the relative distance from the target weighted feature representation to the cluster center exceeds the anomaly diagnosis threshold, then the audio signal of the real-time detected target escalator is determined to be abnormal audio.

[0022] Furthermore, determining the anomaly diagnostic threshold by performing statistical analysis on the set of relative distances based on box plots includes:

[0023] Calculate the upper quartile and interquartile distance of the set of relative distances, calculate the base reference relative distance based on the upper quartile and interquartile distance, and set the anomaly diagnosis threshold as the product of the base reference relative distance and the threshold coefficient.

[0024] Furthermore, the method also includes rolling updates of the anomaly diagnosis threshold, wherein the update method includes:

[0025] During the continuous operation of the target escalator, the relative distance values ​​of the accumulated normal audio signals form a dynamic distance set. When the number of the relative distance values ​​reaches a first threshold, a sliding time window is used to obtain a certain number of relative distance values.

[0026] For the relative distance values ​​within the sliding time window, weighted quantiles are calculated to construct box plot statistics and update the anomaly diagnosis threshold.

[0027] Furthermore, for the relative distance values ​​within the sliding time window, weighted quantiles are calculated to construct boxplot statistics, and the anomaly diagnostic thresholds are updated, including:

[0028] Calculate the weighted upper quartile and weighted lower quartile; calculate the weighted quartile distance; update the base reference relative distance based on the weighted upper quartile and weighted quartile distance; update the anomaly diagnosis threshold as the product of the updated base reference relative distance and the threshold coefficient.

[0029] The escalator abnormal sound detection system based on domain-invariant feature transfer and clustering described in this invention includes:

[0030] The data acquisition unit is used to acquire audio signal samples and fault labels of escalators to construct a source domain dataset, and to acquire audio signal samples of target escalators to construct a target domain dataset.

[0031] The source domain data domain adversarial training unit, for the source domain and target domain datasets, preprocesses the audio signal samples to obtain the audio time-domain signal of each frame, calculates several time-domain features and frequency-domain features of each audio time-domain signal, constructs a feature matrix for each audio signal sample based on the time-domain features and frequency-domain features, and trains a domain adversarial network using the feature matrix. The domain adversarial network includes an attention weight generator, which learns the attention weight matrix of the feature matrix through a neural network.

[0032] The threshold setting unit is used to apply the attention weight matrix to audio signal samples in the target domain dataset to obtain weighted feature representations, and to determine the anomaly detection and diagnosis threshold through a clustering algorithm.

[0033] The target domain data anomaly detection unit is used to detect the audio signal of the target escalator in real time, construct the target weighted feature representation of the audio signal, and determine whether the audio signal is abnormal audio by combining the anomaly detection and diagnosis threshold.

[0034] The electronic device of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the escalator abnormal sound detection method based on domain-invariant feature transfer and clustering.

[0035] The computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering.

[0036] The computer program product of the present invention includes a computer program that, when executed by a processor, implements the escalator abnormal sound detection method based on domain-invariant feature transfer and clustering.

[0037] Beneficial Effects: Addressing the problems of poor cross-device generalization ability of models and over-reliance on labeled fault data and expert experience of target devices in escalator acoustic monitoring, this invention proposes a domain-invariant attention-weighted method and system for diagnosing abnormal escalator sounds, belonging to the unsupervised diagnostic approach. By constructing a manually generated feature matrix, employing a domain adversarial attention mechanism, and using weighted feature diagnosis, accurate identification and early warning of abnormal escalator sounds are achieved. Compared with existing technologies, the advantages of this invention are:

[0038] (1) Significantly improve cross-device generalization ability. By constructing a domain adversarial training framework, domain-invariant time frame weights are learned for handmade feature matrices through an attention weight generator and gradient inversion mechanism. This method effectively eliminates the acoustic characteristics differences between different escalator devices, enabling the model to maintain stable detection performance on target devices with unlabeled data.

[0039] (2) Significantly reduces reliance on labeled data and expert experience. An attention-weighted diagnostic method is adopted, which learns a consistent discrimination pattern across devices through domain adversarial training. This method only requires labeled data from the source domain devices for model training and can be directly applied to the anomaly diagnosis of the target device. It does not require fault label data or manual annotation of the target device, which significantly reduces data collection costs and the need for expert intervention.

[0040] (3) The temporal handcrafted features weighted by the attention mechanism maintain good physical interpretability. The handcrafted features include indicators with clear physical meaning such as mean, effective value, peak-to-peak value, kurtosis, and skewness. Combined with the intelligent weighting of the attention weight generator, the physical interpretability of the features is preserved, and the discriminative feature segments that are stable across devices are highlighted, forming a feature representation with strong interpretability and outstanding discriminative ability.

[0041] (4) End-to-end adaptive process ensures long-term effectiveness: From audio preprocessing, manual feature extraction, domain-invariant weight learning to weighted diagnosis, the entire process achieves complete adaptive diagnosis. The attention-weighted feature representation can adapt to changes in device state. By highlighting stable discriminative segments, it ensures that the model maintains its diagnostic capability in long-term operation and adapts to the challenges of device aging and environmental changes.

[0042] (5) High engineering practicality and low deployment cost: The trained attention weight generator can be deployed independently, combined with lightweight manual feature calculation, balancing computational efficiency and detection accuracy. The manual feature-based approach avoids the computational overhead of complex deep networks, is easy to deploy on edge devices, and ensures cross-device generalization capability through domain adversarial training, significantly reducing system deployment and maintenance costs while ensuring detection accuracy. Attached Figure Description

[0043] Figure 1 This is a flowchart of the escalator abnormal sound detection method according to Embodiment 1 of the present invention.

[0044] Figure 2 This is a flowchart of the escalator abnormal sound detection method according to Embodiment 2 of the present invention.

[0045] Figure 3 This is a flowchart of the domain-invariant weight generator in Embodiment 2 of the present invention.

[0046] Figure 4 This is the original time-domain waveform diagram of the sound sample in Embodiment 3 of the present invention.

[0047] Figure 5 This is a time-domain waveform diagram of the preprocessed sound sample from Embodiment 3 of the present invention.

[0048] Figure 6The weight distribution map generated for a target domain sample in Embodiment 3 of the present invention.

[0049] Figure 7 This is the confusion matrix of the diagnostic results without domain adversarial training in Embodiment 3 of the present invention.

[0050] Figure 8 This is the confusion matrix of the diagnostic results of the adversarial training in the domain of Embodiment 3 of the present invention. Detailed Implementation

[0051] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0052] Example 1

[0053] like Figure 1 As shown, the method for detecting abnormal noises in escalators includes the following steps:

[0054] Step 1: Systematically deploy acoustic sensors in key parts of the escalator to build a multi-source audio monitoring network and establish source and target domain datasets across devices.

[0055] Specifically, the audio signals collected in step 1 of this embodiment include: audio of the motor, drive chain and reducer operation collected in the upper stair pit; abnormal sound of step offset and compression monitored by dual measuring points in the middle of the escalator; and audio of the step chain operation collected in the lower stair pit.

[0056] Step 2 involves standardizing, pre-emphasizing, and preprocessing the acquired normal operating audio signals by frame-by-frame windowing. Handcrafted features are then calculated frame-by-frame to form a handcrafted feature matrix for each sample. Frame-by-frame attention weights for these handcrafted features are learned through domain adversarial training to generate a domain-invariant weight matrix that highlights key cross-device segments.

[0057] Specifically, step 2 of this embodiment includes the following steps:

[0058] S2.1, each normal audio sample in the source domain and target domain datasets is sequentially standardized, pre-emphasized, framed, and windowed to obtain the time domain signal.

[0059] S2.2 calculates time-domain features for each normal-running audio sample in the source and target domain datasets, specifically for each frame of the original time-domain signal. These features include mean, RMS value, peak-to-peak value, variance, kurtosis, skewness, peak factor, impulse factor, zero-crossing rate, energy, and the frequency-domain feature Mel-spectral coefficients. A feature vector is output for each frame. These feature vectors are stacked chronologically to form a handcrafted feature matrix for each sample.

[0060] S2.3 Construct a domain adversarial network containing a gradient inversion layer. Take the handcrafted feature matrix as input and force the attention mechanism to learn a consistent weight allocation pattern across devices through adversarial training between the attention weight generator, domain discriminator, and classifier. Finally, output a domain-invariant time frame weight matrix, where high weights correspond to stable discriminative feature segments in cross-device diagnosis.

[0061] Step 3: Apply the trained attention weighter to the normal operation data of the target domain, and perform cluster analysis based on the enhanced feature representation to achieve abnormal escalator sound recognition in the target domain.

[0062] Specifically, step 3 of this embodiment includes the following steps:

[0063] S3.1 The trained domain-invariant attention weight generator is applied to the normal operation data of each measurement point of the target escalator. The learned attention weight matrix is ​​multiplied frame by frame with the original temporal feature matrix of the target domain data to obtain the weighted features of the key segments. After extracting the weighted feature representation, a clustering algorithm based on relative distance is used to identify the unique cluster center.

[0064] S3.2 Calculate the Euclidean distance from each sample point's feature to the cluster center, forming a distance set. By obtaining the median distance, calculate the ratio of each sample point's distance to the median to obtain a relative distance metric. This relative distance effectively eliminates the influence of absolute scale and better characterizes the degree to which samples deviate from the normal pattern.

[0065] S3.3 performs statistical analysis on the set of relative distances based on the box plot principle. Using the median position as the basic reference relative distance, an adjustable threshold coefficient is introduced to set the anomaly detection and diagnosis threshold.

[0066] S3.4, Real-time detection of the target escalator's operating audio data; calculation of the time-domain and frequency-domain features of the operating audio data according to the method in step 2.2, forming a first hand-crafted feature matrix; multiplication of the domain-invariant weight matrix calculated in step 2.3 with the first hand-crafted feature matrix to obtain a first weighted feature representation; calculation of the relative distance between the first weighted feature representation and the cluster centers obtained in step 3.2; operating audio data with a relative distance exceeding a diagnostic threshold are judged as abnormal sounds, triggering the corresponding alarm mechanism. This method achieves accurate identification and timely warning of abnormal sounds in the target domain through relative distance measurement and adaptive threshold setting.

[0067] In a preferred embodiment, step S3.5, threshold update based on sample size accumulation, is also included. To overcome the limitation of static thresholds in adapting to the slow evolution of equipment status, this embodiment further constructs a sample size-driven threshold baseline rolling update mechanism based on the box plot statistics set in S3, ensuring that the diagnostic threshold always reflects the latest data distribution characteristics. During the continuous operation of the target escalator, the system accumulates the relative distance values ​​of normal samples in real time, forming a dynamic distance set. When the accumulated sample size N reaches the statistically stable threshold, the system no longer relies on the initially calculated fixed quantiles, but instead uses a sliding time window mechanism to dynamically update the threshold.

[0068] Example 2

[0069] like Figure 2 As shown, the method for detecting abnormal noises in escalators includes the following steps:

[0070] Step 1: Construct a multi-source audio monitoring network and dataset.

[0071] Acoustic sensors are systematically deployed at key locations on escalators to construct a multi-source audio monitoring network. Audio data from escalators of different stations and models are collected to form a source domain dataset. ,in For audio samples, For the corresponding normal or abnormal label (e.g., (0 represents normal, 1 represents abnormal), where This indicates the number of samples collected from the source domain, with the superscript 's' representing the source domain. Simultaneously, audio data of the target escalator's normal operation is collected to form the target domain dataset. The superscript t represents the source domain. This indicates the number of samples collected from the target domain, where i and j represent samples from the two data domains, respectively.

[0072] Step 2: Audio preprocessing and training of the domain-invariant attention weight generator.

[0073] Step 2 is the core step of this invention. Although unsupervised learning does not rely on labeled data, a large amount of irrelevant or redundant information in the high-dimensional features of the original audio can interfere with model learning, leading to poor clustering results, ambiguous state identification, and reduced model robustness in subsequent steps. Traditional dimensionality reduction methods, while simplifying the feature space, often lose fault-sensitive information and physical meaning, making it difficult to retrospectively interpret diagnostic results and hindering engineering understanding and system optimization. Therefore, this embodiment uses attention-weighted temporal handcrafted features, which retains the physical interpretability of the features while highlighting cross-device stable discriminative feature segments, forming a feature representation with strong interpretability and outstanding discriminative ability. The specific implementation process is as follows: Figure 3 As shown, it includes the following steps.

[0074] Step 2.1, audio signal preprocessing: For each source and target domain support sample with a duration of 5 seconds, since the preprocessing process is consistent, symbols are used. replace and For audio samples Perform the following processing:

[0075] (1) Standardization

[0076]

[0077] in, For signal The mean of the signal amplitude is normalized to the range [-1, 1].

[0078] (2) Pre-emphasis: A first-order high-pass filter is used to boost the high-frequency components. Its transfer function is: ,in This is the pre-emphasis factor, typically taken as 0.95~0.97. Its calculation in the time domain is as follows:

[0079]

[0080] (3) Framing and Windowing: The pre-emphasized signal is divided into frames with a length of... Frame shift is In this embodiment, L is 40ms and R is 20ms. For the first... Frame signal The time-domain signal is obtained by applying the Hamming window:

[0081]

[0082] Among them, the Hamming window function .

[0083] Step 2.2: Construct the feature matrix manually frame by frame for each frame of time-domain signal. Calculate multiple handcrafted features to form a feature vector. (in (For feature dimensions). For all Frame (for a 5-second audio file) After repeating this process, the feature vectors of each frame are stacked in chronological order to form a handcrafted feature matrix for each sample. The formulas for calculating some manual features are as follows:

[0084] (a) Formula for calculating kurtosis:

[0085]

[0086] in, The number of sampling points per frame. This is the arithmetic mean of the frame signal. The kurtosis formula quantifies the intensity of abnormal impulse components in the signal distribution by the ratio of the fourth moment to the square of the second moment. In the abnormal sound diagnosis of this embodiment, it is a powerful indicator for identifying non-stationary, transient sounds generated by mechanical faults in escalators.

[0087] (b) Mel-frequency cepstral coefficients (MFCCs): For each frame, after windowing, a discrete Fourier transform is performed on each window signal, and then its spectral energy is calculated. The discrete Fourier transform formula is shown below:

[0088]

[0089] in, The result is the Discrete Fourier Transform. For signal length, This represents the number of discrete Fourier points. After the Fourier transform, the spectrum of the frame signal is squared to obtain the spectral energy of each frame. The calculation formula is as follows:

[0090]

[0091] After the calculation is complete, a Mel filter is used for filtering. Before filtering, the frame signal spectrum is first converted to Mel scale to convert the linear frequency... Mapped to Mel frequency The formula is as follows:

[0092]

[0093] Mel filter banks are filter banks developed for low-frequency components. They perform dense filtering at low frequencies and sparse filtering at high frequencies. A Mel filter bank consists of multiple triangular filters, each with a center frequency response of 1, which decays from 1 to 0. Except for the start and end points, the center frequency of each Mel filter is the lower cutoff frequency of the preceding filter and the upper cutoff frequency of the following filter. The frequency response formulas for each filter are as follows. for:

[0094]

[0095] in, The number of filters. For Mel frequency, For the first The frequency response of each filter, after being filtered by the filter bank, is the Mel-energy. for:

[0096]

[0097] After the spectral energy is processed by Mel filtering, it needs to undergo logarithmic and discrete cosine transform to obtain the Mel cepstral coefficients. The formula is as follows:

[0098]

[0099] Where u is the number of frames. For each frame's Mel energy, The number of Mel filters, It is the order of MFCC.

[0100] Ultimately, u can be obtained The coefficient matrix of order 2. In the field of acoustic signal processing, more low-frequency components are usually selected to meet the recognition criteria of the human ear. The 2nd to 13th order Mel-spectral coefficients are often chosen because the values ​​corresponding to the other orders represent the rapid changing trend of the Mel filter and contribute less to signal feature recognition.

[0101] Step 2.3: Domain adversarial training generates a domain-invariant weight matrix and constructs a domain adversarial network containing a gradient inversion layer.

[0102] Domain adversarial networks include attention weight generators Classifier Domain discriminator The attention weight generator employs a temporal frame attention mechanism: global pooling is performed along the feature dimension of the feature matrix, and the importance weights of each temporal frame are learned through a two-layer fully connected network. The temporal frame attention vector is output via a sigmoid function, enabling the network to adaptively select discriminative temporal segments that are stable across devices. These are then multiplied element-wise with the original features to obtain a weighted feature representation. The classifier uses a bottleneck layer plus two layers of fully connected networks. After inputting the weighted features, the bottleneck layer performs feature dimensionality reduction, followed by a fully connected layer, ReLU activation function, and Dropout regularization. Finally, a fully connected layer and Softmax activation function output a normal or abnormal classification prediction. The domain discriminator uses a three-layer fully connected network. After inputting the weighted features processed by the gradient inversion layer, the first fully connected layer, ReLU activation function, and Dropout regularization are applied sequentially; the second fully connected layer, ReLU activation function, and Dropout regularization are applied again; and the third fully connected layer and sigmoid activation function output a domain prediction for the source or target domain. The inversion layer is a special operation layer. During forward propagation, it performs an identity transformation on the input features. During backward propagation, it multiplies the input gradient by a coefficient and then propagates it in the reverse direction, forcing the attention weight generator to learn an attention weight allocation pattern with an unchanged learning domain.

[0103] Attention weight generator The input is a handcrafted feature matrix. This module learns attention weights in the time dimension through a neural network:

[0104]

[0105] in It is an attention weight matrix, each element The weights are non-negative. The weighted feature representation is obtained by element-wise multiplication. .

[0106] Classifier The input is The output is a classification prediction. Its loss function is the cross-entropy loss over the source domain:

[0107]

[0108] Domain discriminator The input is The output is the domain prediction. (Source domain is 0, target domain is 1). Its loss function is the binary cross-entropy loss:

[0109]

[0110] in, It is the domain label of the k-th sample. It is the domain prediction probability output by the domain discriminator.

[0111] The inversion layer is an identity transformation during forward propagation: During backpropagation, it inverts the incoming gradient and multiplies it by a coefficient. : .

[0112] The training objective of a domain adversarial network is a minimax game problem, with the objective function as follows:

[0113]

[0114] Through this confrontation Trained to generate a "deceptive" weight, so that The inability to distinguish which domain the weighted features originate from forces Attention patterns that learn domain-invariant features.

[0115] Step 3: Target domain abnormal sound recognition based on clustering algorithm.

[0116] Step 3.1, target domain feature extraction and single-class clustering modeling.

[0117] The trained domain-invariant attention weight generator The weighted feature representation of the audio data applied to all measurement points in the target domain is obtained. .

[0118] Considering that the target domain typically lacks known outlier samples in the initial stages, a uniclass clustering approach is adopted, treating the entire target domain dataset as a primary "normal" operating pattern for modeling. The K-Means algorithm is used, and the number of clusters is set accordingly. The process aims to find unique cluster centers. The objective function is: [The goal is to minimize the sum of the distances from all samples in the target domain to the center.]

[0119] This cluster center This represents the baseline normal state of the target escalator in the domain-invariant feature space.

[0120] Step 3.2, calculate the relative distance.

[0121] Weighted feature representation for each sample in the target domain Calculate its unique cluster center. Euclidean distance:

[0122]

[0123] This distance Quantified the sample The degree of deviation from the baseline normal state. To eliminate the influence of the absolute scale of the features and to make the threshold setting more universal, the relative distance is calculated. :

[0124]

[0125] in, It is the median distance between all target domain samples. This relative distance indicates how many times greater a sample's deviation is compared to the typical deviation of the entire system.

[0126] Step 3.3, adaptive diagnostic threshold setting.

[0127] Based on the box plot principle, the set of relative distances Perform statistical analysis to automatically determine a reasonable threshold for anomaly diagnosis:

[0128] (1) Calculate the upper quartile of the set. and lower quartiles .

[0129] (2) Calculate the interquartile range .

[0130] (3) Set the base reference relative distance to .

[0131] In statistics, this is usually the boundary for judging data points as "outliers".

[0132] Final diagnostic threshold Set as:

[0133]

[0134] in, This is an adjustable threshold coefficient. It can be set during initial deployment. Operations personnel can fine-tune this parameter later based on the actual accuracy and recall requirements of the alarms (e.g., improving accuracy and recall). To reduce false alarms and lower To improve sensitivity to potential anomalies.

[0135] The core advantage of step 3 lies in: through single-class clustering ( This method establishes a unique normal state benchmark for the target escalator without requiring any anomaly labels. Combined with relative distance based on data distribution and adaptive thresholds, it can sensitively capture any abnormal sounds that deviate from this normal pattern, making it highly suitable for the challenges of limited anomaly samples and insufficient prior knowledge in real-world industrial scenarios.

[0136] Step 4: Abnormal diagnosis and early warning.

[0137] Real-time detection of the target escalator's operating audio data; calculation of the time-domain and frequency-domain features of the operating audio data according to the method in step 2.2 to form a first handcrafted feature matrix; multiplication of the domain-invariant weight matrix calculated in step 2.3 with the first handcrafted feature matrix to obtain a first weighted feature representation; calculation of the relative distance between the first weighted feature representation and the cluster centers obtained in step 3.2. If the system detects an abnormal audio signal, it will determine that the audio data is abnormal. The system will automatically trigger an alarm mechanism, notifying maintenance personnel via monitoring center screen notifications, SMS, or email, and recording the time, location (measurement point), and audio data of the anomaly.

[0138] Step 5: Rolling update of the threshold baseline based on the accumulated sample size.

[0139] In long-term industrial equipment condition monitoring, equipment performance undergoes slow, gradual changes (concept drift) due to factors such as natural aging and wear. Static diagnostic thresholds set based on the initial dataset cannot adapt to these dynamic changes, leading to increased false alarm rates or missed alarm risks. This invention aims to solve the mismatch between static baselines and such slow, time-varying processes. The core of this method is a dynamic calculation process based on a sliding window of recent data to adjust the threshold baseline. It can track the long-term evolution of equipment status.

[0140] Step 5.1, define the sliding window.

[0141] The system maintains a fixed-capacity first-in-first-out (FIFO) data buffer, i.e., a sliding window:

[0142]

[0143] Where W is the window size. This represents the total number of normal samples collected in real time at the current moment. Let n be the relative distance value of the idx-th normal state sample within the window. This design ensures that the calculation is always based on the latest n. win A normal state sample effectively eliminates interference from outdated historical data, ensuring that the threshold baseline remains sensitive to the recent state of the equipment.

[0144] Step 5.2, time decay weight.

[0145] To assign greater importance to recent data, this embodiment introduces an exponentially decaying weighting function:

[0146]

[0147] in The attenuation coefficient is... For the current moment, This represents the acquisition time of the idx-th sample. This function calculates the acquisition time for each sample within the window. Assign a weight This weight varies with the age of the sample. The decay is exponential as the value increases. This mechanism implicitly incorporates the prior knowledge that "recent data is more representative of the current device state" in the calculation.

[0148] Step 5.3, Calculate the weighted quantiles.

[0149] For window The samples are sorted in ascending order of their relative distance values ​​to obtain an ordered sequence. The corresponding weights of the sequence are denoted as . .

[0150] Based on the above weights, the weighted quantiles of the samples within the window are calculated to construct dynamic boxplot statistics:

[0151] Weighted upper quartiles

[0152] Weighted lower quartiles

[0153] Among them, the function Implemented weighted sequence The p-quantile is calculated. This step yields statistics reflecting the recent normal fluctuation range of the equipment.

[0154] The weighted quantile calculation function used in this embodiment This can be achieved through the following steps:

[0155] 1) Data preprocessing: Sort the original sequence in the window in ascending order to obtain the ordered sequence and its sorted weight sequence.

[0156] 2) Weight normalization: Calculate the sum of weights Normalize each weight .

[0157] 3) Calculate the cumulative weight sequence:

[0158] 4) Quantile determination: finding those that meet the conditions If the smallest index idx is found, then the weighted p-quantile is: , This represents the distance value at the idx-th position in the sorted sequence. Accumulate its weight.

[0159] Step 5.4, Dynamic threshold function.

[0160] Ultimately, the dynamic threshold baseline and diagnostic threshold are determined by the following formula:

[0161]

[0162]

[0163]

[0164] in, The weighted interquartile range reflects the degree of dispersion of recent normal fluctuations in equipment. The dynamic threshold baseline is calculated by taking the upper quartile plus 1.5 times the interquartile range, covering the vast majority of normal samples. Sensitivity coefficient. Inherited from the settings in step 3.3. The final dynamic diagnostic threshold varies with the cumulative sample size. Evolution, adapting to changes in device state.

[0165] Example 3

[0166] This embodiment verifies the method described in Embodiment 2 through specific experiments.

[0167] In this embodiment, an application of domain adaptation for audio anomaly detection employs a method based on feature importance weighting and adaptive threshold adjustment to achieve accurate anomaly detection in cross-domain environments.

[0168] Figure 4 It is the original time-domain signal waveform of the sound sample in the source domain. The horizontal axis represents time, and the vertical axis represents amplitude, i.e., the energy intensity of the signal. Figure 5 This represents the time-domain waveform of the sample after preprocessing, with the horizontal and vertical axes showing the time-domain waveform. Figure 4 Consistent.

[0169] As shown in Table 1, the basic model performance is as follows: The basic anomaly diagnosis model trained directly on normal samples from the source domain achieves 100% accuracy (8 / 8) in anomaly detection in the target domain. However, when applied to normal data in the target domain, the model completely fails, with an accuracy of 0% (0 / 500), meaning all 500 normal samples are misclassified as anomalies. This indicates a severe distribution shift between the source and target domains, and the basic model lacks cross-domain generalization ability.

[0170] As shown in Table 2, the performance after domain adaptation optimization is as follows: After adopting the domain adaptation method, the model calculates the importance weight of each frame's features through an attention mechanism, adaptively weighting the target domain features, effectively aligning the feature distributions of the source and target domains. After optimization, all 500 normal samples in the target domain were correctly identified (100% accuracy, 500 / 500), achieving perfect differentiation from abnormal samples. This indicates that the domain adaptation mechanism successfully eliminated inter-domain differences and significantly improved the model's cross-domain detection performance.

[0171] Figure 6 It is the weight generated by the attention weight generator after domain adversarial training for a certain target domain sample, where the horizontal axis represents the frame index and the vertical axis represents the weight assigned to that frame; Figure 7 and Figure 8 These are the confusion matrices of diagnostic results before and after domain adversarial training.

[0172] Conclusion: The above results fully verify the effectiveness and superiority of the proposed method in solving the problem of distribution differences between domains, and provide a reliable cross-domain solution for abnormal sound detection in the target domain.

[0173] Table 1. Diagnostic accuracy of the basic model

[0174]

[0175] The diagnostic accuracy of the model after domain adversarial training on the example data is shown in Table 2.

[0176] Table 2. Model diagnostic accuracy after domain adversarial training

[0177]

[0178] Example 4

[0179] The escalator abnormal sound detection system based on domain-invariant feature transfer and clustering described in this invention includes:

[0180] The data acquisition unit is used to acquire audio signal samples and fault labels of escalators to construct a source domain dataset, and to acquire audio signal samples of target escalators to construct a target domain dataset.

[0181] The source domain data domain adversarial training unit, for the source domain and target domain datasets, preprocesses the audio signal samples to obtain the audio time-domain signal of each frame, calculates several time-domain features and frequency-domain features of each audio time-domain signal, constructs a feature matrix for each audio signal sample based on the time-domain features and frequency-domain features, and trains a domain adversarial network using the feature matrix. The domain adversarial network includes an attention weight generator, which learns the attention weight matrix of the feature matrix through a neural network.

[0182] The threshold setting unit is used to apply the attention weight matrix to audio signal samples in the target domain dataset to obtain weighted feature representations, and to determine the anomaly detection and diagnosis threshold through a clustering algorithm.

[0183] The target domain data anomaly detection unit is used to detect the audio signal of the target escalator in real time, construct the target weighted feature representation of the audio signal, and determine whether the audio signal is abnormal audio by combining the anomaly detection and diagnosis threshold.

[0184] Example 5

[0185] The electronic device of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the escalator abnormal sound detection method based on domain-invariant feature transfer and clustering.

[0186] The processor is used to execute a computer program stored in memory to implement the various steps in the methods described in the above embodiments.

[0187] Example 6

[0188] The computer-readable storage medium of the present invention stores a computer program, which, when executed by a processor, implements the method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering.

[0189] The computer-readable storage medium may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store program code in the form of instructions or data structures and is accessible by a computer.

[0190] The processor is used to execute a computer program stored in memory to implement the various steps in the methods described in the above embodiments.

[0191] Example 7

[0192] The computer program product of the present invention includes a computer program that, when executed by a processor, implements the escalator abnormal sound detection method based on domain-invariant feature transfer and clustering.

[0193] The processor is used to execute a computer program stored in memory to implement the various steps in the methods described in the above embodiments.

Claims

1. A method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering, characterized in that, Includes the following steps: Audio signal samples and fault labels of escalators are obtained to construct a source domain dataset, and audio signal samples of target escalators are obtained to construct a target domain dataset. For the source and target domain datasets: the audio signal samples are preprocessed to obtain the audio time-domain signal of each frame, and several time-domain features and frequency-domain features of each audio time-domain signal are calculated. A feature matrix of each audio signal sample is constructed based on the time-domain features and frequency-domain features. A domain adversarial network is trained using the feature matrix. The domain adversarial network includes an attention weight generator, and the attention weight matrix of the feature matrix is ​​learned through a neural network. The attention weight matrix is ​​applied to audio signal samples in the target domain dataset to obtain a weighted feature representation, and the anomaly detection and diagnosis threshold is determined by a clustering algorithm. The audio signal of the target escalator is detected in real time, a target weighted feature representation of the audio signal is constructed, and the abnormal detection and diagnosis threshold is used to determine whether the audio signal is abnormal.

2. The method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering according to claim 1, characterized in that, The source domain dataset was constructed by obtaining audio signal samples and fault labels from escalators, including: Acoustic sensors are installed at key parts of the escalator, including acquiring the operating audio of the motor, drive chain and reducer in the upper pit, acquiring abnormal sound of step offset and compression at two measuring points in the middle of the escalator, and acquiring the operating audio of the step chain in the lower pit.

3. The method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering according to claim 1, characterized in that, Preprocessing the audio signal samples to obtain the audio time-domain signal for each frame includes: After normalizing the audio signal samples, a high-pass filter is used to boost the high-frequency components. Then, the samples are divided into short time frames and windowed using a Hamming window to obtain the audio time-domain signal of each frame.

4. The method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering according to claim 1, characterized in that, The time-domain features include one or more of the following: mean, RMS value, peak-to-peak value, variance, kurtosis, skewness, peak factor, impulse factor, zero-crossing rate, and energy. The frequency-domain features include the frequency-domain feature Mel-spectral coefficients.

5. The method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering according to claim 1, characterized in that, The domain adversarial network includes an attention weight generator, a classifier, a domain discriminator, and an inversion layer. Through adversarial training between the attention weight generator, the domain discriminator, and the classifier, the attention weight generator learns attention patterns of domain-invariant features.

6. The method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering according to claim 1, characterized in that, Determining anomaly detection and diagnostic thresholds using clustering algorithms includes: Calculate the unique cluster center of the set of weighted feature representations, and calculate the relative distance from each weighted feature representation to the cluster center; perform statistical analysis on the set of relative distances based on box plots to determine the anomaly diagnosis threshold; If the relative distance from the target weighted feature representation to the cluster center exceeds the anomaly diagnosis threshold, then the audio signal of the real-time detected target escalator is determined to be abnormal audio.

7. The escalator abnormal sound detection method based on domain-invariant feature transfer and clustering according to claim 6, characterized in that, Statistical analysis of the set of relative distances based on box plots to determine anomaly diagnostic thresholds includes: Calculate the upper quartile and interquartile distance of the set of relative distances, calculate the base reference relative distance based on the upper quartile and interquartile distance, and set the anomaly diagnosis threshold as the product of the base reference relative distance and the threshold coefficient.

8. The method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering according to claim 6, characterized in that, The method further includes rolling updates of the anomaly diagnosis threshold, wherein the update method includes: During the continuous operation of the target escalator, the relative distance values ​​of the accumulated normal audio signals form a dynamic distance set. When the number of the relative distance values ​​reaches a first threshold, a sliding time window is used to obtain a certain number of relative distance values. For the relative distance values ​​within the sliding time window, weighted quantiles are calculated to construct box plot statistics and update the anomaly diagnosis threshold.

9. The method for detecting abnormal sounds in escalators based on domain-invariant feature transfer and clustering according to claim 8, characterized in that, For the relative distance values ​​within the sliding time window, weighted quantiles are calculated to construct box plot statistics, and the anomaly diagnosis threshold is updated including: Calculate the weighted upper quartile and weighted lower quartile; calculate the weighted quartile distance; update the base reference relative distance based on the weighted upper quartile and weighted quartile distance; update the anomaly diagnosis threshold as the product of the updated base reference relative distance and the threshold coefficient.

10. An escalator abnormal sound detection system based on domain-invariant feature transfer and clustering, characterized in that, include: The data acquisition unit is used to acquire audio signal samples and fault labels of escalators to construct a source domain dataset, and to acquire audio signal samples of target escalators to construct a target domain dataset. The source domain data domain adversarial training unit, for the source domain and target domain datasets, preprocesses the audio signal samples to obtain the audio time-domain signal of each frame, calculates several time-domain features and frequency-domain features of each audio time-domain signal, constructs a feature matrix for each audio signal sample based on the time-domain features and frequency-domain features, and trains a domain adversarial network using the feature matrix. The domain adversarial network includes an attention weight generator, which learns the attention weight matrix of the feature matrix through a neural network. The threshold setting unit is used to apply the attention weight matrix to audio signal samples in the target domain dataset to obtain weighted feature representations, and to determine the anomaly detection and diagnosis threshold through a clustering algorithm. The target domain data anomaly detection unit is used to detect the audio signal of the target escalator in real time, construct the target weighted feature representation of the audio signal, and determine whether the audio signal is abnormal audio by combining the anomaly detection and diagnosis threshold.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements the escalator abnormal sound detection method based on domain-invariant feature transfer and clustering according to any one of claims 1-9.

12. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the escalator abnormal sound detection method based on domain-invariant feature transfer and clustering according to any one of claims 1-9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the escalator abnormal sound detection method based on domain-invariant feature transfer and clustering according to any one of claims 1-9.

Images