A self-supervised device operational anomaly sound detection method

Abstract

The application discloses a self-supervised device running abnormal sound detection method, characterized in that single-channel running normal sound signals of multiple different types of devices in a running state are collected, and frequency spectrum graphs and time spectrum graphs are obtained by conversion; the frequency spectrum graphs and the time spectrum graphs are spliced to obtain double-channel time-frequency feature graphs, the double-channel time-frequency feature graphs after feature enhancement are input into a dynamic feature extraction module, so that final one-dimensional audio feature vectors of different types of devices are extracted, and a probability density function of the final one-dimensional audio feature vectors is fitted; single-channel running sound signals of a device to be detected in a running state are collected, and the above steps are repeated to obtain the probability density of one-dimensional audio feature vectors of the device to be detected; finally, the probability density is input into a classifier to complete the discrimination of normal and abnormal sounds; the method can learn effective features of audio of different types of devices through time-frequency fusion and dynamic feature extraction, and the accuracy and reliability of abnormal sound detection are improved.

Description

Technical Field

[0001] This invention relates to the field of equipment testing technology, and in particular to a method for detecting abnormal sounds in self-monitoring equipment operation. Background Technology

[0002] In industrial production, various types of machinery and equipment typically operate continuously for extended periods. Any malfunction or failure of critical components can lead to decreased production efficiency, or even equipment damage and safety accidents. Therefore, building an efficient and reliable equipment condition monitoring system to enable early diagnosis and timely maintenance of equipment faults is crucial for ensuring the stable operation of industrial systems.

[0003] Among various methods for monitoring equipment operating status, sound signal-based fault detection methods have received widespread attention in the field of industrial equipment health monitoring due to their non-contact, non-embedded acquisition and ease of access. These methods can obtain equipment operating status information without interfering with normal equipment operation. Traditional methods for detecting abnormal equipment sounds typically rely on manually designed acoustic features, combined with statistical analysis or traditional machine learning algorithms for fault identification. However, in complex industrial environments, these methods often struggle to extract features with good discriminative capabilities and have limited adaptability to different equipment types and operating conditions.

[0004] In recent years, deep learning-based abnormal sound detection methods have gradually become a research hotspot. These methods can automatically learn the feature representation of sound signals through data-driven approaches, improving detection performance to some extent. However, due to the complexity and diversity of abnormal sounds from equipment operation, and the difficulty in exhaustively labeling all abnormal types, it is often difficult to obtain a large number of labeled fault samples in real-world industrial scenarios. Therefore, abnormal sound detection often adopts unsupervised methods. Furthermore, the spectral structure and temporal characteristics of sounds from different equipment vary significantly, and existing methods often struggle to simultaneously consider both time and frequency domain information during feature extraction, resulting in low detection accuracy. Additionally, multi-channel feature extraction also faces the problem of high computational complexity. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method for detecting abnormal sounds in self-monitored equipment operation, which can improve the detection accuracy, efficiency and reliability of abnormal sounds, and also improve the adaptability to abnormal sound detection of different types of equipment.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows: a method for detecting abnormal operating sounds of a self-monitoring device, comprising the following specific steps: (1) Collect single-channel normal operating sound signals of various types of equipment in operation; (2) Input each single-channel normal operating sound signal into the adaptive filtering module and the time architecture module respectively, and convert them to obtain the spectrogram and the time spectrum; (3) The spectrum and the time spectrum are spliced ​​together along the channel dimension to obtain a dual-channel time-frequency feature map. Then, the dual-channel time-frequency feature map is fed into the parameterless attention module for attention weighting to enhance the time-frequency features. (4) Input the enhanced dual-channel time-frequency feature map into the dynamic feature extraction module. Through the collaborative modeling of the large kernel perception stage and the small kernel aggregation stage, capture global and local feature information respectively to extract the initial one-dimensional audio feature vector z of the dual-channel time-frequency feature map; (5) Use the single-channel normal operating sound signals of different types of devices collected in step (1) as training samples to train the dynamic feature extraction module to extract the final one-dimensional audio feature vector z of different types of devices. * And fit the data to obtain the final one-dimensional audio feature vector z. * The probability density function; (6) Collect the single-channel operating sound signal of the device under test in the running state, and input the single-channel operating sound signal into the adaptive filtering module and the time architecture module respectively. Repeat steps (2) to (5) to obtain the one-dimensional audio feature vector of the device under test. The probability density; (7) Input the probability density obtained in step (6) into the classifier to complete the discrimination between normal and abnormal sounds.

[0007] Further, in step (1), a single-channel normal operating sound signal is acquired through an audio sensor, and the sampling rate is set to 16kHz. The acquired single-channel normal operating sound signal is recorded as the original audio sequence x: , Where: T represents the total number of sampling points in the original audio sequence, x t This represents the amplitude at the t-th sampling point.

[0008] Furthermore, in step (2), the method for generating the spectrum is as follows: The adaptive filtering module performs frame segmentation and windowing on the original audio sequence, and then performs short-time Fourier transform on the framed and windowed data. For devices that exhibit sound abnormalities across the entire frequency band, the adaptive filtering module selects a linear filter to generate a spectrogram. For devices that exhibit sound faults only in the low-frequency band, the adaptive filtering module selects a Mel filter to generate a spectrogram.

[0009] Furthermore, in step (2), the temporal architecture module includes a one-dimensional convolutional neural network, an autocorrelation module, and a temporal decomposition module. The method for generating the temporal spectrogram is as follows: (2.1) The original audio sequence is subjected to one-dimensional convolution operation by a one-dimensional convolutional neural network to obtain time-coded features; (2.2) The autocorrelation module performs frame segmentation and windowing operations on the time coding features, divides the time coding features into multiple time segments, sets each time segment as the current time segment in turn, searches for the k time segments most similar to the current time segment, and performs weighted fusion of the current time segment and its corresponding k time segments to obtain the fused time coding features, so as to extract the time information in the audio. (2.3) The fused time-coded features are decomposed by the time-series decomposition module. That is, the periodic fluctuation component is extracted from each time segment of the fused time-coded features by global average pooling. Then, the periodic fluctuation component is subtracted from the fused time-coded features to obtain the trend change component. Finally, the trend change component is used as the final output time spectrum.

[0010] Furthermore, in step (3), the method for attention-weighting the dual-channel time-frequency feature map is as follows: The dual-channel time-frequency feature map is input into a parameterless attention module. Based on the response distribution of the dual-channel time-frequency feature map within and between single channels, the importance of each feature at each position on the dual-channel time-frequency feature map is calculated. Adaptive attention weighting is then applied to each feature at each position, multiplying each feature at its corresponding weight to obtain the feature-enhanced dual-channel time-frequency feature map F. A .

[0011] Furthermore, in step (4), the extraction process of the initial one-dimensional audio feature vector z of the dual-channel time-frequency feature map is as follows: (4.1) Enhance the dual-channel time-frequency feature map F A The input dynamic feature extraction module performs large and small kernel convolution. In the large kernel perception stage, the dual-channel time-frequency feature map F A First, pointwise convolution is performed to obtain the channel compressed feature map F. pw Then compress the channel feature map F pw Perform large-kernel depthwise separable convolution to obtain a large-kernel perceptual feature map F. lk Then, the large kernel sensing feature map F lk Perform 1×1 convolution to generate a spatially adaptive weight map, and finally use the spatially adaptive weight map to process the large kernel perceptual feature map F. lk By performing position-by-position adjustments, the weighted dual-channel time-frequency characteristic map F is obtained. wa ; (4.2) The weighted dual-channel time-frequency feature map F wa Entering the small kernel aggregation stage, the weighted dual-channel time-frequency feature map F is first...wa The system is divided into G channel sub-images. Then, within each channel sub-image, a small kernel convolution operation (3×3 convolution) is applied to each pixel to obtain a local aggregated feature map F after small kernel aggregation. sk ; (4.3) Aggregate the local feature map F sk With the enhanced dual-channel time-frequency feature map F A The residual fusion feature map F is obtained by fusion using a residual method. res ; (4.4) Fuse the residual feature map F res The input is fed into a feedforward neural network to obtain a feedforward fused feature map F. ffn Then the feedforward fused feature map F ffn As the output of the current big.me and small kernel convolutional networks, after being stacked through multiple layers of big.me and small kernel convolutional networks, the high-dimensional output feature map F of the dynamic feature extraction module is finally obtained. out The high-dimensional output feature map F out The mapping is to an initial one-dimensional audio feature vector z.

[0012] Furthermore, in step (5), the final one-dimensional audio feature vector z * The extraction process is as follows: (5.1) Define the set of parameters to be trained in the dynamic feature extraction module as the module parameter R, which includes pointwise convolution, large kernel depthwise separable convolution, 1×1 convolution, small kernel convolution, residual fusion, and feedforward neural network. Use the single-channel normal operating sound signals of various types of devices collected in step (1) as training samples to train the model of the dynamic feature extraction module. The training process takes the self-supervised classification of different types of devices as the training task. Specifically, a Softmax layer is constructed on the initial one-dimensional audio feature vector to identify the type of device it belongs to. The parameters to be trained in the Softmax layer are defined as parameters W. Backpropagation is performed on the dynamic feature extraction module and the Softmax layer using cross-entropy loss. The module parameter R of the dynamic feature extraction module and the parameters W of the Softmax layer are continuously updated during the training process until the model converges, and the updated module parameter R of the dynamic feature extraction module is obtained. * ; (5.2) The dual-channel time-frequency feature map F, which is the feature enhancement of each single-channel normal sound signal. A The input is fed into the trained dynamic feature extraction module to obtain the final one-dimensional audio feature vector z. * .

[0013] Furthermore, in step (5), the final one-dimensional audio feature vector z * The fitting process for the probability density function is as follows: Gaussian mixture model is used to calculate the final one-dimensional audio feature vector z of different types of devices. * By fitting the probability density function, assuming the Gaussian mixture model consists of M Gaussian components, the final one-dimensional audio feature vector z is obtained. * probability density p(z) * ) is represented as: , Where: r represents the index of the Gaussian component. This represents the mixing weight of the r-th Gaussian component. Let represent the mean vector of the r-th Gaussian component. Let r be the covariance matrix of the r-th Gaussian component. This represents the final one-dimensional audio feature vector z. * The corresponding Gaussian probability density function.

[0014] Furthermore, the process of distinguishing between normal and abnormal sounds in step (7) is as follows: (7.1) The one-dimensional audio feature vector of the device to be detected probability density The negative log-likelihood value is defined as the outlier score. ,Right now: ; (7.2) Set the threshold for judging normal and abnormal sounds as δ, then: , in: This indicates an abnormal sound. This indicates that the sound is normal.

[0015] Compared with the prior art, the advantages of the present invention are: (1) Through time-frequency fusion and dynamic feature extraction, effective features of audio from different types of devices can be learned. Compared with the traditional spectrogram, the time-frequency fusion process not only effectively extracts the frequency information of the sound signal, but also fully preserves the temporal information of the sound signal, thus improving the integrity of the information. The dynamic feature extraction module can extract useful information from the dual-channel time-frequency feature map after time-frequency fusion more comprehensively through operations such as large kernel convolution and small kernel aggregation, thereby improving the accuracy and reliability of abnormal sound detection. This method is applicable to a variety of different types of devices and has strong generalization ability. (2) The present invention constructs a dual-channel audio feature representation structure that integrates autocorrelation time-series features of time spectrum and adaptive filter spectrum. In the time-series branch, the ability of time spectrum to represent time-series patterns is enhanced while taking into account computational efficiency. In the frequency domain branch, for devices that have anomalies in the entire frequency band, a linear filter is used to generate the spectrum. For devices that have anomalies only in the low frequency band, a Mel filter is used to generate the spectrum. This fully preserves the key signal change frequency band of the single-channel operating sound signal and improves the sensitivity and robustness of subsequent anomaly detection. (3) In the dynamic feature extraction module, the present invention integrates the large kernel perception and small kernel aggregation strategy, and achieves high detection efficiency and accuracy while maintaining the lightweight model and eliminating the need for pre-training. It is suitable for real-time abnormal sound detection deployment in industrial edge computing scenarios. Attached Figure Description

[0016] Figure 1 This is a flowchart of the present invention; Figure 2 This is a visualization of the detection results of six types of equipment according to the present invention. Detailed Implementation

[0017] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0018] like Figure 1 As shown, a method for detecting abnormal sounds during the operation of a self-monitoring device includes the following specific steps: (1) Acquire single-channel normal operating sound signals of various types of devices in operation using an audio sensor, with a sampling rate set to 16kHz. Record the acquired single-channel normal operating sound signals as the original audio sequence x: , Where: T represents the total number of sampling points in the original audio sequence, x t This represents the amplitude at the t-th sampling point; (2) Input each single-channel normal operating sound signal into the adaptive filtering module to obtain the corresponding spectrum diagram, specifically: The adaptive filtering module performs frame segmentation and windowing on the original audio sequence, and then performs a short-time Fourier transform on the framed and windowed data. For devices that exhibit sound abnormalities across the entire frequency range, the adaptive filtering module selects a linear filter to generate a spectrogram, ensuring that the generated spectrogram maintains consistent focus on the low-frequency and high-frequency components of the signal, preventing it from ignoring signal abrupt changes in the high-frequency range. For devices that only exhibit sound faults in the low-frequency range, the adaptive filtering module selects a Mel filter to generate a spectrogram. The generated spectrogram primarily focuses on the low-frequency components of the sound signal, while compressing the high-frequency components to amplify signal changes in the low-frequency range and weaken signal changes in the high-frequency range. Each single-channel, normally functioning audio signal is input into the time architecture module to generate the corresponding time spectrogram, thus compensating for the lack of temporal dimension information in the spectrogram. The time architecture module includes a one-dimensional convolutional neural network, an autocorrelation module, and a time-series decomposition module. The specific method for generating the time spectrogram is as follows: (2.1) The original audio sequence is subjected to one-dimensional convolution operation by a one-dimensional convolutional neural network to obtain time-coded features; (2.2) The autocorrelation module performs frame segmentation and windowing operations on the time coding features, divides the time coding features into multiple time segments, sets each time segment as the current time segment, and then searches for the k time segments most similar to the current time segment, that is, sorts them from high to low similarity, takes the top k, and performs weighted fusion of the current time segment and its corresponding k time segments to obtain the fused time coding features, so as to extract the time information in the audio. (2.3) The fused time-coded features are decomposed by the time-series decomposition module. That is, the periodic fluctuation component is extracted from each time segment of the fused time-coded features by global average pooling. Then, the periodic fluctuation component is subtracted from the fused time-coded features to obtain the trend change component. Finally, the trend change component is used as the final output time spectrum. (3) The spectrum diagram and time spectrum diagram obtained by converting the same single-channel normal sound signal are spliced ​​together along the channel dimension to obtain a dual-channel time-frequency feature map. Then, the dual-channel time-frequency feature map is sent to the parameterless attention module for attention weighting to enhance the time-frequency features. Specifically: The dual-channel time-frequency feature map is input into a parameterless attention module. Based on the response distribution of the dual-channel time-frequency feature map within and between single channels, the importance of each feature at each position on the dual-channel time-frequency feature map is calculated. Adaptive attention weighting is then applied to each feature at each position, multiplying each feature at its corresponding weight to obtain the feature-enhanced dual-channel time-frequency feature map F. A This feature enhancement method can improve the effectiveness and stability of feature representation without increasing the additional training burden. (4) Enhance the dual-channel time-frequency feature map F A The input dynamic feature extraction module performs big and small kernel convolution. Through collaborative modeling of the big kernel perception stage and the small kernel aggregation stage, global and local feature information are captured respectively to extract the initial one-dimensional audio feature vector z of the dual-channel time-frequency feature map, specifically: (4.1) In the large kernel perception stage, the dual-channel time-frequency feature map F after feature enhancement A First, pointwise convolution is performed to obtain the channel compressed feature map F. pw Then compress the channel feature map F pw Perform large-kernel depthwise separable convolution to obtain a large-kernel perceptual feature map F. lk Then, the large kernel sensing feature map F lk Perform 1×1 convolution to generate a spatially adaptive weight map, and finally use the spatially adaptive weight map to process the large kernel perceptual feature map F. lk By performing position-by-position adjustments, the weighted dual-channel time-frequency characteristic map F is obtained. wa ; (4.2) The weighted dual-channel time-frequency feature map F wa Entering the small kernel aggregation stage, the weighted dual-channel time-frequency feature map F is first... wa The system is divided into G channel sub-images. Then, within each channel sub-image, a small kernel convolution operation (3×3 convolution) is applied to each pixel to obtain a local aggregated feature map F after small kernel aggregation. sk Because of the small kernel size, this stage is better at capturing local details, such as local time-frequency textures, small-scale energy abrupt changes, edge variations, and fine-grained anomaly patterns. (4.3) Aggregate the local feature map F sk With the enhanced dual-channel time-frequency feature map F A The residual fusion feature map F is obtained by fusion using a residual method. res ; (4.4) Fuse the residual feature map F res The input is fed into a feedforward neural network to obtain a feedforward fused feature map F. ffn Then the feedforward fused feature map F ffn As the output of the current big.me and small kernel convolutional networks, after being stacked through multiple layers of big.me and small kernel convolutional networks, the high-dimensional output feature map F of the dynamic feature extraction module is finally obtained. out The high-dimensional output feature map F out Mapped to an initial one-dimensional audio feature vector z; (5) Use the single-channel normal operating sound signals of different types of devices collected in step (1) as training samples to train the dynamic feature extraction module to extract the final one-dimensional audio feature vector z of different types of devices.* And fit the data to obtain the final one-dimensional audio feature vector z. * The probability density function is as follows: (5.1) The parameters of pointwise convolution, large kernel depthwise separable convolution, 1×1 convolution, small kernel convolution, residual fusion and feedforward neural network in the dynamic feature extraction module are used as training parameters. The set of training parameters is defined as module parameters R. The single-channel normal operating sound signals of various types of devices collected in step (1) are used as training samples to train the model of the dynamic feature extraction module. The training process takes the self-supervised classification of different types of devices as the training task. Specifically, a Softmax layer is constructed on the initial one-dimensional audio feature vector to identify the type of device it belongs to. The training parameters of the Softmax layer are defined as parameters W. The cross-entropy loss is used to perform backpropagation operation on the dynamic feature extraction module and the Softmax layer. The module parameters R of the dynamic feature extraction module and the parameters W of the Softmax layer are continuously updated during the training process until the model converges, and the updated module parameters R of the dynamic feature extraction module are obtained. * ; (5.2) The dual-channel time-frequency feature map F, which is the feature enhancement of each single-channel normal sound signal. A The input is fed into the trained dynamic feature extraction module to obtain the final one-dimensional audio feature vector z. * ; (5.3) The Gaussian mixture model is used to analyze the final one-dimensional audio feature vector z of different types of devices. * By fitting the probability density function, assuming the Gaussian mixture model consists of M Gaussian components, the final one-dimensional audio feature vector z is obtained. * probability density p(z) * ) is represented as: , Where: r represents the index of the Gaussian component. This represents the mixing weight of the r-th Gaussian component. Let represent the mean vector of the r-th Gaussian component. Let r be the covariance matrix of the r-th Gaussian component. This represents the final one-dimensional audio feature vector z. * The corresponding Gaussian probability density function; (6) Collect the single-channel operating sound signal of the device under test in the running state, and input the single-channel operating sound signal into the adaptive filtering module and the time architecture module respectively. Repeat steps (2) to (3) to obtain the dual-channel time-frequency feature map after feature enhancement. Then input the dual-channel time-frequency feature map after feature enhancement into the trained dynamic feature extraction module to obtain a one-dimensional audio feature vector. Then, the one-dimensional audio feature vector Input the probability density function obtained from step (5.3) to obtain the one-dimensional audio feature vector of the device to be detected. probability density ; (7) The probability density obtained in step (6) The sound is fed into a classifier to distinguish between normal and abnormal sounds, specifically: (7.1) Transform the one-dimensional audio feature vector probability density The negative log-likelihood value is defined as the outlier score. ,Right now: ; (7.2) Set the threshold for judging normal and abnormal sounds as δ. This threshold is usually determined based on human experience. Then: , in: This indicates that the operating sound of the device under test is abnormal; This indicates that the operating sound of the device under test is normal.

[0019] The abnormal sound detection method of this invention was compared with four existing detection methods—autoencoders, iteratively dilated convolutional networks, lightweight convolutional networks, and masked autoencoders—on a public dataset of the Acoustic Scene and Event Detection and Classification Challenge. The experimental subjects included six devices: fans, pumps, sliders, valves, toy cars, and toy trains. Normal and abnormal audio samples from the above six devices were input to each method, and each method output the abnormality score for the corresponding sample. Then, receiver operating characteristic (ROC) curves were plotted based on the discrimination results of normal and abnormal samples, and the area under the curve (AUC) and partial area under the curve (pAUC) were further calculated. The experimental results are shown in the table below:

[0020] The table above shows the performance comparison of this invention with four existing methods on six devices. AUC represents the area under the receiver operating characteristic (ROC) curve, and pAUC represents the area under the ROC curve within a specified low false alarm range. Higher AUC and pAUC values ​​indicate stronger ability to distinguish between normal and abnormal samples, and better anomaly detection performance. As shown in the table, the average AUC of this invention on the six devices is 95.10%, and the average pAUC is 90.25%, both higher than the other four comparison methods, indicating that this invention has superior overall performance in anomaly sound detection tasks across multiple devices.

[0021] The one-dimensional audio feature vectors extracted from the above six devices using the method of this invention are visualized using the t-distributed random neighborhood embedding method. The results are as follows: Figure 2 As shown, Figure 2 Image (a) is a visualization of a fan, image (b) is a visualization of a pump, image (c) is a visualization of a slider, image (d) is a visualization of a toy car, image (e) is a visualization of a toy train, and image (f) is a visualization of a valve. In the images, different colors represent different device numbers, circles represent normal samples, and crosses represent abnormal samples. Figure 2 It is evident that normal samples and abnormal samples exhibit significant distribution differences in the two-dimensional feature space, indicating that the present invention can effectively learn the key features of abnormal device states, improve the anomaly detection effect, verify the effectiveness of the present invention on public datasets, and demonstrate its promising prospects for engineering applications.

[0022] The scope of protection of this invention includes, but is not limited to, the above embodiments. The scope of protection is defined by the claims. Any substitutions, modifications, or improvements to this technology that are easily conceived by those skilled in the art fall within the scope of protection of this invention.

Claims

1. A method for detecting abnormal sounds during the operation of a self-monitoring device, characterized in that... The specific steps include the following: (1) Collect single-channel normal operating sound signals of various types of equipment in operation; (2) Input each single-channel normal operating sound signal into the adaptive filtering module and the time architecture module respectively, and convert them to obtain the spectrogram and the time spectrum; (3) The spectrum and the time spectrum are spliced ​​together along the channel dimension to obtain a dual-channel time-frequency feature map. Then, the dual-channel time-frequency feature map is fed into the parameterless attention module for attention weighting to enhance the time-frequency features. (4) Input the enhanced dual-channel time-frequency feature map into the dynamic feature extraction module. Through the collaborative modeling of the large kernel perception stage and the small kernel aggregation stage, capture global and local feature information respectively to extract the initial one-dimensional audio feature vector z of the dual-channel time-frequency feature map; (5) Use the single-channel normal operating sound signals of different types of devices collected in step (1) as training samples to train the dynamic feature extraction module to extract the final one-dimensional audio feature vector z of different types of devices. * And fit the data to obtain the final one-dimensional audio feature vector z. * The probability density function; (6) Collect the single-channel operating sound signal of the device under test in the running state, and input the single-channel operating sound signal into the adaptive filtering module and the time architecture module respectively. Repeat steps (2) to (5) to obtain the one-dimensional audio feature vector of the device under test. The probability density; (7) Input the probability density obtained in step (6) into the classifier to complete the discrimination between normal and abnormal sounds.

2. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 1, characterized in that: In step (1), a single-channel normal operating sound signal is acquired through an audio sensor, with the sampling rate set to 16kHz, and the acquired single-channel normal operating sound signal is recorded as the original audio sequence x: ， Where: T represents the total number of sampling points in the original audio sequence, x t This represents the amplitude at the t-th sampling point.

3. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 2, characterized in that: In step (2), the method for generating the spectrum is as follows: The adaptive filtering module performs frame segmentation and windowing on the original audio sequence, and then performs short-time Fourier transform on the framed and windowed data. For devices that exhibit sound abnormalities across the entire frequency band, the adaptive filtering module selects a linear filter to generate a spectrogram. For devices that exhibit sound faults only in the low-frequency band, the adaptive filtering module selects a Mel filter to generate a spectrogram.

4. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 2, characterized in that: In step (2), the temporal architecture module includes a one-dimensional convolutional neural network, an autocorrelation module, and a temporal decomposition module. The method for generating the temporal spectrogram is as follows: (2.1) The original audio sequence is subjected to one-dimensional convolution operation by a one-dimensional convolutional neural network to obtain time-coded features; (2.2) The autocorrelation module performs frame segmentation and windowing operations on the time coding features, divides the time coding features into multiple time segments, sets each time segment as the current time segment in turn, searches for the k time segments most similar to the current time segment, and performs weighted fusion of the current time segment and its corresponding k time segments to obtain the fused time coding features, so as to extract the time information in the audio. (2.3) The fused time-coded features are decomposed by the time-series decomposition module. That is, the periodic fluctuation component is extracted from each time segment of the fused time-coded features by global average pooling. Then, the periodic fluctuation component is subtracted from the fused time-coded features to obtain the trend change component. Finally, the trend change component is used as the final output time spectrum.

5. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 1, characterized in that: In step (3), the method for attention-weighting the dual-channel time-frequency feature map is as follows: The dual-channel time-frequency feature map is input into a parameterless attention module. Based on the response distribution of the dual-channel time-frequency feature map within and between single channels, the importance of each feature at each position on the dual-channel time-frequency feature map is calculated. Adaptive attention weighting is then applied to each feature at each position, multiplying each feature at its corresponding weight to obtain the feature-enhanced dual-channel time-frequency feature map F. A .

6. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 5, characterized in that: In step (4), the extraction process of the initial one-dimensional audio feature vector z of the dual-channel time-frequency feature map is as follows: (4.1) Enhance the dual-channel time-frequency feature map F A The input dynamic feature extraction module performs large and small kernel convolution. In the large kernel perception stage, the dual-channel time-frequency feature map F A First, pointwise convolution is performed to obtain the channel compressed feature map F. pw Then compress the channel feature map F pw Perform large-kernel depthwise separable convolution to obtain a large-kernel perceptual feature map F. lk Then, the large kernel sensing feature map F lk Perform 1×1 convolution to generate a spatially adaptive weight map, and finally use the spatially adaptive weight map to process the large kernel perceptual feature map F. lk By performing position-by-position adjustments, the weighted dual-channel time-frequency characteristic map F is obtained. wa ; (4.2) The weighted dual-channel time-frequency feature map F wa Entering the small kernel aggregation stage, the weighted dual-channel time-frequency feature map F is first... wa The system is divided into G channel sub-images. Then, within each channel sub-image, a small kernel convolution operation (3×3 convolution) is applied to each pixel to obtain a local aggregated feature map F after small kernel aggregation. sk ; (4.3) Aggregate the local feature map F sk With the enhanced dual-channel time-frequency feature map F A The residual fusion feature map F is obtained by fusion using a residual method. res ; (4.4) Fuse the residual feature map F res The input is fed into a feedforward neural network to obtain a feedforward fused feature map F. ffn Then the feedforward fused feature map F ffn As the output of the current big.me and small kernel convolutional networks, after being stacked through multiple layers of big.me and small kernel convolutional networks, the high-dimensional output feature map F of the dynamic feature extraction module is finally obtained. out The high-dimensional output feature map F out The mapping is to an initial one-dimensional audio feature vector z.

7. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 6, characterized in that: In step (5), the final one-dimensional audio feature vector z * The extraction process is as follows: (5.1) Define the set of parameters to be trained in the dynamic feature extraction module as the module parameter R, which includes pointwise convolution, large kernel depthwise separable convolution, 1×1 convolution, small kernel convolution, residual fusion, and feedforward neural network. Use the single-channel normal operating sound signals of various types of devices collected in step (1) as training samples to train the model of the dynamic feature extraction module. The training process takes the self-supervised classification of different types of devices as the training task. Specifically, a Softmax layer is constructed on the initial one-dimensional audio feature vector to identify the type of device it belongs to. The parameters to be trained in the Softmax layer are defined as parameters W. Backpropagation is performed on the dynamic feature extraction module and the Softmax layer using cross-entropy loss. The module parameter R of the dynamic feature extraction module and the parameters W of the Softmax layer are continuously updated during the training process until the model converges, and the updated module parameter R of the dynamic feature extraction module is obtained. * ; (5.2) The dual-channel time-frequency feature map F, which is the feature enhancement of each single-channel normal sound signal. A The input is fed into the trained dynamic feature extraction module to obtain the final one-dimensional audio feature vector z. * .

8. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 1, characterized in that: In step (5), the final one-dimensional audio feature vector z * The fitting process for the probability density function is as follows: Gaussian mixture model is used to calculate the final one-dimensional audio feature vector z of different types of devices. * By fitting the probability density function, assuming the Gaussian mixture model consists of M Gaussian components, the final one-dimensional audio feature vector z is obtained. * probability density p(z) * ) is represented as: ， Where: r represents the index of the Gaussian component. This represents the mixing weight of the r-th Gaussian component. Let represent the mean vector of the r-th Gaussian component. Let r be the covariance matrix of the r-th Gaussian component. This represents the final one-dimensional audio feature vector z. * The corresponding Gaussian probability density function.

9. The method for detecting abnormal operating sounds of a self-monitoring device as described in claim 1, characterized in that: The process of distinguishing between normal and abnormal sounds in step (7) is as follows: (7.1) The one-dimensional audio feature vector of the device to be detected probability density The negative log-likelihood value is defined as the outlier score. ,Right now: ; (7.2) Set the threshold for judging normal and abnormal sounds as δ, then: ， in: This indicates an abnormal sound. This indicates that the sound is normal.

Images