A domain-transferred self-supervised machine abnormal sound detection method

Abstract

The present invention relates to the field of computer technology, and in particular to a domain-shifted self-supervised machine abnormal sound detection method, wherein a time-frequency domain feature extraction network, a dynamic graph convolution network, and a domain adaptation network are respectively designed; the spatial feature information of a sound signal and the feature information of time-domain interaction are captured by the time-domain feature extraction network to obtain a feature vector, thereby improving the performance stability of machine abnormal sound detection under domain transfer; the dependency relationship between the domain transfers of the feature vectors is captured by the dynamic graph convolution network to improve the model's ability to perceive domain transfer features; based on the dependency relationship, the feature vector is compensated by a domain adaptation network to obtain a detection result, which can compensate for the degradation of model performance caused by domain movement, improves the domain transfer adaptation ability of the model under self-supervised environment for abnormal sounds, and solves the problem that the existing detection method learns sound features of different domain transfers and the detection effect is unstable.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a domain-transferred self-supervised machine abnormal sound detection method. Background Art

[0002] The abnormal sound detection task is to judge whether the machine is faulty by the sound it makes. The model is trained using normal sound signals so that the model can learn better probability distribution, fit abnormal sound signals, detect abnormal sound signals, and perform machine fault detection. However, in practical training scenarios, due to the physical parameter changes of the machine's operating conditions, the normal sound signal fluctuates, causing the normal sound signal to be detected as abnormal sound.

[0003] Recently, a large number of deep learning-based self-supervised abnormal sound detection methods have been applied to detection tasks. They are mainly divided into autoencoder-based methods, flow-based self-supervised density estimation methods and self-supervised classification methods. The autoencoder-based method first maps the input features to low-dimensional vectors through the encoder, and then the decoder restores the feature vectors in the low-dimensional vector space to the input features. Although this type of method is effective in detection, it will be greatly limited when the training process involves abnormal or unstable sound signals; the flow-based self-supervised density estimation method uses a normalized flow to distinguish between normal sound signals and abnormal sound signals of the machine identity, thereby improving the model's ability to detect abnormal sounds of a single type of machine. However, this type of method requires the same distribution sampling for different machine identities under each type of machine, resulting in limited detection stability of this type of method. The self-supervised detection method is to pass all machine types and machine identities into the model for training, and use log-likelihood or classification confidence to calculate the anomaly score. Although this type of method is better than the autoencoder-based method and the self-supervised density estimation-based method in performance, this type of method has a low ability to learn the domain offset features of the sound signal, resulting in unstable detection results. Summary of the invention

[0004] The purpose of the present invention is to provide a domain transfer self-supervised machine abnormal sound detection method, aiming to solve the problem that the existing detection method learns the sound features of different domain transfers and the detection effect is unstable.

[0005] To achieve the above object, the present invention provides a domain transfer self-supervised machine abnormal sound detection method, comprising the following steps:

[0006] Design time-frequency domain feature extraction network, dynamic graph convolution network and domain adaptation network respectively;

[0007] The spatial feature information of the feature vector and the feature information of the time-domain interaction are captured by the time-frequency domain feature extraction network to obtain a feature vector;

[0008] Capturing the dependencies between the domain transfers of the feature vectors through the dynamic graph convolutional network;

[0009] The feature vector is compensated by the domain adaptation network based on the dependency relationship to obtain a detection result.

[0010] The specific method of obtaining the feature vector by capturing the spatial feature information of the feature vector and the feature information of the time-domain interaction through the time-frequency domain feature extraction network is as follows:

[0011] Extracting the time domain features of the feature vector using short-time Fourier transform and log-Mel;

[0012] The time series feature network is used to capture the time domain features lost by short-time Fourier transform and logarithmic Mel spectrum, and the lost features and time-frequency domain features are added and passed into the token converter network;

[0013] The time-frequency domain features are transmitted to the token converter network for spatial transformation and spatial relationship modeling to obtain a feature vector.

[0014] The specific method of capturing the dependency between the domain transfers of the feature vectors through the dynamic graph convolutional network is as follows:

[0015] Using an invariant domain transfer module to perform perceptual modeling on the coarse label dependency between machine IDs from the feature vector to obtain a perceptual model;

[0016] Based on the perception model, a variable domain transfer module is used to capture the subtle relationship between domain transfer features in the machine ID.

[0017] The specific method of compensating the feature vector by the domain adaptation network is as follows:

[0018] Calculating the second-order covariance distance between the feature vectors under different domain transfers to obtain a distance difference;

[0019] The distance difference is reduced using a back gradient propagation algorithm.

[0020] Wherein, when calculating the second-order covariance distance, source domain features and target features after domain transfer are defined.

[0021] The present invention discloses a self-supervised machine abnormal sound detection method for domain transfer, which respectively designs a time-frequency domain feature extraction network, a dynamic graph convolution network and a domain adaptation network; the spatial feature information of the sound signal and the feature information of the time-domain interaction are captured by the time-frequency domain feature extraction network to obtain a feature vector, thereby improving the performance stability of machine abnormal sound detection under domain transfer; the dependency between the domain transfers of the feature vectors is captured by the dynamic graph convolution network, thereby improving the model's ability to perceive domain transfer features and deepening the model's sensitivity to domain transfer features; the feature vectors are compensated by the domain adaptation network, thereby compensating for the model performance degradation caused by domain movement, thereby improving the model's ability to adapt to the domain transfer of abnormal sounds under the self-supervised environment, and solving the problem that the existing detection method learns sound features of different domain transfers and the detection effect is unstable. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a schematic diagram of a domain-transferred self-supervised machine abnormal sound detection model provided by the present invention.

[0024] Figure 2 This is a feature extraction network structure diagram provided by the present invention.

[0025] Figure 3 It is a schematic diagram of the feature extraction network provided by the present invention.

[0026] Figure 4 It is a schematic diagram of the Transformer provided by the invention.

[0027] Figure 5 It is the overall architecture of the dynamic graph convolutional network provided by the invention.

[0028] Figure 6 It is a spectrogram of the Transformer and temporal feature network provided by the invention.

[0029] Figure 7 It is a schematic diagram of the visualization of the time-frequency domain characteristics of each model provided by the invention.

[0030] Figure 8 The invention provides a schematic diagram that uses t-SNE to visualize the feature graph extracted by the model.

[0031] Fig. 9It is a flow chart of a domain-transferred self-supervised machine abnormal sound detection provided by the present invention. DETAILED DESCRIPTION

[0032] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0033] See also Figures 1 to 9 The present invention provides a domain transfer self-supervised machine abnormal sound detection method, comprising the following steps:

[0034] S1 designs the time-frequency domain feature extraction network, dynamic graph convolution network and domain adaptation network respectively;

[0035] S2 captures the spatial feature information of the sound signal and the feature information of the time domain interaction through the time-frequency domain feature extraction network to obtain a feature vector;

[0036] Specific method:

[0037] S21 extracts the time domain features of the feature vector using short-time Fourier transform and log-Mel;

[0038] Specifically, the short-time Fourier transform and the logarithmic Mel spectrum are used to transform the sound signal x v ∈R 1 ×T After transformation, the time-frequency domain features of appropriate size are obtained. The expression is as follows:

[0039]

[0040]

[0041] Where h(t) is the window function and g(t,f) is the time-frequency domain feature about time t and frequency f. represents the Mel filter bank, i represents the dimension of the spectral feature, and b represents the number of frequency bins of the spectrogram obtained by short-time Fourier transform. m Represents the time-frequency domain features after logarithmic Mel spectrum enhancement.

[0042] S22 passes the time domain features into a time domain feature extraction network for spatial transformation and spatial relationship modeling to obtain a feature vector.

[0043] Specifically, the time series feature network is then used to capture the time domain features that are abnormally lost in the short-time Fourier transform and logarithmic Mel spectrum, and the lost features are added and passed into the token converter network. Finally, the token converter network is used to perform local and global adaptive spatial transformation of the time-frequency domain features and model the intrinsic connection of the space-time relationship.

[0044] Among them, the time series feature network: short-time Fourier transform and logarithmic Mel spectrum extraction of time-frequency domain features are prone to lose abnormal time series information in the sound. TgramNet network extracts time-frequency domain features from the sound signal x v Although TgramNet uses one-dimensional convolution with a large kernel to extract temporal features, it is difficult for one-dimensional convolution to find reliable temporal feature dependencies in long sequences. Recently, ETDNet uses Transformer to capture the dependencies between temporal features, but Transformer uses a sparse attention mechanism, which not only has quadratic complexity in terms of computation, but also cannot fully model the local and global relationships of the temporal features of sound signals. Therefore, we propose that subsequences at the same position in different periods of the temporal signal often have the same trend, and the theoretical basis for capturing temporal features can be based on the similarity of subsequences. So we designed a temporal feature network, such as Figure 2 As shown in the figure, the network can better utilize the similarity of subsequences to capture abnormal time series features with less computation. In the time series feature network, a one-dimensional convolution is first used to encode the sound signal along the time dimension. Then the autocorrelation encoding module is used to aggregate similar subsequence information and replace the point-to-point connection self-attention mechanism in the Transformer. This mechanism not only captures the intrinsic time feature similarity of the encoded sound signal, but also reduces the computational complexity to O(L log L), as shown below.

[0045]

[0046]

[0047]

[0048] Where arg Topk(·) represents the parameter for obtaining k autocorrelation subsequences. Represents a subsequence τ of length k cycles sliding according to the time window 1 ,…,τ k . is the autocorrelation weight between time series Q and K. It represents the alignment of subsequences with similar periods and phases.

[0049] Finally, for the encoded sound sequence x of length dimension LA ∈R L×C , Series Decomp Block is used to extract the high-order time information of the input and the high-order time information hidden in the middle of the model. The expression is as shown in the formula below.

[0050] x s =AvgPool(Padding(x A ))

[0051] x l =x A -x s

[0052] where x s ∈R L×C represents the short-term fluctuation of the time series, Padding represents the use of sliding windows to fill the ends of the sound sequence features with 0, AvgPoo represents the global average pooling operation, x l ∈R L×C Indicates the periodic fluctuation of the time series. In addition, we also added a fully connected feedforward network layer to the time series feature network according to the design of Transformer, so that the network can better cluster the extracted time series features together.

[0053] Among them, the token converter: feature extraction network aims to fully model the global and local information of the input time-frequency domain features with fewer parameters when extracting time-frequency domain features. Therefore, we designed the token converter network, as shown in Figure 4. In the local spatial information modeling of time-frequency domain features, we used the Focus module to convert the time-frequency domain features x m ∈R H×W×c Projection to high-dimensional space x L ∈R H×W×d ( d＞c ). In order to enable the token converter to have spatial inductive global representation capabilities, we transform x L Expand to x u ∈R P×N×d , so that the model does not lose the order between patches of time-frequency features. Where P = wh, N = HW / P is the number of patches, h and w are the height and width of the patch respectively. In addition, in order not to lose the spatial order between tokens in the patch, we use the Token Learner structure, which uses the spatial attention mechanism to calculate the spatial weights between tokens in the patch and selects t important tokens for modeling. Finally, by applying Transformer to encode the relationship pairs between patches and the relationship pairs of tokens in the patch, the global information relationship modeling of time-frequency domain features is performed, and its expression is as follows.

[0054] x G (p)=Transformer(TokenLearner(x U (p))), 1≤p≤P

[0055] In addition, in order to promote the model to capture more domain transfer representations, we introduce a channel attention mechanism in the multi-layer perceptron (MLP) module in Tranformer, such as Figure 5 This method can take more comprehensive consideration of the channels of the Transformer aggregated features and make full use of the channel grouping information to mine more domain transfer representations. For the output features of the Transformer, we use the TokenFuser module to convert x G =R P×N×t Mapped to x G =R P×N×d And x G Fold to x F ∈R H ×W×d As output of the tokenizer.

[0056] S3 captures the dependency between the domain transfers of the feature vectors through the dynamic graph convolutional network;

[0057] Specific method:

[0058] S31 uses an invariant domain transfer module to perform perceptual modeling on the coarse label dependency relationship between machine IDs from the feature vector to obtain a perceptual model;

[0059] Specifically, the invariant domain transfer module GCN (IDT-GCN) constructs a static GCN module that performs content-aware modeling on the coarse relationship between machine IDs from the extracted feature maps.

[0060] In IDT-GCN, we use the traditional GCN to define the static GCN in our model and calculate the coarse dependency of machine ID labels. Given a set of features F∈R C×d As the input node, using the correlation matrix M s C×C and the state update matrix Calculate the rough relationship between machine IDs. The IDT-GCN calculation method is as follows.

[0061] S=δ(M s FW s )

[0062] Among them, M s and W sis randomly initialized by the gradient descent algorithm. s The time-frequency domain features of all sounds share the same weight value, so our goal is to make M s It can capture the global coarse classification dependency between different machine IDs. δ represents the LeakyRelu activation function.

[0063] S32 uses a variable domain transfer module based on the perception model to capture subtle relationships between domain transfer features within the machine ID.

[0064] Specifically, the variable domain transfer module GCN (VDT-GCN) constructs a dynamic GCN module that captures the subtle dependencies of domain transfer features within the machine ID from the content-aware features output by the IDT-GCN module.

[0065] Among them, in the VDT-GCN module, we adaptively estimate the corresponding M according to the input feature S d Matrix. Because different domain transfer features of each type of machine ID have different M d Matrix, so M d The matrix can enable the model to have better feature extraction capabilities under different domain transfer conditions. The calculation method of VDT-GCN is as follows.

[0066] P'=δ(M d SW d ),where M d =Sigmoid(W A S')

[0067] in is the updated weight value, Is to construct the dynamic incidence matrix M d The weight value of the convolutional layer. S' is composed of S and S g ∈R d1 Connect in series to get S g It is calculated by the global average pooling layer (GAP) and the convolution layer.

[0068] Finally, we use the generated label P' to update the feature F using the cross entropy loss function so that the feature F can accurately capture the dependency between domain transfers. The calculation method is as shown in the formula below. It is worth noting that Dy-GCN is only used to update features during training, and the model is not used during prediction.

[0069]

[0070] S4 compensates the feature vector through the domain adaptation network.

[0071] Specifically, in the detection of abnormal machine sounds, the training set and the test set have different domain shifts, resulting in different distributions of the mixture density between the two, which makes it easy for the model to detect normal sound signals as abnormal sound signals. We hope that the features learned in the end can also be well applied to the test set. Traditional solutions use relative entropy, Bregman divergence and contrastive learning to estimate their distribution density. The above methods can learn the domain invariance of low-level features to a certain extent, but they are still easily affected by domain transformation, and the calculation is complex and inefficient. Therefore, we designed a adaptive network (DAN) to enable the model to minimize the second-order statistical difference between source features and target features. In this network, we first randomly extract sound signals of the same machine ID transferred to different domains from the training set, then perform data augmentation operations on the sound signals transferred to different domains so that the same machine ID has more features transferred from different domains. Finally, the second-order covariance of the feature vectors for different domains is calculated, and the distance difference is reduced by the back gradient propagation algorithm to reduce the distance between features with different domain shifts.

[0072] Data augmentation: We use a variety of transformations to generate more domain-shifted sound signals. For example, we use a pitch shifter to increase or decrease the pitch of the sound signal; a time shifter to randomly move the sound signal forward or backward; a time stretcher to change the speed of the sound signal; a Gaussian white noise injection sound signal; and an enhancement filter to apply different weights on the frequency band to simulate an acoustic filter.

[0073] Specific method:

[0074] S41 calculates the second-order covariance distance between the feature vectors under different domain transfers to obtain a distance difference;

[0075] Specifically, when calculating the second-order covariance distance, the source domain feature F is defined as S ={f S i} and the target feature F after domain transfer T ={f T i}, the calculation method is as follows:

[0076]

[0077]

[0078]

[0079] Where C represents the covariance matrix of the feature, n represents the number of input target domain and source domain features, and d represents the dimension of the feature vector.

[0080] Autoencoder loss function (AE Loss) has been widely used and has achieved excellent performance in ASR tasks. In this paper, in order to enable the model to adapt to the sound features after domain shift, we also added AE Loss to the model. The calculation method is as shown in the formula below.

[0081]

[0082] where x m Extract time-frequency domain features for short-time Fourier transform and log-Mel spectrum. It is the feature reconstructed by Autoencoder. MSE represents the mean square error loss function.

[0083] In this paper, in order to make the sound features between domains more compact, we use the Additive Angular Margin loss function (AAM Loss) in Arcface to update the features and obtain highly discriminable features. The calculation formula of AAM Loss is shown below.

[0084]

[0085] AAMLoss distributes the features F on a hyperplane of radius s to learn the inter-domain similarities within machine IDs and the differences between machine IDs, and performs higher feature embedding optimization. θ is the angle between the weight W and the feature. Finally, we use PolyLoss to replace the cross entropy loss function. PolyLoss fits the probability distribution of the sound signal by adding Taylor expansion, and achieves better results than the cross entropy loss function. The calculation method of PolyLoss is as follows.

[0086] l poly = l CE +∈ 1 (1-P C )

[0087] Among them l CE Represents the cross entropy loss function. ∈ 1 Represents the penalty coefficient term. In this task, we set ∈ 1 is 1. C Represents the true label. Finally, the loss function of our model is calculated as shown below.

[0088] l total = l AE +l gcn +l arc +l se rco +l poly

[0089] S42 uses a back gradient propagation algorithm to reduce the distance difference.

[0090] Dataset: We used the datasets from DCASE 2020 Task2 and DCASE 2022 Task2 to evaluate the effectiveness of our method. There are 6 machine types on the DCASE 2020 Task2 dataset. Among them, ToyConveyor has 4 different machine IDs. ToyCar, fan, pump, slider, and valve have 3 different machine IDs. There are 7 different types of machines on the DCASE 2022 Task2 dataset. The 7 different types of machines include ToyCar, ToyTrain, bearing, fan, gearbox, slider, and valve. DCASE 2022 has 3 different machine IDs for each type.

[0091] The DCASE 2022 Task2 dataset and the DCASE2020 Task2 dataset mainly differ in the following aspects:

[0092] The training set and test set of the DCASE 2022 Task2 dataset have domain shifts under different conditions, while the training set and test set of the DCASE2020 Task2 dataset have domain shifts under the same conditions.

[0093] The test set of the DCASE 2022 Task2 dataset mixes the source data that is not affected by domain shift with the target data that is affected by domain shift, and does not specify the target data and source domain. However, the DCASE2020 Task2 distinguishes the target domain from the source domain in the test set. Therefore, detection on the DCASE 2022 Task2 dataset is more difficult.

[0094] Evaluation indicators: Indicators for evaluating the detection effect of the model. We chose Area Under Curve (AUC) and partial area under AUC (pAUC) to evaluate the performance of the model. AUC is defined as the area under the Receiver Operation Characteristics (ROC) curve and the coordinate axis. PAUC is calculated as AUC in the low false-positive rate (FPR) range [0, p], where p is equal to 0.1.

[0095] DCASE 2020 Task2: To verify the effectiveness of our model, we list the AUC and pAUC results of various advanced models for detecting all machines on DCASE 2020 Task2 in Table 1. From the experimental results in Table 1, it can be seen that our proposed method improves the AUC by 2.32% and the pAU by 8.1% compared with the current advanced STgram-MFN model, significantly improving the detection performance of abnormal machine sounds, proving that the TranSelf-DyGCN model has a strong representation capability for the time-frequency domain features transferred from different domains.

[0096] DCASE 2022 Task2: In order to fully demonstrate the acoustic features for different domain shift conditions, the TranSelf-DyGCN model can also perform well in detecting abnormal sounds. We conducted experiments on DCASE 2022 Task2 and compared them with some advanced methods. The experimental results are shown in Table 2. From Table 2, it can be seen that the AUC and pAUC of our method on the average of seven machine types reached 70.41% and 62.46% respectively. Although the TranSelf-DyGCN model deviates from the results of some advanced model detections in the Toycar, ToyTrain and Fan machine categories, for example, in the Toycar category, our model is 7.89% lower in AUC and 4.14% lower in pAUC than the AE-based model, the overall detection effect of our model is better than that of the AE-based model. From these results, it can be seen that our model can fully capture the time-frequency domain characteristics of different domain transfers of sound signals and can be well applied in the task of abnormal sound detection.

[0097] Table 1 Comparison of AUC (%) and pAUC (%) of different types of machines on the DCASE2020 Task2 dataset. The best results are shown in bold.

[0098] Table 1

[0099]

[0100]

[0101]

[0102] Table 2 Comparison of AUC (%) and pAUC (%) of different types of machines on the DCASE2022 Task2 dataset. The best results are shown in bold.

[0103] Table 2

[0104]

[0105]

[0106] To show the effectiveness of our method, we conducted an ablation experiment on the proposed model on the DCASE2022 Task2 dataset to verify each key component of TranSelf-DyGCN. (including loss function, temporal feature network, token converter, dynamic graph convolutional network (DyGCN), domain adaptation network (DDN)

[0107] Loss function: We first verified the impact of the loss function on the model prediction results. We compared the Autoencoder loss function (AE Loss), Additive Angular Margin loss function (AAM Loss), cross entropy loss function, Focal loss function and Ploy loss function. The comparison results are shown in Table 3. As can be seen from Table 3, the AUC of the model with the addition of AAM, AE and Poly loss functions reached 70.41% and the PAUC reached 62.46%. The results obtained are better than the performance of other loss functions in all indicators. It shows that the addition of AAM, AE and Ploy loss functions can increase the adaptive domain shift capability of the model to a certain extent.

[0108] Temporal feature extraction network: To explore the importance of the temporal feature network module, we compared the effects of different temporal feature extraction networks in Table 4. From these results, we can see that after adding the temporal feature network to the model, our model's AUC increased by 1.45% and pAUC increased by 1.5% compared to Transformer, proving that the temporal feature network can well capture abnormally lost temporal feature information. Finally, we visualized the spectrogram of the sound signal extracted by adding the temporal feature network and Transformer, as shown in the figure below. Figure 5 As shown in the figure, we found that the time-frequency domain features extracted by the time series feature network are not only more periodic but also have richer semantic information than those extracted by the Transformer.

[0109] Time-frequency domain feature extraction network: In order to explore the modeling ability of the token converter network module for time-frequency domain features, we compare different time-frequency domain feature extraction networks in Table 5. From the third row of Table 5, it can be seen that the AUC of the token converter increased by 3.4% and the pAUC increased by 1.24% compared with MobileNetv2. From the fourth row of Table 5, it can be seen that the addition of Token Learner (TL) to the token converter increases the AUC of the token converter by 1.03% and the pAUC by 0.55%, indicating that TokenLearner can fully encode the spatial order of Patch sequence pairs in the time-frequency domain features and the spatial order of Token sequence pairs within the Patch. From the fifth row of Table 5, it can be seen that the addition of channel attention to the token converter increases the AUC of the token converter by 0.61% and the pAUC by 0.6%, indicating that the addition of the channel attention mechanism (SE) to the token converter can enable the token converter to mine more domain transfer representations. Finally, we visualized the dependency relationship of the time-frequency domain feature information after modeling the token converter, MobileNetv2, and Transformer with the addition of the channel attention mechanism and Token Learner, as shown in Figure 7 As shown. Figure 7 It can be seen that the token converter can capture the rise and fall process of each periodic time-frequency feature more correctly than MobileNetv2 and Transformer, and there are few missed recognitions and misrecognitions, indicating that the token converter can better capture global structural information and local detail features.

[0110] DyGCN and DAN: To study the importance of DyGCN and DAN, we experimented with DyGCN and DAN in Table 6. As can be seen from the third row of Table 6, we added DyGCN to the model, and the AUC and pAUC of the model increased by 1.64% and 1.98% compared with the model without DyGCN in the first row of Table 6, indicating that the model can capture the subtle dependencies between domain transfer features using content-aware features between machine IDs. As can be seen from the fourth row of Table 6, we added DAN to the TranSelf-DyGCN model, and the AUC and pAUC of the model increased by 2.36% and 0.99% compared with the model without DAN in the second row of Table 6, indicating that DAN can reduce the distance between features of different domain shifts. Finally, we compared the differences in the features of sound signals extracted by the TranSelf-DyGCN model with DyGCN and DAN and the TranSelf-DyGCN model without DyGCN and DAN, and used t-distributed stochastic neighbor embedding (t-SNE) clustering visualization, as shown in Figure 2. Figure 7 As shown. Figure 7As can be seen in (a), the features extracted by the TranSelf-DyGCN model without DyGCN and DAN overlap. Figure 8 The features extracted by the TranSelf-DyGCN model with DyGCN and DAN in (b) are easier to distinguish. This finding shows that adding DyGCN and DAN to the model enables the model to fully extract domain transfer features.

[0111] Table 3 Comparison of AUC (%) and pAUC (%) of the models with different loss functions added, and the best results are shown in bold.

[0112] Table 3

[0113]

[0114] Table 4 Comparison of AUC (%) and pAUC (%) of models with different temporal feature extraction networks added, with the best results in bold. (“-” indicates that no temporal feature extraction network is used).

[0115] Table 4

[0116]

[0117] Table 5 Comparison of AUC (%) and pAUC (%) of the models with different time-frequency domain feature extraction networks added, and the best results are shown in bold.

[0118] Table 5

[0119]

[0120] Table 6 Comparison of AUC (%) and pAUC (%) of the model with DyGCN and DAN, the best result is shown in bold. (“-” means that DyGCN and DAN are not added to the model)

[0121] Table 6

[0122]

[0123]

[0124] Figure 6 We visualized the spectrograms of Transformer and temporal feature network. (a) shows the spectrogram of sound signal extracted by Transformer. (b) shows the spectrogram of sound signal extracted by temporal feature network.

[0125] Figure 7Visualization of the time-frequency domain features of each model. Line 1 (yellow line) and line 2 (red line) represent the attention weights of the current frame. The yellow line represents the correct attention weight of the current frame, and the red line represents the incorrect attention weight of the current frame.

[0126] Figure 8 The figure shows the visualization of the feature map extracted by TranSelf-DyGCN using t-SNE. (a) shows the features extracted by TranSelf-DyGCN without DyGCN and DAN. (b) shows the features extracted by TranSelf-DyGCN with DyGCN and DAN. Different colors represent different machine IDs. Indicates the characteristics of a normal machine. × indicates the characteristics of an abnormal machine.

[0127] What is disclosed above is only a preferred embodiment of the patent name of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment are implemented, and equivalent changes made according to the claims of the present invention are still within the scope of the invention.

Claims

1. A domain-transferred self-supervised machine abnormal sound detection method, characterized in that: The following steps are involved: Design time-frequency domain feature extraction network, dynamic graph convolution network and domain adaptation network respectively; The spatial feature information of the sound signal and the feature information of the time-domain interaction are captured by the time-frequency domain feature extraction network to obtain a feature vector; Capturing the dependencies between the domain transfers of the feature vectors through the dynamic graph convolutional network; Based on the dependency relationship, compensating the feature vector through the domain adaptation network to obtain a detection result; The specific method of obtaining the feature vector by capturing the spatial feature information of the feature vector and the feature information of the time-domain interaction through the time-frequency domain feature extraction network is as follows: Extracting the time-frequency domain features of the feature vector using short-time Fourier transform and log-Mel; The time-frequency domain feature extraction network is used to capture the time-domain features lost by the short-time Fourier transform and the logarithmic Mel spectrum, and the lost features and the time-frequency domain features are added and passed into the token converter network; The time-frequency domain features are transmitted to the token converter network for spatial transformation and spatial relationship modeling to obtain a feature vector.

2. The method for self-supervised machine abnormal sound detection with domain transfer as claimed in claim 1, characterized in that: The specific manner of capturing the dependency between the domain transfers of the feature vectors through the dynamic graph convolutional network is as follows: Using an invariant domain transfer module to perform perceptual modeling on the coarse label dependency between machine IDs from the feature vector to obtain a perceptual model; Based on the perception model, a variable domain transfer module is used to capture the subtle relationship between domain transfer features in the machine ID.

3. The method for self-supervised machine abnormal sound detection with domain transfer as claimed in claim 1, characterized in that: The specific manner of compensating the feature vector by the domain adaptation network is: Calculating the second-order covariance distance between the feature vectors under different domain transfers to obtain a distance difference; The distance difference is reduced using a back gradient propagation algorithm.

4. The method for self-supervised machine abnormal sound detection with domain transfer as claimed in claim 3, characterized in that: When calculating the second-order covariance distance, source domain features and target features after domain transfer are defined.

Images