Sound anomaly detection method and device based on Transform model, equipment and medium

Abstract

The invention relates to the technical field of sound anomaly detection, in particular to a sound anomaly detection method and device based on a Transform model, equipment and a medium, and the method comprises the steps: collecting a sound signal during the operation of the equipment through a data collection interface, and obtaining an original sound signal; resampling is carried out on the collected sound signals, and normalization processing is carried out on the resampled data; mel-frequency cepstrum coefficient features are extracted from the sound signals after normalization processing, and the sound signals after normalization processing are input into a pre-training module to output high-dimensional features including time sequence and semantic information; splicing the Mel-frequency cepstrum coefficient features with the high-dimensional features to form a comprehensive feature vector; inputting the comprehensive feature vector into a support vector machine model, and performing abnormal sound recognition through a trained classification hyperplane; and detected abnormal information is fed back to the user in real time. Multi-feature fusion enables the model to identify abnormal sound more accurately.

Description

Technical Field

[0001] This application relates to the technical field of sound anomaly detection, and particularly to a sound anomaly detection method, device, equipment and medium based on the Transformer model. Background Art

[0002] In many fields such as industrial production, the stable operation of equipment is crucial. Traditional sound anomaly detection methods, such as those based on simple threshold judgment, only judge whether there is an anomaly based on the amplitude of the sound signal. This method is too simple and crude to effectively cope with the complex and changeable industrial environment, and is prone to a large number of misjudgments. With the continuous complexity and intelligent development of industrial equipment, higher requirements are put forward for the accuracy, real-time performance and detail level of fault information in sound anomaly detection.

[0003] The current mainstream sound anomaly detection methods are mainly divided into two categories: methods based on traditional signal processing, which rely on manually designed acoustic features. Feature extraction depends on expert experience, and it is difficult to capture non-linear anomaly patterns under complex working conditions. Sensitive to noise, with limited generalization ability, and difficult to adapt to changes in different equipment or environments. Methods based on deep learning, which use convolutional neural networks (CNNs) or recurrent neural networks (RNNs) to learn sound features end-to-end, such as Log-Mel spectrograms or raw waveforms. Require a large amount of labeled data, and abnormal samples are scarce in industrial scenarios, resulting in the model being prone to overfitting. High computational complexity, making it difficult to be deployed in real time on edge devices. Summary of the Invention

[0004] In view of the above problems existing in the current mainstream sound anomaly detection methods, the present invention provides a sound anomaly detection method, device, equipment and medium based on the Transformer model.

[0005] In a first aspect, the technical solution of the present invention provides a sound anomaly detection method based on the Transformer model, including the following steps: Collect the sound signal during the operation of the equipment through the data acquisition interface to obtain the original sound signal; Resample the collected sound signal and normalize the resampled data; Extract Mel-frequency cepstral coefficients features from the normalized sound signal to convert the time-domain signal into a frequency-domain feature; Input the normalized sound signal into the pre-trained Transformer-Encoder module to output high-dimensional features containing temporal and semantic information; Concatenate the Mel-frequency cepstral coefficients features with the high-dimensional features to form a comprehensive feature vector; Input the comprehensive feature vector into the support vector machine model, and perform abnormal sound recognition through the trained classification hyperplane; When an abnormal sound is detected, the abnormal information is fed back to the user in real time, and the abnormal information includes the abnormal time, the device location, and the fault type.

[0006] It can feed back detailed information such as the time of the abnormality occurrence, the device location, and the fault type to the user in real time. This enables the device maintenance personnel to quickly understand the abnormal situation, take targeted measures in a timely manner, avoid the further deterioration of the device fault, reduce the production interruption and economic losses caused by the device fault, and improve the continuity and stability of industrial production.

[0007] As an optimization of the technical solution of the present invention, the steps of resampling the collected sound signal and normalizing the resampled data include: Set the target sampling rate; Resample the original sound data according to the target sampling rate; Map the resampled data to the range of [-1, 1] to complete the normalization process.

[0008] As an optimization of the technical solution of the present invention, the steps of extracting Mel frequency cepstral coefficients features from the normalized sound signal and converting the time-domain signal into a frequency-domain feature include: Perform frame processing on the normalized sound signal, obtain the spectral distribution information of the signal through discrete Fourier transform, and calculate the energy spectrum; Pass the calculated energy spectrum through a set of Mel filter banks, and obtain the output signal of the Mel filter banks; Take the logarithm of the obtained output signal of the Mel filter banks to obtain the logarithmic energy output by the Mel filters; Calculate Mel frequency cepstral coefficients based on the logarithmic energy output by the Mel filters as the sound feature.

[0009] As an optimization of the technical solution of the present invention, the spectral distribution information of the ; energy spectrum ; Pass the energy spectrum through a set of N Mel filter banks , and obtain the output signal of the Mel filter banks ; Take the logarithm of the output signal of the Mel filter banks to obtain the logarithmic energy output by the th Mel filter ; Based on the logarithmic energy output by the Mel filters , calculate Mel frequency cepstral coefficients through discrete cosine transform As a voice feature;

[0010] Wherein, is the value of the th dimension in the voice feature of the th frame, M is the frame length, is the th frame signal th logarithm energy output by the Mel filter, N is the number of Mel filters, and L is the dimension of the Mel frequency cepstrum coefficient feature vector; The Mel frequency cepstrum coefficient feature vector of the th frame of voice signal is .

[0011] As a preference of the technical solution of the present invention, the method further includes: training the Transformer-Encoder module; specifically including: Construct a Transformer-Encoder model, including an input layer, a multi-head attention layer, a feed-forward neural network layer, a residual connection layer, and a normalization layer; Use labeled voice data to input the Encoder layer of the Transformer model for supervised pre-training. The input data of the model uses the short-time Fourier transform spectrogram, and maps the spectrogram to the model dimension through a linear projection layer as a sequence input; add positional encoding to the input sequence before training; Adopt the multi-head self-attention mechanism to capture the long-distance dependence relationship of the voice sequence; Adopt the cross-entropy loss function to optimize the model, calculate the error between the predicted category and the true label, and backpropagate to update the parameters of the model; After the training is completed, fix the weights of the Transformer-Encoder for extracting high-dimensional temporal and semantic features.

[0012] As a preference of the technical solution of the present invention, the steps of splicing the Mel frequency cepstrum coefficient feature and the high-dimensional feature to form a comprehensive feature vector include: Align the Mel frequency cepstrum coefficient feature sequence and the high-dimensional feature sequence according to the set time step; If the dimension of the Mel frequency cepstrum coefficient feature is not equal to the dimension of the high-dimensional feature, map the Mel frequency cepstrum coefficient feature to the high-dimensional feature dimension through a fully connected layer, or reduce the dimension of the high-dimensional feature to the Mel frequency cepstrum coefficient feature dimension; Splice the aligned Mel frequency cepstrum coefficient feature and the high-dimensional feature along the feature dimension; Perform mean pooling on the spliced temporal features along the time axis to compress them into a comprehensive feature vector with a fixed dimension.

[0013] Preferably, as a technical solution of the present invention, the method further includes: training a support vector machine model to obtain a trained classification hyperplane; the specific steps include: Dividing the comprehensive feature vectors into a training set and a validation set; Initializing the support vector machine model and selecting a kernel function according to the characteristics of the sound data; Determining the objective function and the constraint conditions; Based on the selected kernel function and the objective function, training the support vector machine model based on the training set, using parameter grid search, iteratively updating the model parameters, and minimizing the objective function; in each iteration, updating the slack variables according to the constraint conditions; Evaluating the model during the training process using the validation set, and adjusting the penalty factor and the kernel function parameters according to the evaluation results; After the training is completed, determine the final parameters to obtain a hyperplane capable of classifying new data.

[0014] Preferably, as a technical solution of the present invention, the objective function is , and the constraint . In the formula, is the normal vector of the hyperplane, is the penalty factor, is the slack variable, is the sample label, is the feature vector after kernel function mapping.

[0015] In a second aspect, a technical solution of the present invention provides a sound anomaly detection device based on a Transformer model, including a data acquisition module, a preprocessing module, a sound feature extraction module, a high-dimensional feature acquisition module, a splicing module, a sound recognition module, and a feedback module; The data acquisition module is used to acquire the sound signal during the operation of the device through the data acquisition interface to obtain the original sound signal; The preprocessing module is used to resample the acquired sound signal and perform normalization processing on the resampled data; The sound feature extraction module is used to extract Mel frequency cepstral coefficient features from the normalized sound signal and convert the time-domain signal into a frequency-domain feature; The high-dimensional feature acquisition module is used to input the normalized sound signal into the pre-trained Transformer-Encoder module to output high-dimensional features containing temporal and semantic information; The splicing module is used to splice the Mel frequency cepstral coefficient features and the high-dimensional features to form a comprehensive feature vector; The sound recognition module is used to input the comprehensive feature vector into the support vector machine model and perform abnormal sound recognition through the trained classification hyperplane; A feedback module, which is used to, when an abnormal sound is detected, feedback the abnormal information to the user in real time, and the abnormal information includes the abnormal time, the device location, and the fault type.

[0016] The Transformer-Encoder learns general acoustic representations from normal sound data through pre-training, reducing the dependence on abnormal samples; the SVM can still maintain stable classification performance in the small-sample scenario, avoiding the overfitting problem of deep learning models. The Transformer-Encoder is only used for feature extraction, with fixed weights in the inference stage and controllable computational complexity; the SVM classifier is computationally efficient and suitable for deployment on edge devices (such as industrial gateways) to achieve real-time monitoring.

[0017] As an optimization of the technical solution of the present invention, the preprocessing module is specifically configured to set a target sampling rate; resample the original sound data according to the target sampling rate; and map the resampled data to the range of [-1, 1] to complete the normalization process.

[0018] As an optimization of the technical solution of the present invention, the sound feature extraction module includes a framing processing unit, an energy calculation unit, and a sound feature extraction unit; The framing processing unit is configured to perform framing processing on the normalized sound signal, obtain the spectral distribution information of the signal through discrete Fourier transform, and calculate the energy spectrum; The energy calculation unit is configured to pass the calculated energy spectrum through a set of Mel filter banks and obtain the output signal of the Mel filter banks; take the logarithm of the obtained output signal of the Mel filter banks to obtain the logarithmic energy output by the Mel filter banks; The sound feature extraction unit is configured to calculate the Mel frequency cepstral coefficients based on the logarithmic energy output by the Mel filter banks as the sound features.

[0019] As an optimization of the technical solution of the present invention, the spectral distribution information of the nth frame signal; the energy spectrum is passed through a set of N Mel filter banks to obtain the output signal of the Mel filter banks ; take the logarithm of the output signal of the Mel filter banks to obtain the logarithmic energy output by the nth Mel filter; Based on the logarithmic energy output by the Mel filter banks the Mel frequency cepstral coefficients are calculated through discrete cosine transform as the sound features;

[0020] Among them, is the value of the -th dimension in the sound feature of the -th frame. M is the frame length. is the logarithmic energy output by the -th Mel filter for the -th frame signal. N is the number of Mel filters, and L is the dimension of the Mel-frequency cepstral coefficient feature vector. The Mel-frequency cepstral coefficient feature vector of the -th frame of the sound signal is .

[0021] As an optimization of the technical solution of the present invention, the device further includes a first model training module for training the Transformer-Encoder module; specifically, it is used to construct a Transformer-Encoder model, including an input layer, a multi-head attention layer, a feed-forward neural network layer, a residual connection layer, and a normalization layer; use labeled sound data to input the Encoder layer of the Transformer model for supervised pre-training. The input data of the model uses the short-time Fourier transform spectrogram, and maps the spectrogram to the model dimension through a linear projection layer as the sequence input; add positional encoding to the input sequence before training; adopt the multi-head self-attention mechanism to capture the long-range dependencies of the sound sequence; adopt the cross-entropy loss function to optimize the model, calculate the error between the predicted category and the true label, and backpropagate to update the parameters of the model; after the training is completed, fix the weights of the Transformer-Encoder for extracting high-dimensional time series and semantic features.

[0022] As an optimization of the technical solution of the present invention, the splicing module is specifically used to align the Mel-frequency cepstral coefficient feature sequence and the high-dimensional feature sequence according to the set time step; if the dimension of the Mel-frequency cepstral coefficient feature is not equal to the dimension of the high-dimensional feature, map the Mel-frequency cepstral coefficient feature to the high-dimensional feature dimension through a fully connected layer, or reduce the dimension of the high-dimensional feature to the dimension of the Mel-frequency cepstral coefficient feature; splice the aligned Mel-frequency cepstral coefficient feature and the high-dimensional feature along the feature dimension; perform mean pooling on the spliced time series feature along the time axis to compress it into a comprehensive feature vector with a fixed dimension.

[0023] Preferably, as a technical solution of the present invention, the device further includes a second model training module, which is used to train the support vector machine model to obtain a trained classification hyperplane; specifically, it is used to divide the comprehensive feature vectors into a training set and a validation set; initialize the support vector machine model, and select a kernel function according to the characteristics of the sound data; determine the objective function and the constraint conditions; based on the selected kernel function and the objective function, train the support vector machine model based on the training set, adopt parameter grid search, iteratively update the model parameters, and minimize the objective function; in each iteration, update the slack variables according to the constraint conditions; use the validation set to evaluate the model during the training process, and adjust the penalty factor and the kernel function parameters according to the evaluation results; after the training is completed, determine the final parameters to obtain a hyperplane that can classify new data.

[0024] The objective function is , and the constraint , where, is the normal vector of the hyperplane, is the penalty factor, is the slack variable, is the sample label, is the feature vector after kernel function mapping.

[0025] In a third aspect, the technical solution of the present invention further provides an electronic device, where the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the sound anomaly detection method based on the Transformer model as described in the first aspect.

[0026] In a fourth aspect, the technical solution of the present invention further provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the sound anomaly detection method based on the Transformer model as described in the first aspect.

[0027] As can be seen from the above technical solutions, the present application has the following advantages: By virtue of the self-supervised learning ability of the Transformer-Encoder module, the feature representation of abnormal sounds is adaptively learned from the original data. This characteristic effectively solves the problem that it is difficult to obtain sufficient abnormal sound samples due to the low incidence rate and diverse types of abnormal sounds in industrial scenarios. The Mel Frequency Cepstral Coefficient (MFCC) features are concatenated with the high-dimensional features output by the Transformer-Encoder module to form a comprehensive feature vector. This multi-feature fusion method provides more comprehensive and rich information for the support vector machine model, enabling the model to more accurately identify abnormal sounds. At the same time, the support vector machine model can effectively solve the problem of unbalanced sample data distribution by finding the optimal classification hyperplane. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] To more clearly illustrate the technical solutions of the present application, the drawings required for description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0029] Figure 1 It is a flowchart of the method provided by the embodiment of the present invention.

[0030] Figure 2 It is a block diagram of the device provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] To make the application purpose, features, and advantages of the present application more obvious and understandable, the technical solutions protected by the present application will be clearly and completely described below by using specific embodiments and the accompanying drawings. Obviously, the embodiments described below are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in this patent, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of this patent.

[0032] Transformer: A deep learning model based on the attention mechanism, consisting of two parts: an encoder and a decoder. During the model training process, through the self-attention mechanism and positional encoding, Transformer can efficiently process sequence data and capture long-range dependency relationships.

[0033] Encoder: One of the core components in the Transformer model, responsible for mapping the input sequence into a series of high-dimensional vector representations. Through multiple layers of self-attention mechanisms and feed-forward neural networks, the encoder can extract key information from the input data and provide effective feature representations for subsequent tasks.

[0034] High-dimensional features: In data science and machine learning, high-dimensional features refer to the feature vectors obtained after feature engineering, where the data is mapped into a high-dimensional space. These feature vectors contain a large amount of information in the original data, which helps the model better understand and classify the data.

[0035] Feature engineering: A key step in data preprocessing, aiming to extract useful features for model training from the original data through methods such as selection, construction, transformation, and combination. Good feature engineering can significantly improve the performance and accuracy of the model. Support vector machine: A supervised machine learning algorithm that can be used for classification and regression analysis and is widely applied in fields such as pattern recognition and data mining.

[0036] As Figure 1 shown, an embodiment of the present invention provides a method for detecting abnormal sounds based on a Transformer model, including the following steps: S1: Collect the sound signal during the operation of the device through a data acquisition interface to obtain the original sound signal; The professional audio acquisition devices supported by the data acquisition interface include, but are not limited to, microphone arrays and piezoelectric sensors to adapt to the sound acquisition requirements of different industrial devices.

[0037] S2: Resample the collected sound signal and perform normalization processing on the resampled data; In this step, first set the target sampling rate; resample the original sound data according to the target sampling rate; map the resampled data to the range [-1, 1] to complete the normalization processing. The preset standard frequency for resampling is set according to different model processing requirements, and common values are 8 kHz, 16 kHz, or 44.1 kHz.

[0038] S3: Extract Mel-frequency cepstral coefficient features from the normalized sound signal to convert the time-domain signal into a frequency-domain feature; The Mel-frequency cepstral coefficient (MFCC) method can be used to obtain the frequency-domain information of the sound and convert the sound into a region sensitive to the human ear. After obtaining a sound sample, frame it and apply a window, then for each short-time analysis window, the corresponding spectrum can be obtained by Fourier transform, and then the above spectrum can be passed through a Mel filter to obtain the Mel spectrum, and finally, cepstral analysis is performed on the obtained Mel spectrum to obtain the Mel-spectrum cepstral coefficients; specifically including: S31: Perform framing processing on the normalized sound signal, obtain the spectrum distribution information of the signal through discrete Fourier transform, and calculate the energy spectrum; S32: Pass the calculated energy spectrum through a set of Mel filter banks and obtain the output signal of the Mel filter banks; S33: Take the logarithm of the obtained output signal of the Mel filter banks to obtain the logarithmic energy output by the Mel filter; S34: Calculate the Mel-frequency cepstral coefficients based on the logarithmic energy output by the Mel filter bank as the voice feature.

[0039] In the embodiments of the present invention, the spectral distribution information of the nth frame signal ; energy spectrum ; Pass the energy spectrum through a set of N Mel filter banks to obtain the output signal of the Mel filter bank ; Take the logarithm of the output signal of the Mel filter bank to obtain the logarithmic energy output by the nth Mel filter ; ; Based on the logarithmic energy output by the Mel filter bank , calculate the Mel-frequency cepstral coefficients through discrete cosine transform as the voice feature;

[0040] wherein, is the value of the mth dimension in the voice feature of the nth frame, M is the frame length, is the logarithmic energy output by the mth Mel filter of the nth frame signal, N is the number of Mel filters, and L is the dimension of the Mel-frequency cepstral coefficient feature vector; The Mel-frequency cepstral coefficient feature vector of the nth frame voice signal is The Mel-frequency cepstral coefficient feature vector of the nth frame voice signal is .

[0041] S4: Input the normalized voice signal into the pre-trained Transformer-Encoder module to output high-dimensional features containing temporal and semantic information; It should be noted that the general steps for training the Transformer-Encoder module include: constructing a Transformer-Encoder model, which includes an input layer, a multi-head attention layer, a feed-forward neural network layer, a residual connection layer, and a normalization layer; using labeled voice data to input the Encoder layer of the Transformer model for supervised pre-training. The input data of the model uses the short-time Fourier transform spectrogram, and the spectrogram is mapped to the model dimension through a linear projection layer as the sequence input; adding positional encoding to the input sequence before training; adopting the multi-head self-attention mechanism to capture the long-distance dependencies of the voice sequence; using the cross-entropy loss function to optimize the model, calculating the error between the predicted category and the true label and backpropagating to update the parameters of the model; after training is completed, fixing the weights of the Transformer-Encoder for extracting high-dimensional temporal and semantic features.

[0042] With its unique attention mechanism, the Transformer model demonstrates powerful feature extraction capabilities when processing sequence data. In this system, only the Encoder part of the Transformer model is used. First, we will select a part of the data with true labels to perform supervised pre-training on the Transformer model. During the pre-training process, we use the Adam optimizer and set the learning rate to 0.01. After 30 epochs of training, we will separately extract the Encoder part of the Transformer and use it as an encoder for high-dimensional features. Input the normalized voice data features into the Encoder layer of the Transformer model. This layer contains multiple multi-head attention mechanisms and feed-forward neural networks, which can fully explore the long-distance dependencies in the voice data. The pre-trained Transformer-Encoder can effectively distinguish abnormal voices at the high-dimensional level, and the high-dimensional features it outputs can guide the subsequent classifier and increase its accuracy.

[0043] S5: Concatenate the Mel-frequency cepstral coefficient features and the high-dimensional features to form a comprehensive feature vector; The specific implementation process is as follows: Align the Mel-frequency cepstral coefficient feature sequence and the high-dimensional feature sequence according to the set time steps; if the dimensions of the Mel-frequency cepstral coefficient features and the high-dimensional features are not equal, map the Mel-frequency cepstral coefficient features to the high-dimensional feature dimension through a fully connected layer, or reduce the dimensions of the high-dimensional features to the Mel-frequency cepstral coefficient feature dimension; concatenate the aligned Mel-frequency cepstral coefficient features and the high-dimensional features along the feature dimension; perform mean pooling on the concatenated temporal features along the time axis to compress them into a comprehensive feature vector with a fixed dimension.

[0044] S6: Input the comprehensive feature vector into the support vector machine model, and perform abnormal sound recognition through the trained classification hyperplane; In the embodiment of the present invention, after splicing the extracted sound features and the high-dimensional features obtained by the Transformer-Encoder model, they are input into the support vector machine model for abnormal sound recognition together. The support vector machine is a binary classification model. By finding an optimal classification hyperplane, it separates the data samples of normal sounds and abnormal sounds, and has good generalization ability and classification performance. It usually has three steps: selecting a kernel function, constructing an objective function, and finding the optimal solution. The role of the kernel function is to map the original data to a high-dimensional space so as to find a linearly separable hyperplane in the high-dimensional space. According to the training data and the selected kernel function, the objective function of the SVM is constructed. For the linearly separable case, the goal is to find a hyperplane that maximizes the margin between the two classes of data. This margin is defined as the sum of the distances from the points closest to the hyperplane in the two classes of data to the hyperplane. For the non-linearly separable case, by introducing slack variables, allowing some data points to violate the margin constraint, the objective function maximizes the margin while minimizing the degree of violation of the constraint. The process of finding the objective function is the training process of the support vector machine. In the present invention, the parameter grid search is selected to update and iterate the model parameters to reach the optimal solution. In addition, during the training process of the support vector machine, the parameter grid search method is used to find the best parameters, and the parameter combination with the highest F1_code score is found after multiple trainings.

[0045] Specifically, train the support vector machine model to obtain a trained classification hyperplane; the specific steps include: dividing the comprehensive feature vector into a training set and a validation set; initializing the support vector machine model and selecting a kernel function according to the characteristics of the sound data; determining the objective function and constraint conditions; based on the selected kernel function and objective function, training the support vector machine model based on the training set, using the parameter grid search to iteratively update the model parameters and minimize the objective function; in each iteration, updating the slack variables according to the constraint conditions; using the validation set to evaluate the model during the training process, and adjusting the penalty factor and kernel function parameters according to the evaluation results; after the training is completed, determining the final parameters to obtain a hyperplane that can classify new data.

[0046] The objective function is , the constraint , where is the normal vector of the hyperplane, is the penalty factor, is the slack variable, is the sample label, is the feature vector after kernel function mapping.

[0047] S7: When abnormal sound is detected, the abnormal information is fed back to the user in real time, and the abnormal information includes the abnormal time, the device location, and the fault type.

[0048] It can feed back detailed information such as the time when the abnormality occurs, the device location, and the fault type to the user in real time. This enables the device maintenance personnel to quickly understand the abnormal situation, take targeted measures in a timely manner, avoid the further deterioration of the device fault, reduce the production interruption and economic losses caused by the device fault, and improve the continuity and stability of industrial production.

[0049] In the embodiment of the present invention, once the support vector machine model detects abnormal sound, the relevant information of the abnormal sound, such as the time when the abnormality occurs, the device location, the abnormal type, etc., is fed back to the user so that the user can take measures in a timely manner to avoid greater losses caused by the device fault. Generally, the system clock is used to synchronously record the abnormal trigger moment, the physical location of the device is obtained by associating the device ID with the pre-stored topology database, and based on the historical fault database, the possible fault types are matched through the acoustic feature similarity. Specifically, it is necessary to calculate the cosine similarity between the MFCC / Transformer features of the current abnormal sound and the templates in the fault database. The 3 fault types with the highest similarity and probabilities are returned.

[0050] As Figure 2 shown, the embodiment of the present invention provides a sound abnormality detection device based on a Transformer model, including a data acquisition module, a preprocessing module, a sound feature extraction module, a high-dimensional feature acquisition module, a splicing module, a sound recognition module, and a feedback module; The data acquisition module is used to acquire the sound signal during the operation of the device through the data acquisition interface to obtain the original sound signal; The preprocessing module is used to resample the acquired sound signal and perform normalization processing on the resampled data; The sound feature extraction module is used to extract Mel frequency cepstral coefficients features from the normalized sound signal to convert the time-domain signal into a frequency-domain feature; The high-dimensional feature acquisition module is used to input the normalized sound signal into the pre-trained Transformer-Encoder module to output high-dimensional features containing temporal and semantic information; The splicing module is used to splice the Mel frequency cepstral coefficients features and the high-dimensional features to form a comprehensive feature vector; The sound recognition module is used to input the comprehensive feature vector into the support vector machine model to identify abnormal sound through the trained classification hyperplane; The feedback module is used to, when abnormal sound is detected, feed back the abnormal information to the user in real time, and the abnormal information includes the abnormal time, the device location, and the fault type.

[0051] The Transformer-Encoder learns general acoustic representations from normal sound data through pre-training, reducing the dependence on abnormal samples; the SVM can still maintain stable classification performance in small-sample scenarios, avoiding the overfitting problem of deep learning models. The Transformer-Encoder is only used for feature extraction, with fixed weights during the inference phase and controllable computational complexity; the SVM classifier is computationally efficient and suitable for deployment on edge devices (such as industrial gateways) for real-time monitoring.

[0052] In some embodiments, the preprocessing module is specifically configured to set the target sampling rate; resample the original sound data according to the target sampling rate; and map the resampled data to the range of [-1, 1] to complete the normalization process.

[0053] In some embodiments, the sound feature extraction module includes a framing processing unit, an energy calculation unit, and a sound feature extraction unit; The framing processing unit is configured to frame the normalized sound signal, obtain the spectral distribution information of the signal through discrete Fourier transform, and calculate the energy spectrum; The energy calculation unit is configured to pass the calculated energy spectrum through a set of mel filter banks and obtain the output signal of the mel filter banks; take the logarithm of the obtained output signal of the mel filter banks to obtain the logarithmic energy output by the mel filter; The sound feature extraction unit is configured to calculate the mel-frequency cepstral coefficients based on the logarithmic energy output by the mel filter as the sound features.

[0054] The spectral distribution information of the ; energy spectrum ; Pass the energy spectrum through a set of N mel filter banks to obtain the output signal of the mel filter banks ; Take the logarithm of the output signal of the mel filter banks to obtain the logarithmic energy output by the th mel filter ; Based on the logarithmic energy output by the mel filter calculate the mel-frequency cepstral coefficients through discrete cosine transform as the sound features;

[0055] where is the value of the th dimension in the sound features of the th frame, M is the frame length, is the log energy output of the frame signal for the th Mel filter, where N is the number of Mel filters and L is the dimension of the Mel-frequency cepstral coefficient feature vector; The Mel-frequency cepstral coefficient feature vector of the -th frame of the voice signal is

[0056] In some embodiments, the device further includes a first model training module for training the Transformer-Encoder module; specifically for constructing a Transformer-Encoder model, including an input layer, a multi-head attention layer, a feed-forward neural network layer, a residual connection layer, and a normalization layer; using labeled voice data to input the Encoder layer of the Transformer model for supervised pre-training, and the input data of the model uses the short-time Fourier transform spectrogram, which is mapped to the model dimension through a linear projection layer as a sequence input; adding positional encoding to the input sequence before training; adopting a multi-head self-attention mechanism to capture the long-range dependencies of the voice sequence; adopting a cross-entropy loss function to optimize the model, calculating the error between the predicted category and the true label and backpropagating to update the parameters of the model; after training is completed, fixing the weights of the Transformer-Encoder for extracting high-dimensional temporal and semantic features.

[0057] In some embodiments, the splicing module is specifically used to align the Mel-frequency cepstral coefficient feature sequence and the high-dimensional feature sequence according to a set time step; if the dimension of the Mel-frequency cepstral coefficient feature is not equal to the dimension of the high-dimensional feature, the Mel-frequency cepstral coefficient feature is mapped to the high-dimensional feature dimension through a fully connected layer, or the high-dimensional feature is reduced to the Mel-frequency cepstral coefficient feature dimension; splicing the aligned Mel-frequency cepstral coefficient feature and the high-dimensional feature along the feature dimension; performing mean pooling on the spliced temporal features along the time axis to compress them into a comprehensive feature vector with a fixed dimension.

[0058] In some embodiments, the device further includes a second model training module for training a support vector machine model to obtain a trained classification hyperplane; specifically for dividing the comprehensive feature vector into a training set and a validation set; initializing the support vector machine model and selecting a kernel function according to the characteristics of the voice data; determining the objective function and constraints; based on the selected kernel function and objective function, training the support vector machine model based on the training set, using parameter grid search to iteratively update the model parameters and minimize the objective function; in each iteration, updating the slack variable according to the constraints; using the validation set to evaluate the model during the training process, and adjusting the penalty factor and kernel function parameters according to the evaluation results; after training is completed, determining the final parameters to obtain a hyperplane that can classify new data.

[0059] The objective function is , and the constraint , where is the hyperplane normal vector, is the penalty factor, is the slack variable, is the sample label, is the feature vector after kernel function mapping.

[0060] An embodiment of the present invention further provides an electronic device, which includes: a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus. The communication bus can be used for information transmission between the electronic device and the sensor. The processor can call the logical instructions in the memory to execute the following method: S1: Collect the sound signal during device operation through the data acquisition interface to obtain the original sound signal; S2: Resample the collected sound signal and perform normalization processing on the resampled data; S3: Extract the Mel-frequency cepstral coefficient features from the normalized sound signal to convert the time-domain signal into a frequency-domain feature; S4: Input the normalized sound signal into the pre-trained Transformer-Encoder module to output high-dimensional features containing timing and semantic information; S5: Concatenate the Mel-frequency cepstral coefficient features and the high-dimensional features to form a comprehensive feature vector; S6: Input the comprehensive feature vector into the support vector machine model, and perform abnormal sound recognition through the trained classification hyperplane; S7: If an abnormal sound is detected, the abnormal information is fed back to the user in real time, and the abnormal information includes the abnormal time, the device location, and the fault type.

[0061] In addition, when the logical instructions in the above-mentioned memory are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0062] An embodiment of the present invention provides a non-transitory computer-readable storage medium that stores computer instructions, which cause a computer to execute the method provided in the above method embodiment. For example, it includes: S1: Collect the sound signal during device operation through a data acquisition interface to obtain the original sound signal; S2: Resample the collected sound signal and perform normalization processing on the resampled data; S3: Extract Mel-frequency cepstral coefficient features from the normalized sound signal to convert the time-domain signal into a frequency-domain feature; S4: Input the normalized sound signal into a pre-trained Transformer-Encoder module to output high-dimensional features containing temporal and semantic information; S5: Concatenate the Mel-frequency cepstral coefficient features with the high-dimensional features to form a comprehensive feature vector; S6: Input the comprehensive feature vector into a support vector machine model to perform abnormal sound recognition through a trained classification hyperplane; S7: If an abnormal sound is detected, the abnormal information is fed back to the user in real time, and the abnormal information includes the abnormal time, device location, and fault type.

[0063] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting abnormal sounds based on the Transformer model, characterized in that It includes the following steps: Collect the sound signal during device operation through the data acquisition interface to obtain the original sound signal; Resample the collected sound signal and normalize the resampled data; Extract Mel-frequency cepstral coefficients features from the normalized sound signal to convert the time-domain signal into frequency-domain features; Input the normalized sound signal into the pre-trained Transformer-Encoder module to output high-dimensional features containing temporal and semantic information; Concatenate the Mel-frequency cepstral coefficients features and the high-dimensional features to form a comprehensive feature vector; Input the comprehensive feature vector into the support vector machine model to perform abnormal sound recognition through the trained classification hyperplane; When an abnormal sound is detected, the abnormal information is fed back to the user in real time, and the abnormal information includes the abnormal time, device location, and fault type.

2. The method for detecting abnormal sound based on the Transformer model according to claim 1, wherein, The steps of extracting Mel-frequency cepstral coefficients features from the normalized sound signal to convert the time-domain signal into frequency-domain features include: Perform frame processing on the normalized sound signal, obtain the spectral distribution information of the signal through discrete Fourier transform, and calculate the energy spectrum; Pass the calculated energy spectrum through a set of Mel filter banks and obtain the output signal of the Mel filter banks; Take the logarithm of the obtained output signal of the Mel filter banks to get the logarithmic energy output by the Mel filter; Calculate Mel-frequency cepstral coefficients based on the logarithmic energy output by the Mel filter as the sound feature.

3. The method for detecting abnormal sound based on the Transformer model according to claim 2, wherein, The spectrum distribution information of the frame signal ; energy spectrum ; The energy spectrum is passed through a set of N Mel filter banks to obtain the Mel filter bank output signal ; For the output signal of the Mel filter bank Take the logarithm to obtain the logarithmic energy output by the th Mel filter; Logarithmic energy based on the output of the Mel filter , calculate the Mel-frequency cepstral coefficients through the discrete cosine transform as the voice feature; Among them, is the value of the -th dimension in the acoustic features of the -th frame, M is the frame length, is the logarithmic energy output by the -th mel filter for the -th frame signal, N is the number of mel filters, and L is the dimension of the mel-frequency cepstral coefficient feature vector; The Mel-frequency cepstral coefficient feature vector of the -th frame sound signal is 4. The method for detecting abnormal sound based on the Transformer model according to claim 3, wherein, This method also includes: training the Transformer-Encoder module; specifically including: Construct a Transformer-Encoder model, including an input layer, a multi-head attention layer, a feed-forward neural network layer, a residual connection layer, and a normalization layer; Use the labeled sound data to input the Encoder layer of the Transformer model for supervised pre-training. The input data of the model uses the short-time Fourier transform spectrogram, and maps the spectrogram to the model dimension through a linear projection layer as the sequence input; add positional encoding to the input sequence before training; Adopt the multi-head self-attention mechanism to capture the long-range dependence relationship of the sound sequence; Adopt the cross-entropy loss function to optimize the model, calculate the error between the predicted category and the true label, and backpropagate to update the parameters of the model; After training is completed, fix the weights of the Transformer-Encoder for extracting high-dimensional temporal and semantic features.

5. The method for detecting voice anomalies based on the Transformer model according to claim 4, characterized in that, The steps of concatenating the Mel-frequency cepstral coefficients features and the high-dimensional features to form a comprehensive feature vector include: Align the Mel-frequency cepstral coefficients feature sequence and the high-dimensional feature sequence according to the set time step; If the dimension of the Mel-frequency cepstral coefficients features is not equal to the dimension of the high-dimensional features, map the Mel-frequency cepstral coefficients features to the high-dimensional feature dimension through a fully connected layer, or reduce the dimension of the high-dimensional features to the dimension of the Mel-frequency cepstral coefficients features; Concatenate the aligned Mel-frequency cepstral coefficients features and the high-dimensional features along the feature dimension; Perform mean pooling on the concatenated temporal features along the time axis to compress them into a comprehensive feature vector with a fixed dimension.

6. The method for detecting abnormal sound based on the Transformer model according to claim 5, characterized in that The method further includes: training a support vector machine model to obtain a trained classification hyperplane; the specific steps include: Dividing the comprehensive feature vector into a training set and a validation set; Initializing the support vector machine model and selecting a kernel function according to the characteristics of the sound data; Determining the objective function and the constraint conditions; Based on the selected kernel function and the objective function, training the support vector machine model based on the training set, using parameter grid search, iteratively updating the model parameters, and minimizing the objective function; in each iteration, updating the slack variable according to the constraint conditions; Using the validation set to evaluate the model during the training process, and adjusting the penalty factor and the kernel function parameters according to the evaluation results; After the training is completed, determining the final parameters to obtain a hyperplane that can classify new data.

7. The method for detecting abnormal sound based on the Transformer model according to claim 6, characterized in that, The objective function is , and the constraint . In the formula, is the normal vector of the hyperplane, is the penalty factor, is the slack variable, is the sample label, is the eigenvector after the kernel function mapping.

8. An abnormal sound detection device based on the Transformer model, characterized in that, It includes a data acquisition module, a preprocessing module, a sound feature extraction module, a high-dimensional feature acquisition module, a splicing module, a sound recognition module, and a feedback module; The data acquisition module is used to acquire the sound signal during the operation of the device through the data acquisition interface to obtain the original sound signal; The preprocessing module is used to resample the acquired sound signal and perform normalization processing on the resampled data; The sound feature extraction module is used to extract the Mel-frequency cepstral coefficient features from the normalized sound signal and convert the time-domain signal into a frequency-domain feature; The high-dimensional feature acquisition module is used to input the normalized sound signal into the pre-trained Transformer-Encoder module to output high-dimensional features containing temporal and semantic information; The splicing module is used to splice the Mel-frequency cepstral coefficient features and the high-dimensional features to form a comprehensive feature vector; The sound recognition module is used to input the comprehensive feature vector into the support vector machine model and perform abnormal sound recognition through the trained classification hyperplane; The feedback module is used to, when an abnormal sound is detected, feedback the abnormal information to the user in real time, and the abnormal information includes the abnormal time, the device location, and the fault type.

9. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the Transformer model-based sound anomaly detection method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the Transformer model-based sound anomaly detection method according to any one of claims 1 to 7.

Images