Anomaly detection methods, devices, and storage media based on machine operating audio.

Abstract

The application provides an abnormality detection method and device based on machine working audio and a storage medium, the method comprising: obtaining a machine audio sample; extracting a mel filter bank energy feature of the machine audio sample; extracting deep features of all mel filter bank energy features through a transformer encoder to obtain multiple deep general features and multiple deep to-be-tested features; constructing a training feature set according to all deep general features, and determining an abnormality threshold value from the training feature set by using a K nearest neighbor algorithm; obtaining distance measurement values of the deep to-be-tested features and each deep general feature in the training feature set one by one, and generating an abnormality score of the deep to-be-tested features based on the distance measurement values; comparing the abnormality score with the abnormality threshold value, and determining whether a machine to be detected has a fault according to a comparison result. The application only determines a detection threshold value through an audio sample when the machine is normally working, and the fault detection can be realized without the participation of an abnormality sample in the judgment process.

Description

Technical Field

[0001] This invention relates to the field of industrial equipment fault detection technology, and in particular to an anomaly detection method, device and storage medium based on machine operating audio. Background Technology

[0002] In modern industry and manufacturing, the proper functioning of equipment is crucial for ensuring production efficiency and product qualification rates. However, equipment is often affected by various factors during operation, such as mechanical wear, electrical faults, and environmental noise. These problems can lead to a decline in equipment performance and a decrease in product qualification rates. Therefore, timely monitoring of the operating status of production machines or equipment is an indispensable part of industrial production.

[0003] In existing technologies, methods for determining the operating status of machines based on their operating sounds mainly include segmentation and classification models based on neural networks. These methods use normal and abnormal sound features as input, rely on label-supervised training, and perform well when both positive and negative samples are plentiful. However, in real-world industrial scenarios, while normal machine operating sounds are readily available, abnormal sound samples are often scarce, limiting the practical application of this type of method.

[0004] Therefore, how to achieve machine fault detection without requiring abnormal audio samples is a key problem that audio-based machine fault detection methods urgently need to solve. Summary of the Invention

[0005] The present invention aims to at least solve one of the technical problems existing in the prior art. To this end, the present invention provides an anomaly detection method, apparatus and storage medium based on machine operating audio, which can realize fault detection by using only audio samples of the machine in normal state and the current audio of the machine under test, without relying on audio data of the machine in abnormal state.

[0006] In a first aspect, embodiments of the present invention provide an anomaly detection method based on machine operating audio, comprising:

[0007] Obtain machine audio samples, which include training samples and test samples. The training samples are audio samples when the machine is working normally, and the test samples are audio samples when the machine to be tested is working.

[0008] The Mel filter bank energy features of the machine audio samples are extracted, wherein the Mel filter bank energy features include a first Fbank feature and a second Fbank feature, the Mel filter bank energy features extracted from the training samples are the first Fbank features, and the Mel filter bank energy features extracted from the test samples are the second Fbank features;

[0009] The deep features of all the energy features of the Mel filter bank are extracted by the transformer encoder to obtain multiple deep general features and multiple deep features to be measured. The deep general features are extracted from the first Fbank features and the deep features to be measured are extracted from the second Fbank features.

[0010] A training feature set is constructed based on all the aforementioned deep general features, and an anomaly threshold is determined from the training feature set using the K-nearest neighbor algorithm;

[0011] The distance metric between the depth feature to be tested and each of the deep general features in the training feature set is obtained one by one, and an anomaly score of the depth feature to be tested is generated based on the distance metric.

[0012] The abnormal score is compared with the abnormal threshold, and the machine under test is determined to be faulty based on the comparison result.

[0013] In some embodiments of the present invention, the step of extracting the Mel filter bank energy features of the machine audio samples includes:

[0014] The machine audio samples are divided into frames according to a preset frame length, and adjacent frames are overlapped by a preset overlap length.

[0015] After applying a Hanning window to each frame of the signal, a short-time Fourier transform is performed to obtain the spectrum;

[0016] Calculate the power spectrum of the spectrum;

[0017] The power spectrum is filtered through a triangular filter bank based on the Mel scale, and the output energy of each filter is calculated to obtain the Mel filter bank energy characteristics of the machine audio sample.

[0018] Wherein, when the machine audio sample is the training sample, the obtained Mel filter bank energy feature is the first Fbank feature; when the machine audio sample is the test sample, the obtained Mel filter bank energy feature is the second Fbank feature.

[0019] In some embodiments of the present invention, after the step of filtering the power spectrum through a Mel-scale-based triangular filter bank and calculating the output energy of each filter to obtain the Mel filter bank energy characteristics of the machine audio sample, the method further includes:

[0020] The energy characteristics of the Mel filter bank are normalized by using a preset mean and a preset standard deviation.

[0021] In some embodiments of the present invention, prior to the step of converting the machine audio sample into Mel filter bank energy features, the method further includes:

[0022] If the audio length of the machine audio sample is greater than the preset audio length, then the machine audio sample is segmented according to the preset audio length.

[0023] In some embodiments of the present invention, the step of extracting deep features of all the energy features of the Mel filter bank by a transformer encoder to obtain multiple deep general features and multiple deep features to be measured includes:

[0024] The energy features of the Mel filter bank are divided into multiple sequence blocks and input into the transformer encoder to obtain output features containing information of the sequence blocks;

[0025] The dimensions of the output features are reshaped to obtain multidimensional features associated with the time-frequency structure of the corresponding Mel filter bank energy features;

[0026] Pooling is performed on the multidimensional features along the time dimension to obtain dimensionality-reduced features;

[0027] The dimensionality reduction features are fused with the feature dimensions of each sequence block in the remaining spectrum to obtain one-dimensional features; wherein, the one-dimensional features include deep general features and deep test features. The deep general features are obtained when the segmented Mel filter bank energy features are first Fbank features, and the deep test features are obtained when the segmented Mel filter bank energy features are first Fbank features.

[0028] In some embodiments of the present invention, the step of constructing a training feature set based on all the deep general features and determining anomaly thresholds from the training feature set using the K-nearest neighbor algorithm includes:

[0029] Obtain the Euclidean distance between each deep general feature in the training feature set and all other deep general features in the set to obtain a first distance matrix;

[0030] Obtain the minimum distance of the K nearest neighbors of each deep general feature in the training feature set, and calculate the arithmetic mean of all the minimum distances as the average nearest neighbor distance of the deep general feature, where K is a preset positive integer;

[0031] The anomaly threshold is obtained based on the average nearest neighbor distance.

[0032] In some embodiments of the present invention, after the step of obtaining the anomaly threshold based on the average nearest neighbor distance, the method further includes:

[0033] Obtain the mean and standard deviation of the average nearest neighbor distance for all the aforementioned deep general features;

[0034] Based on the mean and standard deviation, the average distance of each of the K nearest neighbors is normalized to obtain a standardized outlier score;

[0035] All standardized outlier scores are linearly mapped to the closed interval [0,1] to obtain the mapped quantiles. The outlier threshold is determined by selecting the mapped quantiles according to a preset standard.

[0036] In some embodiments of the present invention, the step of obtaining the distance metric between each of the depth features to be tested and each of the deep general features in the training feature set, and generating anomaly scores for the depth features to be tested based on the distance metric, includes:

[0037] Obtain the Euclidean distance between the deep feature to be tested and each of the deep general features in the training feature set, and select the minimum value among all Euclidean distances as the initial anomaly score;

[0038] Based on the mean and standard deviation of all the deep general features in the training feature set, the initial anomaly score is normalized to generate the anomaly score of the deep feature to be tested.

[0039] In a second aspect, embodiments of the present invention provide a computer device including a memory and a processor, the memory being used to store at least one program, and the processor being used to load the at least one program to execute the anomaly detection method based on machine operating audio described in the above-mentioned embodiments.

[0040] Thirdly, a computer-readable storage medium stores computer-executable instructions for causing a computer to perform the anomaly detection method based on machine operating audio as described in the embodiments above.

[0041] This invention provides an anomaly detection method based on machine operating audio. It utilizes the Mel filter bank energy characteristics of audio samples from the machine producing qualified products and audio samples from the machine under test. Based on these Mel filter bank energy characteristics, it derives deep general features and deep test features. An anomaly threshold is determined based on the deep general features, and an anomaly score is obtained based on the deep test features. Finally, by comparing the anomaly score with the anomaly threshold, it determines whether the machine under test has a fault. This invention provides the following advantages: it can determine a reasonable detection threshold using only audio samples from the machine producing qualified products, eliminating the need for abnormal samples in the judgment process, thus achieving accurate and efficient detection of abnormal sounds under simple sound insulation conditions. Attached Figure Description

[0042] Figure 1 This is a flowchart of an anomaly detection method based on machine operating audio according to an embodiment of the present invention;

[0043] Figure 2 This is a logic diagram of an anomaly detection method based on machine operating audio according to an embodiment of the present invention;

[0044] Figure 3 This is a structural diagram of a computer device provided in another embodiment of the present invention. Detailed Implementation

[0045] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0046] In the description of this invention, it should be understood that the orientation descriptions, such as up, down, front, back, left, right, etc., are based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0047] In the description of this invention, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0048] In the description of this invention, unless otherwise explicitly defined, terms such as "set up," "install," and "connect" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this invention in conjunction with the specific content of the technical solution.

[0049] The method provided by the embodiments of the present invention has the following beneficial effects: The embodiments of the present invention can determine a reasonable detection threshold only through audio samples when the machine produces qualified products, without the need for abnormal samples in the judgment process, thereby achieving accurate and efficient detection of abnormal sounds under simple sound insulation conditions.

[0050] The concepts mentioned in the embodiments of the present invention are explained below:

[0051] F-bank features: Mel-Filterbank Energy Features, designed to transform the raw audio signal into a lower-dimensional numerical matrix that better reflects human hearing, facilitating subsequent processing by machine learning models (such as neural networks).

[0052] K-Nearest Neighbors (KNN) algorithm: Given a training dataset, for a new input instance, find the K nearest neighbors (the K nearest neighbors) in the training dataset. If the majority of these K instances belong to a certain class, then classify the input instance into that class.

[0053] Transformer is a deep learning model based on self-attention mechanisms, which completely eliminates the limitations of recurrent and convolutional structures, achieving efficient parallel computation. Its core component, the Transformer encoder, consists of a multi-layered stacked structure. Each layer captures global dependencies in the sequence through multi-head self-attention, then performs nonlinear transformations via a feedforward network, supplemented by residual connections and layer normalization to ensure training stability.

[0054] The control method of the present invention will be further described below with reference to the accompanying drawings.

[0055] Reference Figure 1 , Figure 2 , Figure 1 A flowchart illustrating an anomaly detection method based on machine operating audio provided in an embodiment of the present invention. According to a first aspect of the present invention, an anomaly detection method based on machine operating audio is provided, comprising:

[0056] Step S100: Obtain machine audio samples. The machine audio samples include training samples and test samples. The training samples are audio samples when the machine is working normally, and the test samples are audio samples when the machine to be tested is working.

[0057] Step S200: Extract the Mel filter bank energy features of the machine audio samples, wherein the Mel filter bank energy features include the first Fbank feature and the second Fbank feature. The Mel filter bank energy features extracted from the training samples are the first Fbank features, and the Mel filter bank energy features extracted from the test samples are the second Fbank features.

[0058] Step S300: Extract deep features of all Mel filter bank energy features by the transformer encoder to obtain multiple deep general features and multiple deep features to be tested. The deep general features are extracted from the first Fbank features and the deep features to be tested are extracted from the second Fbank features.

[0059] Step S400: Construct a training feature set based on all deep general features, and use the K-nearest neighbor algorithm to self-train the feature set to determine the anomaly threshold;

[0060] Step S500: Obtain the distance metric between the deep feature to be tested and each deep general feature in the training feature set, and generate the anomaly score of the deep feature to be tested based on the distance metric.

[0061] Step S600: Compare the abnormal score with the abnormal threshold, and determine whether the machine under test has a fault based on the comparison result.

[0062] It should be noted that in step S100, before determining whether the machine is operating normally, it is necessary to first acquire the audio recorded when the same model of the machine under test is operating normally, as well as the audio recorded when the machine under test is operating. This audio can be acquired through a microphone or any other device with audio acquisition and recording capabilities. In this embodiment of the invention, it is not necessary to acquire the audio of the machine under test operating abnormally. In step S200, before fault determination, the acquired machine audio samples need to be preprocessed. The main preprocessing method is to extract Fbank features from both types of audio samples: audio samples recorded during normal operation and audio samples from the machine under test. In step S300, although Fbank features can describe the spectral energy distribution of audio within each short time window in the acoustic field, Fbank features are essentially a low-to-medium level acoustic feature. While they retain rich audio details, they fail to abstract higher-level, deeper features strongly correlated with machine faults (such as 'metallic friction sound', 'regular impact sound', etc.). Therefore, this limits the model's ability to distinguish unknown anomaly types. Furthermore, industrial environments are noisy, and the sound of the same normal machine fluctuates under different loads and temperatures. F-bank features record all subtle differences, resulting in an overly dispersed feature distribution of normal samples, making it difficult to learn a compact normal pattern boundary. Therefore, this invention requires further extraction of deep features from the F-bank features, namely deep general features and deep test features. In step S400, this embodiment of the invention can automatically calculate and process the deep general features obtained in step S300 using the K-nearest neighbor algorithm to determine the anomaly threshold. The anomaly threshold is used to determine whether the machine's operating state is normal, i.e., to construct a normal baseline based on the audio under normal machine operating conditions. In step S500, the method provided by this embodiment of the invention can automatically calculate and obtain anomaly scores based on the distance metric between the deep general features and the deep test features. Since both the deep test features and the deep general features are processed by a transformer encoder, their vector dimensions are the same. The degree of deviation between the normal centers of the deep test features and the deep general features can be measured by calculating the distance metric. It is understood that in this embodiment of the invention, the weight of the distance metric includes Euclidean distance, Manhattan distance, Minkowski distance, Mahalanobis distance, and other distance metrics that can measure the correlation between the deep general features and the deep test features. In step S600, the Euclidean distance between the aforementioned deep feature to be tested and all deep general features in the training feature set is transformed into an anomaly score. This score is used to compare with an anomaly threshold to determine whether the operating status of the machine to be tested is normal. It is understood that, in the embodiments of the present invention, when the anomaly score is greater than the anomaly threshold, it can be considered that the operating status of the machine to be tested is abnormal and requires maintenance.

[0063] It should be noted that the abnormal thresholds in steps S100 to S700 can be calculated and obtained solely from audio samples recorded during normal machine operation, without the need to acquire a large number of abnormal audio samples during abnormal machine operation, thus broadening the application scope. Furthermore, the method provided in this embodiment requires no training; it automatically determines the threshold by calculating the quantiles of the distance between normal samples, eliminating the need for manual labeling of abnormal samples and complex hyperparameter tuning. This adapts to the current situation in industrial scenarios where abnormal samples are scarce and manual parameter tuning is costly, while also avoiding the threshold failure problem caused by data distribution offset in neural networks, resulting in stronger robustness. Further, the aforementioned thresholds are automatically calculated based on the statistical characteristics of the distances across all samples, fully utilizing the distance distribution patterns of normal samples. The thresholds more closely match the inherent characteristics of the data, accurately distinguishing between normal and abnormal data, and avoiding misjudgments caused by relying too heavily on the experience of debugging personnel when manually setting thresholds. This application leverages the advantage of calculating thresholds based on deep general features. Further, this application further processes the fbank features through a Transformer encoder, dimensionality reshaping, temporal pooling, and feature flattening operations to obtain deep general features. The Transformer encoder can capture long-distance dependencies of features. Subsequent operations such as dimensionality reshaping and pooling integrate multi-dimensional information such as time and frequency domains. Compared with directly using fbank features, it can more accurately and comprehensively characterize the essential features of audio, making the threshold calculated based on these features more accurate in distinguishing between normal and abnormal audio.

[0064] Additionally, in one embodiment, reference is made to Figure 1 ,exist Figure 1 Step S200 of the illustrated embodiment also includes, but is not limited to, the following steps:

[0065] Step S210: Divide the machine audio samples into frames according to the preset frame length, and overlap adjacent frames according to the preset overlap length.

[0066] Step S220: Apply a Hanning window to each frame of signal after framing and then perform a short-time Fourier transform to obtain the spectrum;

[0067] Step S230: Calculate the power spectrum of the spectrum;

[0068] Step S240: Filter the power spectrum through a triangular filter bank based on the Mel scale, and calculate the output energy of each filter to obtain the Mel filter bank energy characteristics of the machine audio sample.

[0069] Specifically, when the sample is a training sample, the obtained Mel filter bank energy feature is the first Fbank feature; when the sample is a test sample, the obtained Mel filter bank energy feature is the first Fbank feature.

[0070] It should be noted that steps S210 to S240 are preprocessing steps for the training samples and the samples to be tested. Their main purpose is to convert the one-dimensional waveform signal into a two-dimensional time-frequency representation (spectral graph), and further refine it into F-bank features that conform to human auditory perception. The specific steps of F-bank feature conversion are as follows:

[0071] S21 Framing and Windowing: A frame length is set, and the audio signal is divided into multiple frames with a certain overlap rate. Each frame signal is multiplied by a Hannine window function to reduce spectral leakage at frame edges. In this embodiment of the invention, the frame length is typically set to 20-30ms, and the signal overlap rate is 50%.

[0072] ·····(1),

[0073] In equation (1), L is the frame length and n is the sampling point index. The weight of the Hanning window at index n is used to convert continuous audio signals into overlapping short segments (frames) to analyze the stability of the signal over short periods. Windowing is applied to reduce abrupt changes at the beginning and end of each frame, preventing "spectral leakage" during spectral analysis.

[0074] S22 Short-Time Fourier Transform: Perform a Fourier transform on each windowed frame of the signal to obtain the spectrum. :

[0075] ·····(2),

[0076] In equation (2), These are the sampled values ​​of the original signal. For window function values, Let k be the frequency index of the transformed spectrum. The purpose of this step is to convert the time-domain signal of each frame into a frequency-domain signal, obtain the spectrum of that frame, and thus see the energy distribution of the signal at different frequencies.

[0077] S23 Power Spectrum Calculation: Taking the square of the modulus of the complex spectrum obtained from the Fourier transform yields the power spectrum.

[0078] ·····(3),

[0079] In equation (3), Power spectrum, The spectrum obtained after transformation in step 2) is denoted by k, where k is the frequency index. The purpose of this step is to calculate the power (energy) of the signal at different frequency components, obtain a spectrum that is more consistent with human hearing, and prepare for subsequent filter bank processing.

[0080] S24 Mel filter bank filtering: The power spectrum in step S23 is filtered through a set (M) of Mel-scale-based triangular filter banks. Filtering is performed, and the output energy of each filter is calculated. This set of energy values ​​represents the F-bank features of the training or test samples. :

[0081] ·····(4),

[0082] In equation (4), Let be the response (weight) of the m-th triangular filter at frequency k. The power at frequency k, The output energy of the filter is represented by the M-dimensional Fbank feature vector. The purpose of this step is to eliminate the differences caused by factors such as volume and recording conditions among different audio samples, making the feature data distribution more stable.

[0083] In addition, in one embodiment, after step S240 of the above-described aspect embodiment, the following steps are included, but are not limited to:

[0084] Step S250: Normalize the energy characteristics of the Mel filter bank using preset mean and preset standard deviation.

[0085] It should be noted that step S250 involves the normalization of the Fbank feature necklaces obtained in steps S210 to S240, specifically as follows:

[0086] S25 uses preset average and standard deviation standardization: ,in For the number of time frames, For the Mel spectrum dimension.

[0087] ·····(5),

[0088] The purpose of step (5) is to eliminate the differences caused by factors such as volume and recording conditions among different audio samples, so as to make the distribution of feature data more stable.

[0089] Additionally, in one embodiment, in Figure 1 Before step S200 in the illustrated embodiment, the following steps may also be included, but are not limited to:

[0090] Step S201: If the audio length of the machine audio sample is greater than the preset audio length, then the machine audio sample is segmented according to the preset audio length.

[0091] It should be noted that, in some embodiments of the present invention, if the audio duration in the obtained training samples or test samples exceeds 20 seconds, it is sliced ​​into audio segments of 10 to 20 seconds.

[0092] Additionally, in one embodiment, referring to the figure, in Figure 1 Step S300 of the illustrated embodiment also includes, but is not limited to, the following steps:

[0093] Step S310: Divide the energy features of the Mel filter bank into multiple sequence blocks and input them into the transformer encoder to obtain output features containing information of the sequence blocks;

[0094] Step S320: Reshape the dimensions of the output features to obtain multidimensional features associated with the time-frequency structure of the corresponding Mel filter bank energy features;

[0095] Step S330: Pool the multidimensional features along the time dimension to obtain dimensionality-reduced features;

[0096] Step S340: The dimensionality-reduced features are fused with the remaining spectrum and the feature dimension of each sequence block to obtain one-dimensional features;

[0097] Among them, the one-dimensional features include deep general features and deep test features. When the energy features of the segmented Mel filter bank are the first Fbank features, the deep general features are obtained, and when the energy features of the segmented Mel filter bank are the first Fbank features, the deep test features are obtained.

[0098] It should be noted that in the embodiments of the present invention, steps S310 to S340 are the specific processing procedures for the first Fbank feature and the second Fbank feature, that is, the process in equation (5) Input is fed into an open-source pre-trained model. The following is the detailed process for obtaining deep, general features:

[0099] S31) Deep feature extraction:

[0100] The preprocessed fbank features The system is divided into time-frequency blocks into sequence patches (i.e., sequence blocks); each patch consists of N time frames and N Mel bands, and the number of patches is:

[0101] ·····(6),

[0102] In equation (6), The length is the time dimension. Where N is the length of the frequency dimension, and N is the size of each patch.

[0103] Then, this patch sequence is input into an open-source pre-trained Transformer encoder, and the output features are obtained after processing by the Transformer encoder:

[0104] ·····(7),

[0105] in, The total number of sequence blocks, For each patch, the feature dimension is defined, and B is the batch size, which is the number of samples contained in each batch in deep learning. The purpose of the deep feature extraction step is to divide the continuous spectrogram into small patches so that they can be used as sequence inputs to the Transformer model, capturing long-range dependencies between patches.

[0106] S32) Dimensional Reshaping: [This section appears to be incomplete and requires further context.] The dimension is reconstructed into a two-dimensional matrix of time and spectrum, which is also... The dimensions are reshaped as follows: ·····(8),

[0107] The time-frequency distribution of associated Fbank features; Number of segments for the time dimension Let be the number of spectral dimension segments, satisfying This corresponds to the temporal and spectral structure of the Fbank features. The purpose of the dimensionality reshaping step is to restore the sequence structure (L) output by the Transformer back to its original two-dimensional spatiotemporal structure (time T × spectrum S) so as to align with the structure of the original spectrogram and prepare for subsequent pooling operations.

[0108] S33) Time pooling: Pooling based on the time dimension Average pooling is used to compress redundant temporal information, retaining core content, removing subtle variations, and improving the robustness of features.

[0109] ·····(9),

[0110] The output dimension is obtained after pooling:

[0111] ·····(10),

[0112] In equations (9) and (10), Indicates the first The first time block, the first The first spectrum block, the first The values ​​of the dimensional features; The corresponding position features after pooling. For the time dimension after pooling, Number of spectrum blocks For feature dimensions.

[0113] S34) Feature Flattening and General Feature Construction: This involves flattening the pooled features... Along the spectral dimension With feature dimension Flattening and fusing spectral and feature information, we obtain the general features of the training set audio:

[0114] ·····(11),

[0115] Among them, the flattening dimension For the time dimension after pooling, Number of spectrum blocks (For feature dimensions).

[0116] Traverse all samples in the training set to construct a training feature set:

[0117] ·····(12),

[0118] in, The total number of training samples, Corresponding to the The deep general features of a normal sample. Feature flattening and general feature construction: The purpose of this step is to transform the pooled three-dimensional tensor [1, S, D] into a one-dimensional feature vector to obtain the final feature representation of the audio segments of the entire training sample.

[0119] It is understandable that, based on the second Fbank deep features of the audio samples of the machine to be detected, the extraction of deep test features needs to have the same dimension as the deep general features in order to perform subsequent comparisons of anomaly scores and anomaly thresholds. Therefore, it is necessary to convert the second Fbank features into deep test features through the same steps as steps S31) to S34), specifically including:

[0120] S31') Deep feature extraction: based on the same block parameters The sequence is divided into patches, which are then processed by a Transformer encoder. This represents the total number of the second sequence blocks in the sample to be tested. The feature dimensions for each patch;

[0121] S32') Dimensional Reshaping: Remodeling ,in, Let be the number of spectral dimension segments, satisfying ;

[0122] S33') Feature Flattening: General Feature Construction: Temporal Pooling and Feature Flattening, focusing on the time dimension. Perform average pooling, then flatten into general features, finally obtaining the general features of the test set:

[0123] ·····(13),

[0124] In equation (13) The vector dimension is consistent with that of the deep general features obtained from the training set.

[0125] It is understood that the above processing is the inference process of the open-source pre-trained model, not the training process. Therefore, the embodiments of the present invention do not require a large number of audio samples when the machine is working normally and a large number of audio samples when the machine is malfunctioning to train the model. The requirement for samples is small, and the application scope is wider than that of methods that require training models (such as methods based on neural networks, such as segmentation models and classification models, which use normal and abnormal sound characteristics as input and use labels to guide the training process).

[0126] Additionally, in one embodiment, referring to the figure, in Figure 1 Step S400 of the illustrated embodiment also includes, but is not limited to, the following steps:

[0127] Step S410: Obtain the Euclidean distance between each deep general feature in the training feature set and all other deep general features in the set, and obtain the first distance matrix;

[0128] Step S420: Obtain the minimum distance of the K nearest deep general features of each deep general feature in the training feature set, and calculate the arithmetic mean of all minimum distances as the average nearest neighbor distance of the deep general feature, where K is a preset positive integer;

[0129] Step S430: Obtain the anomaly threshold based on the average nearest neighbor distance.

[0130] It should be noted that the core idea behind the anomaly threshold obtained in this embodiment of the invention is that normal samples tend to cluster tightly in the feature space. Therefore, the distance from each normal sample to its neighbors should be relatively small. By statistically analyzing the distribution of these distances, an upper limit for a "normal distance" can be determined; distances exceeding this limit are considered abnormal. After obtaining the deep general features, the anomaly threshold can be automatically calculated using open-source and trained models. The specific calculation steps for the anomaly threshold include:

[0131] Based on the general features of the training set The process of automatically calculating the anomaly detection threshold is as follows:

[0132] S41) Calculate the K-nearest neighbor distance for normal samples: Calculate the distance for each sample in the training set. With all other samples The Euclidean distance is used to obtain the first distance matrix:

[0133] ·····(14),

[0134] in, Let be the Euclidean distance between sample i and sample j. ' represents the total dimension of the feature vector. Let be the eigenvalues ​​of the feature vectors of samples i and j in the d-th dimension. The purpose of this step is to calculate the Euclidean distance from each normal sample in the training set to all other normal samples, preparing for finding the K nearest neighbors of each sample. The purpose of obtaining the first distance matrix is ​​to provide a basis for selecting the K nearest neighbor distances and determining the anomaly detection score threshold.

[0135] S42) Determine the K nearest neighbors for each sample and calculate the average distance:

[0136] For each sample Distance to other samples Sort in ascending order and select the top... The minimum distance (i.e., the nearest neighbor) (for each sample), calculate the average as the base distance for that sample:

[0137] ······(15)

[0138] in, Let K be the average K-nearest neighbor distance of sample i. This is the k-th distance value after sorting the distances from sample i to all other samples in ascending order. The purpose of this step is to filter out the influence of a few outliers or distant samples, using a more stable "local density" index to represent the normal region of each sample. Based on this average distance, an anomaly threshold can be determined.

[0139] In addition, in one embodiment, after step S430 shown in the above-described embodiment, the following steps are included, but are not limited to:

[0140] Step S440: Calculate the mean and standard deviation of the average K-nearest neighbor distances for all deep common features;

[0141] Step S450: Based on the mean and standard deviation, normalize the average distance of each K nearest neighbor to obtain a standardized outlier score;

[0142] Step S460: Linearly map all standardized outlier scores to the closed interval [0,1], sort the mapped scores in ascending order, and select the score value corresponding to the specified quantile as the outlier threshold.

[0143] It should be noted that, in order to obtain a more accurate anomaly threshold, the embodiments of the present invention also modify the value obtained from equation (15). The following processing was performed:

[0144] S43) ZERO Normalization:

[0145] The base distance for collecting all samples Calculate its mean and standard deviation :

[0146] , ·····(16),

[0147] in, The mean, Standard deviation, Let be the average K-nearest neighbor distance of sample i.

[0148] For each Perform ZERO normalization to obtain standardized outlier scores. :

[0149] ······(17)

[0150] The purpose of this step is to determine the base distance of all samples. Standardization is performed to a distribution with a mean of 0 and a standard deviation of 1, eliminating the influence of the absolute value of the original distance and making it comparable.

[0151] S44) Mapped to the interval [0,1]:

[0152] Will Mapping to the [0,1] interval unifies the score range:

[0153] ······(18)

[0154] in, , Each for all The minimum and maximum values. The purpose of this step is to linearly map the scores obtained after normalization to a fixed, uniform interval [0,1], so as to facilitate the subsequent setting of an easy-to-understand threshold.

[0155] S45) Determination of quantile threshold:

[0156] right Sort in ascending order and select the score corresponding to a specific quantile (e.g., 95%, 99%) as the threshold. :

[0157] ·····(19),

[0158] The purpose of this step is to define a scientific, data-driven decision boundary based on business needs and tolerance for false positives. For example, This means that the abnormal score of 99% of normal samples is ≤ This is used to define the boundary between normal and abnormal.

[0159] Additionally, in one embodiment, reference is made to Figure 1 ,exist Figure 1 Step S500 of the illustrated embodiment also includes, but is not limited to, the following steps:

[0160] Step S510: Obtain the Euclidean distance between the deep feature to be tested and each deep general feature in the training feature set;

[0161] Step S520: Select the minimum value from all obtained Euclidean distances as the initial anomaly score of the depth feature to be measured;

[0162] Step S530: Based on the mean and standard deviation of the deep general features in the training feature set, the initial anomaly score is normalized to obtain the anomaly score of the deep feature to be tested.

[0163] It should be noted that this is based on the features of the test set. and the anomaly score threshold determined in step S4 Perform the following testing procedures:

[0164] S61) Calculate the characteristics of the test samples With training feature set The Euclidean distance of all samples in the dataset is calculated using the following formula:

[0165] ·····(20),

[0166] Reuse the mean of the training set features and standard deviation Normalization is performed to obtain outlier scores. :

[0167] ······(twenty one),

[0168] S62) Determine whether a sample is abnormal based on the comparison between the abnormal score and the abnormal threshold. If the abnormal score exceeds the abnormal threshold, it is abnormal (product is unqualified); otherwise, it is normal (product is qualified).

[0169] like Figure 3 As shown, Figure 3 This is a structural diagram of a computer device provided in one embodiment of the present invention. The present invention also provides a computer device, comprising:

[0170] The processor 801 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0171] The memory 802 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 802 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 802 and is called and executed by the processor 801 to execute the abnormal detection method based on machine operating audio of the embodiments of this application.

[0172] The 803 input / output interface is used to implement information input and output.

[0173] The communication interface 804 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0174] Bus 805 transmits information between various components of the device (e.g., processor 801, memory 802, input / output interface 803, and communication interface 804);

[0175] The processor 801, memory 802, input / output interface 803, and communication interface 804 are connected to each other within the device via bus 805.

[0176] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separate, and may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0178] The above provides a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for anomaly detection based on machine working audio, characterized by, The method comprises the following steps: obtaining machine audio samples, wherein the machine audio samples comprise training samples and to-be-tested samples, the training samples are audio samples when the machine is working normally, and the to-be-tested samples are audio samples when the machine to be detected is working; extracting mel filter bank energy features of the machine audio samples, wherein the mel filter bank energy features comprise first Fbank features and second Fbank features, the mel filter bank energy features extracted from the training samples are the first Fbank features, and the mel filter bank energy features extracted from the to-be-tested samples are the second Fbank features; extracting deep features of all the mel filter bank energy features through a transformer encoder to obtain a plurality of deep general features and a plurality of deep to-be-tested features, wherein the deep general features are extracted from the first Fbank features, and the deep to-be-tested features are extracted from the second Fbank features, and the step specifically comprises: dividing the mel filter bank energy features into a plurality of sequence blocks and inputting the sequence blocks into the transformer encoder to obtain output features containing information of the sequence blocks; reshaping the dimensions of the output features to obtain multi-dimensional features associated with a time-frequency structure corresponding to the mel filter bank energy features; pooling the multi-dimensional features in the time dimension to obtain reduced-dimensional features; fusing the reduced-dimensional features in the remaining frequency spectrum and the feature dimensions of each sequence block to obtain one-dimensional features; wherein the one-dimensional features comprise deep general features and deep to-be-tested features, the deep general features are obtained when the divided mel filter bank energy features are the first Fbank features, and the deep to-be-tested features are obtained when the divided mel filter bank energy features are the first Fbank features; constructing a training feature set according to all the deep general features, and determining an abnormal threshold from the training feature set by using a K-nearest neighbor algorithm, and the step specifically comprises: obtaining the Euclidean distances between each deep general feature in the training feature set and all other deep general features in the set to obtain a first distance matrix; obtaining the minimum distances of each deep general feature to the K nearest deep general features in the training feature set, calculating the arithmetic mean of all the minimum distances as the nearest neighbor average distance of the deep general features, wherein K is a predetermined positive integer; obtaining the abnormal threshold based on the nearest neighbor average distance; obtaining distance measurement values of the deep to-be-tested features and each deep general feature in the training feature set one by one, and generating an abnormal score of the deep to-be-tested features based on the distance measurement values; comparing the abnormal score with the abnormal threshold, and determining whether the machine to be detected has a fault according to the comparison result.

2. The machine working sound based anomaly detection method of claim 1, wherein, The step of extracting the mel filter bank energy features of the machine audio samples comprises the following steps: frame the machine audio samples according to a preset frame length, and make adjacent frames overlap by a preset overlap length; performing short-time Fourier transform on each frame signal after applying a Hanning window to obtain a frequency spectrum; calculating a power spectrum of the frequency spectrum; filtering the power spectrum through a triangular filter bank based on a mel scale, and calculating an output energy of each filter to obtain a mel filter bank energy feature of the machine audio sample; wherein, when the machine audio sample is the training sample, the obtained mel filter bank energy feature is the first Fbank feature; and when the machine audio sample is the to-be-tested sample, the obtained mel filter bank energy feature is the second Fbank feature.

3. The machine working sound based anomaly detection method of claim 2, wherein, After the step of filtering the power spectrum through a triangular filter bank based on a mel scale, and calculating an output energy of each filter to obtain a mel filter bank energy feature of the machine audio sample, the method further comprises: normalizing the mel filter bank energy feature based on a preset mean value and a preset standard deviation.

4. The machine working sound based anomaly detection method of claim 1, wherein, Before the step of converting the machine audio sample into a mel filter bank energy feature, the method further comprises: if the audio length of the machine audio sample is greater than a preset audio length, splitting the machine audio sample according to the preset audio length.

5. The machine working sound based anomaly detection method of claim 1, wherein, After the step of obtaining the abnormal threshold value based on the nearest neighbor average distance, the method further comprises: obtaining a mean value and a standard deviation of the nearest neighbor average distance of all the deep general features; normalizing each nearest neighbor average distance based on the mean value and the standard deviation to obtain a standardized abnormal score; linearly mapping all the standardized abnormal scores into a closed interval of [0, 1] to obtain a mapped quantile, and selecting the mapped quantile according to a preset standard to determine the abnormal threshold value.

6. The machine working sound based anomaly detection method of claim 1, wherein, The step of obtaining distance measurement values between the deep to-be-tested feature and each deep general feature in the training feature set one by one, and generating an abnormal score of the deep to-be-tested feature based on the distance measurement values, comprises: obtaining an Euclidean distance between the deep to-be-tested feature and each deep general feature in the training feature set, and selecting a minimum value of all the Euclidean distances as an initial abnormal score; normalizing the initial abnormal score based on a mean value and a standard deviation of all the deep general features in the training feature set to generate the abnormal score of the deep to-be-tested feature.

7. A computer apparatus, comprising: The computer readable storage medium stores computer executable instructions for causing a computer to execute the machine working audio based anomaly detection method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions for causing a computer to execute the machine working audio based anomaly detection method according to any one of claims 1 to 6.

Images