A method and system for detecting abnormal sound based on neural network with multiple receptive fields

Abstract

The present invention provides a method and system for detecting abnormal sound based on a neural network with multiple receptive fields, and relates to the technical field of industrial intelligent diagnosis and quality control. The method comprises the steps of collecting audio signal data of industrial equipment in various working states, and marking the collected audio signal data; constructing and dividing a data set based on the marked data; preprocessing the audio signal in the data set, and extracting Mel spectrum features from the preprocessed audio signal; inputting the Mel spectrum features into a pre-constructed abnormal sound detection model for training, and obtaining a trained abnormal sound detection model; inputting a sample of the audio signal to be tested into the trained abnormal sound detection model for prediction, and obtaining a prediction result, and classifying the industrial equipment according to the prediction result. The present invention can dynamically capture multi-scale audio features, and significantly improve the accuracy and robustness of abnormal sound detection through rich feature representation capabilities.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial intelligent diagnosis and quality control, and in particular to a method and system for detecting abnormal sounds based on a neural network with multiple receptive fields. Background Art

[0002] With the growing demand for sound signal analysis in the industrial field, traditional noise detection methods, such as those based on statistical analysis, Fourier transform and spectral analysis, have been unable to meet the detection requirements in complex acoustic environments to a certain extent. These traditional methods usually rely on manually designed features and show insufficient detection accuracy when faced with non-stationary and complex noise backgrounds. In addition, these methods often require a lot of domain knowledge and experience, and it is difficult to achieve ideal detection results without being familiar with specific application scenarios.

[0003] In recent years, deep learning-based technologies have developed rapidly, especially in the fields of computer vision and natural language processing. Deep learning provides new data representation and automatically extracts features, including nonlinear structures. As a typical classification problem, foreign sound detection has begun to introduce deep learning methods for improvement. Although deep learning technology has shown great potential in foreign sound detection, its performance is often affected by the feature extraction module, especially when dealing with complex and diverse data. Different audio features may have different time scales and frequency characteristics. Traditional convolutional neural networks (CNNs) usually use a fixed-size receptive field, which may not be able to fully capture the multi-scale features involved in foreign sounds. At the same time, traditional neural networks use scalars to represent features, which often ignores the structural information of samples (such as position, direction, size, etc.).

[0004] Therefore, there is an urgent need for a noise detection method with high feature extraction accuracy and strong robustness. Summary of the invention

[0005] To this end, an embodiment of the present invention provides an abnormal sound detection method and system based on a neural network with multiple receptive fields, which are used to solve the problem that the traditional abnormal sound detection method in the prior art is insufficient in accuracy and the feature extraction of deep learning technology is limited when processing complex abnormal sound data.

[0006] In order to solve the above problem, an embodiment of the present invention provides a method for detecting abnormal sound based on a neural network with multiple receptive fields, the method comprising:

[0007] Step S1: collecting audio signal data of industrial equipment in various working states, including audio signals of industrial equipment under normal and abnormal operating conditions, and marking the collected audio signal data;

[0008] Step S2: construct a data set based on the labeled data, and divide the data set into a training set and a validation set;

[0009] Step S3: preprocessing the audio signal in the data set, and extracting Mel spectrum features from the preprocessed audio signal;

[0010] Step S4: inputting the Mel spectrum features into the previously constructed abnormal sound detection model for training to obtain a trained abnormal sound detection model, specifically including:

[0011] The Mel spectrum features are input into the pre-built foreign sound detection model. First, the primary features are extracted through the convolutional neural network. Then, the multi-scale feature extraction and fusion are performed using the selective convolution kernel network module. Then, the adaptive selection of features is achieved through feature dimensionality reduction and soft attention mechanism. Finally, the classification task is completed through the capsule network module.

[0012] During the training process, five-fold cross validation is used to evaluate the performance of the foreign sound detection model and optimize its parameters to finally obtain a trained foreign sound detection model.

[0013] Step S5: input the audio signal sample to be tested into the trained abnormal sound detection model for prediction, obtain the prediction result, and classify the industrial equipment according to the prediction result.

[0014] Preferably, the method for preprocessing the audio signal in the data set includes removing background noise and normalizing processing.

[0015] Preferably, the method for extracting Mel spectrum features from the preprocessed audio signal is:

[0016] Divide the continuous audio signal into multiple short-time frames and perform windowing on each frame signal;

[0017] Perform fast Fourier transform on each frame of windowed signal to convert the time domain signal into a frequency domain signal to obtain the spectrum representation of the frame;

[0018] Square the spectrum obtained by fast Fourier transform and calculate the power spectrum of each frequency component;

[0019] Process the power spectrum through a set of Mel filters;

[0020] Perform logarithmic operation on the energy output by the Mel filter bank to obtain the logarithmic energy spectrum.

[0021] Preferably, the method of inputting the Mel frequency spectrum feature into a pre-built foreign sound detection model for training to obtain the trained foreign sound detection model is:

[0022] Divide the continuous audio signal into multiple short-time frames and perform windowing on each frame signal;

[0023] Perform fast Fourier transform on each frame of windowed signal to convert the time domain signal into a frequency domain signal to obtain the spectrum representation of the frame;

[0024] Square the spectrum obtained by fast Fourier transform and calculate the power spectrum of each frequency component;

[0025] Process the power spectrum through a set of Mel filters;

[0026] Perform logarithmic operation on the energy output by the Mel filter bank to obtain the logarithmic energy spectrum.

[0027] Preferably, the method of extracting primary features by convolutional neural network is:

[0028] Use a layer of convolutional neural network to preliminarily process the Mel spectrum features to obtain primary features , , where H, C, and W represent the height, number of channels, and width of the feature, respectively. Represents the set of real numbers.

[0029] Preferably, the method for extracting and fusing multi-scale features using a selective convolution kernel network module is:

[0030] Use convolution kernels of different sizes and ReLU activation functions to transform primary features Perform multi-scale feature extraction to obtain different feature maps and , , ;

[0031] Mapping the features and Fusion to obtain comprehensive features ;

[0032] The integrated features after fusion Perform global average pooling operation to obtain global feature representation .

[0033] Preferably, the method for implementing adaptive feature selection through feature dimensionality reduction and soft attention mechanism is:

[0034] The global feature representation is achieved through the fully connected layer Perform dimensionality reduction to obtain the compressed feature vector ,in is the length of the compressed vector;

[0035] Using the compressed feature vector Guide the cross-channel soft attention mechanism to adaptively adjust the degree of attention to information at different spatial scales and obtain the soft attention vector and ;

[0036] According to the soft attention vector and ,right and Perform weighted fusion to obtain tensor features .

[0037] Preferably, the method for completing the classification task through the capsule network module is:

[0038] The tensor features Converted into vector features and encapsulated into primary capsule units ;

[0039] The primary capsule units are aggregated using a dynamic routing mechanism to generate a classification probability vector , where the modulus of the vector represents the probability of a certain category;

[0040] The weight parameters in the capsule unit are updated through back propagation until the weight parameters are fixed, completing the training of the entire network model.

[0041] Preferably, the dynamic routing mechanism specifically includes:

[0042] Each primary capsule unit is a vector representing different attributes of local features in the input data;

[0043] For each primary capsule unit And each target capsule unit , calculate the prediction vector ;

[0044] The routing coefficient Initialized to equal values, representing primary capsule units and the target capsule unit The strength of the connection between

[0045] According to the routing coefficient and the prediction vector , calculate the target capsule unit Input ;

[0046] To the target capsule unit Input Apply the Squash activation function to get the target capsule unit Output ;

[0047] According to the target capsule unit Output and the prediction vector The inner product between them updates the routing coefficient .

[0048] Preferably, when training the network, the loss of the foreign voice detection model It includes reconstruction loss and marginal loss, where the reconstruction loss is calculated by three fully connected layers according to The reconstructed Mel spectrum features and the original Mel spectrum features are calculated, and the specific losses are as follows:

[0049] ;

[0050] in, is the marginal loss; To rebuild the losses, is the loss coefficient.

[0051] The embodiment of the present invention further provides a system for detecting abnormal sounds based on a neural network with multiple receptive fields. The system is used to implement the above-mentioned method for detecting abnormal sounds based on a neural network with multiple receptive fields, and specifically includes:

[0052] The audio signal acquisition module is used to collect audio signal data of industrial equipment in various working states, covering the audio signals of industrial equipment under normal and abnormal operating conditions, and annotating the collected audio signal data;

[0053] The data set partitioning module is used to construct a data set based on the labeled data and divide the data set into a training set and a validation set;

[0054] The audio preprocessing and feature extraction module is used to preprocess the audio signals in the data set and extract Mel spectrum features from the preprocessed audio signals;

[0055] The model training and optimization module is used to input the Mel spectrum features into the pre-built foreign sound detection model for training to obtain a trained foreign sound detection model. Specifically, it includes:

[0056] The Mel spectrum features are input into the pre-built foreign sound detection model. First, the primary features are extracted through the convolutional neural network. Then, the multi-scale feature extraction and fusion are performed using the selective convolution kernel network module. Then, the adaptive selection of features is achieved through feature dimensionality reduction and soft attention mechanism. Finally, the classification task is completed through the capsule network module.

[0057] During the training process, five-fold cross validation is used to evaluate the performance of the foreign sound detection model and optimize its parameters to finally obtain a trained foreign sound detection model.

[0058] The model prediction and application module is used to input the audio signal sample to be tested into the trained abnormal sound detection model for prediction, obtain the prediction result, and classify the industrial equipment according to the prediction result.

[0059] It can be seen from the above technical solutions that the present invention has the following beneficial effects:

[0060] (1) Multi-scale feature capture capability: The present invention realizes dynamic capture of different scale features in audio signals by combining a selective convolution kernel network module. By processing the input signal with multiple branches with different kernel sizes and fusing the information of these branches together, this method can flexibly adjust the size of the neuron's receptive field and effectively capture a variety of audio features from subtle changes to complex frequency features. This capability is particularly suitable for processing complex and diverse audio signals of industrial equipment, improving the accuracy and applicability of abnormal sound detection.

[0061] (2) Rich feature representation capability: By introducing the capsule network module, the present invention realizes a multi-dimensional vector representation of audio signal features. Each capsule can not only capture the relationship between different features, but also contains structural information (such as position, direction, size, etc.). This representation method enables the network to better retain and utilize the spatial information and directionality in the audio signal. This rich feature representation capability significantly enhances the accuracy and robustness of the model in the process of abnormal sound detection, enabling the model to better distinguish between normal and abnormal audio signals.

[0062] (3) Efficient abnormal sound detection performance: Combining the advantages of selective convolution kernel network and capsule network, the present invention constructs an efficient abnormal sound detection model. The model achieves accurate detection of audio signals of industrial equipment through multi-scale feature extraction and fusion, feature adaptive selection and completion of classification tasks. During the training process, the model performance is evaluated using five-fold cross validation and its parameters are optimized to ensure the stability and reliability of the model. Finally, the model can achieve fast and accurate classification of industrial equipment in practical applications, providing strong technical support for status monitoring and fault diagnosis of industrial equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0064] Figure 1A flowchart of a method for detecting abnormal sounds based on a neural network with multiple receptive fields provided in an embodiment;

[0065] Figure 2 It is a structural schematic diagram of the abnormal sound detection model in the embodiment;

[0066] Figure 3 A block diagram of an abnormal sound detection system based on a neural network with multiple receptive fields provided in an embodiment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1

[0068] In order to solve the problem that the traditional noise detection method in the prior art is not accurate enough and the deep learning technology is limited in feature extraction when processing complex noise data. Figure 1 As shown, an embodiment of the present invention proposes a method for detecting abnormal sounds based on a neural network with multiple receptive fields, the method comprising:

[0069] Step S1: collecting audio signal data of industrial equipment in various working states, including audio signals of industrial equipment under normal and abnormal operating conditions, and marking the collected audio signal data;

[0070] Step S2: construct a data set based on the labeled data, and divide the data set into a training set and a validation set;

[0071] Step S3: preprocessing the audio signal in the data set, and extracting Mel spectrum features from the preprocessed audio signal;

[0072] Step S4: inputting the Mel spectrum features into the previously constructed abnormal sound detection model for training to obtain a trained abnormal sound detection model, specifically including:

[0073] The Mel spectrum features are input into the pre-built foreign sound detection model. First, the primary features are extracted through the convolutional neural network. Then, the multi-scale feature extraction and fusion are performed using the selective convolution kernel network module. Then, the adaptive selection of features is achieved through feature dimensionality reduction and soft attention mechanism. Finally, the classification task is completed through the capsule network module.

[0074] During the training process, five-fold cross validation is used to evaluate the performance of the foreign sound detection model and optimize its parameters to finally obtain a trained foreign sound detection model.

[0075] Step S5: input the audio signal sample to be tested into the trained abnormal sound detection model for prediction, obtain the prediction result, and classify the industrial equipment according to the prediction result.

[0076] It can be seen from the above technical scheme that the present invention provides a method for detecting abnormal sound based on a neural network with multiple receptive fields, by collecting and labeling audio signal data of industrial equipment, constructing and dividing a data set, preprocessing the audio signal to extract Mel spectrum features, inputting the abnormal sound detection model (combining convolutional neural network, selective convolution kernel network module, feature dimension reduction and soft attention mechanism, capsule network module) for training, using a five-fold cross-validation optimization model, and finally predicting and classifying the audio signal to be tested. Its corresponding advantage is that, through the selective convolution kernel network module, the receptive field size is dynamically adjusted to effectively capture multi-scale audio features, which is particularly suitable for subtle changes and complex frequency features in abnormal sounds; at the same time, the capsule network module uses multi-dimensional vectors to represent features, which can not only capture the relationship between different features, but also retain spatial and structural information, enriching the feature representation capability, and enhancing the accuracy and robustness of detection. This method of integrating multi-scale feature capture and enriching feature representation significantly improves the classification accuracy of the abnormal sound detection model, and provides an efficient and accurate technical means for abnormal sound detection of industrial equipment.

[0077] In step S1, professional audio acquisition equipment is used to collect audio signal data of industrial equipment in various working states, covering audio signals of industrial equipment under normal (OK) and abnormal (NG) operating conditions, and professionals accurately label the collected audio signal data to ensure the accuracy of audio signal sample classification.

[0078] In step S2, a data set is constructed based on the labeled data, and the data set is divided into a training set and a validation set.

[0079] In step S3, the audio signal in the data set is preprocessed, and the Mel spectrum feature is extracted from the preprocessed audio signal.

[0080] Mel Spectrogram is an audio signal processing method whose main purpose is to imitate the human ear's perception of sounds of different frequencies. Human hearing is more sensitive to low-frequency sounds and less sensitive to high-frequency sounds. In order to reflect this perception characteristic, the Mel scale uses a nonlinear frequency scale that has a higher resolution in the low-frequency range and a relatively low resolution in the high-frequency range. This frequency conversion is closer to the auditory characteristics of the human ear. The conversion relationship between Mel frequency and traditional Hertz (Hz) frequency can be expressed by the following formula:

[0081] ;

[0082] In the formula, represents the Mel frequency, is the frequency in Hertz (Hz). This formula demonstrates the nonlinear conversion characteristics of frequency, which enables the Mel spectrum to more accurately capture the audio characteristics perceived by the human auditory system.

[0083] Specifically, the audio features of the preprocessed audio signal are extracted using the Mel spectrum, including the following steps:

[0084] (1) Split the continuous audio signal into multiple short-time frames, with each frame usually 20-40 milliseconds long; perform windowing on each frame. Common window functions include Hanning window or Hamming window to reduce the boundary effect generated during the frequency conversion process.

[0085] (2) Perform a fast Fourier transform (FFT) on the windowed signal of each frame. FFT converts the time domain signal into a frequency domain signal, thereby obtaining the spectrum representation of the frame.

[0086] (3) Square the spectrum obtained by FFT and calculate the power spectrum of each frequency component. The purpose of this step is to quantify the energy distribution of the signal at different frequencies.

[0087] (4) The power spectrum is processed through a set of Mel filters. The Mel filter bank is usually composed of a series of triangular filters whose center frequencies and bandwidths are distributed according to the Mel scale. The Mel scale is a frequency scale that simulates the auditory perception of the human ear, with the low frequency part being finer and the high frequency part being coarser.

[0088] (5) Perform a logarithmic operation on the energy output by the Mel filter bank to obtain a logarithmic energy spectrum. The logarithmic operation simulates the human ear's perception characteristics of sounds of different intensities.

[0089] In step S4, the Mel spectrum features are input into a pre-built foreign sound detection model for training to obtain a trained foreign sound detection model.

[0090] like Figure 2 As shown in the figure, the Mel spectrum features are input into the pre-built foreign sound detection model. First, the primary features are extracted through the convolutional neural network, and then the multi-scale feature extraction and fusion are performed using the selective convolution kernel network module. Then, the adaptive selection of features is achieved through feature dimensionality reduction and soft attention mechanism, and finally the classification task is completed through the capsule network module.

[0091] Specifically, the method of extracting primary features through convolutional neural network is:

[0092] Use a layer of convolutional neural network to preliminarily process the Mel spectrum features to obtain primary features , , where H, C, and W represent the height, number of channels, and width of the feature, respectively. Represents the set of real numbers.

[0093] Specifically, 1x1 convolution (called point-by-point convolution) and ReLU activation function are used to convert the two-dimensional feature map into a three-dimensional feature map to increase the expressiveness of the feature. The 1x1 convolution operation expands the feature vector of each pixel from two dimensions to three dimensions. This conversion increases the expressiveness of the feature and can expand the number of channels of the input feature from a smaller dimension to a higher dimension, thereby better representing complex feature combinations.

[0094] Furthermore, the method of using the selective convolution kernel network module to extract and fuse multi-scale features is as follows:

[0095] Using convolution kernels of different sizes (3×3 and 5×5) and ReLU activation functions, the primary features are Perform multi-scale feature extraction to obtain different feature maps and , , .

[0096] By aggregating multi-scale information, different receptive fields in multiple branches enable the network to capture various information in the input data more comprehensively and finely during feature extraction. Certain details or local features may be ignored or faded under a larger receptive field, but can be extracted more clearly under a smaller receptive field. A larger receptive field helps capture global context or features of a larger range. By combining the features extracted from these different receptive fields, the network can generate richer and more recognizable feature representations, thereby more accurately and reliably analyzing and classifying complex data. The present invention fuses the features extracted from branches of different receptive fields to generate a comprehensive feature representation, that is, feature mapping. and Fusion to obtain comprehensive features + .

[0097] This fusion can integrate multi-scale information from different receptive fields to form a more expressive feature vector, allowing the network to more comprehensively understand the diversity of input data. Perform a global average pooling operation ( ), and obtain the global feature representation .in It can be obtained by the following formula:

[0098] .

[0099] Furthermore, the method of realizing adaptive feature selection through feature dimensionality reduction and soft attention mechanism is as follows:

[0100] The global feature representation is achieved through the fully connected layer Perform dimensionality reduction to obtain the compressed feature vector , to improve the computational efficiency:

[0101] ;

[0102] in, represents the fully connected layer, is the ReLU activation function, represents batch normalization, , Express The weight matrix for linear transformation, is the length of the compressed vector, where the compressed length is determined by the reduction ratio control, .

[0103] Using the compressed feature vector Guide the cross-channel soft attention mechanism to adaptively adjust the degree of attention to information at different spatial scales and obtain the soft attention vector and ;

[0104] ;

[0105] ;

[0106] in, They represent the linear transformation matrices used to generate the attention scores. Their function is to convert the compressed feature vector Convert to weight space to calculate the attention coefficient, ; and They are and Soft attention vector of ; , Respectively represent the feature vector Through the matrix and The result of the linear transformation is used to generate attention scores associated with different widths.

[0107] According to the soft attention vector and ,right and Perform weighted fusion to obtain tensor features :

[0108] ;

[0109] in, ; , Respectively and Feature representation taken along a specific width dimension.

[0110] Furthermore, the present invention uses a capsule network module to extract tensor features The transformation from primary capsule to advanced capsule is performed to complete the final classification. The method to complete the classification task through the capsule network module is as follows:

[0111] First, the tensor features Converted into vector features and encapsulated into primary capsule units Specifically, for each position (i.e., a pixel in the feature map), the values ​​of all channels at that position are combined together to form a vector.

[0112] Then, the primary capsule units are aggregated using a dynamic routing mechanism to generate a classification probability vector , where the modulus of the vector represents the probability of a certain category.

[0113] Finally, the weight parameters in the capsule unit are updated through back propagation until the weight parameters are fixed, completing the training of the entire network model.

[0114] The dynamic routing mechanism specifically includes the following steps:

[0115] (1) Primary capsule layer: Each primary capsule unit is a vector that represents different attributes of local features in the input data (e.g., location, orientation, etc.).

[0116] (2) Calculate the prediction vector: For each primary capsule unit And each target capsule unit , calculate the prediction vector :

[0117] ;

[0118] In the formula, Represents the primary capsule unit To the prediction vector The weight matrix of .

[0119] (3) Initialize the routing coefficient: Initialized to equal values, representing primary capsule units and the target capsule unit The strength of the connection between them.

[0120] (4) Calculate the input of the target capsule: according to the routing coefficient and the prediction vector , calculate the target capsule unit Input :

[0121] .

[0122] (5) Apply nonlinear activation function: to the target capsule unit Input Apply the Squash activation function to get the target capsule unit Output :

[0123] .

[0124] (6) Update dynamic routing coefficient: according to the target capsule unit Output and the prediction vector The inner product between them updates the routing coefficient :

[0125] ;

[0126] The softmax function is used to normalize these coefficients so that the sum of the routing coefficients of each primary capsule is 1:

[0127] .

[0128] (7) Iteration process: The above steps (from calculating the prediction vector to updating the routing coefficient) are iterated 3 times. After each iteration, the routing coefficients gradually converge to those primary capsules that are most relevant to the target capsule, thereby effectively capturing complex patterns.

[0129] The loss of the foreign voice detection model when training the network It includes reconstruction loss and marginal loss, where the reconstruction loss is calculated by three fully connected layers according to The reconstructed Mel spectrum features and the original Mel spectrum features are calculated, and the specific losses are as follows:

[0130] ;

[0131] in, The marginal loss is designed to ensure that the model can effectively distinguish between correct and incorrect classifications; To rebuild the losses, is the loss coefficient.

[0132] For each class of capsule, the loss function calculates the loss for the positive and negative classes separately and then sums them up:

[0133] ;

[0134] in, is a binary indicator variable, when the category It is 1 if it is the target category, otherwise it is 0. Yes Category The modulus of the capsule output. and are two hyperparameters, usually set to 0.9 and 0.1. is the weight used to balance the loss between positive and negative classes and is usually set to a small value (0.5).

[0135] In order to further improve the performance of the foreign sound detection model, the network also introduces reconstruction loss. The input data is reconstructed by decoding the capsule output of the target class. Reconstruction loss The mean square error (MSE) is usually used to measure the difference between the reconstructed image and the original input image:

[0136] ;

[0137] in, is the Mel-spectrogram feature of the original input audio, is the reconstructed Mel spectrum feature.

[0138] The loss function of the proposed network combines marginal loss and reconstruction loss. Margin loss ensures the classification ability of the model by controlling the modulus of the capsule output, while reconstruction loss enhances the model's understanding of features through the reconstruction process of the input. This design enables the network to better capture and represent complex patterns and improve the generalization ability of the model.

[0139] In addition, during the training process, five-fold cross validation is used to evaluate the performance of the foreign sound detection model and optimize its parameters to finally obtain a trained foreign sound detection model.

[0140] In step S5, the audio signal sample to be tested is input into the trained abnormal sound detection model for prediction to obtain the prediction result, and the industrial equipment is classified according to the prediction result.

[0141] In our study, we conducted ablation experiments to verify the effectiveness of the proposed method and the contribution of each module. Ablation experiments are a key analytical method that aims to evaluate the specific impact of each component or module in the model on the final performance. By systematically removing or replacing specific parts of the model, we can accurately quantify the contribution of each module to the overall model.

[0142] After training and parameter tuning for the model of the method of the present invention, the selective convolution kernel network module maintains the same configuration as when used alone. Similarly, the capsule network module also maintains its settings when used independently.

[0143] For the selective convolution kernel network module (using the fully connected layer as a classifier), we only use it for feature extraction and use it with the fully connected layer for classification. This setting is intended to demonstrate the ability of the selective convolution kernel network in capturing multi-scale features and its effectiveness as a base model.

[0144] For the selective convolution kernel network module, we directly apply it as an independent model to process raw data. The network structure focuses on capturing the pose information of features and demonstrates its performance in processing data with complex structures and diversity.

[0145] In our proposed method, the selective convolution kernel network module and the selective convolution kernel network module are used in combination, the former is responsible for multi-scale feature extraction, and the latter focuses on capturing the pose information of the feature. This combination aims to integrate the advantages of the two models to improve the accuracy and robustness of foreign voice detection.

[0146] The experimental results are shown in Table 1. The selective convolution kernel network module (with a fully connected layer as a classifier) ​​can effectively extract multi-scale features from the input data, and achieves an accuracy of 90.05% when used independently, highlighting its excellent ability in capturing multi-scale features. When the capsule network module is used alone for classification, the accuracy is 91.74%, showing the strong strength of the capsule network in maintaining feature posture information and processing complex feature structures. When the selective convolution kernel network module is combined with the capsule network module, the classification accuracy of the model is increased to 93.61%, which is significantly improved compared to using either network alone. This shows that the combination of the two can complement each other and make full use of their respective advantages to enhance the overall performance of the model.

[0147] Table 1 Ablation experiment

[0148]

[0149] Application: By applying the method of the present invention, factories can identify early signs of equipment failure in advance, and then implement preventive maintenance strategies, effectively avoid major losses, and significantly reduce maintenance costs. The new system has the ability to operate automatically around the clock, significantly reducing dependence on manual labor. At the same time, it reduces unplanned equipment downtime and failures through timely warnings, thereby improving production efficiency. In addition, the continuous operation of the system has accumulated a wealth of audio data resources, which can be used to further optimize the model and enhance the accuracy and robustness of detection. This not only enables factories to efficiently perform equipment maintenance without affecting the production process, but also promotes the development of equipment management and production processes to a higher level of intelligence. Embodiment 2

[0150] like Figure 3 As shown, the present invention provides an abnormal sound detection system based on a neural network with multiple receptive fields, and the system is used to implement the abnormal sound detection method based on a neural network with multiple receptive fields of the above-mentioned embodiment 1, and specifically includes:

[0151] The audio signal acquisition module 100 is used to collect audio signal data of industrial equipment in various working states, including audio signals of industrial equipment under normal and abnormal operating conditions, and to mark the collected audio signal data;

[0152] A data set partitioning module 200 is used to construct a data set based on the labeled data and to partition the data set into a training set and a validation set;

[0153] The audio preprocessing and feature extraction module 300 is used to preprocess the audio signal in the data set and extract the Mel spectrum feature from the preprocessed audio signal;

[0154] The model training and optimization module 400 is used to input the Mel spectrum features into the pre-built abnormal sound detection model for training to obtain a trained abnormal sound detection model, which specifically includes:

[0155] The Mel spectrum features are input into the pre-built foreign sound detection model. First, the primary features are extracted through the convolutional neural network. Then, the multi-scale feature extraction and fusion are performed using the selective convolution kernel network module. Then, the adaptive selection of features is achieved through feature dimensionality reduction and soft attention mechanism. Finally, the classification task is completed through the capsule network module.

[0156] During the training process, five-fold cross validation is used to evaluate the performance of the foreign sound detection model and optimize its parameters to finally obtain a trained foreign sound detection model.

[0157] The model prediction and application module 500 is used to input the audio signal sample to be tested into the trained abnormal sound detection model for prediction, obtain the prediction result, and classify the industrial equipment according to the prediction result.

[0158] The present embodiment provides an abnormal sound detection system based on a neural network with multiple receptive fields, which is used to implement the aforementioned abnormal sound detection method based on a neural network with multiple receptive fields. Therefore, the specific implementation of the abnormal sound detection system based on a neural network with multiple receptive fields can be seen in the embodiment of the abnormal sound detection method based on a neural network with multiple receptive fields. For example, the audio signal acquisition module 100, the data set division module 200, the audio preprocessing and feature extraction module 300, the model training and optimization module 400, and the model prediction and application module 500 are respectively used to implement steps S1, S2, S3, S4, and S5 in the abnormal sound detection method based on a neural network with multiple receptive fields. Therefore, its specific implementation can refer to the description of the corresponding embodiments of each part. In order to avoid redundancy, it will not be repeated here.

[0159] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0161] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0162] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A method for detecting abnormal sounds based on a neural network with multiple receptive fields, characterized in that: include: Step S1: collecting audio signal data of industrial equipment in various working states, including audio signals of industrial equipment under normal and abnormal operating conditions, and marking the collected audio signal data; Step S2: construct a data set based on the labeled data, and divide the data set into a training set and a validation set; Step S3: preprocessing the audio signal in the data set, and extracting Mel spectrum features from the preprocessed audio signal; Step S4: inputting the Mel spectrum features into the previously constructed abnormal sound detection model for training to obtain a trained abnormal sound detection model, specifically including: The Mel spectrum features are input into the pre-built foreign sound detection model. First, the primary features are extracted through the convolutional neural network. Then, the multi-scale feature extraction and fusion are performed using the selective convolution kernel network module. Then, the adaptive selection of features is achieved through feature dimensionality reduction and soft attention mechanism. Finally, the classification task is completed through the capsule network module. The method of using the selective convolution kernel network module to extract and fuse multi-scale features is as follows: Using convolution kernels of different sizes and ReLU activation functions, multi-scale feature extraction is performed on the primary feature Y1 to obtain different feature maps. and Among them, H, C, and W represent the height, number of channels, and width of the feature, respectively. represents the set of real numbers; Mapping the features and Fusion to obtain comprehensive features Perform global average pooling operation on the fused comprehensive feature U Get global feature representation Where s is obtained by the following formula: In the formula, s w represents the value of the global feature representation s in the width dimension, U w It represents the feature representation of comprehensive features in the width dimension, U w (i,j) is the value at height i and channel j; The method for realizing adaptive feature selection through feature dimension reduction and soft attention mechanism is as follows: The global feature representation s is reduced in dimension through the fully connected layer to obtain the compressed feature vector The eigenvector z is expressed as: in, represents the fully connected layer, δ is the ReLU activation function, represents batch normalization, W represents the weight matrix for linear transformation of s, d is the length of the compressed vector, where the compressed length is controlled by the reduction ratio r, d = W / r; The compressed feature vector z is used to guide the cross-channel soft attention mechanism to adaptively adjust the degree of attention to information of different spatial scales, and obtain the soft attention vector a w and b w : Among them, A and B represent the linear transformation matrices used to generate attention scores. Their function is to convert the compressed feature vector z into the weight space for calculating the attention coefficient. a w and b w The feature maps are and The soft attention vector of A w z, B w z represents the result of linearly transforming the feature vector z through matrices A and B, respectively, which is used to generate attention scores associated with different widths; According to the soft attention vector a w and b w , for feature mapping and Perform weighted fusion to obtain the tensor feature V: Where V=[V1,V2,...,V W ], Represents feature maps and Feature representation taken at a specific width dimension; During the training process, five-fold cross validation is used to evaluate the performance of the foreign sound detection model and optimize its parameters to finally obtain a trained foreign sound detection model. Step S5: input the audio signal sample to be tested into the trained abnormal sound detection model for prediction, obtain the prediction result, and classify the industrial equipment according to the prediction result.

2. The method for detecting abnormal sound based on a neural network with multiple receptive fields according to claim 1, characterized in that: The method for preprocessing the audio signal in the data set includes removing background noise and normalizing processing.

3. The method for detecting abnormal sound based on a neural network with multiple receptive fields according to claim 1, characterized in that: The method for extracting Mel spectrum features from the preprocessed audio signal is: Divide the continuous audio signal into multiple short-time frames and perform windowing on each frame signal; Perform fast Fourier transform on each frame of windowed signal to convert the time domain signal into a frequency domain signal to obtain the spectrum representation of the frame; Square the spectrum obtained by fast Fourier transform and calculate the power spectrum of each frequency component; Process the power spectrum through a set of Mel filters; Perform logarithmic operation on the energy output by the Mel filter bank to obtain the logarithmic energy spectrum.

4. The method for detecting abnormal sound based on a neural network with multiple receptive fields according to claim 1, characterized in that: The method for extracting primary features through a convolutional neural network is: A layer of convolutional neural network is used to preliminarily process the Mel spectrum features to obtain the primary feature Y1. Among them, H, C, and W represent the height, number of channels, and width of the feature, respectively. Represents the set of real numbers.

5. The method for detecting abnormal sound based on a neural network with multiple receptive fields according to claim 1, characterized in that: The method for completing the classification task through the capsule network module is: Convert the tensor feature V into a vector feature and encapsulate it into a primary capsule unit u i ; The primary capsule units are aggregated using a dynamic routing mechanism to generate a classification probability vector v j , where the modulus of the vector represents the probability of a certain category; The weight parameters in the capsule unit are updated through back propagation until the weight parameters are fixed, completing the training of the entire network model.

6. The method for detecting abnormal sound based on a neural network with multiple receptive fields according to claim 5, characterized in that: The dynamic routing mechanism specifically includes: Each primary capsule unit u i is a vector representing different attributes of local features in the input data; For each primary capsule unit u i And each target capsule unit u j , calculate the prediction vector The routing coefficient c ij Initialized to equal values, representing the primary capsule unit u i and the target capsule unit u j The strength of the connection between According to the routing coefficient c ij and the prediction vector Calculate the target capsule unit u j Inputs j ; For the target capsule unit u j Inputs j Apply the Squash activation function to get the target capsule unit u j The output v j ; According to the target capsule unit u j The output v j and the prediction vector The inner product between them updates the routing coefficient c ij .

7. The method for detecting abnormal sound based on a neural network with multiple receptive fields according to claim 6, characterized in that: When training the network, the loss of the foreign voice detection model includes reconstruction loss and marginal loss. The reconstruction loss is calculated by the three fully connected layers according to v j The reconstructed Mel spectrum features and the original Mel spectrum features are calculated, and the specific losses are as follows: Loss=L margin +a·L recon ; Among them, L margin is the marginal loss; L recon is the reconstruction loss, and a is the loss coefficient.

8. A noise detection system based on a neural network with multiple receptive fields, characterized in that: The system is used to implement the method for detecting abnormal sounds based on a neural network with multiple receptive fields as described in any one of claims 1 to 7, and specifically comprises: The audio signal acquisition module is used to collect audio signal data of industrial equipment in various working states, covering the audio signals of industrial equipment under normal and abnormal operating conditions, and annotating the collected audio signal data; The data set partitioning module is used to construct a data set based on the labeled data and divide the data set into a training set and a validation set; The audio preprocessing and feature extraction module is used to preprocess the audio signals in the data set and extract Mel spectrum features from the preprocessed audio signals; The model training and optimization module is used to input the Mel spectrum features into the pre-built foreign sound detection model for training to obtain a trained foreign sound detection model. Specifically, it includes: The Mel spectrum features are input into the pre-built foreign sound detection model. First, the primary features are extracted through the convolutional neural network. Then, the multi-scale feature extraction and fusion are performed using the selective convolution kernel network module. Then, the adaptive selection of features is achieved through feature dimensionality reduction and soft attention mechanism. Finally, the classification task is completed through the capsule network module. During the training process, five-fold cross validation is used to evaluate the performance of the foreign sound detection model and optimize its parameters to finally obtain a trained foreign sound detection model. The model prediction and application module is used to input the audio signal sample to be tested into the trained abnormal sound detection model for prediction, obtain the prediction result, and classify the industrial equipment according to the prediction result.

Images