Environmental sound classification target model construction method and system

Abstract

The invention discloses an environmental sound classification target model construction method and system. The method comprises the following steps: 1, constructing an environmental sound classification model based on Wigner-Ville distribution and improved ResNet50; 2, training a network model; according to the method, the Inception Conversion structure is creatively fused into the ResNet50 model, the efficient feature learning capability of the ResNet50 in deep learning is inherited, and the multi-dimensional feature expression of the model on the sound signal is greatly enhanced through the multi-scale parallel convolution design of the Inception module. The fusion of the structure enables the model to simultaneously capture the subtle change and the macroscopic trend of the sound signal, and both the rapid frequency change and the continuous tone can be accurately identified and analyzed. In addition, the improvement also significantly improves the adaptability of the model to a complex sound environment, and can maintain high recognition accuracy and stability even in the presence of background noise.

Description

Technical Field

[0001] The present invention relates to the technical field of sound signal processing, and in particular to a method and system for constructing an environmental sound classification target model. Background Art

[0002] With the development of deep learning, environmental sound classification has attracted more and more attention. Environmental sound classification specifically refers to the ability to accurately identify environmental sounds in a complex acoustic environment and accurately classify the identified environmental sounds. In real life, people also have demands for environmental sound classification tasks in many aspects. For example, in smart homes, by identifying different sound events (such as knocking on the door, breaking glass, etc.), automatic alarms for home safety can be realized. With the development of machine learning, environmental sound classification has now surpassed humans.

[0003] Existing environmental sound classification methods rely on traditional signal processing techniques, such as Mel-frequency cepstral coefficients (MFCC), which have limitations when processing non-stationary signals and are difficult to accurately capture the instantaneous characteristics of sound signals. With the development of deep learning technology, convolutional neural networks (CNNs) have shown great potential in sound signal processing, but existing methods still have room for improvement in feature extraction and model generalization capabilities. Summary of the invention

[0004] The technical problem to be solved by the present invention is how to accurately identify various types of sounds in a scene with a complex acoustic environment.

[0005] The present invention solves the above technical problems through the following technical means:

[0006] The method for constructing an environmental sound classification target model includes the following steps:

[0007] Step 1: Build an improved ResNet50 model containing the Inception Convolution structure:

[0008] Step 1.1: Use the EDO algorithm to determine the optimal parameter configuration based on the data:

[0009] Initialize the pre-trained network and set each convolutional layer to include all possible dilation modes, where the weights of the convolutional filters are defined as W i represents the weight of the i-th convolution filter, and the sampling position is defined as in and Represents the sampling position of the i-th channel;

[0010] by The objective function is to minimize the error between the expected output of the pre-trained weights W and the expected output of the sampled inflated weights The sampling position range is

[0011] Step 1.2: Incorporate the Inception Convolution architecture into the ResNet50 model;

[0012] Step 1.3: Improve the ResNet50 model:

[0013] The convolution kernel size of the convolution layer is changed from the initial 3×3 to (2d max +1)×(2d max +1); select the best expansion mode to minimize the error;

[0014] Step 1.4: Recombining filters by merging filters with the same dilation pattern;

[0015] Step 2: Use the time-frequency feature map to train the improved ResNet50 model to obtain the target model.

[0016] Furthermore, the step 2 specifically includes:

[0017] Step 2.1: Resize the image to 224x224 to meet the training requirements of IC-ResNet;

[0018] Step 2.2: Apply Z-score normalization to normalize the pixel values ​​to the specified range;

[0019] Step 2.3: Update the network weights using the back-propagation algorithm.

[0020] Furthermore, the acquisition process of the time-frequency feature map is as follows:

[0021] The audio data under various environments are acquired, the audio data are accurately labeled, and then the labeled audio data is transformed into a time-frequency signal using the Wigner-Ville distribution to generate a time-frequency feature map.

[0022] Furthermore, the specific process of using Wigner-Ville distribution to perform time-frequency transformation on the sound signal is as follows:

[0023] Define Wigner-Ville distribution;

[0024]

[0025] Performing Hilbert transform on the preprocessed audio signal to convert it into the complex domain;

[0026] Use WVD to quantify instantaneous frequency and amplitude changes on different time scales to capture the inherent non-stationary characteristics of sound signals;

[0027] The time-frequency resolution of the frequency components is adjusted by the weighted window function method, the sparse representation theory is used to enhance the time-frequency feature representation of the signal, and the time-frequency distribution method is used to improve the concentration of time-frequency energy.

[0028] Furthermore, the improved ResNet50 model optimizes the expansion mode of the Inception Convolution structure through the Efficient Dilation Optimization algorithm.

[0029] The present invention also provides an environmental sound classification target model construction system, comprising:

[0030] Improved ResNet50 model building module: Build an improved ResNet50 model containing the Inception Convolution structure:

[0031] Step 1.1: Use the EDO algorithm to determine the optimal parameter configuration based on the data:

[0032] Initialize the pre-trained network and set each convolutional layer to include all possible dilation modes, where the weights of the convolutional filters are defined as W i represents the weight of the i-th convolution filter, and the sampling position is defined as in and Represents the sampling position of the i-th channel;

[0033] by The objective function is to minimize the error between the expected output of the pre-trained weights W and the expected output of the sampled inflated weights The sampling position range is

[0034] Step 1.2: Incorporate the Inception Convolution architecture into the ResNet50 model;

[0035] Step 1.3: Improve the ResNet50 model:

[0036] The convolution kernel size of the convolution layer is changed from the initial 3×3 to (2d max +1)×(2d max +1); select the best expansion mode to minimize the error;

[0037] Step 1.4: Recombining filters by merging filters with the same dilation pattern;

[0038] Training module: Use the time-frequency feature map to train the improved ResNet50 model to obtain the target model.

[0039] Furthermore, the training module specifically includes:

[0040] Step 2.1: Resize the image to 224x224 to meet the training requirements of IC-ResNet;

[0041] Step 2.2: Apply Z-score normalization to normalize the pixel values ​​to the specified range;

[0042] Step 2.3: Update the network weights using the back-propagation algorithm.

[0043] Furthermore, the acquisition process of the time-frequency feature map is as follows:

[0044] The audio data under various environments are acquired, the audio data are accurately labeled, and then the labeled audio data is transformed into a time-frequency signal using the Wigner-Ville distribution to generate a time-frequency feature map.

[0045] Furthermore, the specific process of using Wigner-Ville distribution to perform time-frequency transformation on the sound signal is as follows:

[0046] Define Wigner-Ville distribution;

[0047]

[0048] Performing Hilbert transform on the preprocessed audio signal to convert it into the complex domain;

[0049] Use WVD to quantify instantaneous frequency and amplitude changes on different time scales to capture the inherent non-stationary characteristics of sound signals;

[0050] The time-frequency resolution of the frequency components is adjusted by the weighted window function method, the sparse representation theory is used to enhance the time-frequency feature representation of the signal, and the time-frequency distribution method is used to improve the concentration of time-frequency energy.

[0051] Furthermore, the improved ResNet50 model optimizes the expansion mode of the Inception Convolution structure through the Efficient Dilation Optimization algorithm.

[0052] The advantages of the present invention are:

[0053] 1. The present invention utilizes the WVD transform to effectively capture the instantaneous frequency and amplitude changes of sound signals, providing a new technical means for sound signal analysis. The WVD transform can analyze the signal in both time and frequency dimensions at the same time, revealing the instantaneous characteristics of the signal, which is crucial for understanding the complex dynamic behavior of sound signals. This transform is particularly suitable for processing nonlinear and non-stationary signals, such as speech, music, and biomedical signals, and can significantly improve the accuracy and efficiency of signal processing. The present invention utilizes this technology to more accurately identify and separate the various components in the sound signal, providing strong technical support for the classification, recognition, and feature extraction of sound signals.

[0054] 2. The present invention innovatively integrates the Inception Convolution structure into the ResNet50 model, which not only inherits the efficient feature learning ability of ResNet50 in deep learning, but also greatly enhances the model's multi-dimensional feature expression of sound signals through the multi-scale parallel convolution design of the Inception module. The fusion of this structure enables the model to capture subtle changes and macro trends in sound signals at the same time, and both rapid frequency changes and continuous tones can be accurately identified and analyzed. In addition, this improvement also significantly improves the model's adaptability to complex sound environments, and maintains a high recognition accuracy and stability even in the presence of background noise.

[0055] 3. The present invention successfully achieves high-accuracy classification of environmental sounds in complex environments through optimized algorithms and deep learning models. This technological breakthrough not only relies on in-depth analysis of sound signals and noise suppression technology, but also involves innovative feature extraction methods and powerful classifier designs. Through these comprehensive measures, the present invention can accurately identify and distinguish different sound sources in various environmental noises, greatly improving the robustness and accuracy of the sound recognition system in practical applications, and providing an efficient and reliable technical means for scenarios such as environmental monitoring, intelligent security, and human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 An example of a time-frequency diagram of Wigner-Ville distribution in sound signal processing in an embodiment of the present invention;

[0057] Figure 2 This is a schematic diagram of the network structure of the improved ResNet50 model in an embodiment of the present invention;

[0058] Figure 3 Flow chart of the environmental sound classification method of the present invention in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] In this embodiment, a method for constructing an environmental sound classification target model based on Wigner-Ville distribution and improved ResNet50 is provided. The method is to identify and classify environmental sounds through a deep learning method, and to assist in analyzing the dynamic changes of sound signals through Wigner-Ville distribution. By improving the network structure of ResNet50, there are significant improvements in classification accuracy, generalization ability and real-time performance, thereby providing a new solution for environmental sound classification. The environmental sound classification method based on Wigner-Ville distribution and improved ResNet50 specifically includes: data collection and preprocessing, Wigner-Ville distribution transformation, improved ResNet50 model construction, network model training, and sound classification;

[0061] like Figure 3 The figure shows the framework of the environmental sound classification method based on Wigner-Ville distribution and improved ResNet50. First, the surrounding sound is obtained, and after preprocessing, the sound feature extraction process is completed through WVD transformation. After that, the extracted features are input into the training model for modeling, and finally the result output is obtained through the classifier. The specific steps are as follows:

[0062] Step 1: Data preprocessing:

[0063] Step 1.1: Obtain a sound dataset with sound category labels. In this example, the dataset used includes audio data from more than 30 scenarios, such as urban traffic, home environment, and natural environment. The total number of audio samples exceeds 10,000, ensuring data diversity and coverage of practical applications. 80% of the data is a training set, 10% is a validation set, and 10% is a test set.

[0064] Step 1.2: Have a professional team of acoustic analysts accurately annotate the audio data, including information such as sound category, duration, intensity, etc., to support the needs of supervised learning.

[0065] Step 2: Use Wigner-Ville distribution to perform time-frequency transformation on the sound signal to generate a time-frequency feature map;

[0066] Step 2.1: Define Wigner-Ville distribution;

[0067]

[0068] Where s(t) is the input signal sequence, τ is the time delay, and s * (t) is the complex conjugate of s(t), and ω is the angular frequency.

[0069] Step 2.2: Perform Hilbert transform on the preprocessed audio signal to convert it into the complex domain, thereby enhancing the clarity of the signal envelope and phase information and eliminating negative frequency components to improve the performance of the signal in the time domain and frequency domain;

[0070] Step 2.3: Use WVD to quantify the instantaneous frequency and amplitude changes on different time scales to capture the inherent non-stationary characteristics of the sound signal;

[0071] Step 2.4: Use the weighted window function method to adjust the time-frequency resolution of the frequency component, use the sparse representation theory to enhance the time-frequency feature representation of the signal, and use the time-frequency distribution method to improve the concentration of time-frequency energy;

[0072] Step 3: Build an improved ResNet50 model containing the Inception Convolution structure:

[0073] Step 3.1: Use the Efficient Dilation Optimization (EDO) algorithm to determine the optimal parameter configuration based on the data.

[0074] Step 3.1.1: Initialize the pre-trained network and set each convolutional layer to include all possible dilation modes, where the weights of the convolutional filters are defined as W i represents the weight of the i-th convolution filter, and the sampling position is defined as in and Represents the sampling position of the i-th channel.

[0075] Step 3.1.2: The objective function is to minimize the error between the expected output of the pre-trained weights W and the expected output of the sampled inflated weights The sampling position range is

[0076] Step 3.2: Incorporate the Inception Convolution architecture into the ResNet50 model

[0077] Step 3.3: Improve the ResNet50 model

[0078] Step 3.3.1: Change the convolution kernel size of the convolution layer from the initial 3×3 to (2d max +1)×(2d max +1), in this embodiment, d max =2. Select d max = 2 is to obtain a larger receptive field, so as to better capture long-range dependencies, and for larger convolution kernels, it will not ignore regional details, so as to obtain better local information.

[0079] Step 3.3.2: Select the best expansion pattern to minimize the error.

[0080] Step 3.4: Recombine the filters by merging filters with the same dilation pattern.

[0081] Step 4: Use the time-frequency feature map to train the improved ResNet50 model.

[0082] Step 4.1: Resize the image to 224x224 to meet the training requirements of IC-ResNet

[0083] Step 4.2: Apply Z-score normalization to normalize the pixel values ​​to the specified range.

[0084] Step 4.3: Use the back-propagation algorithm to update the network weights. In this embodiment, components such as the cross-entropy loss function, optimizer, regularization technology, and learning rate scheduling strategy are used to enhance the generalization ability of the model and prevent overfitting.

[0085] Step 5: Classify the new sound signal to identify the sound type.

[0086] One feature of this embodiment is that the use of Wigner-Ville distribution transformation includes performing Hilbert transformation on the sound signal and converting it into the complex domain.

[0087] A feature of this embodiment is to improve the ResNet50 model by optimizing the expansion mode of the Inception Convolution structure through the Efficient Dilation Optimization algorithm.

[0088] Model evaluation and testing of this embodiment After training, the sound images in the test set are converted into a network-readable format through a preprocessing step, and the trained IC-ResNet50 model is used for classification prediction. The prediction results are recorded and compared with the actual labels. By calculating the accuracy, precision, recall rate, F1 score and average precision (AP) of the sound classification task, the performance and generalization ability of the model are comprehensively evaluated, and the sound images processed by WVD are combined with the IC-ResNet50 structure and the initial convolution layer optimized by the EDO algorithm to establish an efficient and robust sound classification model. This model makes full use of the rich information of sound signals across time and frequency domains, and can accurately identify the type of sound.

[0089] like Figure 3 As shown, Figure 3 The left side is the original model, and the right side is the improved module. The improvement is that the original two 5*5 convolution blocks are replaced with 3*3 Inception Convolution.

[0090] In order to verify the performance advantage of the improved IC-resNet50 algorithm based on ResNet-50 under low signal-to-noise ratio and make it more convincing, this embodiment conducts comparative experiments between VGG-11, VGG-19, ResNet-18, ResNet-50 and IC-resNet-50. The experimental data in this part uses WVD time-frequency images with a signal-to-noise ratio between -16 dB and 4 dB.

[0091] Table 1. Comparative experimental results

[0092]

[0093] As can be seen from Table 1, the recognition accuracy of the algorithm is better than other conventional methods to a certain extent. Even when the signal-to-noise ratio is -14 dB, the overall average recognition accuracy of 8 sound signal modulations can still reach 96.39%. The analysis shows that the model based on WVD time-frequency analysis and IC-ResNet50 method can automatically mine the time-frequency characteristics of the signal, has strong representation ability, and fully mines the time-frequency characteristics of the signal, enabling the algorithm to extract more distinguishing feature representations. The model has good learning ability and anti-noise ability, and the recognition performance is improved to a certain extent, indicating that the initial convolution structure helps to improve the recognition accuracy.

[0094] This embodiment proposes a new environmental sound classification method IC-ResNet50. This method first performs a WVD transform on the environmental sound data to capture complex time-frequency features. These features are then converted into frequency domain images as input for further classification. By seamlessly integrating the Inception Convolution (IC) structure into the ResNet50 framework, both feature representation and classification accuracy are improved. Experimental results verify that IC-ResNet50 outperforms other methods in classification performance. In the future, in order to meet application requirements, the proposed environmental sound classification method will be lightweight studied to increase classification speed without affecting accuracy.

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing an environmental sound classification target model, characterized in that: The following steps are involved: Step 1: Build an improved ResNet50 model containing the Inception Convolution structure: Step 1.1: Use the EDO algorithm to determine the optimal parameter configuration based on the data: Initialize the pre-trained network and set each convolutional layer to include all possible dilation modes, where the weights of the convolutional filters are defined as W i represents the weight of the i-th convolution filter, and the sampling position is defined as in and Represents the sampling position of the i-th channel; by The objective function is to minimize the error between the expected output of the pre-trained weights W and the expected output of the sampled inflated weights The sampling position range is Step 1.2: Incorporate the Inception Convolution architecture into the ResNet50 model; Step 1.3: Improve the ResNet50 model: The convolution kernel size of the convolution layer is changed from the initial 3×3 to (2d max +1)×(2d max +1); select the best expansion mode to minimize the error; Step 1.4: Recombining filters by merging filters with the same dilation pattern; Step 2: Use the time-frequency feature map to train the improved ResNet50 model to obtain the target model.

2. The method for constructing an environmental sound classification target model according to claim 1, characterized in that: The step 2 specifically includes: Step 2.1: Resize the image to 224x224 to meet the training requirements of IC-ResNet; Step 2.2: Apply Z-score normalization to normalize the pixel values ​​to the specified range; Step 2.3: Update the network weights using the back-propagation algorithm.

3. The method for constructing an environmental sound classification target model according to claim 1 or 2, characterized in that: The acquisition process of the time-frequency feature map is as follows: The audio data under various environments are acquired, the audio data are accurately labeled, and then the labeled audio data is transformed into a time-frequency signal using the Wigner-Ville distribution to generate a time-frequency feature map.

4. The method for constructing an environmental sound classification target model according to claim 3, characterized in that: The specific process of using Wigner-Ville distribution to transform the sound signal into time-frequency is as follows: Define Wigner-Ville distribution; Performing Hilbert transform on the preprocessed audio signal to convert it into the complex domain; Use WVD to quantify instantaneous frequency and amplitude changes on different time scales to capture the inherent non-stationary characteristics of sound signals; The time-frequency resolution of the frequency components is adjusted by the weighted window function method, the sparse representation theory is used to enhance the time-frequency feature representation of the signal, and the time-frequency distribution method is used to improve the concentration of time-frequency energy.

5. The method for constructing an environmental sound classification target model according to claim 1 or 2, characterized in that: The improved ResNet50 model optimizes the expansion mode of the Inception Convolution structure through the Efficient Dilation Optimization algorithm.

6. A system for constructing an environmental sound classification target model, characterized in that: include: Improved ResNet50 model building module: Build an improved ResNet50 model containing the Inception Convolution structure: Step 1.1: Use the EDO algorithm to determine the optimal parameter configuration based on the data: Initialize the pre-trained network and set each convolutional layer to include all possible dilation modes, where the weights of the convolutional filters are defined as W i represents the weight of the i-th convolution filter, and the sampling position is defined as in and Represents the sampling position of the i-th channel; by The objective function is to minimize the error between the expected output of the pre-trained weights W and the expected output of the sampled inflated weights The sampling position range is Step 1.2: Incorporate the Inception Convolution architecture into the ResNet50 model; Step 1.3: Improve the ResNet50 model: The convolution kernel size of the convolution layer is changed from the initial 3×3 to (2d max +1)×(2d max +1); select the best expansion mode to minimize the error; Step 1.4: Recombining filters by merging filters with the same dilation pattern; Training module: Use the time-frequency feature map to train the improved ResNet50 model to obtain the target model.

7. The environmental sound classification target model construction system according to claim 6, characterized in that: The training module specifically includes: Step 2.1: Resize the image to 224x224 to meet the training requirements of IC-ResNet; Step 2.2: Apply Z-score normalization to normalize the pixel values ​​to the specified range; Step 2.3: Update the network weights using the back-propagation algorithm.

8. The method for constructing an environmental sound classification target model according to claim 6 or 7, characterized in that: The acquisition process of the time-frequency feature map is as follows: The audio data under various environments are acquired, the audio data are accurately labeled, and then the labeled audio data is transformed into a time-frequency signal using the Wigner-Ville distribution to generate a time-frequency feature map.

9. The environmental sound classification target model construction system according to claim 8, characterized in that: The specific process of using Wigner-Ville distribution to transform the sound signal into time-frequency is as follows: Define Wigner-Ville distribution; Performing Hilbert transform on the preprocessed audio signal to convert it into the complex domain; Use WVD to quantify instantaneous frequency and amplitude changes on different time scales to capture the inherent non-stationary characteristics of sound signals; The time-frequency resolution of the frequency components is adjusted by the weighted window function method, the sparse representation theory is used to enhance the time-frequency feature representation of the signal, and the time-frequency distribution method is used to improve the concentration of time-frequency energy.

10. The method for constructing an environmental sound classification target model according to claim 6 or 7, characterized in that: The improved ResNet50 model optimizes the expansion mode of the Inception Convolution structure through the Efficient Dilation Optimization algorithm.

Images