Environmental noise source type identification method and device based on deep convolutional neural network

Abstract

The invention relates to the technical field of environment noise source type recognition, and particularly discloses an environment noise source type recognition method and device based on a deep convolutional neural network, and the method comprises the steps: collecting environment noise data through a microphone array, carrying out the signal processing of the environment noise data, and generating an environment noise Mel-spectrum image; and performing fine-grained spectral analysis on the Mel-spectrum image of the environmental noise data by using a deep convolutional neural network to capture deep semantic features of the environmental noise, thereby realizing intelligent recognition of the sound source type of the environmental noise data. Through the mode, the method can achieve the quick and accurate recognition of the types of the environmental noise sources, reduces the dependence on the design of artificial features, improves the recognition accuracy and efficiency, and provides powerful technical support for noise control and environmental protection.

Description

Technical Field

[0001] The present application relates to the technical field of environmental noise source type recognition, and more specifically, to an environmental noise source type recognition method and device based on a deep convolutional neural network. Background Art

[0002] With the acceleration of urbanization and the improvement of people's living standards, environmental noise pollution has become one of the important factors affecting the quality of life of residents. Accurately identifying different sound sources in the environment is not only of great significance for noise control and environmental protection, but also has broad application prospects in the fields of intelligent transportation, smart homes, smart construction sites, security monitoring, etc. For example, in intelligent transportation systems, the ability to identify road noise in real time can help traffic management departments optimize traffic flow; in smart construction site environments, sound source identification technology can be used to monitor abnormal sounds and improve construction safety.

[0003] Traditional sound source identification methods mainly rely on physical models or manually designed features based on feature engineering. However, the sound source identification method based on physical models relies on a deep understanding and modeling of the physical characteristics of the sound source, but the complex acoustic conditions in the actual environment greatly limit the application of this method. The sound source identification method based on feature engineering describes the sound signal by manually designing features, and then uses machine learning algorithms for classification. However, this process of manually designing features is not only time-consuming and laborious, but also often lacks sufficient expression ability when faced with complex and changeable environmental noise.

[0004] In recent years, with the development of deep learning technology, new possibilities have been provided for the intelligent identification of environmental noise. Therefore, a method and device for identifying the type of environmental noise source based on a deep convolutional neural network are expected. Summary of the invention

[0005] The present application provides a method and device for identifying the type of environmental noise sources based on a deep convolutional neural network, which can quickly and accurately identify the types of environmental noise sources, thereby providing strong technical support for noise control and environmental protection, while reducing dependence on artificial feature design and improving recognition accuracy and efficiency.

[0006] In a first aspect, a method for identifying the type of environmental noise source based on a deep convolutional neural network is provided, comprising: Collect environmental noise data in real time through microphone array; Extracting Mel spectrum features of the environmental noise data to obtain an environmental noise Mel spectrum image; Performing a local scale-based spectrum analysis on the environmental noise Mel spectrum image to obtain a sequence of local semantic feature encoding vectors of the environmental noise Mel spectrum; Performing dynamic feature optimization aggregation on the sequence of the environmental noise Mel spectrum local semantic feature encoding vectors to obtain an environmental noise Mel spectrum significant aggregation feature vector; Based on the significantly aggregated feature vector of the environmental noise Mel spectrum, a recognition result of the type of the environmental noise source is determined.

[0007] In a second aspect, a device for identifying the type of environmental noise source based on a deep convolutional neural network is provided, comprising: An environmental noise data collection module is used to collect environmental noise data in real time through a microphone array; A Mel spectrum feature extraction module, used to extract the Mel spectrum features of the environmental noise data to obtain an environmental noise Mel spectrum image; An environmental noise spectrum analysis module is used to perform a spectrum analysis based on a local scale on the environmental noise Mel spectrum image to obtain a sequence of local semantic feature encoding vectors of the environmental noise Mel spectrum; An environmental noise feature dynamic optimization aggregation module is used to perform feature dynamic optimization aggregation on the sequence of the environmental noise Mel spectrum local semantic feature encoding vectors to obtain an environmental noise Mel spectrum significant aggregation feature vector; The noise recognition result determination module is used to determine the recognition result of the ambient noise source type based on the significant aggregation feature vector of the ambient noise Mel spectrum.

[0008] The present application provides a method and device for identifying the type of environmental noise source based on a deep convolutional neural network. The method collects environmental noise data through a microphone array, performs signal processing on the data to generate a mel spectrum image of environmental noise, and then uses a deep convolutional neural network to perform fine-grained spectrum analysis on the mel spectrum image of environmental noise data to capture the deep semantic features of environmental noise, thereby realizing intelligent identification of the type of environmental noise source data. In this way, rapid and accurate identification of environmental noise source types can be achieved, thereby providing strong technical support for noise control and environmental protection, while reducing dependence on artificial feature design and improving recognition accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0010] Figure 1 This is a schematic flowchart of a method for identifying environmental noise source types based on a deep convolutional neural network according to an embodiment of the present application.

[0011] Figure 2This is a data flow diagram of an environmental noise source type identification method based on a deep convolutional neural network according to an embodiment of the present application.

[0012] Figure 3 This is a schematic flowchart of step S3 in the method for identifying the type of environmental noise source based on a deep convolutional neural network in an embodiment of the present application.

[0013] Figure 4 This is a schematic flowchart of step S4 in the method for identifying the type of environmental noise source based on a deep convolutional neural network in an embodiment of the present application.

[0014] Figure 5 This is a schematic flowchart of step S41 in the method for identifying the type of environmental noise source based on a deep convolutional neural network in an embodiment of the present application.

[0015] Figure 6 This is a schematic flowchart of step S42 in the method for identifying the type of environmental noise source based on a deep convolutional neural network in an embodiment of the present application.

[0016] Figure 7 This is a schematic block diagram of an environmental noise source type recognition device based on a deep convolutional neural network according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0018] In response to the above technical problems, the technical concept of this application is: collect environmental noise data through a microphone array, perform signal processing on it to generate a mel spectrum image of environmental noise, and then use a deep convolutional neural network to perform fine-grained spectrum analysis on the mel spectrum image of environmental noise data to capture the deep semantic features of environmental noise, thereby realizing intelligent recognition of the sound source type of environmental noise data. In this way, rapid and accurate recognition of the sound source type of environmental noise can be achieved, thereby providing strong technical support for noise control and environmental protection, while reducing dependence on artificial feature design and improving recognition accuracy and efficiency.

[0019] Based on this, in the technical solution of this application, if Figure 1 and Figure 2As shown, the method for identifying the type of environmental noise source based on a deep convolutional neural network includes: S1, collecting environmental noise data in real time through a microphone array; S2, extracting the Mel spectrum features of the environmental noise data to obtain an environmental noise Mel spectrum image; S3, performing a local scale-based spectrum analysis on the environmental noise Mel spectrum image to obtain a sequence of environmental noise Mel spectrum local semantic feature coding vectors; S4, performing feature dynamic optimization aggregation on the sequence of environmental noise Mel spectrum local semantic feature coding vectors to obtain an environmental noise Mel spectrum significant aggregation feature vector; S5, determining the identification result of the environmental noise source type based on the environmental noise Mel spectrum significant aggregation feature vector.

[0020] Exemplarily, in step S1, environmental noise data is collected in real time through a microphone array. It should be understood that a microphone array can capture sound signals from different directions. Compared with a single microphone, a microphone array can provide multi-channel sound information. By combining data from multiple microphones, the signal of the target sound source can be enhanced and background noise and other interference sources can be suppressed, thereby improving signal quality. In an embodiment of the present application, the microphone array integrates a beamforming technology based on MVDR (minimum variance distortionless response) to minimize the total energy of the output signal (i.e., reduce noise) to improve the acquisition quality of the sound signal and provide a clearer input for subsequent sound source type identification. For example, the microphone array can be a spherical microphone array, and the specific structure or layout can refer to the prior art and is not limited here.

[0021] Exemplarily, in step S2, the Mel spectrum features of the ambient noise data are extracted to obtain the ambient noise Mel spectrum image. It should be understood that considering that the ambient noise signal is a non-stationary, nonlinear time series signal, it is difficult to analyze it directly. Therefore, in the technical solution of the present application, the ambient noise data is further converted into a two-dimensional time-frequency representation, that is, the ambient noise Mel spectrum image. It should be understood that the Mel spectrum is a spectrum representation method based on the perception characteristics of the human ear, which can not only retain the frequency information of the sound signal, but also show the changing characteristics of the sound signal in the frequency and time dimensions in a more intuitive way, and also better simulate the perception characteristics of the human ear to the sound frequency through the Mel scale conversion, so that the subsequent feature extraction and pattern recognition are more effective. In addition, since the Mel spectrum diagram emphasizes the spectrum structure rather than the absolute amplitude value, the Mel spectrum diagram has higher robustness to ambient noise and other interference factors than the original waveform. Even if there is a certain background noise in the signal, as long as the spectrum structure remains stable, it will not significantly affect the final recognition effect.

[0022] Exemplarily, in step S3, the ambient noise Mel spectrum image is subjected to a spectrum analysis based on a local scale to obtain a sequence of local semantic feature encoding vectors of the ambient noise Mel spectrum. It should be understood that the ambient noise signal is complex and diverse, and a direct global analysis of it may ignore some key but weak local characteristics. Through spectrum analysis based on a local scale, subtle changes in different time periods or frequency intervals can be captured more carefully, thereby enriching the feature representation. This fine-grained feature capture is crucial to improving recognition accuracy, especially when dealing with non-stationary, transient or low signal-to-noise ratio sound signals. Analysis at a local scale helps the model learn more types of acoustic patterns, including those that are not easily perceived from a global perspective. This enables the model to perform well not only on the training data set, but also to have stronger adaptability and robustness when facing unknown scenes or new types of sound sources. The sequence of local semantic feature encoding vectors extracted in this way can be used as a high-level representation to help the model better understand and distinguish different types of sound sources. Decomposing the entire Mel spectrum image into smaller local areas (tiles) can simplify the problem space to a certain extent. Analyzing small chunks of data one by one allows for more efficient algorithms and techniques than processing large amounts of data all at once.

[0023] In one embodiment, Figure 3 As shown, the ambient noise Mel spectrum image is subjected to a local scale-based spectrum analysis to obtain a sequence of ambient noise Mel spectrum local semantic feature coding vectors, including: S31, using an image segmentor to perform image block processing on the ambient noise Mel spectrum image to obtain a sequence of ambient noise Mel spectrum image blocks; S32, each ambient noise Mel spectrum image block in the sequence of ambient noise Mel spectrum image blocks is respectively input into a sound Mel spectrum feature extractor based on a deep convolutional neural network model to obtain a sequence of ambient noise Mel spectrum local semantic feature coding vectors.

[0024] Exemplarily, in step S31, an image segmenter is used to perform image block processing on the ambient noise Mel spectrum image to obtain a sequence of ambient noise Mel spectrum image blocks. It should be understood that in order to more accurately identify specific acoustic events or patterns, such as short-term pitch changes, transient noise, etc., the present application further uses an image segmenter to perform image block processing on the ambient noise Mel spectrum image, and divides the long-term Mel spectrum image into multiple short-term Mel spectrum image blocks to form a sequence of ambient noise Mel spectrum image blocks, so as to focus more on capturing local features in each time-frequency region, thereby improving the ability to capture local detail features in the sound signal and effectively reducing the complexity of the calculation.

[0025] In one embodiment, an image segmenter is used to perform image segmentation processing on the ambient noise Mel spectrum image to obtain a sequence of ambient noise Mel spectrum image blocks, including: selecting a suitable segmentation method and parameters, including a segmentation size (width x height) and an overlap ratio. The selection of the segmentation size depends on the needs of the specific application and the type of sound features expected to be captured; and setting an appropriate overlap can avoid information loss at the boundary while ensuring the continuity between adjacent areas. Then, a sliding window is used to traverse the entire Mel spectrum image, starting from a corner of the image, and the window is moved according to a set step size until the entire image is covered. Each window position corresponds to a "tile", that is, a small data fragment in the time-frequency domain. For each sliding window position, the corresponding image sub-area is extracted as a tile. These tiles constitute the final ambient noise Mel spectrum image tile sequence. In order to ensure that all tiles have a consistent size, the edge portion can be padded or cropped when necessary.

[0026] Exemplarily, in step S32, each environmental noise Mel spectrum image block in the sequence of environmental noise Mel spectrum image blocks is respectively input into a sound Mel spectrum feature extractor based on a deep convolutional neural network model to obtain a sequence of environmental noise Mel spectrum local semantic feature encoding vectors. It should be understood that considering that the spectral structure in the environmental noise Mel spectrum image is the key to identifying the type of sound source, the present application adopts a deep convolutional neural network model with significant advantages in local structure pattern recognition to perform feature extraction on each environmental noise Mel spectrum image block in the sequence of environmental noise Mel spectrum image blocks, so as to automatically extract and learn effective features in the environmental noise Mel spectrum image block through a sliding convolution operation, and abstract the spectral components in each block layer by layer, such as frequency peaks, harmonic structures, resonance peaks, spectral contours, etc., to generate a sequence of environmental noise Mel spectrum local semantic feature encoding vectors, thereby achieving an in-depth understanding of the complex spectral structure in the environmental noise signal.

[0027] Specifically, within the DCNN, each layer automatically learns and extracts feature representations at different levels in the tile through a series of convolution operations. The convolution layer uses multiple filters to scan the input tile, identify low-level features such as edges and textures, and gradually abstract higher-level structural patterns as the network depth increases, such as frequency peaks, harmonic structures, resonance peaks, and spectral contours. These high-level features are crucial for understanding the type of sound source because they can capture the complex spectral characteristics of the sound signal. After multiple layers of convolution and pooling operations, the network generates a compact and expressive feature vector, which is the local semantic feature encoding vector of the ambient noise Mel spectrum. It not only condenses the spatial information of the original tile, but also integrates the dynamic changes in the time-frequency domain, providing a profound description of the local sound characteristics. In order to ensure the consistency and comparability of the output features, fully connected layers or global average pooling layers are usually added to the last few layers to further refine the features and map them to a fixed-dimensional space. When all tiles have been processed through the above process, a set of local semantic feature encoding vector sequences of the ambient noise Mel spectrum can be obtained. This set of sequences preserves the temporal order of the original audio signal, and each encoded vector represents the unique acoustic information contained in the corresponding patch.

[0028] Exemplarily, in step S4, the sequence of the local semantic feature coding vectors of the ambient noise Mel spectrum is subjected to dynamic feature optimization aggregation to obtain a significantly aggregated feature vector of the ambient noise Mel spectrum. It should be understood that, although the local feature representations of each spectrum block are extracted separately by the deep convolutional neural network model, these local feature representations are dispersed and lack an overall understanding of the sound signal. Therefore, the present application further aggregates the sequence of the local semantic feature coding vectors of the ambient noise Mel spectrum to obtain an overall feature description of the original sound signal. In particular, in order to pay more attention to the essential attributes of the sound signal and reduce the impact of specific noise or interference, the present application proposes a feature dynamic optimization aggregation method, which learns the distribution of the local semantic features of each Mel spectrum block in the high-dimensional feature space and learns the feature association pattern between each Mel spectrum block, so as to perform feature integration, thereby realizing the global spectrum feature extraction of the ambient noise signal.

[0029] In one embodiment, Figure 4As shown, the sequence of the environmental noise Mel spectrum local semantic feature coding vectors is dynamically optimized and aggregated to obtain the environmental noise Mel spectrum significant aggregation feature vector, including: S41, based on the feature static potential energy distribution of the sequence of the environmental noise Mel spectrum local semantic feature coding vectors, calculating the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector; S42, based on the aggregation movement direction of each environmental noise Mel spectrum local semantic feature coding vector in the sequence of the environmental noise Mel spectrum local semantic feature coding vectors relative to the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector, dynamically aggregate the sequence of the environmental noise Mel spectrum local semantic feature coding vectors to obtain the environmental noise Mel spectrum significant aggregation feature vector.

[0030] In one embodiment, Figure 5 As shown, based on the feature static potential energy distribution of the sequence of the environmental noise Mel spectrum local semantic feature coding vectors, the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector is calculated, including: S411, each environmental noise Mel spectrum local semantic feature coding vector in the sequence of the environmental noise Mel spectrum local semantic feature coding vector is input into the feature static potential energy measurement network to obtain a sequence of environmental noise Mel spectrum local semantic feature static potential energy measurement coefficients; S412, the sequence of environmental noise Mel spectrum local semantic feature static potential energy measurement coefficients is input into the feature energy level screening gating unit to obtain a sequence of environmental noise Mel spectrum local semantic feature static potential energy weight factors; S413, based on the sequence of environmental noise Mel spectrum local semantic feature static potential energy weight factors, the weighted sum of the sequence of environmental noise Mel spectrum local semantic feature coding vectors is calculated to obtain the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector.

[0031] Exemplarily, in step S411, each environmental noise Mel spectrum local semantic feature encoding vector in the sequence of environmental noise Mel spectrum local semantic feature encoding vectors is respectively input into a feature static potential energy measurement network to obtain a sequence of environmental noise Mel spectrum local semantic feature static potential energy measurement coefficients.

[0032]

[0033]

[0034] in, represents a sequence of local semantic feature encoding vectors of the environmental noise Mel spectrum, , , and Respectively represent the first, second, and third in the sequence of the environmental noise Mel spectrum local semantic feature encoding vector and A local semantic feature encoding vector of the ambient noise Mel spectrum, is the number of feature vectors in the sequence of the local semantic feature encoding vectors of the environmental noise Mel spectrum, Indicates the The local semantic feature encoding vector of the environmental noise Mel spectrum The eigenvalues ​​at the positions, and Respectively represent the The feature mean and feature variance of the local semantic feature encoding vector of the ambient noise Mel spectrum, is the characteristic scale value of the local semantic feature encoding vector of the environmental noise Mel spectrum, For the Static potential energy metric coefficient of the local semantic feature encoding vector of the environmental noise Mel spectrum.

[0035] That is, the potential energy level of each local semantic feature encoding vector of the ambient noise Mel spectrum under static conditions is evaluated through the feature static potential energy measurement network, so as to reveal the importance and influence of each Mel spectrum block in the global Mel spectrum feature distribution of the current ambient noise signal, as well as the intrinsic connection between each Mel spectrum block, thereby obtaining a sequence of static potential energy measurement coefficients of the local semantic features of the ambient noise Mel spectrum.

[0036] Exemplarily, in step S412, the sequence of static potential energy measurement coefficients of the local semantic features of the ambient noise Mel spectrum is input into the feature energy level screening gating unit to obtain a sequence of static potential energy weight factors of the local semantic features of the ambient noise Mel spectrum.

[0037]

[0038]

[0039] in, For the The static potential energy measurement coefficient of the local semantic feature encoding vector of the environmental noise Mel spectrum, represents a natural constant, Indicates Normalized ambient noise Mel spectrum local semantic feature static potential energy measurement coefficient, represents the gated mask function, represents the gating threshold, Indicates Static potential weight factor of local semantic features of environmental noise Mel spectrum.

[0040] That is, the sequence of static potential energy measurement coefficients of the local semantic features of the ambient noise Mel spectrum is gated and masked by the feature energy level screening gating unit to screen out the Mel spectrum block semantic features that contribute more to the characterization of the ambient noise source type, ensuring that they have a higher potential energy value. In other words, the local Mel spectrum block features with greater influence are retained and assigned corresponding weight factors, while the feature representations that contribute less to the identification of the sound source type are filtered out, thereby reducing the interference of noise and irrelevant information and reducing the risk of overfitting.

[0041] Exemplarily, in step S413, based on the sequence of static potential weight factors of the ambient noise Mel spectrum local semantic features, the weighted sum of the sequence of ambient noise Mel spectrum local semantic feature encoding vectors is calculated to obtain the ambient noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector.

[0042]

[0043] in, respectively represent the first A local semantic feature encoding vector of the ambient noise Mel spectrum, is the number of feature vectors in the sequence of the local semantic feature encoding vectors of the environmental noise Mel spectrum, Indicates The static potential weight factor of the local semantic feature of the environmental noise Mel spectrum, Represents the local semantic pseudo-anchored aggregation center representation vector of environmental noise Mel-spectrogram.

[0044] That is, based on the obtained weight factor, the sequence of the environmental noise Mel spectrum local semantic feature encoding vector is weightedly aggregated to reveal the feature distribution trend center of each Mel spectrum local block, and generate the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector. In this way, the overall spectrum structure pattern of the environmental noise signal can be accurately reflected, thereby providing effective guidance for the subsequent local feature aggregation process.

[0045] In one embodiment, Figure 6As shown, based on the aggregation movement direction of each environmental noise mel spectrum local semantic feature coding vector in the sequence of environmental noise mel spectrum local semantic feature coding vectors relative to the environmental noise mel spectrum local semantic pseudo-anchor aggregation center representation vector, the sequence of environmental noise mel spectrum local semantic feature coding vectors is dynamically aggregated to obtain the environmental noise mel spectrum significant aggregation feature vector, including: S421, calculating the aggregation movement direction of each environmental noise mel spectrum local semantic feature coding vector in the sequence of environmental noise mel spectrum local semantic feature coding vectors relative to the environmental noise mel spectrum local semantic pseudo-anchor aggregation center representation vector to obtain a sequence of environmental noise mel spectrum local semantic aggregation running directions; S422, based on the sequence of environmental noise mel spectrum local semantic aggregation running directions, dynamically aggregate the sequence of environmental noise mel spectrum local semantic feature coding vectors toward the environmental noise mel spectrum local semantic pseudo-anchor aggregation center representation vector to obtain the environmental noise mel spectrum significant aggregation feature vector.

[0046] In one embodiment, the running direction of the ambient noise Mel spectrum local semantic aggregation is the arccosine function value between the ambient noise Mel spectrum local semantic feature encoding vector and the ambient noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector.

[0047] Exemplarily, in step S421, the aggregation movement direction of each ambient noise Mel spectrum local semantic feature coding vector in the sequence of ambient noise Mel spectrum local semantic feature coding vectors relative to the ambient noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector is calculated to obtain a sequence of ambient noise Mel spectrum local semantic aggregation running directions.

[0048]

[0049] in, respectively represent the first The local semantic feature encoding vector of the ambient noise Mel spectrum, Represents the local semantic pseudo-anchor aggregation center representation vector of the ambient noise Mel spectrum, represents the magnitude of a vector, is the arccosine function, Indicates the The local semantic feature encoding vectors of the environmental noise Mel spectrum represent the aggregated movement direction of the vector relative to the local semantic pseudo-anchor aggregation center of the environmental noise Mel spectrum.

[0050] That is, by calculating the aggregation movement direction of each environmental noise Mel spectrum local semantic feature encoding vector relative to the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector, the change trend of the semantic feature of each Mel spectrum block relative to the center point is quantified, revealing the dynamic changes and interactions of each Mel spectrum block in the overall spectrum structure, so as to facilitate the identification of the relative relationship between features and group behavior patterns.

[0051] Exemplarily, in step S422, based on the sequence of the running direction of the local semantic aggregation of the ambient noise Mel spectrum, the sequence of the local semantic feature encoding vectors of the ambient noise Mel spectrum is dynamically aggregated toward the local semantic pseudo-anchor aggregation center representation vector of the ambient noise Mel spectrum to obtain the significant aggregation feature vector of the ambient noise Mel spectrum.

[0052]

[0053] in, For the said The static potential energy measurement coefficient of the local semantic feature encoding vector of the environmental noise Mel spectrum, Indicates the The aggregated motion direction of the local semantic feature encoding vector of the ambient noise Mel spectrum relative to the local semantic pseudo-anchor aggregation center representation vector of the ambient noise Mel spectrum, is the number of feature vectors in the sequence of the local semantic feature encoding vectors of the environmental noise Mel spectrum, Represents the local semantic pseudo-anchor aggregation center representation vector of the ambient noise Mel spectrum, represents the sine function, and denote the weight matrix and bias term respectively, Represents the salient aggregated feature vector of the ambient noise Mel spectrum.

[0054] That is, based on the aggregation running direction and characteristic static potential energy measurement coefficient of each environmental noise Mel spectrum local semantic feature encoding vector, each environmental noise Mel spectrum local semantic feature encoding vector is dynamically aggregated toward the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector to obtain the environmental noise Mel spectrum significant aggregation feature vector, thereby achieving a global understanding of the spectral characteristics of the environmental noise signal. In this way, not only the key spectral components in the sound signal can be captured, but also the interaction and dynamic changes between different spectral components can be revealed, providing richer and more accurate information for sound source type identification.

[0055] Exemplarily, in step S4, the identification result of the type of the source of the ambient noise is determined based on the significantly aggregated feature vector of the mel spectrum of the ambient noise. It should be understood that the identification result of the type of the source of the ambient noise is determined based on the significantly aggregated feature vector of the mel spectrum of the ambient noise because these feature vectors integrate the key information extracted from the original audio signal and provide a deep understanding of the ambient noise. The mel spectrum significantly aggregated feature vector can accurately reflect the essential properties of the ambient noise by capturing and integrating the local semantic features in different time-frequency regions. Specifically, the local semantic feature encoding vector generated by each block in the mel spectrum image after being processed by the deep convolutional neural network (DCNN) already contains important clues about the type of specific sound source. When these local features are further aggregated into a significantly aggregated feature vector of the mel spectrum of the ambient noise, it not only retains the time sequence information of the original audio signal, but also integrates the deep semantic features of multiple local regions. Such a feature vector directly reflects the overall pattern and structure of the ambient noise and is an effective representation of the type of the sound source. Therefore, the identification result of the type of the source of the ambient noise is determined based on this significantly aggregated feature vector of the mel spectrum of the ambient noise, which means that the present application uses the data that best represents the characteristics of the ambient noise to make classification decisions. This approach ensures that the recognition process directly relies on finely analyzed and highly abstracted sound features, rather than simple raw waveforms or low-level statistical features. This enables the system to maintain high recognition accuracy in complex and changing environments, because the information contained in the significantly aggregated feature vector of the environmental noise Mel spectrum has stronger expressiveness and resolution for distinguishing different types of sound sources.

[0056] In one embodiment, based on the significantly aggregated feature vector of the ambient noise Mel spectrum, determining the recognition result of the ambient noise source type includes: inputting the significantly aggregated feature vector of the ambient noise Mel spectrum into a classifier-based ambient noise source type recognition module to obtain the recognition result, and the recognition result is used to represent the sound source type label of the ambient noise data.

[0057] In one embodiment, the environmental noise Mel spectrum significant aggregation feature vector is input into a classifier-based environmental noise source type recognition module to obtain the recognition result, and the recognition result is used to represent the sound source type label of the environmental noise data, including: using the fully connected layer of the classifier-based environmental noise source type recognition module to fully connect the environmental noise Mel spectrum significant aggregation feature vector to obtain a fully connected environmental noise Mel spectrum significant aggregation feature vector; inputting the fully connected environmental noise Mel spectrum significant aggregation feature vector into the Softmax classification function of the classifier-based environmental noise source type recognition module to obtain the recognition result.

[0058] Preferably, the significantly aggregated feature vector of the environmental noise Mel spectrum is passed through a classifier-based environmental noise source type recognition module to obtain a recognition result, which includes: The absolute value sum of all eigenvalues ​​of the environmental noise Mel spectrum significant aggregation eigenvector is calculated to obtain the first environmental noise Mel spectrum significant aggregation spatial structure value, and the square root of the square sum is calculated to obtain the second environmental noise Mel spectrum significant aggregation spatial structure value, that is:

[0059]

[0060] in, The first eigenvector representing the significant aggregated feature vector of the ambient noise Mel spectrum eigenvalues, represents the total number of eigenvalues ​​of the significant aggregated eigenvector of the ambient noise Mel spectrum, Indicates the significant aggregated spatial structure value of the first ambient noise Mel spectrum, Indicates the significant aggregated spatial structure value of the second environmental noise Mel spectrum; For each eigenvalue of the environmental noise Mel spectrum significant aggregation feature vector, the first environmental noise Mel spectrum significant aggregation spatial structure value is calculated by subtracting the product of the eigenvalue and the total number of eigenvalues ​​of the environmental noise Mel spectrum significant aggregation feature vector to obtain the first environmental noise Mel spectrum significant aggregation long-range dependency value ,in, Indicates the significant aggregated spatial structure value of the first ambient noise Mel spectrum, The first eigenvector representing the significant aggregated feature vector of the ambient noise Mel spectrum eigenvalues, represents the total number of eigenvalues ​​of the significant aggregated eigenvector of the ambient noise Mel spectrum, Indicates the significant aggregated long-range dependency value of the first environmental noise Mel spectrum; Calculate the second environmental noise Mel spectrum significant aggregation long-range dependency value obtained by multiplying the square root of the total number of eigenvalues ​​by the product of the eigenvalues ​​minus the second environmental noise Mel spectrum significant aggregation spatial structure value ,in, Indicates the significant aggregated spatial structure value of the second ambient noise Mel spectrum, The first eigenvector representing the significant aggregated feature vector of the ambient noise Mel spectrum eigenvalues, represents the total number of eigenvalues ​​of the significant aggregated eigenvector of the ambient noise Mel spectrum, Indicates the significant aggregated long-range dependency value of the second environmental noise Mel spectrum; The exponential value calculated by taking the first environmental noise Mel spectrum significant aggregation long-range dependency value as the exponent of the natural constant and the inverse of the second environmental noise Mel spectrum significant aggregation long-range dependency value are weighted summed to obtain the optimized eigenvalue corresponding to each eigenvalue ,in, Indicates the significant aggregated long-range dependency value of the first ambient noise Mel spectrum, Indicates the significant aggregation long-range dependency value of the second environmental noise Mel spectrum, represents a natural constant, and represents the weighted hyperparameter, Indicates the optimized eigenvalue corresponding to each eigenvalue; The optimized environmental noise Mel-spectrum significant aggregation feature vector composed of the optimized feature values ​​is passed through an environmental noise source type recognition module based on a classifier to obtain a recognition result.

[0061] Here, when each environmental noise Mel-spectrogram local semantic feature coding vector in the sequence of environmental noise Mel-spectrogram local semantic feature coding vectors respectively represents the Mel-spectrogram local area image semantic coding features determined by image block processing of the environmental noise Mel-spectrogram image, when performing feature aggregation based on feature dynamics analysis, the fuzziness and randomness of the sound dynamics characteristics will cause large fluctuations in the fit of the feature dynamics analysis, resulting in the environmental noise Mel-spectrogram significant aggregation feature vector having a difference in the spatial structure of the fusion distribution of the Mel-spectrogram image semantic features, affecting the classification regression convergence consistency, and thus affecting the accuracy of the recognition result obtained by the environmental noise source type recognition module based on the classifier.

[0062] Based on this, in order to solve the problem that the feature set of the environmental noise Mel-spectrum significant aggregation feature vector may have a spatial structure loss in the high-dimensional space, which leads to the convergence inconsistency of the image semantic classification regression based on the feature-based implicit inference of the spatial structure information, a long-distance feature dependency relationship is established based on the overall feature scale of the environmental noise Mel-spectrum significant aggregation feature vector relative to the spatial structure representation of the environmental noise Mel-spectrum significant aggregation feature vector, so as to establish the feature local connectivity of the environmental noise Mel-spectrum significant aggregation feature vector, and the spatial ambiguous information of the object feature value is captured by predicting the unstructured feature value points of the environmental noise Mel-spectrum significant aggregation feature vector, thereby improving the spatial inductive bias perception ability of the feature set of the environmental noise Mel-spectrum significant aggregation feature vector, improving the classification regression convergence consistency, and improving the accuracy of the recognition result obtained by the environmental noise Mel-spectrum significant aggregation feature vector through the classifier-based environmental noise source type recognition module.

[0063] In summary, the method for identifying the type of environmental noise source based on a deep convolutional neural network according to the embodiment of the present application is explained, and the method and device for identifying the type of environmental noise source based on a deep convolutional neural network collect environmental noise data through a microphone array, and perform signal processing on it to generate a mel spectrum image of environmental noise, and then use a deep convolutional neural network to perform fine-grained spectrum analysis on the mel spectrum image of the environmental noise data to capture the deep semantic features of the environmental noise, thereby realizing intelligent recognition of the type of environmental noise source data. In this way, rapid and accurate recognition of environmental noise sources can be achieved, thereby providing strong technical support for noise control and environmental protection, while reducing dependence on artificial feature design and improving recognition accuracy and efficiency.

[0064] Figure 7 Schematic block diagram of an environmental noise source type recognition device based on a deep convolutional neural network according to an embodiment of the present application. Figure 7 As shown, the device 100 for identifying the type of environmental noise source based on a deep convolutional neural network includes: an environmental noise data acquisition module 110, which is used to collect environmental noise data in real time through a microphone array; a Mel spectrum feature extraction module 120, which is used to extract the Mel spectrum features of the environmental noise data to obtain an environmental noise Mel spectrum image; an environmental noise spectrum analysis module 130, which is used to perform a spectrum analysis based on a local scale on the environmental noise Mel spectrum image to obtain a sequence of environmental noise Mel spectrum local semantic feature coding vectors; an environmental noise feature dynamic optimization aggregation module 140, which is used to perform feature dynamic optimization aggregation on the sequence of environmental noise Mel spectrum local semantic feature coding vectors to obtain an environmental noise Mel spectrum significant aggregation feature vector; a noise recognition result determination module 150, which is used to determine the recognition result of the type of environmental noise source based on the environmental noise Mel spectrum significant aggregation feature vector.

[0065] In one embodiment, the environmental noise spectrum analysis module includes: an image block processing unit, which is used to use an image segmenter to perform image block processing on the environmental noise Mel spectrum image to obtain a sequence of environmental noise Mel spectrum image blocks; a sound Mel spectrum feature extraction unit, which is used to input each environmental noise Mel spectrum image block in the sequence of environmental noise Mel spectrum image blocks into a sound Mel spectrum feature extractor based on a deep convolutional neural network model to obtain a sequence of environmental noise Mel spectrum local semantic feature encoding vectors.

[0066] Here, those skilled in the art can understand that the specific operations of the various modules and units in the above-mentioned environmental noise source type identification device based on deep convolutional neural network have been described in the above reference. Figures 1 to 6The invention has been introduced in detail in the description of the environmental noise source type identification method based on deep convolutional neural network, and therefore, its repeated description will be omitted.

[0067] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0068] It should be understood that the specific examples in this article are only intended to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application.

[0069] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0070] It should also be understood that the various implementation methods described in this specification can be implemented individually or in combination, and the embodiments of the present application are not limited to this.

[0071] Unless otherwise stated, all technical and scientific terms used in the embodiments of the present application are the same as the meanings generally understood by those skilled in the art of the technical field of the present application. The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the scope of the present application. The term "and / or" used in the present application includes any and all combinations of one or more related listed items. The singular forms of "a kind of", "above" and "the" used in the embodiments of the present application and the appended claims are also intended to include majority forms, unless the context clearly indicates other meanings. In addition, the terms "first", "second" etc. are only used for description purposes and cannot be understood as indicating or suggesting relative importance.

[0072] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0073] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0075] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0076] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for identifying environmental noise source types based on deep convolutional neural network, characterized in that: include: Collect environmental noise data in real time through microphone array; Extracting Mel-spectrogram features of the environmental noise data to obtain an environmental noise Mel-spectrogram image; Performing a local scale-based spectrum analysis on the environmental noise Mel spectrum image to obtain a sequence of local semantic feature encoding vectors of the environmental noise Mel spectrum; Performing dynamic feature optimization aggregation on the sequence of the environmental noise Mel spectrum local semantic feature encoding vectors to obtain an environmental noise Mel spectrum significant aggregation feature vector; Based on the significantly aggregated feature vector of the environmental noise Mel spectrum, a recognition result of the type of the environmental noise source is determined.

2. The method for identifying environmental noise source types based on deep convolutional neural network according to claim 1 is characterized in that: Performing a local scale-based spectrum analysis on the ambient noise Mel spectrum image to obtain a sequence of ambient noise Mel spectrum local semantic feature encoding vectors, including: Using an image segmenter to perform image block processing on the environmental noise Mel spectrum image to obtain a sequence of environmental noise Mel spectrum image blocks; Each environmental noise mel spectrum image block in the sequence of environmental noise mel spectrum image blocks is input into a sound mel spectrum feature extractor based on a deep convolutional neural network model to obtain a sequence of environmental noise mel spectrum local semantic feature encoding vectors.

3. The method for identifying environmental noise source types based on deep convolutional neural network according to claim 2 is characterized in that: Performing dynamic feature optimization aggregation on the sequence of the environmental noise Mel spectrum local semantic feature encoding vectors to obtain an environmental noise Mel spectrum significant aggregation feature vector, including: Calculate the local semantic pseudo-anchor aggregation center representation vector of the ambient noise Mel spectrum based on the characteristic static potential energy distribution of the sequence of the ambient noise Mel spectrum local semantic feature encoding vector; Based on the aggregation movement direction of each ambient noise Mel spectrum local semantic feature coding vector in the sequence of ambient noise Mel spectrum local semantic feature coding vectors relative to the ambient noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector, the sequence of ambient noise Mel spectrum local semantic feature coding vectors is dynamically aggregated to obtain the ambient noise Mel spectrum significant aggregation feature vector.

4. The method for identifying environmental noise source types based on deep convolutional neural network according to claim 3 is characterized in that: Based on the characteristic static potential energy distribution of the sequence of the environmental noise Mel spectrum local semantic feature encoding vector, the environmental noise Mel spectrum local semantic pseudo anchor aggregation center representation vector is calculated, including: Inputting each environmental noise Mel spectrum local semantic feature encoding vector in the sequence of environmental noise Mel spectrum local semantic feature encoding vectors into a feature static potential energy measurement network to obtain a sequence of environmental noise Mel spectrum local semantic feature static potential energy measurement coefficients; Inputting the sequence of static potential energy measurement coefficients of the local semantic features of the environmental noise Mel spectrum into the feature energy level screening gating unit to obtain a sequence of static potential energy weight factors of the local semantic features of the environmental noise Mel spectrum; Based on the sequence of static potential weight factors of the environmental noise Mel spectrum local semantic features, the weighted sum of the sequence of environmental noise Mel spectrum local semantic feature encoding vectors is calculated to obtain the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector.

5. The method for identifying environmental noise source types based on deep convolutional neural network according to claim 4 is characterized in that: Based on the aggregation movement direction of each environmental noise Mel spectrum local semantic feature encoding vector in the sequence of environmental noise Mel spectrum local semantic feature encoding vectors relative to the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector, dynamically aggregating the sequence of environmental noise Mel spectrum local semantic feature encoding vectors to obtain the environmental noise Mel spectrum significant aggregation feature vector, including: Calculate the aggregation movement direction of each ambient noise Mel spectrum local semantic feature encoding vector in the sequence of ambient noise Mel spectrum local semantic feature encoding vectors relative to the ambient noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector to obtain a sequence of ambient noise Mel spectrum local semantic aggregation running directions; Based on the sequence of the running direction of the environmental noise Mel spectrum local semantic aggregation, the sequence of the environmental noise Mel spectrum local semantic feature encoding vectors is dynamically aggregated toward the environmental noise Mel spectrum local semantic pseudo-anchor aggregation center representation vector to obtain the environmental noise Mel spectrum significant aggregation feature vector.

6. The method for identifying environmental noise source types based on deep convolutional neural network according to claim 5 is characterized in that: The running direction of the local semantic aggregation of the ambient noise Mel spectrum is the arccosine function value between the local semantic feature encoding vector of the ambient noise Mel spectrum and the local semantic pseudo-anchor aggregation center representation vector of the ambient noise Mel spectrum.

7. The method for identifying environmental noise source types based on deep convolutional neural network according to claim 6 is characterized in that: Determining the recognition result of the type of the ambient noise source based on the significantly aggregated feature vector of the ambient noise Mel spectrum includes: The environmental noise Mel-spectrogram significant aggregation feature vector is input into a classifier-based environmental noise sound source type recognition module to obtain the recognition result, and the recognition result is used to represent the sound source type label of the environmental noise data.

8. The method for identifying environmental noise source types based on deep convolutional neural network according to claim 7 is characterized in that: Inputting the environmental noise Mel-spectrum significant aggregation feature vector into a classifier-based environmental noise sound source type recognition module to obtain the recognition result, which is used to represent the sound source type label of the environmental noise data, including: Use the fully connected layer of the classifier-based environmental noise source type recognition module to perform fully connected encoding on the environmental noise Mel spectrum significant aggregation feature vector to obtain a fully connected environmental noise Mel spectrum significant aggregation feature vector; The fully connected environmental noise Mel-spectrogram significant aggregation feature vector is input into the Softmax classification function of the classifier-based environmental noise source type recognition module to obtain the recognition result.

9. A device for identifying the type of environmental noise source based on a deep convolutional neural network, which can implement the method for identifying the type of environmental noise source based on a deep convolutional neural network as described in any one of claims 1 to 8, characterized in that: The device for identifying the type of environmental noise source based on a deep convolutional neural network comprises: An environmental noise data collection module is used to collect environmental noise data in real time through a microphone array; A Mel spectrum feature extraction module, used to extract the Mel spectrum features of the environmental noise data to obtain an environmental noise Mel spectrum image; An environmental noise spectrum analysis module is used to perform a spectrum analysis based on a local scale on the environmental noise Mel spectrum image to obtain a sequence of local semantic feature encoding vectors of the environmental noise Mel spectrum; An environmental noise feature dynamic optimization aggregation module is used to perform feature dynamic optimization aggregation on the sequence of the environmental noise Mel spectrum local semantic feature encoding vectors to obtain an environmental noise Mel spectrum significant aggregation feature vector; The noise recognition result determination module is used to determine the recognition result of the ambient noise source type based on the significant aggregation feature vector of the ambient noise Mel spectrum.

10. The device for identifying environmental noise source types based on deep convolutional neural network according to claim 9, characterized in that: The environmental noise spectrum analysis module comprises: An image block processing unit, configured to use an image segmentor to perform image block processing on the ambient noise Mel spectrum image to obtain a sequence of ambient noise Mel spectrum image blocks; The sound mel spectrum feature extraction unit is used to input each environmental noise mel spectrum image block in the sequence of environmental noise mel spectrum image blocks into a sound mel spectrum feature extractor based on a deep convolutional neural network model to obtain a sequence of environmental noise mel spectrum local semantic feature encoding vectors.

Images