A high-precision environmental noise identification method and device

Abstract

The present disclosure provides a high-precision environmental noise identification method and device, which include: obtaining noise data to be identified; inputting the noise data to be identified into a preset noise identification model for identification processing to obtain an identification result, wherein the noise identification model is used to extract target local features and target time dependencies corresponding to the target local features from the noise data to be identified, and identify the noise data to be identified based on the target local features and the target time dependencies, and the target time dependencies reflect the correlation between the target local features at different moments in the noise data to be identified in the time dimension. By extracting local features from the noise data to be identified and capturing the time dependencies corresponding to the local features, the noise data can be accurately identified.

Description

Technical Field

[0001] The present disclosure relates to the technical field of noise monitoring, and in particular to a high-precision environmental noise identification method and device. Background Art

[0002] Environmental noise refers to the sound that disturbs the surrounding living environment caused by industrial production, construction, transportation and social life. Environmental noise needs to be classified and controlled, that is, it is necessary to distinguish the types of environmental noise and then carry out supervision and law enforcement according to the types of noise. Therefore, when a noise pollution incident occurs, it is crucial to determine the main noise source and take reasonable and effective control measures.

[0003] Currently, although there are solutions that can be used to identify the types of environmental noise, these solutions generally have the following problems:

[0004] On the one hand, due to the limitations of noise collection conditions, the noise data that can be collected for training usually have many limitations, such as the number of categories, the scope of scene coverage, and the amount of single-category data. It is difficult to train a recognition algorithm or recognition model that can perform well in complex and diverse real-world scenarios based on these noise data, and it is even more difficult to accurately locate the noise source.

[0005] On the other hand, a single and rigid feature extraction method is often used in the training process of recognition algorithms or recognition models, which leads to problems such as limited expressive ability of the trained algorithms or models, fewer feature extraction dimensions, and poor adaptability to complex scenarios. Summary of the invention

[0006] The present invention provides a high-precision environmental noise recognition method and device.

[0007] In a first aspect, an embodiment of the present disclosure provides a high-precision environmental noise recognition method, which includes:

[0008] Acquire noise data to be identified;

[0009] The noise data to be identified is input into a preset noise identification model for identification processing to obtain an identification result, wherein the noise identification model is used to extract target local features and target time dependencies corresponding to the target local features from the noise data to be identified, and the noise data to be identified is identified based on the target local features and the target time dependencies, and the target time dependencies reflect the correlation between the target local features at different times in the noise data to be identified in the time dimension.

[0010] In a second aspect, an embodiment of the present disclosure provides a high-precision environmental noise recognition device, comprising:

[0011] An acquisition module, used for acquiring noise data to be identified;

[0012] An identification module is used to input the noise data to be identified into a preset noise identification model for identification processing to obtain an identification result, wherein the noise identification model is used to extract target local features and target time dependencies corresponding to the target local features from the noise data to be identified, and identify the noise data to be identified based on the target local features and the target time dependencies, and the target time dependencies reflect the correlation between the target local features at different times in the noise data to be identified in the time dimension.

[0013] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor; the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, the high-precision environmental noise identification method described in the first aspect is implemented.

[0014] In the disclosed embodiment, the preset noise recognition model can capture complex features from the noise data, so that the model can be applied to the recognition of noise data in more complex sound environments, thereby effectively coping with the impact of environmental interference on the recognition results of noise data. Based on the captured features, the local features in the noise data and their corresponding time dependencies are analyzed. Since the time dependency can reflect the correlation between the local features at different moments in the noise data in the time dimension, the complex features in the noise data can be captured based on the local features and their time dependencies, and the noise data can be accurately recognized using complex noise.

[0015] In addition, the training process of the model uses the noise data in the training dataset as the source domain data, and the noise data in the same scene category as the noise data in the training dataset as the target domain data, which can fully measure the feature distribution differences under the same scene category. Based on the distribution differences of features between the source domain data and the target domain data, and between sub-domains within the class (that is, between the same scene categories), it can help the model to perform transfer learning, thereby improving the generalization and robustness of the model, that is, enhancing the model's adaptability between sub-domains under the same scene category. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In the accompanying drawings of the embodiments of the present disclosure:

[0017] Figure 1 A flowchart of a high-precision environmental noise recognition method provided by an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of a noise recognition model training process provided by an embodiment of the present disclosure;

[0019] Figure 3 A schematic diagram of a process for obtaining a training data set provided in an embodiment of the present disclosure;

[0020] Figure 4 A schematic diagram of an exemplary process of extracting feature vectors provided in an embodiment of the present disclosure;

[0021] Figure 5 A schematic diagram of a process flow of inputting a feature vector into a preset model provided in an embodiment of the present disclosure;

[0022] Figure 6 A schematic diagram of the structure of an exemplary feature extraction layer and a feature enhancement layer provided in an embodiment of the present disclosure;

[0023] Figure 7 A schematic diagram of a process for calculating a joint loss function value provided in an embodiment of the present disclosure;

[0024] Figure 8 A schematic diagram of an exemplary calculation process of a joint loss function value provided in an embodiment of the present disclosure;

[0025] Fig. 9 A schematic diagram of the structure of a high-precision environmental noise recognition device provided by an embodiment of the present disclosure;

[0026] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0027] Fig.11 A flowchart of an exemplary high-precision environmental noise identification method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0029] The present disclosure will be described more fully below with reference to the accompanying drawings, but the embodiments shown may be embodied in different forms, and the present disclosure should not be construed as being limited to the embodiments set forth below. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete, and will enable those skilled in the art to fully understand the scope of the present disclosure.

[0030] The accompanying drawings of the embodiments of the present disclosure are used to provide a further understanding of the embodiments of the present disclosure, and constitute a part of the specification, and are used together with the detailed embodiments to explain the present disclosure, and do not constitute a limitation of the present disclosure. By describing the detailed embodiments with reference to the accompanying drawings, the above and other features and advantages will become more apparent to those skilled in the art.

[0031] The present disclosure may be described with reference to plan views and / or cross-sectional views by means of ideal schematic views of the present disclosure. Therefore, the exemplary illustrations may be modified according to manufacturing techniques and / or tolerances.

[0032] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0033] The terms used in the present disclosure are only used to describe specific embodiments and are not intended to limit the present disclosure. The term "and / or" as used in the present disclosure includes any and all combinations of one or more related enumerated items. The singular forms "one" and "the" as used in the present disclosure are also intended to include plural forms, unless the context clearly indicates otherwise. The terms "including", "made of..." as used in the present disclosure specify the presence of the features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof.

[0034] Unless otherwise defined, all terms (including technical and scientific terms) used in this disclosure have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly defined in this disclosure.

[0035] The present disclosure is not limited to the embodiments shown in the drawings, but includes modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the drawings have schematic properties, and the shapes of the regions shown in the drawings illustrate the specific shapes of the regions of the elements, but are not intended to be limiting.

[0036] In some related technologies, algorithms or models are used to realize noise recognition and classification. However, due to the limitations of training conditions and training process, it is difficult to achieve good results in complex and diverse real-life scenarios, and it is impossible to accurately locate the type of noise source, which brings important challenges to the identification of environmental noise.

[0037] At present, most noise datasets used for training focus on sub-category segmentation recognition within specific vertical fields, and are difficult to cope with the recognition requirements of application scenarios or data complexity in environmental noise. Since high-accuracy recognition depends on a sufficient amount of high-quality labeled data, building a data sample library that matches the actual scene requires a lot of manpower and material resources, and also involves complex data processing and sample labeling processes. Therefore, due to the conditions for collecting noise data, it is very difficult to form a voiceprint spectrum library covering environmental noise in various scenes in real life, which leads to poor recognition results in real scenes. Although the relevant technology has audio classification datasets that can be realized across scenes, these noise data also have many limitations such as the number of categories, the scope of scene coverage, and the amount of single-category data.

[0038] Moreover, a single and rigid feature extraction method is often used during the training process, which fails to fully utilize the potential and rich information and feature expression capabilities in the collected noise data. This leads to problems such as limited expression capabilities of the trained algorithm or model, fewer feature extraction dimensions, and poor adaptability to complex scenarios. It is difficult to fully learn the implicit common features of intra-class features and capture the high-dimensional features of sound source features between different categories, which results in a significant decrease in recognition performance in complex scenarios such as noisy environments and mixed multiple sound sources.

[0039] In addition, on the one hand, due to the limitations of audio acquisition equipment, which is affected by factors such as microphone quality and circuit noise, the noise data used in training and the noise data in real scenes often have significant distribution differences, resulting in large differences in audio signal-to-noise ratio, sound acquisition frequency, etc. between domains, thus affecting the recognition effect during use; on the other hand, the sound sources in the same scene category are affected by factors such as individual differences, and will also produce significant intra-class differences, which not only leads to a decline in performance when processing noise data in real scenes, but also increases the difficulty of adaptation between sub-domains.

[0040] Therefore, how to accurately identify noise data in a complex sound environment is an urgent problem to be solved.

[0041] In a first aspect, the present disclosure provides a method for identifying environmental noise with high precision. Figure 1 , which includes:

[0042] S1. Obtain noise data to be identified;

[0043] S2. Input the noise data to be identified into a preset noise recognition model for identification processing to obtain an identification result, wherein the noise recognition model is used to extract target local features and target time dependencies corresponding to the target local features from the noise data to be identified, and identify the noise data to be identified based on the target local features and the target time dependencies, wherein the target time dependencies reflect the correlation between the target local features at different times in the noise data to be identified in the time dimension.

[0044] In the embodiments of the present disclosure, the noise data to be identified is identified and processed by a preset noise recognition model, and the target local features and the target time dependencies corresponding to the target local features are extracted from the noise data to be identified. Since the target time dependencies can reflect the correlation between the target local features at different moments in the time dimension, the noise data to be identified can be accurately identified based on the target local features and the target time dependencies, and more complex audio features in the time dimension can be captured, and environmental interference can be effectively coped with, with strong robustness.

[0045] In some embodiments, reference Figure 2 The training process of the noise recognition model in step S2 includes:

[0046] S21', obtaining a training data set; the training data set includes feature vectors corresponding to noise data of multiple scene categories;

[0047] S22', selecting first noise data that has not been selected from each noise data in the training data set, inputting a feature vector corresponding to the first noise data into a preset model for processing, and obtaining an intermediate recognition result;

[0048] S23', determining a joint loss function value according to the intermediate recognition result;

[0049] S24', judging whether the value of the joint loss function reaches the convergence condition;

[0050] S251', when the value of the joint loss function does not reach the convergence condition, the model parameters of the preset model are updated using the value of the joint loss function, and the step of selecting the first noise data that has not been selected from each noise data in the training data set, inputting the first noise data into the preset model for processing, and obtaining the intermediate recognition result;

[0051] S252': when the value of the joint loss function reaches a convergence condition, determine the preset model as the trained noise recognition model.

[0052] In this embodiment, the training data set includes noise data of multiple scene categories, and the feature vector corresponding to the first noise data is input into the preset model for processing to obtain an intermediate recognition result, wherein the first noise data is noise data selected from the training data set that has not been input into the model; based on the intermediate recognition result, the joint loss function value is calculated, wherein the joint loss function value is determined by the LMMD (Local Maximum Mean Discrepancy) loss function value and the classification loss function value, and the LMMD loss function value can reflect the distribution difference between the source domain noise data and the target domain noise data in the same scene category, and each noise data in the training data set is traversed until the joint loss function value reaches the convergence condition, and it is determined that the training of the preset model is completed. The iterative training process of the preset model can continuously clarify the feature distribution of each scene category, and the noise recognition model after training is fully learned and optimized, which can effectively overcome the problems of excessive inter-domain differences caused by inconsistent data distribution in the noise data in the same scene category, unclear inter-class differences between sound sources, and fuzzy intra-class differences between the same type of data, resulting in poor recognition effect.

[0053] In some embodiments, reference Figure 3 , step S21' comprises:

[0054] S211', obtaining noise data of different scene categories;

[0055] S212', for the noise data of each scene category, extracting a feature vector corresponding to the noise data;

[0056] S213', dividing the feature vectors corresponding to the noise data of each scene category according to a preset ratio to obtain the training data set.

[0057] In the embodiments of the present disclosure, the noise data of different scene categories have the characteristics of large coverage, large data volume, multiple scene categories of sound sources, and high data quality. Feature vectors are extracted from the noise data of each scene category, and the noise data of the scene category and its feature vectors are divided into a training data set and a test data set according to a preset ratio. The training data set is used to train the environmental noise recognition model, which can provide reliable data support for the training of the noise recognition model and ensure the effectiveness and reliability of the model training. The test data set is used to evaluate the generalization ability of the model. It is worth noting that the embodiments of the present disclosure do not impose special restrictions on the preset ratio. For example, the data ratio in the training data set and the test data set can be 7:3.

[0058] In some embodiments, each noise data is accurately labeled in combination with its scene category to obtain label data of the noise data.

[0059] In some embodiments, the noise data collection process has high quality, and the sampling rate, bit depth and signal-to-noise ratio all meet the preset audio quality requirements. For example, a high-performance voice recorder is used to collect noise data with a sampling rate of 44.1kHz, a bit depth of 16 bits, and a signal-to-noise ratio of not less than 30dB.

[0060] In some embodiments, a voiceprint spectrum library can also be constructed based on noise data of multiple scene categories, and the noise data in the voiceprint spectrum library can be used to train the noise recognition model, which consciously avoids the problem of imbalanced scene analogy of noise data during model training. This can ensure the diversity of noise data collection scenarios and the high accuracy and robustness of the model in a variety of environments, thereby effectively improving the recognition ability and adaptability to different sound sources.

[0061] In some embodiments, the scene category is at least one of construction noise, industrial noise, traffic noise, social life noise, and natural noise, and these scene categories can be further subdivided. For example, these scene categories are further divided into 60 subdivided scenes, and based on construction noise, the noise data is divided into electric saw sounds, electric hammer sounds, etc. under the scene category, and based on traffic noise, the noise data is divided into car horns, motorcycle sounds, etc. under the scene category. The embodiments of the present disclosure do not impose special restrictions on scene categories and their subdivisions, and may also include other scene categories.

[0062] In some embodiments, the feature vector includes at least one of MFCC (Mel-Frequency Cepstral Coefficients), chrominance features, Mel spectrum, and spectral contrast.

[0063] In some embodiments, step S212' includes:

[0064] Divide the noise data into frames to obtain a plurality of noise data subframes;

[0065] Performing fast Fourier transform processing on the multiple noise data subframes to obtain spectrum data;

[0066] The frequency spectrum data is subjected to frequency transformation to obtain a feature vector; wherein the feature vector includes: Mel frequency cepstral coefficients MFCC, chromaticity features, Mel spectrum and spectrum contrast, so as to constitute at least one item of the multi-dimensional features.

[0067] In an embodiment of the present disclosure, noise data is loaded, and the noise data is framed to obtain a plurality of noise data subframes, wherein each noise data subframe has the same length. The noise data subframes are subjected to fast Fourier transform processing to obtain spectrum data. The spectrum data can show the frequency component characteristics in the noise data, which is convenient for extracting feature vectors. The spectrum data is subjected to frequency transformation to obtain a feature vector, which includes at least MFCC, and the extracted MFCC can better simulate the human ear's perception of sound frequency.

[0068] It is worth noting that other feature vectors can be further extracted based on the noise data. For example, the chromaticity feature of the noise data is extracted, and the chromaticity feature can characterize the energy distribution of different line frequency bands in the noise data subframe; the frequency axis of the noise data is adjusted to a scale that conforms to the auditory perception characteristics of the human ear using the Mel frequency transform, thereby obtaining a spectrum representation that can better reflect the human ear's hearing, namely the Mel spectrum; the spectrum contrast that captures the energy difference of different frequency intervals in the noise data can characterize the multi-dimensional feature information in the noise data. Through MFCC, chromaticity features, Mel spectrum and spectrum contrast, the spectrum characteristics, tone information and dynamic changes of sound can be fully captured from the noise data to form a comprehensive multi-dimensional feature vector to adapt to the complex noise environment.

[0069] In some embodiments, after extracting the feature vector corresponding to the noise data, the method further includes:

[0070] The extracted feature vectors are normalized to eliminate the differences caused by factors such as recording distance and make the feature values ​​more uniform and standardized.

[0071] As an example of an embodiment of the present disclosure, refer to Figure 4 , perform data preprocessing on the noise data to obtain intermediate noise data, divide the intermediate noise data into N intermediate noise data subframes, each intermediate noise data subframe corresponds to a D-dimensional feature vector, and perform fast Fourier transform processing on each intermediate noise data subframe to obtain spectrum data. Perform frequency transformation on the spectrum data to obtain MFCC to form an N×D two-dimensional matrix. Based on the N×D two-dimensional matrix, further extract chromaticity features, Mel spectrum and spectrum contrast to obtain an N×D×4 feature matrix. Reduce the dimension of the feature matrix to form an N×4D feature matrix.

[0072] In some embodiments, reference Figure 5 , step S22' comprises:

[0073] S221', extracting a first local feature and its corresponding first time dependency from a feature vector corresponding to the first noise data;

[0074] S222': Determine an intermediate recognition result according to the first local feature and the first time dependency.

[0075] In an embodiment of the present disclosure, a first local feature and a first time dependency relationship corresponding to the first local feature are extracted from a feature vector corresponding to the first noise data. The first local feature is extracted according to different time resolutions, and can reflect the spatial position characteristics of the sound source of the noise data. The first time dependency relationship can reflect the time dependency relationship of each first local feature in the first noise data in the time dimension. Among them, the first time dependency relationship is obtained based on the feature enhancement processing of the first local feature. The first time dependency relationship can train a preset model to better capture the dynamic characteristics of the first noise data that change over time, so that the intermediate recognition result of the first noise data can be identified according to the first local feature and the first time dependency relationship, and the recognition result has high accuracy.

[0076] In some embodiments, step S221' includes:

[0077] Inputting the feature vector corresponding to the first noise data into the feature extraction layer for processing to obtain a first local feature;

[0078] The first local feature is input into a feature enhancement layer for processing to obtain a first time dependency relationship corresponding to the first local feature.

[0079] Among them, the feature extraction layer is composed of a CNN (Convolutional Neural Network) with an activation function ReLU (Rectified Linear Unit), an SCConv (Spatial and Channel-wise Convolution) module, a CBAM (Convolutional Block Attention Module) module, a global pooling and a fully connected layer.

[0080] The feature enhancement layer includes an LSTM (Long Short-Term Memory) network.

[0081] In an embodiment of the present disclosure, the feature vector is input into the feature extraction layer for feature processing. Since the feature extraction layer includes a CNN with an activation function ReLU, the feature vector can capture local features of different time resolutions (i.e., the first local features) through the convolution operation of the multi-layer CNN, thereby distinguishing different audio components; the feature extraction layer also includes an attention module (including an SCConv module, a CBAM module, and a global pooling and fully connected layer), wherein the CBAM module can adjust the weight according to the energy ratio of each frequency band of the noise data, thereby highlighting the channel of the frequency band where the sound source is located; the SCConv module pays more attention to the position information of the sound source, highlighting the spatial position characteristics of the sound source. Through the processing of the SCConv module and the CBAM module, the problem of uneven weight distribution in different channels and spatial positions in the related technology can be solved, thereby enhancing the features of convolution extraction.

[0082] The extracted first local features are input into the feature enhancement layer to enhance the expressive power of local features and suppress non-critical features to capture the audio time dependency between the first local features. The trained model can handle the long-term dependency problem in the sequence noise data. It is especially suitable for processing long sequence noise, and can accurately associate the acoustic features of noise data at various stages such as startup, continuous operation, and pause, thereby avoiding the problem of gradient disappearance and improving the robustness and performance of the trained model.

[0083] It is worth noting that due to the complex characteristics of noise data, the size of the convolution kernel of each layer of convolution operation can be set according to actual needs. A smaller convolution kernel can effectively capture high-frequency and short-term features, such as momentary whistle sounds, and a larger convolution kernel can capture low-frequency and continuous features, such as the hum of an engine.

[0084] In some embodiments, the formula for the convolution operation is: .in, is the convolution output of the lth layer, σ(·) represents the activation function (e.g., ReLU), W (l) is the convolution kernel, * indicates the convolution operation, is the feature map of the convolution output of the l-1th layer, b (l) For bias.

[0085] In some embodiments, the formula corresponding to the CBAM module in the attention module is: . Among them, σ(·) represents the activation function, W2 and W1 are the weight matrices of the fully connected layer, F is the input value, and GAP(F) is the global average pooling operation on the input value F.

[0086] The formula corresponding to the SCConv module in the attention module is: Among them, Conv (7×7)(F) is a 7×7 convolution operation, and σ(·) represents the activation function.

[0087] The CBAM module can analyze channel features, and the SCConv module can analyze spatial information and enhance the expressiveness of features.

[0088] In some embodiments, the feature enhancement layer includes an LSTM network. The update formula of the LSTM network is:

[0089]

[0090] Among them, f t is the forget gate in the LSTM network, i t is the input gate in the LSTM network, o t is the output gate in the LSTM network, C t is the memory unit, h t is the output of the LSTM network, and ⊙ represents element-wise multiplication.

[0091] According to the local features and time dependencies, the intermediate recognition results are determined through the fully connected layer. The operation formula of the fully connected layer is: y=W fc ·h+b fc Among them, W fc is the weight of the fully connected layer, h is the output of the LSTM network, and b fc In some embodiments, the output nodes of the fully connected layer are 60, corresponding to 60 scene categories (ie, recognition results).

[0092] In some embodiments, local features of different lengths and regular dimensions are mapped to fixed dimensions through a pooling layer to ensure that the dimensions of each local feature remain consistent. lstm (X), the feature representation after adaptive pooling is: .

[0093] After pooling, F pool (X) Classification is performed through the fully connected layer, which converts the multi-dimensional local features into various noise probability distributions based on the weight matrix and bias parameters: y=W fc ·F pool (X)+b fc , where W fc is the weight matrix of the fully connected layer, b fc is the bias vector, and y is the intermediate recognition result (i.e., the predicted value).

[0094] In some embodiments, reference Figure 6 ,The feature extraction layer includes CNN, SCConv module, CBAM module, global pooling and fully connected layer.

[0095] The feature vector is sequentially input into convolution layer 1, SCConv module 1, batch normalization layer 1, CBAM module, convolution layer 2, maximum pooling layer 1, convolution layer 3, SCConv module 3, batch normalization layer 2, convolution layer 4 and maximum pooling layer 2 to extract the intermediate local features.

[0096] The intermediate local features are sequentially input into the global maximum pooling layer 1, the fully connected layer 1, the batch normalization layer 3, the fully connected layer 2, and the batch normalization layer 4 to extract the local features.

[0097] The feature enhancement layer includes LSTM network and fully connected layer.

[0098] The local features are sequentially input into the LSTM network to obtain the temporal dependency.

[0099] In some embodiments, reference Figure 7 , step S23' comprises:

[0100] S231', determining a classification loss function value according to the intermediate recognition result;

[0101] S232', acquiring second noise data of the same scene category as the first noise data, and extracting a second local feature and its corresponding second time dependency from the second noise data;

[0102] S233′, taking the first local feature and the first time dependency of the first noise data as source domain features, taking the second local feature and the second time dependency as target domain features, calculating the LMMD loss function value between the first noise data and the second noise data;

[0103] S234', determining a joint loss function value according to the LMMD loss function value and the classification loss function value.

[0104] In an embodiment of the present disclosure, a classification loss function value is calculated based on the intermediate recognition result. In some embodiments, the classification loss function value is calculated by a cross entropy loss function.

[0105] Obtain second noise data of the same scene category as the first noise data, and extract a second local feature and its corresponding second time dependency from the second noise data, then use the first local feature and the first time dependency as source domain features, and the second local feature and the second time dependency as target domain features, the source domain features and the target domain features can be used to measure the distribution difference of local features, and calculate the LMMD loss function value between the first noise data and the second noise data through the LMMD loss function. The LMMD loss function value pays more attention to the difference in local features between the first noise data and the second noise data, providing a more refined alignment effect. In some embodiments, the LMMD loss function uses a Gaussian kernel function as a mapping kernel, which can keenly capture the subtle differences between the noise data in the source domain and the noise data in the target domain, assist the preset model to quickly align the feature distribution, and finely measure the difference in feature distribution, thereby achieving a better domain adaptation effect.

[0106] According to the LMMD loss function value and the classification loss function value, the joint loss function value is calculated to jointly optimize the classification performance and cross-domain alignment effect of the preset model. Among them, the source domain data and the target domain data can fully measure the difference in feature distribution under the same scene category. Based on the distribution difference of features between the source domain data and the target domain data, and between sub-domains within the class (that is, between the same scene categories), it can help the model to perform transfer learning, thereby improving the generalization ability and robustness of the model, that is, enhancing the model's adaptability between sub-domains under the same scene category.

[0107] The second noise data is data of the same scene category as the first noise data. In some embodiments, the first noise data is data in a voiceprint library constructed based on noise data of multiple scene categories, and the second noise data is data of the same scene category as the first noise data and collected in an actual environment. For example, the first noise data is bird call data in the voiceprint library, and the second noise data is bird call data collected in actual scenes such as forest environments, social living environments, and road traffic environments.

[0108] It is worth noting that, in the embodiment of the present disclosure, there is no special limitation on the method for collecting the second noise data.

[0109] In some embodiments, the source domain feature is determined according to the first noise data as: Among them, F lstm (X s ) is the source domain feature of the first noise data (i.e., the first local feature and the first time dependency), X s is the source domain data;

[0110] The source domain features are determined according to the second noise data: Among them, F lstm (Xt ) is the target domain feature of the second noise data (i.e., the second local feature and the second time dependency), X t is the second noise data.

[0111] In some embodiments, calculating the LMMD loss function value between the first noise data and the second noise data by using the LMMD loss function includes:

[0112] By formula: Calculate the LMMD loss function value. Among them, is the kernel function used to map source domain features or target domain features to a high-dimensional kernel space, n s is the number of samples of source domain features, n t is the number of samples of target domain features, is the source domain feature, is the target domain feature.

[0113] In some embodiments, step S234' includes:

[0114] By formula: , calculate the joint loss function value. Among them, L task is the classification loss function value, L LMMD is the LMMD loss function value, and λ is a hyperparameter used to control the weights of the classification loss function value and the LMMD loss function value.

[0115] As an example of an embodiment of the present disclosure, refer to Figure 8 , taking the intermediate recognition result as the pre-value label and the data label pre-labeled by the noise data as the true value label, the cross entropy loss function value is calculated. Based on the cross entropy loss function value, the error between the prediction and the true value label can be predicted by the quantization model. Especially in the early stage of training the preset model, when the predicted intermediate recognition result is wrong, the cross entropy loss function value is high, which can drive the preset model to update parameters and improve the prediction accuracy.

[0116] According to the source domain data 1 to L and their corresponding target domain data 1 to L, the LMMD loss function values ​​of the source domain data 1 to L and the target domain data 1 to L are calculated in turn. The LMMD loss function value is a quantification of the difference in feature distribution between the source domain data and the target domain data, which can help the preset model in training to perform transfer learning between different environmental noise data sets, and help enhance the cross-scene category adaptability of the preset model. The preset model can adapt to various new scenes, and can also improve the generalization ability and robustness of the model in practical applications, and enhance the adaptability in similar or identical scene categories.

[0117] In some embodiments, in step S251', updating the model parameters of the preset model using the joint loss function value includes:

[0118] Update model parameters through the backpropagation algorithm.

[0119] Specifically, the total loss value is minimized by performing gradient calculation on the value of the joint loss function and using an optimization algorithm to update the model parameters. The disclosed embodiment does not impose any special restrictions on the optimization algorithm, which may be an Adam (Adaptive Moment Estimation) algorithm or an SGD (Stochastic Gradient Descent) algorithm.

[0120] In some embodiments, by the formula: Minimize the loss value. Among them, θ is the model parameter, η is the learning rate, is the gradient of the joint loss function with respect to the parameters.

[0121] In some embodiments, the model training process also includes: using a test data set to evaluate the generalization ability of the current preset model, thereby dynamically adjusting model parameters and hyperparameters to optimize the performance of the model.

[0122] Furthermore, in some embodiments, the recognition result of the test data set is saved in a CSV file format for subsequent analysis or use.

[0123] In some embodiments, when the value of the joint loss function tends to be stable, it is determined that the convergence condition is reached, that is, the performance of the preset model is no longer significantly improved. At the same time, the model parameters of the current preset model are saved, and when the noise recognition model is applied to actual tasks, the noise recognition model is deployed with the model parameters to process the noise data to be recognized.

[0124] In the above embodiments of the present disclosure, in order to address the problems existing in current audio recognition and classification methods, a method is provided that can not only extract local features from noise data, but also capture the time dependency corresponding to the local features. The time dependency can reflect the correlation between local features at different moments in the noise data in the time dimension. Based on the local features and their time dependency, the noise data can be accurately identified. The model is not only applicable to the identification of noise data in complex sound environments, but also can capture the complex features in the noise data, effectively deal with the environmental interference of the noise data, and the model is highly robust in practical applications.

[0125] Moreover, the scene categories of the noise data used in training the model are diverse. Through multi-feature fusion training, the significance of the potential relationship between local features can be effectively improved, thereby enhancing the trained model's ability to learn implicit representations and its adaptability to different scene categories.

[0126] In addition, the feature extraction layer and feature enhancement layer can simplify the model structure, reduce redundant calculations, and further analyze the distribution differences between data. The distribution differences between the source domain and the target domain can be calculated using the joint loss function value. This enables the preset model to fully learn the commonality of data feature distribution between the same scene categories, strengthen the data feature distribution based on the maximum mean difference between data in the kernel Hilbert mapping space, readjust the intra-class features and divide them into different sub-fields, and reduce the impact of feature distribution differences on model recognition.

[0127] In a second aspect, the embodiment of the present disclosure provides a high-precision environmental noise identification device 900, which includes:

[0128] An acquisition module 901 is used to acquire noise data to be identified;

[0129] The identification module 902 is used to input the noise data to be identified into a preset noise identification model for identification processing to obtain an identification result, wherein the noise identification model is used to extract the target local features and the target time dependency corresponding to the target local features from the noise data to be identified, and identify the noise data to be identified based on the target local features and the target time dependency, and the target time dependency reflects the correlation between the target local features at different times in the noise data to be identified in the time dimension.

[0130] In a third aspect, the present disclosure provides an electronic device, referring to Fig.10 , which includes a memory 1002 and a processor 1001; the memory 1002 stores a computer program that can be executed by the processor 1001, and when the computer program is executed by the processor 1001, the high-precision environmental noise recognition method described in the first aspect is implemented.

[0131] The processor 1001 and the memory 1002 are connected via one or more I / O interfaces 1003 , and the interface 1003 is configured to implement information interaction between the processor 1001 and the memory 1002 .

[0132] Among them, the processor 1001 is a device with data processing capabilities, including but not limited to a central processing unit (CPU), etc.; the memory 1002 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 1003 is connected between the processor 1001 and the memory 1002, and can realize information interaction between the processor 1001 and the memory 1002, including but not limited to a data bus (Bus), etc.

[0133] In order to enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of the present disclosure, the technical solutions provided by the embodiments of the present disclosure are described in detail below through specific embodiments:

[0134] As a specific form of the embodiment of the present disclosure, refer to Fig.11 , the model training process for high-precision environmental noise recognition includes the following steps 1101 to 1106.

[0135] In step 1101, noise data of different scene categories are obtained, and the noise data of each scene category is classified and labeled.

[0136] In step 1102, data preprocessing is performed on the noise data.

[0137] In step 1103, for each noise data of each scene category, the feature vector corresponding to the noise data is extracted, and a training data set and a test data set are divided. For each noise data in the training data set, the noise data is input into a preset model for processing to obtain an intermediate recognition result. In the process of processing the noise data by the preset model, the local features and time dependency of the noise data are extracted.

[0138] In step 1104, the classification loss function value corresponding to the recognition result of each intermediate noise data is calculated based on the intermediate recognition result and the preset data label, and the noise data is used as the source domain data, and the noise data of the same scene category as the noise data is obtained as the target domain data to calculate the LMMD loss function value. The joint loss function value is calculated based on the classification loss function value and the LMMD loss function value.

[0139] In step 1105, model parameters in the preset model are updated by a computer optimizer according to the value of the joint loss function.

[0140] In step 1106, the generalization ability of the preset model is evaluated using the test data set.

[0141] The noise recognition model trained by the above embodiment can more accurately recognize various types of noise data, and the model has good generalization ability and robustness in actual application scenarios.

[0142] Among them, the processor is a device with data processing capabilities, including but not limited to the central processing unit (CPU); the memory is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) is connected between the processor and the memory, and can realize information exchange between the memory and the processor, including but not limited to the data bus (Bus), etc.

[0143] Those skilled in the art will appreciate that all or some of the steps, systems, and functional modules / units in the apparatus disclosed above may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0144] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation.

[0145] Some or all physical components may be implemented as software executed by a processor, such as a central processing unit (CPU), a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is well known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH) or other disk storage; read-only compact disk (CD-ROM), digital versatile disk (DVD) or other optical disk storage; magnetic cassettes, tapes, disk storage or other magnetic storage; any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0146] The present disclosure has disclosed example embodiments, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly stated, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A high-precision environmental noise recognition method, characterized in that: It includes: Acquire noise data to be identified; The noise data to be identified is input into a preset noise identification model for identification processing to obtain an identification result, wherein the noise identification model is used to extract target local features and target time dependencies corresponding to the target local features from the noise data to be identified, and the noise data to be identified is identified according to the target local features and the target time dependencies, wherein the target time dependencies reflect the correlation between the target local features at different moments in the noise data to be identified in the time dimension; The training process of the noise recognition model at least includes: Acquire a training data set; the training data set includes feature vectors corresponding to noise data of multiple scene categories; Determine a classification loss function value for each intermediate recognition result corresponding to each first noise data in the training data set; obtain second noise data of the same scene category as the first noise data, and extract a second local feature and its corresponding second time dependency from the second noise data; use the first local feature and the first time dependency of the first noise data as source domain features, and use the second local feature and the second time dependency as target domain features, and calculate a local maximum mean difference (LMMD) loss function value between the first noise data and the second noise data; determine a joint loss function value based on the LMMD loss function value and the classification loss function value; The first noise data is data that has not been selected from the noise data in the training data set, and the intermediate recognition result is obtained by inputting a feature vector corresponding to the first noise data into a preset model for processing.

2. The method according to claim 1, characterized in that The training process of the noise recognition model also includes: Determine whether the joint loss function value reaches a convergence condition; In the case that the value of the joint loss function does not reach the convergence condition, the model parameters of the preset model are updated using the value of the joint loss function, and the step of selecting unselected first noise data from each noise data in the training data set, inputting the first noise data into the preset model for processing, and obtaining an intermediate recognition result is performed; When the value of the joint loss function reaches a convergence condition, the preset model is determined as the trained noise recognition model.

3. The method according to claim 2, characterized in that The obtaining of the training data set comprises: Obtain noise data for different scene categories; For the noise data of each scene category, extracting a feature vector corresponding to the noise data; The feature vectors corresponding to the noise data of each scene category are divided according to a preset ratio to obtain the training data set.

4. The method according to claim 3, characterized in that The extracting the feature vector corresponding to the noise data includes: Divide the noise data into frames to obtain a plurality of noise data subframes; Performing fast Fourier transform processing on the multiple noise data subframes to obtain spectrum data; The frequency spectrum data is subjected to frequency transformation to obtain a feature vector; wherein the feature vector includes: Mel frequency cepstral coefficients MFCC, chromaticity features, Mel spectrum and spectrum contrast, so as to constitute at least one item of the multi-dimensional features.

5. The method according to claim 2, characterized in that: The step of inputting the feature vector corresponding to the first noise data into a preset model for processing to obtain an intermediate recognition result includes: Extracting a first local feature and a first time dependency relationship corresponding to the first noise data from a feature vector corresponding to the first noise data; An intermediate recognition result is determined according to the first local feature and the first time dependency.

6. The method according to claim 5, characterized in that The extracting the first local feature and its corresponding first time dependency from the feature vector corresponding to the first noise data includes: Inputting the feature vector corresponding to the first noise data into the feature extraction layer for processing to obtain a first local feature; The first local feature is input into a feature enhancement layer for processing to obtain a first time dependency relationship corresponding to the first local feature.

7. The method according to claim 1, characterized in that The calculating the local maximum mean difference (LMMD) loss function value between the first noise data and the second noise data comprises: By formula: , calculate the LMMD loss function value; in, is the LMMD loss function value, is a kernel function for mapping the source domain features and the target domain features to a high-dimensional kernel space, is the number of samples of source domain features, is the number of samples of target domain features, is the source domain feature, is the target domain feature.

8. A high-precision environmental noise recognition device, characterized in that: It includes: An acquisition module, used for acquiring noise data to be identified; an identification module, used for inputting the noise data to be identified into a preset noise identification model for identification processing to obtain an identification result, wherein the noise identification model is used for extracting target local features and target time dependencies corresponding to the target local features from the noise data to be identified, identifying the noise data to be identified according to the target local features and the target time dependencies, wherein the target time dependencies reflect the correlation between the target local features at different moments in the noise data to be identified in the time dimension; The training process of the noise recognition model at least includes: Acquire a training data set; the training data set includes feature vectors corresponding to noise data of multiple scene categories; For each first noise data in the training data set, determine a classification loss function value according to an intermediate recognition result corresponding to the first noise data; obtain second noise data of the same scene category as the first noise data, and extract a second local feature and its corresponding second time dependency from the second noise data; use the first local feature and the first time dependency of the first noise data as source domain features, and use the second local feature and the second time dependency as target domain features, and calculate a local maximum mean difference (LMMD) loss function value between the first noise data and the second noise data; determine a joint loss function value according to the LMMD loss function value and the classification loss function value; The first noise data is data that has not been selected from the noise data in the training data set, and the intermediate recognition result is obtained by inputting a feature vector corresponding to the first noise data into a preset model for processing.

9. An electronic device, characterized in that: The method comprises a memory and a processor; the memory stores a computer program executable by the processor, and the computer program, when executed by the processor, implements the method described in any one of claims 1 to 7.

Images