Method and system for smart recognition of environmental noise class, and storage medium

Abstract

Disclosed in the present invention are a method and system for smart recognition of an environmental noise class, and a storage medium. The method comprises: classifying environmental noise into a plurality of environmental noise classes; acquiring sound files for respective environmental noise classes, and manually applying noise class labels; preprocessing the sound files on which the noise class labels have been applied, to obtain a plurality of preprocessed sound segments; performing feature extraction on the preprocessed sound segments, to obtain sound segment features; constructing a noise class recognition model on the basis of the environmental noise classes and the sound segment features; and inputting the sound segment features into the noise class recognition model for model training, and using the trained noise class recognition model to recognize a noise type of a sound segment to be recognized and a confidence corresponding to the noise type. The present invention perfects existing environmental noise monitoring indexes and can provide a basis for noise management.

Description

Methods, systems, and storage media for intelligent identification of environmental noise categories Technical Field

[0001] This invention belongs to the field of environmental noise treatment technology, specifically relating to a method, system, and storage medium for intelligently identifying environmental noise categories. Background Technology

[0002] Noise monitoring is a crucial aspect of social governance. The comprehensiveness of monitoring indicators is essential for effective noise control. Currently, noise monitoring only measures the equivalent sound pressure level (SPL) and simply concludes that noise exceeds the prescribed value. This leads to problems such as: even if a location's noise level is simply due to natural elements like wind, rain, or birdsong exceeding the prescribed decibel level, it faces the risk of being deemed to be exceeding noise limits and even being penalized. Therefore, in practice, defining and automatically identifying environmental noise categories, and providing identification of major noise types in addition to equivalent sound pressure level, is highly significant.

[0003] Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method, system, and storage medium for intelligently identifying environmental noise categories, aiming to improve current environmental noise monitoring indicators and provide a basis for noise control.

[0005] According to a first aspect of the present disclosure, a method for intelligently identifying environmental noise categories is provided, the method comprising the following steps:

[0006] Environmental noise is classified into multiple environmental noise categories;

[0007] For each environmental noise category, a sound file was collected and manually labeled with the noise category.

[0008] Preprocess each audio file that has been labeled with a noise category to obtain several preprocessed audio segments;

[0009] Feature extraction is performed on each preprocessed audio segment to obtain the features of each audio segment;

[0010] A noise category identification model is constructed based on the environmental noise category and the sound segment features;

[0011] The features of each sound segment are input into the noise category recognition model for model training, and the trained noise category recognition model is used to identify the noise type of the sound segment to be identified and its corresponding confidence level.

[0012] A further technical feature of this invention is that the preprocessing of each audio file already labeled with a noise category tag includes:

[0013] The audio file is divided into several audio segments based on a specified duration.

[0014] Assign a unique filename to each of the aforementioned sound segments.

[0015] The aforementioned sound segments are then generalized.

[0016] A further technical feature of the present invention is that the sound file is divided into several sound segments based on a specified duration. The process includes: using a sliding window to sample the sound file in real time to obtain sound segments with a duration of t seconds, and the two sound segments before and after sampling have an overlap of s seconds, where s < t.

[0017] A further technical feature of this invention is that the noise category recognition model identifies the noise type of each sound segment and its corresponding confidence level. When the confidence level is greater than a specified threshold, the corresponding noise type will be used as the final output result. The specified threshold is set by plotting the noise category recognition accuracy under different thresholds based on the verification dataset, and the threshold corresponding to the highest accuracy is determined as the specified threshold.

[0018] A further technical feature of this invention is: feature extraction is performed on each preprocessed sound segment to obtain the features of each sound segment, the process including:

[0019] Extract the M x N Mel spectrogram matrix for each sound segment;

[0020] The original time-domain signal of each sound segment is scaled into a time-domain signal vector containing N data points. The time-domain signal vector is then merged with the Mel spectrogram matrix to form a mixed signal matrix with M+1 rows and N columns. The time-domain signal vector is the first row of the mixed signal matrix.

[0021] Based on the signals contained in the hybrid signal matrix, sound segment features that are discriminative for different sound types are extracted.

[0022] A further technical feature of this invention is that: for multiple environmental noise categories, a noise category identification model is constructed respectively, specifically including:

[0023] The M+1 row and N column mixed signal matrix is ​​converted into a row vector containing (M+1)*N elements;

[0024] The R sound segments used for training are arranged into an R-row, (M+1)*N-column matrix P;

[0025] Select the sound segments containing a certain category from R sound segments, and set the category label to a two-dimensional row vector with element values ​​of 1 and 0. For the sound segments that do not contain a certain category, set their category label to a two-dimensional row vector with element values ​​of 0 and 1. Finally, construct an R-row, 2-column matrix Q from the category label vectors of the R sound segments.

[0026] Let P and Q have a relationship Py = Q, where y represents the unknown. Then, using y = (P... T P+τI) -1 P T Q finds an approximate value for y, where P T τ represents the transpose of matrix P, τ represents a small positive number, and I represents the identity matrix;

[0027] The process of finding an approximate value for y is the training process of the noise category recognition model.

[0028] Before inputting the sound segment to be identified into each of the noise category recognition models that have been trained, the sound segment to be identified is first converted into a row vector g containing (M+1)*N elements;

[0029] Using g as the input to each of the noise category recognition models, based on gy i =h i The relationship is used to calculate the corresponding model output h. i ;

[0030] Based on h i Given a row vector containing two elements, compare h. i The order of the two elements is determined by the size of the first element. If the first element is larger and its corresponding confidence score is greater than a specified threshold, then the current audio segment belongs to the i-th category, where the confidence score is defined as h. i The ratio of the absolute values ​​of the first element to the second element.

[0031] A further technical feature of this invention is: data augmentation of multi-labeled sound segments based on a data synthesis method for training the noise category recognition model, specifically including any one or more of the following methods a and b:

[0032] a. Data synthesis of two or three distinct sound types;

[0033] b. Combine data from one salient sound type with one insignificant sound type;

[0034] In the latter, the sound segments synthesized in methods a and b are categorized as the significant sound types used for synthesis; the significant sound types refer to sound types that are clearly distinguishable by normal human ears, and the non-significant sound types refer to sound types that are not distinguishable by normal human ears.

[0035] According to a second aspect of the present disclosure, a system for intelligently identifying environmental noise categories is provided, the system comprising:

[0036] The environmental noise category acquisition module is used to classify environmental noise into multiple environmental noise categories;

[0037] The category label acquisition module is used to collect a sound file for each environmental noise category and manually label the noise category.

[0038] The audio file preprocessing module is used to preprocess each audio file that has been labeled with a noise category to obtain several preprocessed audio segments.

[0039] The sound segment feature acquisition module is used to extract features from each preprocessed sound segment and obtain the features of each sound segment.

[0040] A noise category recognition model building module is used to build a noise category recognition model based on the environmental noise category and the sound segment features;

[0041] The sound segment recognition module is used to input the features of each sound segment into the noise category recognition model for model training, and to use the trained noise category recognition model to identify the noise type of the sound segment to be identified and its corresponding confidence level.

[0042] According to a third aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method for intelligently identifying environmental noise categories.

[0043] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, the storage medium storing computer instructions that, when executed by a processor, implement the steps of the above-described method for intelligently identifying environmental noise categories.

[0044] This disclosure provides a method, system, and storage medium for intelligently identifying environmental noise categories. It defines the main categories of environmental noise and achieves automatic identification of these categories. This invention improves current environmental noise monitoring indicators and provides a basis for noise control.

[0045] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0047] Figure 1 is a schematic flowchart of the method for intelligently identifying environmental noise categories in an embodiment of the present invention;

[0048] Figure 2 is a schematic flowchart of the audio file preprocessing method in an embodiment of the present invention;

[0049] Figure 3 is a schematic diagram of the feature extraction method for sound segments in an embodiment of the present invention;

[0050] Figure 4 is a schematic diagram of the system structure for intelligent identification of environmental noise categories in an embodiment of the present invention;

[0051] Figure 5 is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0052] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present invention are shown in the drawings, not the entire structure.

[0053] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0054] The present invention provides the following embodiments for a method, system, and storage medium for intelligently identifying environmental noise categories:

[0055] Example 1 illustrates a method for intelligently identifying environmental noise categories. Referring to Figure 1, which is a flowchart of the method, the specific steps include:

[0056] S1. Classify environmental noise into multiple environmental noise categories;

[0057] S2. For each type of environmental noise, a sound file is collected and the noise category is manually labeled.

[0058] S3. Preprocess each audio file that has been labeled with a noise category to obtain several preprocessed audio segments.

[0059] S4. Extract features from each preprocessed audio segment to obtain the features of each audio segment;

[0060] S5. Construct a noise category recognition model based on the environmental noise category and the sound segment features;

[0061] S6. Input the features of each sound segment into the noise category recognition model for model training, and use the trained noise category recognition model to identify the noise type of the sound segment to be identified and its corresponding confidence level.

[0062] In the specific implementation of S1 and S2, the main environmental noises are divided into the following categories: chainsaw noise, pile driving noise, (large machinery) excavation noise, airplane noise, engine noise (including engine noise from motorcycles, tractors, large machinery, etc.), other mechanical roaring noise, high-speed rail noise, vehicle horn noise, traffic noise, car horn noise, air raid siren noise, Chinese speaking noise, non-Chinese speaking noise, music noise (including instrumental music), music and singing, a cappella singing, wind noise, birdsong, rooster crowing, dog barking, duck quacking, goose quacking, cat meowing, frog croaking, insect chirping (including cicadas, crickets, etc.), other animal calls, rain sound, stream sound, ocean wave sound, and storm sound. The implementation example divides these into a total of 30 categories. The above definitions are consistent with the environment of major cities, towns, and rural areas. In the practical application stage, for each input sound signal, the output result is one of the above 30 noise categories or an "unknown" category. Furthermore, each of the 30 noise categories belongs to one of the four major noise categories: industrial production noise, construction noise, transportation noise, and social noise (these four categories are referred to as major noise categories in this method). Referring to the specific content of the "Noise Pollution Prevention and Control Law of the People's Republic of China," the system simultaneously outputs the major noise category to which each specific noise category belongs.

[0063] The preprocessing of each audio file labeled with a noise category, as described in S3, is shown in Figure 2, and includes:

[0064] S31. Divide the audio file into several audio segments according to the specified duration;

[0065] S32. Assign a unique filename to each of the aforementioned sound segments.

[0066] S33. Perform generalization processing on each of the aforementioned sound segments.

[0067] Specifically, in this embodiment of the method, automatic environmental noise identification is achieved using machine learning. In a specific example, for each environmental noise category, at least 20 audio files of at least 40 seconds in length are collected, and noise category labels are manually provided. In practice, the collected audio clips may contain only a single noise category or multiple noise categories; the manually provided noise category labels must cover the noise types that are discernible to the human ear.

[0068] In the data preprocessing step, for each audio file, it is segmented into audio segments with a duration of t seconds, and an audio segment containing multiple noise categories is called a "mixed sound" segment. For each audio segment obtained by segmentation, its file name contains its category label and serial number and the file name is unique (the file name of the "mixed sound" segment needs to contain all the category labels contained in the segment). In the actual audio acquisition process, it is not only affected by electromagnetic signals, but also due to differences in devices of different sound cards, the acquired audio signals also have slight differences. Considering these factors, through the following generalization processing of the original audio segment, and using the processed audio segment as a new audio segment with the same label as the original audio segment: A weak white noise is added to each original audio segment. The specific operation is to first transform each original audio segment into the frequency domain, then superimpose the weak white noise on its frequency domain signal and use the superimposed result as the new audio segment (here, the weak white noise means that the energy of the superimposed white noise is less than one-tenth of the energy of the original audio segment).

[0069] It should be noted that in addition to giving the main noise types in the audio segment, the noise category recognition model also gives the confidence levels corresponding to the noise types.

[0070] In S31, based on the specified duration as the standard, the audio file is segmented into several audio segments. The process includes: using a sliding window to perform real-time sampling on the audio file to obtain an audio segment with a duration of t seconds, and there is an overlap of s seconds between the two audio segments before and after sampling, where s < t.

[0071] Specifically, the environmental noise automatic recognition method needs to have high accuracy and good stability. First, use the sound card to perform real-time audio acquisition, and then, in a partially overlapping manner, extract features from each audio segment of t seconds; the specific method is to use a sliding window to perform real-time sampling on the audio to obtain an audio segment with a duration of t seconds (referred to as an audio segment), and make the two audio segments before and after sampling have an overlap of s seconds (s < t). The audio sampling with partially overlapping data is conducive to the more complete utilization of the audio signal. After such audio sampling, the data obtained by using the audio signal mixing scheme described later is used as the feature of each audio segment. Then, use the trained noise category recognition model to identify the noise category. For each noise type recognized by the model, the system reads its corresponding confidence level, and only when the confidence level is greater than the specified threshold, the noise type is finally output to the display screen of the system. The threshold as the optimal confidence level boundary is set according to the experiment. In practice, based on the prepared validation data set (the validation data set contains a total of no less than 50 different audio segments, and the noise category in each audio segment is known), plot the accuracy of the noise type discrimination obtained by the system under different "thresholds", and use the threshold corresponding to the highest accuracy as the final threshold of the system.

[0072] The accuracy of noise type identification is measured using the following two metrics:

[0073] In the above formula, t ij represents the number of sound categories (based on human ear discrimination) in the j-th sound segment of the i-th sound file, while s ij This represents the number of correctly identified sound categories in the j-th sound segment of the i-th sound file. i p represents the total number of audio segments in the i-th audio file. ij This indicates whether the sound category contained in the j-th sound segment of the i-th sound file has been completely and accurately identified. If yes, its value is 1; otherwise, its value is 0. For example, if a sound segment contains sounds of daily life, birdsong, and frog croaks, the corresponding value p is only set when all three sounds are identified. ij The value is 1 only if the condition is met; otherwise, it is 0. acc1 is called the overall discrimination accuracy, and acc2 is called the complete discrimination accuracy. Clearly, acc2... <acc1。

[0074] Furthermore, the noise category recognition model identifies the noise type of each sound segment and its corresponding confidence level. When the confidence level is greater than a specified threshold, the corresponding noise type will be used as the final output result. The specified threshold is set by plotting the noise category recognition accuracy under different thresholds based on the validation dataset, and the threshold corresponding to the highest accuracy is determined as the specified threshold.

[0075] In S4, feature extraction is performed on each preprocessed audio segment to obtain the features of each audio segment. The process is shown in Figure 3, including:

[0076] S41. Extract the M-row N-column Mel spectrogram matrix for each sound segment;

[0077] S42. Scale the original time-domain signal of each sound segment into a time-domain signal vector containing N data points, and merge the time-domain signal vector with the Mel spectrogram matrix into a mixed signal matrix with M+1 rows and N columns, wherein the time-domain signal vector is the first row of the mixed signal matrix;

[0078] S43. Based on the signals contained in the mixed signal matrix, extract the sound segment features that are discriminative for different sound types.

[0079] Specifically, before noise category identification, feature extraction of sound segments is required, both during model training and application. The feature extraction process employs the following sound signal profanity scheme:

[0080] First, extract the Mel spectrogram matrix for each short sound segment. Let the Mel spectrogram matrix be an M x N matrix. Scale the original time-domain signal of each short sound segment into an N-dimensional vector containing N data points (referred to as the time-domain signal vector), and merge it with the corresponding Mel spectrogram matrix into an M+1 x N matrix (referred to as the mixed signal matrix), with the time-domain signal vector as the first row of this matrix. See Figure 3 for the corresponding illustration.

[0081] The advantage of the above sound signal mixing scheme is that the mixed signal matrix not only contains the sound signal in a statistical sense, such as frequency, but also the time-varying signal of sound intensity, which can better extract the sound characterization features that are discriminative for different sound types.

[0082] In S5, a noise category recognition model is constructed based on the environmental noise category and the sound segment features. Specifically, for multiple environmental noise categories, a separate noise category recognition model is constructed, including:

[0083] The M+1 row and N column mixed signal matrix is ​​converted into a row vector containing (M+1)*N elements;

[0084] The R sound segments used for training are arranged into an R-row, (M+1)*N-column matrix P;

[0085] Select the sound segments containing a certain category from R sound segments, and set the category label to a two-dimensional row vector with element values ​​of 1 and 0. For the sound segments that do not contain a certain category, set their category label to a two-dimensional row vector with element values ​​of 0 and 1. Finally, construct an R-row, 2-column matrix Q from the category label vectors of the R sound segments.

[0086] Let P and Q have a relationship Py = Q, where y represents the unknown. Then, using y = (P... T P+τI) -1 P T Q finds an approximate value for y, where P T τ represents the transpose of matrix P, τ represents a small positive number, and I represents the identity matrix;

[0087] The process of finding an approximate value for y is the training process of the noise category recognition model.

[0088] Before inputting the sound segment to be identified into each of the noise category recognition models that have been trained, the sound segment to be identified is first converted into a row vector g containing (M+1)*N elements;

[0089] Using g as the input to each of the noise category recognition models, based on gy i =h i The relationship is used to calculate the corresponding model output h. i ;

[0090] Based on h i Given a row vector containing two elements, compare h. i The order of the two elements is determined by the size of the first element. If the first element is larger and its corresponding confidence score is greater than a specified threshold, then the current audio segment belongs to the i-th category, where the confidence score is defined as h. i The ratio of the absolute values ​​of the first element to the second element.

[0091] For the recognition of the 30 categories in this embodiment, a recognition model is constructed for each category. The features of each short sound segment are an M+1 row N column matrix. For ease of representation, it is converted into a row vector containing (M+1) x N elements. Assuming there are R short sound segments used for training, they constitute an R row (M+1) x N column matrix, denoted as P.

[0092] The steps to construct a category-specific recognition model are as follows: First, select all short-time sound segments belonging to the specified category from all R short-time sound segments (for short-time sound segments containing multiple categories, they belong to the specified category as long as they contain the current category), and set their category labels as two-dimensional row vectors with element values ​​of 1 and 0 respectively; then, set the category labels of all other short-time sound segments as two-dimensional row vectors with element values ​​of 0 and 1 respectively. Clearly, the category label vectors of all R short-time sound segments form an R-row, 2-column matrix, denoted as Q. Assume that there is a relationship Py = Q between P and Q (y is unknown). Then, using y = (Py = Qy), we can further refine the model. T P+τI) -1 P T Q finds an approximate value for y, where P T Let represent the transpose of matrix P, τ represent a small positive number, and I represent the identity matrix. The process of finding an approximate value for y is also called the training process of the recognition model. The training process for the recognition model for each category is as described above. Therefore, there are 30 recognition models for 30 categories, and different recognition models correspond to different y. Let the y values ​​of the first to the 30th recognition models be y1, ..., y2, respectively. 30 .

[0093] The process of identifying the sound category of a short-term sound segment using a recognition model is as follows: For a specific short-term sound segment in an input system, first extract its (M+1)xN row vector features, denoted as g. Then, g is used as the input to 30 recognition models, and the corresponding model outputs are calculated sequentially. For example, for the i-th recognition model, the product of g and matrix yi is calculated, and the product result is hi, which is the model output. Obviously, hi is a row vector containing two elements. By comparison, the size of the two elements in hi is determined. If the first element is larger and the corresponding confidence score is greater than a specified threshold, then the current short-term sound segment is considered to belong to the i-th category. The confidence score is defined as the ratio of the absolute values ​​of the first and second elements in hi. The specific value of the threshold is determined during the training process of the recognition model, based on achieving the highest recognition accuracy for all short-term sound segments used for training.

[0094] Data augmentation of multi-labeled audio segments is performed using data synthesis methods for training the noise category recognition model, specifically including any one or more of the following methods a and b:

[0095] a. Data synthesis of two or three distinct sound types;

[0096] b. Combine data from one salient sound type with one insignificant sound type;

[0097] In the latter, the sound segments synthesized in methods a and b are categorized as the significant sound types used for synthesis; the significant sound types refer to sound types that are clearly distinguishable by normal human ears, and the non-significant sound types refer to sound types that are not distinguishable by normal human ears.

[0098] Specifically, classifying short-term sound segments with multiple labels (i.e., short-term sound segments containing various environmental noises) is a challenging problem. Data augmentation is an effective means to improve model robustness in addressing this issue. The quantity and diversity of short-term sound segments with multiple labels collected in datasets are insufficient for model training. Therefore, this invention proposes two effective data augmentation schemes based on data synthesis: Scheme 1 synthesizes data of two and three salient sound types; Scheme 2 synthesizes data of one salient sound type and one insignificant sound type. A salient sound type refers to a sound type that is clearly distinguishable to the normal human ear; an insignificant sound type refers to a sound type that is difficult for the normal human ear to distinguish. The category labeling result of the sound segment synthesized by Scheme 2 is the salient sound type used for synthesis. The category labeling result of the synthesized sound segment using Scheme 1 is all the significant sound types used for synthesis. In other words, if two different significant sound types are used to synthesize the data, the category labeling result of the synthesized sound segment will be these two significant sound types (i.e., the synthesized sound segment has 2 category labels). If three different significant sound types are used to synthesize the data, the category labeling result of the synthesized sound segment will be these three significant sound types (i.e., the synthesized sound segment has 3 category labels).

[0099] Example 2 illustrates a system for intelligently identifying environmental noise categories. Referring to Figure 4, the system 400 includes:

[0100] The environmental noise category acquisition module 410 is used to classify environmental noise into multiple environmental noise categories.

[0101] The category label acquisition module 420 is used to collect a sound file for each environmental noise category and manually label the noise category.

[0102] The audio file preprocessing module 430 is used to preprocess each audio file that has been labeled with a noise category to obtain several preprocessed audio segments.

[0103] The sound segment feature acquisition module 440 is used to extract features from each preprocessed sound segment and acquire the features of each sound segment.

[0104] The noise category recognition model building module 450 is used to build a noise category recognition model based on the environmental noise category and the sound segment features;

[0105] The sound segment recognition module 460 is used to input the features of each sound segment into the noise category recognition model for model training, and use the trained noise category recognition model to identify the noise type of the sound segment to be identified and its corresponding confidence level.

[0106] In addition to the above module, system 400 may also include other components; however, since these components are not relevant to the content of this disclosure, their illustrations and descriptions are omitted here.

[0107] Other specific working processes of the system 400 for intelligently identifying environmental noise categories are described in the above-described embodiment of the method for intelligently identifying environmental noise categories, and will not be repeated here.

[0108] Another embodiment illustrates that the system of the present invention can also be implemented using the architecture of the computing device shown in FIG. 5. FIG. 5 shows the architecture of the computing device. As shown in FIG. 5, the system includes a computer system 510, a system bus 530, one or more CPUs 540, input / output 520, a memory 550, etc. The memory 550 can store various data or files used by the computer for processing and / or communication, as well as program instructions executed by the CPU, including the method for intelligently identifying environmental noise categories in the embodiments. The architecture shown in FIG. 5 is merely exemplary, and one or more components in FIG. 5 may be adjusted according to actual needs when implementing different devices. The memory 550, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for intelligently identifying environmental noise categories in the embodiments of the present invention (e.g., the environmental noise category acquisition module 410, category label acquisition module 420, sound file preprocessing module 430, sound segment feature acquisition module 440, noise category recognition model construction module 450, and sound segment recognition module 460 in the system 400 for intelligently identifying environmental noise categories). One or more CPUs 540 execute various functional applications and data processing of the system of the present invention by running software programs, instructions, and modules stored in memory 550, that is, to realize the above-mentioned method for intelligent identification of environmental noise categories, the method comprising the following steps:

[0109] Environmental noise is classified into multiple environmental noise categories;

[0110] For each environmental noise category, a sound file was collected and manually labeled with the noise category.

[0111] Preprocess each audio file that has been labeled with a noise category to obtain several preprocessed audio segments;

[0112] Feature extraction is performed on each preprocessed audio segment to obtain the features of each audio segment;

[0113] A noise category identification model is constructed based on the environmental noise category and the sound segment features;

[0114] The features of each sound segment are input into the noise category recognition model for model training, and the trained noise category recognition model is used to identify the noise type of the sound segment to be identified and its corresponding confidence level.

[0115] Of course, the processor of the server provided in the embodiments of the present invention is not limited to performing the method operations described above, but can also perform related operations in the method for intelligently identifying environmental noise categories provided in any embodiment of the present invention.

[0116] The memory 550 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 550 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 550 may further include memory remotely configured relative to one or more CPUs 540, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0117] Input / output 520 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Input / output 520 may also include a display device such as a display screen.

[0118] This invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements the method for intelligently identifying environmental noise categories described in the above embodiments. The computer-readable storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be—but is not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0119] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0120] The program code contained on the storage medium can be transmitted using any suitable medium, including—but not limited to—wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0121] Furthermore, other specific operating procedures of a non-transitory computer-readable storage medium are described in the above-described embodiments of the method for intelligently identifying environmental noise categories, and will not be repeated here.

[0122] Based on the technical solutions provided in the above embodiments, a method, system, and storage medium for intelligently identifying environmental noise categories are disclosed. These methods define the main categories of environmental noise and achieve automatic identification of environmental noise categories. This invention improves current environmental noise monitoring indicators and provides a basis for noise control and management.

[0123] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a step or method that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such a step or method.

[0124] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for intelligently identifying environmental noise categories, characterized in that, The method includes the following steps: Environmental noise is classified into multiple environmental noise categories; For each environmental noise category, a sound file was collected and manually labeled with the noise category. Preprocess each audio file that has been labeled with a noise category to obtain several preprocessed audio segments; Feature extraction is performed on each preprocessed audio segment to obtain the features of each audio segment; A noise category identification model is constructed based on the environmental noise category and the sound segment features; The features of each sound segment are input into the noise category recognition model for model training, and the trained noise category recognition model is used to identify the noise type of the sound segment to be identified and its corresponding confidence level. Feature extraction is performed on each preprocessed audio segment to obtain the features of each audio segment. The process includes: Extract the M x N Mel-spectrum matrix of each sound segment; The original time-domain signal of each sound segment is scaled into a time-domain signal vector containing N data points. The time-domain signal vector is then merged with the Mel spectrogram matrix to form a mixed signal matrix with M+1 rows and N columns. The time-domain signal vector is the first row of the mixed signal matrix. Based on the signals contained in the hybrid signal matrix, sound segment features that are discriminative for different sound types are extracted; For multiple environmental noise categories, noise category identification models are constructed respectively, specifically including: The M+1 row and N column mixed signal matrix is ​​converted into a row vector containing (M+1)*N elements; The R sound segments used for training are arranged into an R-row, (M+1)*N-column matrix P; Select the sound segments containing a certain category from R sound segments, and set the category label to a two-dimensional row vector with element values ​​of 1 and 0. For the sound segments that do not contain a certain category, set their category label to a two-dimensional row vector with element values ​​of 0 and 1. Finally, construct an R-row, 2-column matrix Q from the category label vectors of the R sound segments. Let P and Q be related by Py = Q, where y represents an unknown variable. The function of y is to transform the input matrix P of the model into Q; therefore, it is called the transformation vector. Using y = (P... T P+τI) -1 P T Q finds an approximate value for y, where P T Let represent the transpose of matrix P, τ represent a positive number ranging from 0.001 to 0.01, and I represent the identity matrix. The introduction of τ improves the stability of matrix inversion operations. Clearly, each sound category corresponds to a transformation vector. To distinguish them, the transformation vector of the i-th sound category is represented as y. i ; The process of finding an approximate value for y is the training process of the noise category recognition model. Before inputting the sound segment to be identified into each of the noise category recognition models that have been trained, the sound segment to be identified is first converted into a row vector g containing (M+1)*N elements; Using g as the input to each of the noise category recognition models, based on gy i =h i The relationship is used to calculate the corresponding model output h. i ; Based on h i Given a row vector containing two elements, compare h. i The order of the two elements is determined by the size of the first element. If the first element is larger and its corresponding confidence score is greater than a specified threshold, then the current audio segment belongs to the i-th category, where the confidence score is defined as h. i The ratio of the absolute values ​​of the first element to the second element.

2. The method for intelligently identifying environmental noise categories according to claim 1, characterized in that, The preprocessing of each audio file labeled with a noise category includes: The audio file is divided into several audio segments based on a specified duration. Assign a unique filename to each of the aforementioned sound segments. The aforementioned sound segments are then generalized.

3. The method for intelligently identifying environmental noise categories according to claim 2, characterized in that, The audio file is divided into several audio segments based on a specified duration. The process includes: using a sliding window to sample the audio file in real time to obtain audio segments with a duration of t seconds, and the two audio segments before and after sampling have an overlap of s seconds, where s < t.

4. The method for intelligently identifying environmental noise categories according to claim 1, characterized in that, The noise category recognition model identifies the noise type of each sound segment and its corresponding confidence level. When the confidence level is greater than a specified threshold, the corresponding noise type will be used as the final output result. The specified threshold is set by plotting the noise category recognition accuracy under different thresholds based on the validation dataset, and the threshold corresponding to the highest accuracy is determined as the specified threshold.

5. The method for intelligently identifying environmental noise categories according to any one of claims 1-4, characterized in that, Data augmentation is performed on multi-labeled audio segments using data synthesis methods for training the noise category recognition model. Specific methods include any one or more of the following a and b: a. Data synthesis of two or three distinct sound types; b. Combine data from one salient sound type with one insignificant sound type; In this context, the sound segments synthesized in methods a and b are categorized as significant sound types used for synthesis; the significant sound types refer to sound types that are clearly distinguishable by normal human ears, while the non-significant sound types refer to sound types that are not distinguishable by normal human ears.

6. A system for intelligently identifying environmental noise categories, characterized in that, include: The environmental noise category acquisition module is used to classify environmental noise into multiple environmental noise categories; The category label acquisition module is used to collect a sound file for each environmental noise category and manually label the noise category. The audio file preprocessing module is used to preprocess each audio file that has been labeled with a noise category to obtain several preprocessed audio segments. The sound segment feature acquisition module is used to extract features from each preprocessed sound segment and acquire the features of each sound segment. feature; A noise category recognition model building module is used to build a noise category recognition model based on the environmental noise category and the sound segment features; The sound segment recognition module is used to input the features of each sound segment into the noise category recognition model for model training, and use the trained noise category recognition model to identify the noise type of the sound segment to be identified and its corresponding confidence level.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for intelligently identifying environmental noise categories as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, wherein computer instructions are stored on the storage medium, characterized in that, When the instructions are executed by the processor, they implement the steps of the method for intelligently identifying environmental noise categories as described in any one of claims 1 to 5.

Images