Model training method, sound scene classification method and sound scene classification equipment

Abstract

The invention relates to the technical field of deep learning, in particular to a model training method, a sound scene classification method and sound scene classification equipment. The model training method comprises the steps that a sound scene classification data set is determined, and data in the sound scene classification data set comprises multiple pieces of audio data and city information labels and sound scene category labels corresponding to the audio data; training a teacher model based on the sound scene classification data set to obtain a classification soft label and a deep feature corresponding to the audio data; and training a target model based on the sound scene classification data set, the deep features and the classification soft labels, the parameter quantity of the target model being smaller than that of the teacher model. According to the method and the device, the target model has better generalization ability and classification precision in cross-city sound scene classification, and the target model can be ensured to have higher classification precision while the parameter quantity and the calculation complexity of the target model are ensured to be lower.

Description

Technical Field

[0001] This disclosure relates to the field of deep learning technology, specifically to a model training method, a sound scene classification method, and a sound scene classification device. Background Technology

[0002] Sound scene classification technology can automatically identify the corresponding acoustic environment, such as airports, subway stations, and city parks, based on audio signals. Currently, deep learning-based sound scene classification methods are generally used. However, due to differences in recording equipment, physical characteristics, and acquisition environments, device domain bias and city domain bias exist between the training and test sets of deep learning-based sound scene classification methods, resulting in poor model generalization ability. Furthermore, the acoustic characteristics of the same environment vary across different cities, leading to poor accuracy in sound scene classification. The large number of parameters and high computational complexity of deep learning models make it difficult to deploy sound scene classification models on resource-constrained terminals, resulting in poor adaptability to edge devices. Summary of the Invention

[0003] To overcome the problems existing in related technologies, an exemplary embodiment of this disclosure provides a model training method for sound scene classification in a first aspect. The model training method includes: determining a sound scene classification dataset, wherein the data in the sound scene classification dataset includes multiple audio data and a city information label and a sound scene category label corresponding to each audio data, wherein the city information label corresponds to the city where the audio data was collected, and the sound scene category label corresponds to the sound scene where the audio data was collected; training a teacher model based on the sound scene classification dataset to obtain classification soft labels and deep features corresponding to the audio data; and training a target model based on the sound scene classification dataset, the deep features, and the classification soft labels, wherein the number of parameters in the target model is smaller than that in the teacher model.

[0004] In some embodiments, the teacher model includes: multiple sub-models; the step of training the teacher model based on the sound scene classification dataset to obtain the classification soft label and deep features corresponding to the audio data includes: obtaining the classification soft label corresponding to the audio data by training at least one of the sub-models based on the sound scene classification dataset; and obtaining the deep features corresponding to the audio data by training another sub-model based on the audio data and the corresponding sound scene category label in the sound scene classification dataset.

[0005] In some embodiments, obtaining the classification soft label corresponding to the audio data by training at least one of the sub-models based on the sound scene classification dataset includes: training a first sub-model based on the audio data and corresponding city information labels in the sound scene classification dataset; training a second sub-model based on the first sub-model that has been trained, and the audio data and corresponding sound scene category labels in the sound scene classification dataset, and obtaining the classification soft label.

[0006] In some embodiments, the first sub-model includes a first feature extractor for feature extraction and a first classifier for city classification; the step of training the second sub-model based on the trained first sub-model and the audio data and corresponding sound scene category labels in the sound scene classification dataset, and obtaining the classification soft label, includes: extracting a first feature from the audio data in the sound scene classification dataset using the first feature extractor of the trained first sub-model; adjusting the parameters of the second classifier of the second sub-model based on the first feature and the audio data and corresponding sound scene category labels to obtain the trained second sub-model, wherein the second classifier is used for sound scene classification; and obtaining the classification soft label using the trained second sub-model.

[0007] In some embodiments, obtaining deep features corresponding to the audio data by training another sub-model based on the audio data and corresponding audio scene category labels in the audio scene classification dataset includes: training a third sub-model based on the audio data and corresponding audio scene category labels in the audio scene classification dataset, wherein the third sub-model includes a third feature extractor for feature extraction and a third classifier for audio scene classification; and extracting the third features of the audio data in the audio scene classification dataset through the third feature extractor of the trained third sub-model to obtain the deep features.

[0008] In some embodiments, training a target model based on the sound scene classification dataset, the deep features, and the classification soft labels includes: determining a first loss function based on the collected features of the sound scene classification dataset extracted by the target model and the deep features; determining a second loss function based on the classification result of the sound scene classification dataset by the target model and the classification soft labels; determining a third loss function based on the classification result of the sound scene classification dataset by the target model and the sound scene category labels; and determining a hybrid loss function of the target model based on the first loss function, the second loss function, and the third loss function to train the target model.

[0009] In some embodiments, determining the hybrid loss function of the target model based on the first loss function, the second loss function, and the third loss function includes: determining the hybrid loss function of the target model to train the target model based on a weighted combination of the first loss function, the second loss function, and the third loss function, wherein the weight of the first loss function is 1, and the sum of the weights of the second loss function and the third loss function is 1.

[0010] In some embodiments, determining the sound scene classification dataset includes: acquiring multiple audio data; determining the corresponding city information label and the sound scene category label based on each audio data; performing audio feature processing on each audio data to determine the sound scene classification dataset, wherein the audio feature processing includes at least one of the following: short-time Fourier transform, Mel filtering, logarithmic operation, and discrete cosine transform.

[0011] Secondly, this disclosure also provides a sound scene classification method, which includes: inputting the audio to be tested into a target model to determine the sound scene category corresponding to the audio to be tested, wherein the target model is determined according to the model training method described in the first aspect.

[0012] Thirdly, this disclosure also provides a sound scene classification device, including: a model training module for training a target model based on the model training method described in the first aspect; and a sound scene classification module for determining the sound scene category of the audio to be tested based on the trained target model.

[0013] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure.

[0014] According to the model training method provided in this disclosure, city information tags corresponding to audio data can be introduced into the model training process, enabling the target model to have better generalization ability and classification accuracy in cross-city sound scene classification, effectively improving the sound scene discrimination ability and robustness of audio. The model training method also allows the target model to converge more quickly. Furthermore, the target model determined by the model training method provided in this disclosure has fewer parameters, making the model more lightweight. Therefore, while ensuring a low number of parameters and low computational complexity, it is possible to guarantee high classification accuracy for the target model. Consequently, the target model trained according to the model training method provided in this disclosure can be applied to resource-constrained edge devices, exhibiting better universality and ensuring efficient and accurate sound scene classification in resource-limited environments such as mobile terminals or embedded systems. Attached Figure Description

[0015] This disclosure can be better understood by describing exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart illustrating a model training method according to exemplary embodiments of the present disclosure; Figure 2 This is a flowchart illustrating a model training method according to exemplary embodiments of the present disclosure; Figure 3 This is a flowchart illustrating a model training method according to exemplary embodiments of the present disclosure; Figure 4 This is a flowchart illustrating a model training method according to exemplary embodiments of the present disclosure; Figure 5 This is a flowchart illustrating a model training method according to exemplary embodiments of the present disclosure; Figure 6 This is a flowchart illustrating a model training method according to exemplary embodiments of the present disclosure; Figure 7 This is a flowchart illustrating a model training method according to exemplary embodiments of the present disclosure; Figure 8 This is a schematic diagram of the structure of a sound scene classification device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0016] The following describes specific embodiments of the present invention. It should be noted that, in order to provide a concise description, this specification cannot exhaustively describe all features of the actual embodiments. It should be understood that, in the actual implementation of any embodiment, just as in any engineering or design project, various specific decisions are often made to achieve the developer's specific goals and to meet system-related or business-related constraints, and this can change from one embodiment to another. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this invention, some design, manufacturing, or production modifications based on the technical content disclosed herein are merely conventional technical means and should not be construed as insufficient content of this disclosure.

[0017] Unless otherwise defined, the technical or scientific terms used in the claims and description shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in the patent application description and claims of this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. The terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "comprising" or "including" and similar terms mean that the element or object preceding "comprising" or "including" encompasses the element or object listed following "comprising" or "including" and its equivalents, and do not exclude other elements or objects. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, nor are they limited to direct or indirect connections.

[0018] Sound scene classification technology can extract features and classify audio data to determine the corresponding acoustic environment, such as airports, subway stations, and city parks. Current technologies mostly use deep learning models to achieve feature recognition and sound scene classification of audio data. However, the performance of these models heavily relies on the assumption that the training and test data distributions are consistent. In practical applications, differences in recording equipment type, physical characteristics, and acquisition environment can lead to device domain bias and city domain bias between the training and test sets. This makes it difficult for the model to correctly classify the same acoustic environment in different cities, resulting in poor sound scene recognition accuracy and poor generalization ability. Current technologies often treat the differences in the same sound scene across different cities as noise or bias, failing to utilize the acoustic characteristics of different cities to provide further value for sound scene classification. Furthermore, the large number of parameters and computational demands of deep learning models makes them difficult to apply to resource-constrained devices, such as Bluetooth headsets and speakers, thus resulting in poor universality of the target model.

[0019] An exemplary embodiment of this disclosure provides a model training method, such as Figure 1 As shown, the model training method for sound scene classification may include steps S110 to S130.

[0020] Step S110: Determine the sound scene classification dataset. This dataset includes multiple audio data points and corresponding city information labels and sound scene category labels for each audio data point. The city information labels correspond to the cities where the audio data was collected, and the sound scene category labels correspond to the sound scenes in which the audio data was collected. First, the sound scene classification dataset used for model training can be determined. This dataset can include multiple audio data points, which may contain audio from various acoustic environments collected in different cities. Based on the cities where the audio data was collected and their corresponding acoustic environments, city information labels and sound scene category labels can be determined for the audio data. Thus, the sound scene classification dataset can be determined.

[0021] Step S120: Based on the sound scene classification dataset, train a teacher model to obtain the classification soft labels and deep features corresponding to the audio data. First, a teacher model can be trained based on the sound scene classification dataset. The teacher model can be a deep learning model with a large number of parameters. The trained teacher model has a large number of parameters and high sound scene classification accuracy. The teacher model can be trained based on the audio data and its corresponding city information labels and sound scene category labels. This allows the trained teacher model to fully consider the differences between cities where audio was collected during the sound scene classification process, achieving more accurate cross-city sound scene classification. The sound scene classification dataset can be input into the trained teacher model, and the teacher model determines the classification soft labels and deep features corresponding to the audio data. The deep features are key features extracted by the teacher model from the audio data, which can more accurately represent the sound scene to which the audio data belongs. Based on the deep features, the corresponding sound scene can be determined more accurately. The classification soft labels represent the probability distribution of the audio data belonging to different sound scene categories. The hard labels in the sound scene classification dataset, i.e., the sound scene category labels, represent a specific sound scene category of the audio data. Unlike hard labels, classification soft labels can provide multiple sound scene categories to which audio data may belong, and represent the probability of the audio data belonging to each sound scene category through probability distribution.

[0022] Step S130: Train the target model based on the sound scene classification dataset, deep features, and classification soft labels. The target model has fewer parameters than the teacher model. The target model is trained using the sound scene classification dataset and the deep features and classification soft labels corresponding to each audio element in the dataset determined by the teacher model. This allows the target model to be trained based on the results of the teacher model's training, effectively improving the training efficiency and enabling faster convergence. Furthermore, since the target model's training is based on the deep features and classification soft labels determined by the teacher model, it can consider city information and sound scene categories during the sound scene category determination process, thus effectively improving its performance and achieving higher sound scene classification accuracy. The smaller parameter count of the target model compared to the teacher model allows it to be applied to resource-constrained devices such as Bluetooth headsets, broadening its application scope. Specifically, the target model's parameter count can be one-fifth of the teacher model's.

[0023] In some embodiments, each teacher model and the target model may adopt the same model architecture. For example, it may include a feature extractor with 5 layers of convolutional neural networks for feature extraction, and a classifier with 1 fully connected layer for classifying the sound scene based on the features extracted by the feature extractor.

[0024] According to the model training method provided in this embodiment, by constructing a sound scene classification dataset containing city information labels, the training process can comprehensively cover the acoustic differences between different cities. This enables the target model to have stronger discriminative and generalization abilities when classifying sound scenes across cities, effectively alleviating the device domain bias and city domain bias problems caused by differences in recording equipment and environments in traditional methods. By training a teacher model and having it generate classification soft labels and deep features, richer and more granular supervision information can be provided during the training of the target model. This significantly improves the convergence speed and classification accuracy of the target model in cross-city sound scene classification, enhancing the model's robustness to different city acoustic conditions. By simultaneously introducing the sound scene classification dataset, the deep features extracted by the teacher model, and the classification soft labels into the target model training, the training efficiency of the target model can be effectively improved, allowing the target model to achieve sound scene classification performance close to or even better than traditional large-scale deep models under a lightweight structure. Because the target model has fewer parameters than the teacher model, it can achieve high sound scene classification accuracy while maintaining low computational complexity and storage cost, meeting the needs of resource-constrained edge devices such as Bluetooth headsets, wearable devices, and mobile terminals. Therefore, this embodiment can simultaneously improve model training efficiency, model lightweighting, and cross-city sound scene classification performance, further enhancing the system's adaptability, universality, and reliability in real-world acoustic environments.

[0025] In some embodiments, the teacher model may include multiple sub-models. For example... Figure 2 As shown, step S120, based on the sound scene classification dataset, trains the teacher model to obtain the classification soft label and deep features corresponding to the audio data, which may include steps S121 to S122.

[0026] Step S121: Based on the sound scene classification dataset, at least one sub-model is trained to obtain the classification soft label corresponding to the audio data. One or more sub-models can be trained based on the sound scene classification dataset, or based on the audio data in the sound scene classification set along with the corresponding city information labels and sound scene category labels. This ensures that each trained sub-model can determine its corresponding classification soft label based on the audio data. Audio data is input into the trained sub-model, and the sub-model can output the classification soft label corresponding to the audio data, which is used for training the subsequent target model. The classification soft labels determined by this method can fully consider the differences of the same sound scene in different cities, resulting in higher accuracy.

[0027] Step S122: Based on the audio data and corresponding sound scene category labels in the sound scene classification dataset, another sub-model is trained to obtain the deep features corresponding to the audio data. Sub-models can be trained based on the audio data and corresponding sound scene category labels in the sound scene classification dataset, so that each trained sub-model can determine its corresponding deep features based on the audio data. Inputting audio data into a trained sub-model allows it to output the deep features corresponding to that audio data, which are then used for training the target model. The deep features of the audio data determined by this method can contain higher-level audio information, and determining the sound scene category based on these deep features can achieve higher classification accuracy.

[0028] According to the model training method provided in this embodiment, by training multiple sub-models, the accuracy and robustness of the classification soft labels and deep features output by the teacher model can be further improved. This allows the subsequent target model to achieve higher learning efficiency and more stable cross-city sound scene classification performance even with a lightweight structure. Training at least one sub-model based on the sound scene classification dataset to generate classification soft labels enables the sub-model to fully utilize city information labels and sound scene category labels, thus considering the acoustic differences between different cities when predicting the probability distribution of audio data belonging to different sound scenes. Compared with traditional training methods that rely solely on hard labels, the classification soft labels generated by this embodiment can more accurately describe the inter-class similarity relationship of the same sound scene in different cities, enabling the subsequent target model to better learn complex cross-city acoustic pattern differences during training, further improving the generalization ability and accuracy of cross-city sound scene classification. Training another sub-model based on the sound scene classification dataset to extract deep features allows this sub-model to focus on learning the high-level acoustic representation of audio data and extracting deep features containing more semantic information. This allows the model to still have strong classification ability with fewer parameters during the training of the subsequent target model.

[0029] In some embodiments, such as Figure 3 As shown, step S121, based on the sound scene classification dataset, obtains the classification soft label corresponding to the audio data by training at least one sub-model, which may include steps S1211 and S1212.

[0030] Step S1211: Train the first sub-model based on the audio data and corresponding city information labels in the sound scene classification dataset. Training the first sub-model on the audio data and city information labels in the sound scene classification dataset allows it to learn the ability to classify city categories, taking into account the audio differences between cities. This enables the subsequent sound scene classification operations to incorporate relevant sound scene features from different cities for identification and classification, thus improving the accuracy of sound scene classification by considering the differences between cities.

[0031] Step S1212: Based on the first sub-model that has been trained, and the audio data and corresponding sound scene category labels in the sound scene classification dataset, a second sub-model is trained to obtain classification soft labels. Since the first sub-model can incorporate city information labels into the sound scene classification decision process, the second sub-model can be trained based on the first sub-model and the audio data and corresponding sound scene category labels in the sound scene classification dataset. Therefore, the trained second sub-model can classify the audio data into sound scenes based on the relationship between audio data, city information, and sound scenes.

[0032] According to the model training method provided in this embodiment, a first sub-model can be trained based on audio data and corresponding city information labels, enabling the first sub-model to learn the acoustic environment differences between different cities. Thus, the trained first sub-model can fully consider city domain factors when discriminating audio data. A second sub-model is then trained based on the trained first sub-model and the audio data and sound scene category labels in the sound scene classification dataset, allowing the second sub-model to simultaneously utilize the correlation between city information and sound scene categories during training. Compared to directly using audio data and hard labels to train the model, the training method in this embodiment allows the second sub-model to combine the sound scene differences between different cities and the sound scene category distribution during the decision-making process, thereby obtaining more accurate classification soft labels. According to this embodiment, the accuracy and robustness of classification soft labels can be enhanced, and the impact of sound scene differences between cities on sound scene classification can be fully considered during model training, improving the data accuracy for subsequent training of the target model.

[0033] In some embodiments, the first sub-model may include a first feature extractor for feature extraction and a first classifier for city classification. For example... Figure 4 As shown, step S1212, based on the first sub-model that has been trained, and the audio data and corresponding sound scene category labels in the sound scene classification dataset, trains the second sub-model and obtains classification soft labels, which may include steps S12121 to S12123.

[0034] Step S12121: The first feature extractor of the trained first sub-model extracts the first feature of the audio data in the sound scene classification dataset. The first sub-model can be trained based on the audio data and corresponding city information labels in the sound scene classification dataset. During training, the parameters of the first feature extractor and the first classifier of the first sub-model can be continuously updated, enabling the first feature extractor of the first sub-model to extract features related to the city information labels in the audio data. The updated first sub-model can extract the first feature of the audio data in the sound scene classification dataset based on the audio data. The first feature may include features related to the city where the audio was collected and features related to the sound scene.

[0035] Step S12122: Based on the first feature, audio data, and corresponding sound scene category labels, adjust the parameters of the second classifier of the second sub-model to obtain a trained second sub-model. The second classifier can be used for sound scene classification. The parameters of the second sub-model can be adjusted based on the first feature, and the second sub-model can be trained according to the audio data and corresponding sound scene category labels, thereby adjusting the parameters of the second classifier and obtaining a trained second sub-model. Furthermore, the parameters of the first feature extractor of the first sub-model can be used as the parameters of the second feature extractor of the second classifier, and these parameters are kept frozen during the training of the second sub-model without being updated. Only the parameters of the second classifier are updated during training. This allows the second sub-model to collect and recognize features of audio data, taking into account the city where the audio was collected.

[0036] Step S12123: Obtain classification soft labels by completing the training of the second sub-model. Audio data from the sound scene classification dataset can be input again into the trained second sub-model. The second sub-model then determines the sound scene category of the audio data and outputs the classification result as a classification soft label. The classification soft label can be the probability distribution of the sound scene category of the audio data, which may include all possible sound scene categories corresponding to the audio data determined by the second sub-model, and the probability that the audio data belongs to that sound scene category.

[0037] According to the model training method provided in this embodiment, the first feature extractor of the first sub-model after training extracts the first feature of the audio data. This provides data that takes into account the differences in sound scenes between cities for the subsequent training of the second sub-model, providing a more accurate data foundation and effectively avoiding errors in sound scene classification due to differences in urban sound scenes. The parameters of the second sub-model are adjusted based on the first feature and the sound scene category label, allowing the second sub-model to focus on optimizing the sound scene classification task while preserving urban information. This ensures that the second sub-model retains information related to the city where the audio was collected during the process of learning to determine the sound scene category, effectively improving the accuracy and robustness of sound scene classification. Based on the second sub-model, soft labels for classification are determined, which effectively enhances the model's adaptability and recognition accuracy to audio data distributed in different cities with different acoustic characteristics.

[0038] In some embodiments, such as Figure 5 As shown, step S122, based on the audio data and corresponding audio scene category labels in the sound scene classification dataset, obtains the deep features corresponding to the audio data by training another sub-model, which may include steps S1221 and S1222.

[0039] Step S1221: Based on the audio data and corresponding sound scene category labels in the sound scene classification dataset, train a third sub-model. The third sub-model includes a third feature extractor for feature extraction and a third classifier for sound scene classification. The third sub-model can be trained based on the audio data and corresponding sound scene category labels, thereby updating the parameters of the third feature extractor and the third classifier during training to determine the completed third sub-model. The third feature extractor is used to extract features from the audio data, and the third classifier can determine the sound scene category corresponding to the audio data based on the audio data and the features extracted by the third feature extractor.

[0040] Step S1222: Using the third feature extractor of the trained third sub-model, the third features of the audio data in the sound scene classification dataset are extracted to obtain deep features. After the third sub-model is trained, the audio data from the sound scene classification dataset can be input into the trained third sub-model. Through the third feature extractor of the third sub-model, features are extracted from the input audio data, which can extract the deep features corresponding to the audio data. The deep features can include higher-level audio information features. Determining the sound scene category based on the deep features can achieve higher classification accuracy.

[0041] According to the model training method provided in this embodiment, a third sub-model is trained based on audio data and its corresponding sound scene category labels. This enables the third feature extractor and third classifier in the third sub-model to learn the acoustic features of different sound scenes during training, thereby extracting more representative deep features. Furthermore, this allows for feature-level distillation based on deep features during subsequent target model training, enabling the target model to learn general, highly discriminative acoustic feature representations. Therefore, this effectively improves the efficiency of subsequent target model training and gives the target model better sound scene category discrimination ability and generalization performance.

[0042] In some embodiments, such as Figure 6 As shown, step S130, which trains the target model based on the sound scene classification dataset, deep features, and classification soft labels, may include steps S131 to S134.

[0043] Step S131: Determine the first loss function based on the collected features and deep features of the sound scene classification dataset extracted by the target model. During the training of the target model, a loss function can be determined to measure the difference between the model's predicted values ​​and the true values. This allows for further updates to the target model's parameters based on the loss function, reducing the loss between the true audio data values ​​and the predicted values ​​detected by the target model. This helps the model's predicted values ​​move closer to the true values, thus achieving the learning objective. First, the collected features of the audio data can be extracted based on the target model's feature extractor. Then, the first loss function can be determined based on the collected features and the deep features collected in the aforementioned embodiments. Since the feature information in the audio data is complex, and the target model has a small number of parameters, its feature extractor has a small parameter count and computational cost, resulting in poor feature collection performance and affecting the accuracy of subsequent sound scene classification results. Therefore, the deep features can be used as the standard value, and the collected features extracted based on the target model can be used as the predicted value to determine the first loss function.

[0044] Step S132: Determine the second loss function based on the classification results and soft labels of the sound scene classification dataset from the target model. The target model is trained solely on audio data and corresponding sound scene category labels, without considering the differences in sound scenes between cities. This can easily lead to errors in the target model's determination of sound scene categories. The soft labels, however, incorporate city information labels and other data during the determination process. Therefore, the second loss function can be determined based on the target model's classification results and soft labels, using the soft labels as the standard value and the target model's classification results as the predicted value. This introduces the influence of city information on sound scene category determination during the training process of the target model.

[0045] Step S133: Determine the third loss function based on the classification results and sound scene category labels of the sound scene classification dataset by the target model. For audio data in the sound scene classification dataset, the corresponding sound scene category label is the true value of the sound scene classification, while the classification result of the audio data based on the target model is the predicted value of the target model. Therefore, the third loss function can be determined to determine the difference between the predicted result of the target model for the sound scene category of the audio data and the true result of the sound scene classification of the audio data.

[0046] Step S134: Based on the first loss function, the second loss function, and the third loss function, determine the hybrid loss function of the target model to train the target model. According to the aforementioned first loss function, second loss function, and third loss function, the hybrid loss function can be determined. This allows the target model with fewer parameters to use the teacher model as one of the bases for training, enabling the target model to learn information provided by the teacher model with more parameters. This improves the accuracy of the target model with fewer parameters in classifying sound scenes in audio data.

[0047] According to the model training method provided in this embodiment, a first loss function is constructed based on the features extracted by the target model and the deep features generated by the teacher model. This effectively compensates for the low feature extraction quality caused by the small number of parameters and limited feature extraction capabilities of the target model. This allows the target model to gradually approach the high-quality feature extraction of the teacher model during training, enabling the lightweight target model to extract highly discriminative audio features, thereby significantly improving the stability and accuracy of subsequent sound scene classification. A second loss function is constructed based on the classification output and soft labels of the target model. Since the soft labels not only contain the probability distribution of sound scene categories but also implicitly include the influence of acoustic differences between different cities on category judgment, the second loss function allows the target model to integrate these city-specific features during learning. This results in higher robustness and generalization ability in cross-city sound scene classification tasks, avoiding classification errors caused by ignoring city domain offsets. A third loss function is constructed based on the classification results of the target model and the real sound scene category labels, ensuring the accuracy and stability of the target model in basic sound scene classification tasks. The hybrid loss function is determined by the first loss function, the second loss function, and the third loss function. This enables the target model with a small number of parameters to fully inherit the urban difference information and higher-order and more accurate audio feature recognition provided by the teacher model. This significantly improves the convergence speed, sound scene classification accuracy, and environmental adaptability of the target model, allowing it to maintain stable sound scene recognition accuracy and robustness even on resource-constrained terminal devices.

[0048] In some embodiments, step S134, determining the hybrid loss function of the target model based on the first loss function, the second loss function, and the third loss function, may include: determining the hybrid loss function of the target model to train the target model based on a weighted combination of the first loss function, the second loss function, and the third loss function, wherein the weight of the first loss function is 1, and the sum of the weights of the second loss function and the third loss function is 1. The hybrid loss function can be determined by a weighted combination. Since the first loss function is a loss function established for the target model's feature extractor to extract audio features, its weight can be set to 1. The second and third loss functions are both loss functions established for the target model's target classifier to classify audio scene data; therefore, the sum of their weights can be set to 1. Thus, the hybrid loss function of the target model can be determined by a weighted average. By incorporating the first loss function, differences between cities can be introduced; by incorporating the second and third loss functions, the accuracy of the target model in classifying sound scenes can be improved.

[0049] According to the model training method provided in this embodiment, setting the weight of the first loss function to 1 ensures that the target model fully learns the deep features extracted by the teacher model during training. This allows the lightweight target model to effectively inherit the feature extraction capabilities of the teacher model, thereby effectively improving the feature extraction and recognition accuracy of the target model in recognizing and processing complex audio data. Setting the sum of the weights of the second and third loss functions to 1 allows the target classifier of the target model to take into account the cross-city scene difference features contained in the classification soft label while learning the sound scene category determination. This effectively reduces the classification bias caused by differences in the collection environment and improves the model's generalization ability in different city sound scenes. Through this embodiment, multi-level knowledge distillation can be achieved, efficiently compressing the complex knowledge of a large teacher model into a lightweight target network. This target model integrates rich supervision signals from teachers of different tasks during the training phase, but only needs to maintain its own extremely low parameter scale and computational complexity during the inference phase. This allows the model to maintain high classification performance while its memory usage and computing power requirements fully meet the real-time processing requirements of low-computing-power devices such as Bluetooth headsets and embedded sensors. Thus, a target model with both high accuracy and low resource consumption can be obtained.

[0050] In some embodiments, such as Figure 7 As shown, step S110, determining the sound scene classification dataset, may include steps S111 to S113.

[0051] Step S111: Acquire multiple audio data. First, audio data for multiple sound scenes can be collected, and audio data can be collected for different cities to determine audio data for multiple different sound scenes in multiple different cities.

[0052] Step S112: Based on each audio data point, determine the corresponding city information label and sound scene category label. According to the audio data, a city information label can be added to each audio data point based on the city information of the collected audio data. According to the sound scene of the collected audio data, a corresponding sound scene category label can be added to each audio data point.

[0053] Step S113 involves performing audio feature processing on each audio data point to determine the sound scene classification dataset. The audio feature processing includes at least one of the following: Short-Time Fourier Transform (SFT), Mel filtering, logarithmic operation, and Discrete Cosine Transform (DCT). Based on the audio data, audio feature processing can be performed to determine the preprocessed audio features, thereby determining the sound scene classification dataset. The sound scene classification dataset can include the audio data after audio feature processing and its corresponding city information labels and sound scene category labels. Audio feature processing can include SFT, which obtains information about the audio signal in time and frequency by shifting a window function over time and calculating the spectrum of the audio signal within the window. Audio feature processing can also include Mel filtering, which filters the audio data to facilitate subsequent processing during model training. Audio feature processing can also include logarithmic operation and Discrete Cosine Transform. Specifically, SFT, Mel filtering, logarithmic operation, and DCT can be sequentially performed on the audio data to ultimately determine the Mel frequency cepstral coefficient features of the audio data, which serve as the final input audio data for the model.

[0054] According to the model training method provided in this embodiment, by collecting audio data from multiple different cities and various sound scenes, the audio data can cover a wider range of sound scenes and cities, thereby making the constructed dataset more diverse and representative, and improving the generalization ability of the subsequent model in different urban sound scenes. Adding city information tags and sound scene category tags to each audio data point can clearly distinguish different collection environments and sound scene categories at the data level, allowing the differences between cities to be incorporated into the model's knowledge learning process during subsequent training. This enhances the model's ability to perceive cross-regional acoustic differences, thereby reducing classification bias caused by different collection locations. Performing feature processing on the audio data, such as short-time Fourier transform, Mel filtering, logarithmic operation, and discrete cosine transform, can transform the original waveform data into audio feature representations with more stable dimensions and stronger descriptive capabilities. Through this embodiment, redundant information and noise interference in the original audio data can be reduced, ensuring that the audio data input to the model maintains high stability and discriminability, thereby improving the convergence speed and classification performance of model training. Meanwhile, according to this embodiment, the input quality of the sound scene classification model can be effectively improved, enabling the model to achieve higher accuracy and environmental adaptability under multiple city and scene conditions.

[0055] Based on the same inventive concept, this disclosure also provides a sound scene classification method. The sound scene classification method may include: inputting the audio to be tested into a target model to determine the sound scene category corresponding to the audio to be tested, wherein the target model is determined according to the model training method of any of the foregoing embodiments. The audio data to be tested for sound scene recognition can be acquired; the audio to be tested may be audio collected in real time through a microphone or audio segments stored offline. The audio to be tested may be preprocessed to eliminate the influence of noise and invalid data on sound scene classification, performing the same audio feature processing methods as the sound scene classification dataset, such as short-time Fourier transform, Mel filtering, logarithmic operation, and discrete cosine transform. The audio to be tested can be input into the trained target model. The target model may include a target feature extractor and a target classifier. After receiving the audio to be tested, the target feature extractor can extract the collected features of the audio to be tested; subsequently, the target classifier determines the sound scene category corresponding to the audio to be tested based on the extracted collected features. Because the target model's feature extractor learns the representational ability of deep features from the teacher model through the first loss function, and the target model's classifier is trained under dual supervision of classification soft labels and real sound scene category labels, the target model can achieve high-accuracy sound scene classification without increasing the model size. Finally, based on the target model, the sound scene category of the audio to be tested can be determined with high classification accuracy. The sound scene classification method provided in this embodiment can achieve high accuracy, high robustness, and high efficiency in sound scene recognition while ensuring a lightweight model, making it suitable for application scenarios of various edge devices such as Bluetooth headsets and wearable devices.

[0056] Based on the same inventive concept, such as Figure 8 As shown, this disclosure also provides a sound scene classification device 200, which may include: a model training module and a sound scene classification model.

[0057] The model training module 210 is used to train the target model based on the model training method of any of the foregoing embodiments. The model training module 210 can process audio data, label each audio data point with city information tags and sound scene category tags, and perform audio feature processing such as short-time Fourier transform, Mel filtering, logarithmic operation, and discrete cosine transform to generate feature inputs for model training. The model training module 210 can also train multiple teacher models based on audio data, city information tags, and sound scene category tags. Specifically, the model training module 210 trains a first sub-model based on city information tags and audio data; it can also train a second sub-model based on the first sub-model and sound scene tags to generate classification soft labels; and it can also train a third sub-model and obtain deep features through a third feature extractor. Finally, the model training module 210 can train a target model with a small number of parameters based on the deep features and classification soft labels determined in the aforementioned operations, combined with the audio data.

[0058] The sound scene classification module 220 is used to determine the sound scene category of the audio to be tested based on the trained target model. The sound scene classification module 220 can input the audio to be tested into the target model according to the trained target model, extract the collected features by the target feature extractor of the target model, and output the corresponding sound scene category by the target classifier of the target model.

[0059] The sound scene classification device provided in this embodiment can preprocess audio data through the model training module 210 to determine the sound scene classification dataset and train a first sub-model, a second sub-model, and a third sub-model. The first sub-model can incorporate sound scene differences between cities, and the classification soft labels output by the second sub-model and the deep features extracted by the third sub-model provide richer, higher-quality training information for the target model. This allows the target model with fewer parameters to fully learn multi-source information during the training phase, achieving stronger recognition capabilities while maintaining a lightweight structure. Because the target model simultaneously utilizes deep features, classification soft labels, and real sound scene labels for joint optimization during training, it effectively improves the target model's ability to recognize complex audio data. This enables the finally trained target model to accurately distinguish audio data under multiple city, scene, and noise conditions in actual inference, improving the stability and accuracy of the sound scene classification results.

[0060] This application uses specific terms to describe embodiments of the application. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of the application. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics in one or more embodiments of the application can be appropriately combined.

[0061] In the context of this application, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0062] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this disclosure method does not imply that the subject matter of the present application requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of the single embodiments disclosed above.

[0063] The basic concepts have been described above. Obviously, for those skilled in the art, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore remain within the spirit and scope of the embodiments of this application.

Claims

1. A model training method, characterized in that, The model training method for sound scene classification comprises the following steps: determining a sound scene classification data set, wherein the data in the sound scene classification data set comprises a plurality of audio data, and a city information label and a sound scene category label corresponding to each audio data, wherein the city information label corresponds to a city where the audio data is collected, and the sound scene category label corresponds to a sound scene where the audio data is collected; training a teacher model based on the sound scene classification data set to obtain a classification soft label and a deep feature corresponding to the audio data; training a target model based on the sound scene classification data set, the deep feature, and the classification soft label, wherein the parameter quantity of the target model is less than that of the teacher model.

2. The model training method of claim 1, wherein, The teacher model comprises a plurality of sub-models. The training of the teacher model based on the sound scene classification data set to obtain the classification soft label and the deep feature corresponding to the audio data comprises the following steps: obtaining the classification soft label corresponding to the audio data by training at least one sub-model based on the sound scene classification data set; obtaining the deep feature corresponding to the audio data by training another sub-model based on the audio data and the sound scene category label in the sound scene classification data set.

3. The model training method of claim 2, wherein, The training of the teacher model based on the sound scene classification data set to obtain the classification soft label and the deep feature corresponding to the audio data comprises the following steps: training a first sub-model based on the audio data and the city information label in the sound scene classification data set; training a second sub-model based on the first sub-model after training, the audio data, and the sound scene category label in the sound scene classification data set, and obtaining the classification soft label.

4. The model training method of claim 3, wherein, The first sub-model comprises a first feature extractor for feature extraction and a first classifier for city classification. The training of the second sub-model based on the first sub-model after training, the audio data, and the sound scene category label in the sound scene classification data set, and the obtaining of the classification soft label comprise the following steps: extracting a first feature of the audio data in the sound scene classification data set by the first feature extractor of the first sub-model after training; adjusting parameters of a second classifier of the second sub-model based on the first feature and the audio data and the sound scene category label to obtain the second sub-model after training, wherein the second classifier is used for sound scene classification; obtaining the classification soft label by the second sub-model after training.

5. The model training method of claim 2, wherein, The training of the third sub-model based on the audio data and the sound scene category label in the sound scene classification data set comprises the following steps: The third sub-model comprises a third feature extractor for feature extraction and a third classifier for sound scene classification. The third feature extractor of the third sub-model trained is used to extract third features of the audio data in the sound scene classification data set, to obtain the deep features. 6.The model training method of any of claims 1-5, wherein, The target model is trained based on the sound scene classification data set, the deep features and the classification soft label, including: A first loss function is determined based on the collected features of the sound scene classification data set extracted by the target model and the deep features; A second loss function is determined based on the classification result of the sound scene classification data set by the target model and the classification soft label; A third loss function is determined based on the classification result of the sound scene classification data set by the target model and the sound scene category label; A mixed loss function of the target model is determined based on the first loss function, the second loss function and the third loss function to train the target model.

7. The model training method according to claim 6, characterized in that, The mixed loss function of the target model is determined based on the first loss function, the second loss function and the third loss function, including: The mixed loss function of the target model is determined based on the weighted combination of the first loss function, the second loss function and the third loss function to train the target model, wherein the weight of the first loss function is 1, and the sum of the weights of the second loss function and the third loss function is 1. 8.The method of Claim 1, wherein The sound scene classification data set is determined, including: A plurality of audio data are obtained; Based on each audio data, the corresponding city information label and the sound scene category label are determined; Each audio data is subjected to audio feature processing to determine the sound scene classification data set, wherein the audio feature processing includes at least one of the following: short-time Fourier transform, Mel filtering, logarithmic operation, discrete cosine transform.

9. A sound scene classification method, characterized in that, The sound scene classification method includes: The target model is input with the audio to be tested to determine the sound scene category corresponding to the audio to be tested, wherein the target model is determined according to the model training method of any one of claims 1-8.

10. An acoustic scene classification device, characterized by It includes: The model training module is used to train the target model based on the model training method of any one of claims 1-8; The sound scene classification module is used to determine the sound scene category of the audio to be tested based on the target model trained.

Images