Method for constructing audio environment classification model, audio environment identification method and device

Abstract

The application provides a method for constructing an audio environment classification model, an audio environment recognition method and equipment. The method comprises: collecting audio signals in multiple different sample environments, and dividing each audio signal into multiple signal frames by framing; pre-processing each signal frame, and extracting multi-dimensional acoustic features from the pre-processed signal frames; evaluating the contribution of each acoustic feature to environment classification, and selecting post-selected acoustic features from the multi-dimensional acoustic features based on the contribution; generating multiple groups of feature combinations based on the post-selected acoustic features, determining an optimal feature combination by evaluating the classification performance of each group of feature combinations; constructing and training a linear support vector machine classification model based on the optimal feature combination, and obtaining an audio environment classification model based on a linear support vector machine for multi-environment class recognition. The trained audio environment classification model is used for recognizing the audio environment class, thereby realizing high-precision and low-power consumption recognition of multiple audio environment classes.

Description

Technical Field

[0001] This invention relates to the field of audio environment recognition technology, and in particular to a method for constructing an audio environment classification model based on linear support vector machines, an audio environment recognition method, and an electronic device. Background Technology

[0002] With the rapid development of hearing aids, smart headphones, and wearable audio devices, these devices are placing higher demands on their environmental awareness capabilities. Different acoustic environments present significantly different requirements for audio processing algorithms. For example, in quiet environments, natural sound quality should be maintained, avoiding over-processing; in noisy environments, noise reduction capabilities need to be enhanced; in environments with conversation, the speech signal needs to be highlighted; and in music environments, speech enhancement algorithms need to avoid damaging the music's sound quality.

[0003] Existing audio environment recognition methods mostly adopt threshold judgment, rule matching or simple statistical feature classification. In practical applications, they have the following problems: (1) The granularity of environment distinction is limited, making it difficult to accurately distinguish complex environments such as conversation in quiet or conversation in noisy environments; (2) The robustness to environmental changes is insufficient, and misjudgment is easy to occur in noise fluctuations or mixed speech and music scenarios; (3) The algorithm complexity is high or uncontrollable, and it is not suitable for long-term operation in embedded audio devices with limited computing power and power consumption; (4) The model stability is insufficient, and there is no effective processing mechanism for abnormal data or feature failure.

[0004] Therefore, there is an urgent need for an audio environment recognition solution that can ensure the accuracy of environment recognition while also taking into account low power consumption, stability, and embedded feasibility. Summary of the Invention

[0005] In view of the above problems, the present invention proposes a method for constructing an audio environment classification model based on linear support vector machines, an audio environment recognition method, and an electronic device that overcomes or at least partially solves the above problems.

[0006] One objective of this invention is to achieve high-precision recognition of multiple types of audio environments.

[0007] A further objective of this invention is to achieve low-power operation of the audio environment classification model.

[0008] Another further objective of the present invention is to achieve stable environmental identification output.

[0009] In particular, according to one aspect of the present invention, a method for constructing an audio environment classification model based on a linear support vector machine is provided, comprising: Audio signals from various sample environments are collected, and each audio signal is divided into multiple signal frames. Each signal frame is preprocessed, and multidimensional acoustic features are extracted from the preprocessed signal frames. The contribution of each acoustic feature to environmental classification is evaluated, and the selected acoustic features are selected from the multidimensional acoustic features based on the contribution. Multiple feature combinations are generated based on the selected acoustic features, and the optimal feature combination is determined by evaluating the classification performance of each feature combination. A linear support vector machine (SVM) classification model is constructed and trained based on the optimal feature combination, resulting in an audio environment classification model based on SVM for multi-environment category recognition.

[0010] Optionally, the steps for constructing and training a linear support vector machine classification model based on the preferred feature combination include: Standardize the feature vector of the preferred feature combination; A linear support vector machine classification model is constructed based on standardized feature vectors, where the support vector machine uses a linear kernel function; The linear support vector machine classification model is extended by a multi-class strategy to decompose the multi-class environment recognition problem into multiple binary sub-problems; The linear support vector machine classification model is trained using the labeled optimal feature combination as training samples, and the model performance is evaluated during the training process to determine the optimal model parameters; Save and solidify the preferred model parameters of the linear support vector machine classification model, the feature combination index used, and the statistical parameters required for feature vector standardization.

[0011] Optionally, the steps of evaluating the contribution of each acoustic feature to environmental classification and selecting post-selection acoustic features from the multidimensional acoustic features based on the contribution include: Generate category labels for each acoustic feature, and form a training sample set based on the acoustic features and their category labels; A feature importance assessment classification model is constructed by learning from the training sample set. The input and output of the feature importance assessment classification model are the feature vector and category label of the acoustic feature, respectively. The contribution of each acoustic feature to the environmental classification results is calculated based on the change in model classification performance caused by feature perturbation. Acoustic features that meet the contribution requirements are selected as post-selection acoustic features; The contribution requirements include: The contribution level is greater than or equal to the set contribution threshold; or The contribution ranking is placed before the set ranking.

[0012] Optionally, the step of calculating the contribution of each acoustic feature to the environmental classification result based on the change in model classification performance caused by feature perturbation includes: Obtain the original model accuracy with all acoustic features as input; Calculate the model accuracy after perturbing any acoustic feature; The difference between the accuracy of the original model and the accuracy of the perturbated model is calculated as the contribution of the acoustic feature to the environmental classification results.

[0013] Optionally, the step of determining the preferred feature combination by evaluating the classification performance of each feature combination includes: Under the constraint of the size of feature combinations, a low-complexity performance evaluation method is used to quickly select candidate feature combinations that meet the classification performance requirements from multiple feature combinations. A high-reliability performance evaluation method is used to evaluate the classification accuracy of each candidate feature combination, and the optimal feature combination is determined based on the classification accuracy and computational cost of each candidate feature combination.

[0014] Optionally, low-complexity performance evaluation methods include low-fold cross-validation with fewer than 5 folds; or, Under the constraint of the size of feature combinations, the steps of using a low-complexity performance evaluation method to quickly select candidate feature combinations that meet the classification performance requirements from multiple feature combinations include: For each feature combination, calculate the multiple class centers of each acoustic feature relative to the multi-class sample environment; The inter-class distance and intra-class divergence of each feature combination are calculated based on the multiple class centers of each acoustic feature in each feature combination. The ratio of inter-class distance to intra-class divergence for each feature combination is used as the evaluation index for classification performance, and candidate feature combinations are selected based on the evaluation index.

[0015] Optionally, the high reliability performance evaluation method includes K-fold cross-validation, where K is an integer greater than or equal to 5; The steps for evaluating the classification accuracy of each candidate feature combination using a high-reliability performance evaluation method include: For each candidate feature combination, the feature vector of that candidate feature combination is used as the training sample, and the training sample is divided into K parts; Based on K training samples, the pre-set classification model is trained and tested in a K-fold loop to obtain the classification performance value of each fold; Calculate the K-fold average classification performance value of the candidate feature combination based on the classification performance value of each fold; The classification performance variance is calculated based on the classification performance value of each fold and the average classification performance value of K folds of the candidate feature combination, and is used as the stability index of the candidate feature combination. The classification accuracy of each candidate feature combination is evaluated based on its K-fold average classification performance value and stability index.

[0016] Optionally, the multi-dimensional acoustic features extracted from each preprocessed signal frame constitute a frame-level feature vector; Before assessing the contribution of each acoustic feature to environmental classification, the construction method also includes: Detect whether there are abnormal feature values ​​in the feature vectors of each frame; If it exists, then remove or ignore that frame-level feature vector.

[0017] Optionally, the acoustic features include at least one of energy-related features, spectral structure features, harmonic-related features, speech-related features, and music discrimination-related features; Among these, energy-related characteristics include noise level; Spectral structure features include spectral entropy and spectral centroid; Speech-related features include signal-to-noise ratio features; Music discrimination features include autocorrelation value, number of spectral peaks, and zero-crossing rate; Preprocessing includes at least one of the following: DC component removal processing, filtering processing, energy normalization processing, and pre-emphasis processing; The multiple environment categories include quiet environments, quiet conversation environments, noisy environments, noisy conversation environments, and various music environments.

[0018] According to another aspect of the present invention, an audio environment recognition method based on an audio environment classification model constructed by the aforementioned method is also provided, comprising: The audio signal in the environment to be identified is collected and then divided into multiple signal frames. Each signal frame is preprocessed, and the selected acoustic features are extracted from the preprocessed signal frames. The selected acoustic features of each signal frame are input into the audio environment classification model to obtain the environment category classification result for each signal frame.

[0019] Optionally, the audio environment recognition method further includes: After obtaining the environmental category classification results of multiple consecutive signal frames, the environmental category classification results of multiple consecutive signal frames are stabilized in the time domain. Output the stabilized environment category classification results in frame-level or short-window-level format; The stabilization process includes multi-frame voting decisions, sliding time window statistics, or smooth output of decision results.

[0020] According to another aspect of the present invention, an electronic device is also provided, including a memory, a processor, and a machine-executable program stored in the memory and running on the processor, wherein the processor executes the machine-executable program to implement any of the aforementioned audio environment recognition methods.

[0021] Optionally, the electronic device is an embedded audio device; Embedded audio devices include hearing aids, smart headphones, or wearable audio devices.

[0022] The technical solution provided by this invention involves framing, preprocessing, and extracting multidimensional acoustic features from audio signals collected from multiple sample environments. The extracted acoustic features are then screened and optimized for feature combination. Based on the preferred feature combination, a linear support vector machine classification model is constructed and trained to obtain an audio environment classification model based on linear support vector machines for multi-environment category recognition. The trained audio environment classification model is then used to identify audio environment categories, thereby achieving high-precision recognition of multiple audio environment categories.

[0023] Furthermore, the linear support vector machine classification model uses linear kernel parameters. By constraining the model complexity, the classification stage only involves multiplication and addition operations, thus enabling low-power operation of the audio environment classification model. This makes the audio environment classification model suitable for real-time operation in low-power embedded audio devices.

[0024] Furthermore, by employing a two-stage optimization strategy for feature combinations, the computational complexity of the model can be significantly reduced while ensuring classification accuracy.

[0025] Furthermore, by introducing a feature validity detection mechanism before screening acoustic features, the training data entering the classification model is ensured to be stable and reliable, thereby guaranteeing the numerical stability of the audio environment classification model.

[0026] Furthermore, by stabilizing the environmental category classification results of multiple consecutive signal frames in the time domain, stable and reliable environmental identification results are generated, achieving stable environmental identification output.

[0027] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0028] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings: Figure 1This is a flowchart illustrating a method for constructing an audio environment classification model based on a linear support vector machine according to an embodiment of the present invention. Figure 2 This is a flowchart illustrating a method for constructing an audio environment classification model based on a linear support vector machine according to another embodiment of the present invention. Figure 3 This is a flowchart illustrating the feature importance assessment and screening steps according to an embodiment of the present invention; Figure 4 This is a flowchart illustrating the feature combination optimization steps according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating the steps of constructing and training a linear support vector machine classification model based on a preferred feature combination according to an embodiment of the present invention. Figure 6 This is a flowchart illustrating an audio environment recognition method according to an embodiment of the present invention; Figure 7 This is a flowchart illustrating an audio environment recognition method according to another embodiment of the present invention; Figure 8 This is a schematic structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0029] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0030] In view of the above problems, this invention proposes a method for constructing an audio environment classification model based on linear support vector machines.

[0031] Figure 1 This is a flowchart illustrating a method for constructing an audio environment classification model based on a linear support vector machine according to an embodiment of the present invention. See also... Figure 1 As shown, the method for constructing the audio environment classification model based on linear support vector machines may include at least the following steps S102 to S110.

[0032] Step S102: Collect audio signals from multiple different sample environments and divide each audio signal into multiple signal frames.

[0033] Step S104: Preprocess each signal frame and extract multidimensional acoustic features from the preprocessed signal frames.

[0034] Step S106: Evaluate the contribution of each acoustic feature to environmental classification, and select the final acoustic features from the multidimensional acoustic features based on the contribution.

[0035] Step S108: Generate multiple feature combinations based on the selected acoustic features, and determine the preferred feature combination by evaluating the classification performance of each feature combination.

[0036] Step S110: Construct and train a linear support vector machine classification model based on the preferred feature combination to obtain an audio environment classification model based on linear support vector machine for multi-environment category recognition.

[0037] The method for constructing an audio environment classification model based on linear support vector machines (SVMs) provided by this invention involves framing, preprocessing, and extracting multi-dimensional acoustic features from audio signals collected from multiple types of sample environments. The extracted acoustic features are then filtered and optimized for feature combination. Based on the optimized feature combination, a linear support vector machine classification model is constructed and trained, resulting in an audio environment classification model based on linear support vector machines for multi-environment category recognition. This audio environment classification model can be used for high-precision recognition of multiple types of audio environments.

[0038] In step S102 above, the sample environment can be a preset audio environment, such as a quiet environment, a quiet conversation environment, a noisy environment, a noisy conversation environment, a music environment, etc., which can be simulated through standard sound field environment simulation.

[0039] Audio signals can be acquired using voice acquisition devices (such as microphones). The acquired audio signal can be a discrete-time signal, represented as... .

[0040] In some embodiments, the step of framing each audio signal into multiple signal frames may include performing short-time analysis processing on each audio signal to divide it into multiple signal frames. This framing method is suitable for real-time audio stream processing.

[0041] Specifically, each signal frame contains N sampling points, and there is a preset frame shift H between adjacent frames. Then, the k-th signal frame (also called the k-th frame signal) can be represented as: ,in .

[0042] In step S104 above, the preprocessing of the signal frame may include, but is not limited to, at least one of DC component removal processing, filtering processing, energy normalization processing, and pre-emphasis processing, in order to reduce the impact of equipment differences and environmental changes on feature calculation.

[0043] In a specific embodiment, the DC removal process can be represented as: .

[0044] The preprocessed signal is used for subsequent acoustic feature extraction.

[0045] In other embodiments, the original acquired audio signal may be preprocessed as described above. The preprocessed audio signal is then used for subsequent framing and acoustic feature extraction.

[0046] In some embodiments, the extracted acoustic features may include at least one of energy-related features, spectral structure features, harmonic-related features, speech-related features, and music discrimination-related features. These features are used to describe the statistical and structural characteristics of audio signals under different acoustic environments.

[0047] In some specific embodiments, energy-related features may include noise levels. By estimating the energy noise floor characteristics of a signal frame, a noise level reflecting the current background noise intensity can be obtained, enhancing the ability to discriminate against persistent noise environments. The noise level may be expressed as noise power or its normalized form.

[0048] In some specific embodiments, spectral structure features may include spectral entropy and spectral centroid.

[0049] By normalizing the spectral energy distribution of a signal frame, the spectral entropy of that signal frame can be calculated. Spectral entropy characterizes the dispersion of the spectral energy distribution and can be expressed as:

[0050] in, p i Indicates the first i The normalized energy percentage of each frequency band. Spectral entropy plays an important role in distinguishing between musical, speech, and noisy environments.

[0051] The spectral centroid can be obtained by calculating the weighted average frequency of the spectral energy of the signal frame, expressed as: .

[0052] The centroid of the spectrum is used to reflect the center of gravity of the overall spectrum distribution of a signal, which helps to distinguish between a noise environment dominated by low frequencies and a speech or music environment rich in high-frequency components.

[0053] In some specific embodiments, speech-related features may include signal-to-noise ratio features.

[0054] Based on speech and background noise energy estimation, the signal-to-noise ratio (SNR) of a signal frame can be calculated to distinguish between quiet, noisy, and mixed environments. Its form can be summarized as follows: .

[0055] The signal-to-noise ratio (SNR) characteristic has a significant ability to distinguish between noisy and quiet environments.

[0056] In some specific embodiments, music discrimination features may include autocorrelation value, peak count, and zero-crossing rate.

[0057] The short-time autocorrelation value of the signal frame is calculated to reflect the periodicity of the signal, and its form can be summarized as follows: .

[0058] Selecting representative autocorrelation values ​​as acoustic features can be used to detect signal components with periodic structures, such as speech.

[0059] By performing peak detection on the spectrum of a signal frame and counting the number of local peaks exceeding a preset threshold, the peak quantity feature of the spectrum can be obtained, which is used to characterize the complexity of the spectrum structure. The peak quantity feature of the spectrum has a strong ability to distinguish between musical environments (which have a multi-peak structure) and speech or noisy environments.

[0060] The zero-crossing rate can be obtained by calculating the frequency of signal symbol changes within a signal frame, and its form can be expressed as: .

[0061] Zero-crossing rate plays an auxiliary role in distinguishing between stationary noise, speech, and music signals.

[0062] In some specific embodiments, the acoustic features extracted from each signal frame may include seven types of features: signal-to-noise ratio, noise level, spectral entropy, autocorrelation value, spectral centroid, number of spectral peaks, and zero-crossing rate.

[0063] The multidimensional acoustic features extracted from each preprocessed signal frame can form a frame-level feature vector.

[0064] Figure 2 This is a flowchart illustrating a method for constructing an audio environment classification model based on a linear support vector machine according to another embodiment of the present invention. See also Figure 2 As shown, in some embodiments, before performing step S106 to evaluate the contribution of each acoustic feature to the environment classification, the method for constructing the audio environment classification model based on linear support vector machines may further include the following step S105 (feature validity detection step): detecting whether there are abnormal feature values ​​in each frame-level feature vector; if so, removing or ignoring the frame-level feature vector. In other words, only valid feature vectors without abnormal feature values ​​are retained.

[0065] Specifically, outlier values ​​can be non-finite values, such as Not a Number (NaN) or Infinity (Inf).

[0066] In this embodiment, a feature validity detection mechanism is introduced before the acoustic features enter the screening and classification process to detect whether there are abnormal values ​​(such as non-finite values) in the feature data. When abnormal values ​​are detected in the feature vector, the corresponding samples are removed or ignored, thereby ensuring that the data entering the classification model is stable and reliable, thus ensuring the numerical stability of the audio environment classification model.

[0067] Step S106 can be referred to as the feature importance assessment and screening step. Figure 3 This is a flowchart illustrating the feature importance assessment and screening steps according to an embodiment of the present invention. See also... Figure 3 As shown, in some embodiments, step S106 may specifically include the following steps S1061 to S1064.

[0068] Step S1061: Generate category labels for each acoustic feature, and form a training sample set based on the acoustic features and their category labels.

[0069] Specifically, category labels can be obtained through manual annotation (such as voiceless / voiced sounds), automatic mapping based on phoneme rules, or conversion based on existing annotation data.

[0070] Step S1062: A feature importance evaluation classification model is constructed by learning from the training sample set. The input and output of the feature importance evaluation classification model are the feature vector of the acoustic feature and the category label, respectively.

[0071] Specifically, the feature importance assessment classification model can employ decision tree models, ensemble learning models (such as random forests and boosting trees), or other supervised learning models.

[0072] Step S1063: Calculate the contribution of each acoustic feature to the environmental classification result based on the change in model classification performance caused by feature perturbation.

[0073] In step S1063, the contribution of a feature to the model's decision is quantified by the change in classification performance caused by feature perturbation. Specifically, a single feature is perturbed (e.g., randomly shuffled) to observe changes in model performance; the greater the performance decline, the more important the feature.

[0074] Step S1064: Select acoustic features that meet the contribution requirements as post-selection acoustic features.

[0075] In some specific embodiments, the contribution requirement may include: the contribution is greater than or equal to a set contribution threshold. That is, acoustic features whose contribution is greater than or equal to the set contribution threshold are selected as post-selection acoustic features.

[0076] In some other specific embodiments, the contribution requirement may include: the contribution ranking position is before a set position. That is, the acoustic features are sorted according to their contribution, and acoustic features whose contribution ranking position is before a set position (such as the 5th position) or whose contribution ranking position is within the top preset percentage (such as the top 50%) are selected as the next selected acoustic features.

[0077] In some further embodiments, the step of calculating the contribution of each acoustic feature to the environmental classification result based on the change in model classification performance caused by feature perturbation may include: Obtain the raw model accuracy when all acoustic features are used as input. Acc ; Calculate the effect of any acoustic feature (such as the first) j Accuracy of the model after perturbation of acoustic features ; The difference between the accuracy of the original model and the accuracy of the perturbated model is calculated as the contribution of the acoustic feature to the environmental classification results.

[0078] Specifically, no. j The contribution of each acoustic feature to the environmental classification result can be expressed as: .

[0079] In this embodiment, the importance of acoustic features is assessed before training the support vector machine classification model, which can reduce the impact of irrelevant or redundant features on the system's computational burden and reduce the complexity of subsequent calculations.

[0080] Step S108 can be referred to as the feature combination generation and optimization step. Figure 4 This is a flowchart illustrating the feature combination optimization steps according to an embodiment of the present invention. See also... Figure 4 As shown, in some embodiments, the optimization of feature combinations can be evaluated using a two-stage optimization strategy.

[0081] Specifically, the step of determining the preferred feature combination by evaluating the classification performance of each feature combination may include: Step S1081: Under the constraint of the size of the feature combination, a low-complexity performance evaluation method is used to quickly select candidate feature combinations that meet the classification performance requirements from multiple feature combinations. Step S1082: Use a high reliability performance evaluation method to evaluate the classification accuracy of each candidate feature combination, and determine the preferred feature combination based on the classification accuracy and computational cost of each candidate feature combination.

[0082] Step S1081 is the rapid screening stage. In this stage, the size of the feature combination is constrained, and a low-complexity performance evaluation method is used to quickly screen out feature combinations with poor classification performance.

[0083] In some specific embodiments, low-complexity performance evaluation methods include low-fold cross-validation with fewer than 5 folds. The specific operation of low-fold cross-validation is well-known to those skilled in the art and will not be described further here.

[0084] In some optional embodiments, step S1081 may specifically include: For each feature combination, calculate the multiple class centers of each acoustic feature relative to the multi-class sample environment; The inter-class distance and intra-class divergence of each feature combination are calculated based on the multiple class centers of each acoustic feature in each feature combination. The ratio of inter-class distance to intra-class divergence for each feature combination is used as the evaluation index for classification performance, and candidate feature combinations are selected based on the evaluation index.

[0085] For example, suppose there are C audio environment categories, and the acoustic feature samples for each audio environment category are Xc. Then, for a feature combination F, we first calculate the multiple class centers of each acoustic feature in feature combination F relative to the multi-class sample environments. The class center for each class is represented as: .

[0086] Then, the between-class distance of feature combination F is calculated, with greater distance between class centers being better. The between-class distance is expressed as: .

[0087] Next, the within-class variance of the feature combination F is calculated, expressed as: .

[0088] Subsequently, the following evaluation metric, Score, was constructed: .

[0089] Finally, the feature combinations are sorted according to the evaluation index Score, and the feature combinations that are ranked before the preset ranking or within the preset percentage (such as the top 20%) are retained as candidate feature combinations.

[0090] Step S1082 is the fine evaluation stage. In this stage, a more reliable performance evaluation method is used on the small number of candidate feature combinations selected, taking into account both classification accuracy and computational cost, to determine the final preferred feature combination.

[0091] In some specific embodiments, the high reliability performance evaluation method may include K-fold cross-validation, where K is an integer greater than or equal to 5, such as 5 or 10.

[0092] Specifically, the steps for evaluating the classification accuracy of each candidate feature combination using a high-reliability performance evaluation method may include: For each candidate feature combination, the feature vector of that candidate feature combination is used as the training sample, and the training sample is divided into K parts (the training sample is represented as: ); Based on K training samples, a pre-defined classification model is trained and tested in a K-fold loop to obtain the classification performance value for each fold (represented as: , ,..., ); Calculate the K-fold average classification performance value of the candidate feature combination based on the classification performance value of each fold (represented as: ); The classification performance variance is calculated based on the classification performance value of each fold and the average classification performance value of K folds of the candidate feature combination, and is used as a stability index for the candidate feature combination (expressed as: ); The classification accuracy of each candidate feature combination is evaluated based on its K-fold average classification performance value and stability index.

[0093] A higher K-fold average classification performance value indicates better classification performance; a smaller classification performance variance indicates more stable classification and a better feature combination. The optimal feature combination is selected by comprehensively considering both the K-fold average classification performance value and the classification performance variance.

[0094] In this embodiment, a two-stage optimization strategy for feature combination can significantly reduce the computational complexity of the model while ensuring classification accuracy.

[0095] Figure 5 This is a flowchart illustrating the steps of constructing and training a linear support vector machine classification model based on a preferred feature combination according to an embodiment of the present invention. See also... Figure 5 As shown, in some embodiments, the step of constructing and training a linear support vector machine classification model based on a preferred feature combination may include the following steps S1101 to S1105.

[0096] Step S1101: Standardize the feature vector of the preferred feature combination.

[0097] To reduce the impact of differences in feature dimensions on classification results, the feature vectors are standardized before inputting the optimal feature combination into the support vector machine classification model. The standardization method can be summarized as follows:

[0098] in, μ i and σ i They are respectively i The statistical mean and standard deviation of each feature in the training samples are calculated. When the standard deviation is lower than a preset threshold, a substitute value is used to ensure numerical stability. Through this process, the contribution of each feature to the classification model is more balanced, which is beneficial to improving the model's generalization ability and stability.

[0099] Step S1102: Construct a linear support vector machine classification model based on the standardized feature vectors, wherein the support vector machine uses a linear kernel function.

[0100] Support vector machines use linear kernel functions, and their discriminant function can be expressed as:

[0101] in, w For the model weight vector, b This is a bias term.

[0102] The advantages of using linear kernel functions are: simple model structure and low computational complexity; conducive to real-time operation of embedded devices and low-power systems; and easy to solidify model parameters and subsequent maintenance.

[0103] Step S1103: Extend the linear support vector machine classification model through a multi-classification strategy to decompose the multi-class environment recognition problem into multiple binary sub-problems.

[0104] To address the need for identifying multiple environmental categories, the linear support vector machine classification model is extended using a multi-classification strategy. In practice, a one-vs-one multi-classification encoding method is adopted to decompose the multi-class environmental identification problem into multiple binary sub-problems, thereby forming multiple binary sub-models. Each binary sub-model is used to distinguish between two different environmental categories.

[0105] During the model prediction phase, each binary sub-model makes a decision on the input feature vector, and the final environment category decision is obtained by combining the results through voting or weighted methods. This multi-classification strategy can reduce the complexity of individual classifiers and improve the stability of the overall system while ensuring classification accuracy.

[0106] Step S1104: Use the labeled preferred feature combination as training samples to train the linear support vector machine classification model, and evaluate the model performance during the training process to determine the preferred model parameters.

[0107] A linear support vector machine (SVM) classification model is trained using labeled training samples. During training, the model performance is evaluated using cross-validation, and the optimal model parameter configuration, which strikes a good balance between accuracy and computational complexity, is selected as the preferred model parameter configuration.

[0108] Step S1105: Save and solidify the preferred model parameters of the linear support vector machine classification model, the feature combination index used, and the statistical parameters required for feature vector standardization.

[0109] After the model training is completed, the following information will be permanently saved: the optimal model parameters of the support vector machine; the feature combination index used; and the statistical parameters required for feature standardization.

[0110] The model solidification process enables the model to be directly deployed on the target device and reuse the parameters obtained during the training phase during actual operation.

[0111] In this embodiment, the linear support vector machine classification model uses linear kernel parameters. By constraining the model complexity, the classification stage only involves multiplication and addition operations, thus achieving low-power operation of the audio environment classification model. This makes the audio environment classification model suitable for real-time operation in low-power embedded audio devices.

[0112] The audio environment classification model based on linear support vector machines constructed in this invention can be used to distinguish multiple environment categories for environmental audio. Specifically, the multiple environment categories may include quiet environment, quiet conversation environment, noisy environment, noisy conversation environment, and music environment.

[0113] Based on the same technical concept, the present invention also provides an audio environment recognition method, which recognizes the audio environment based on a linear support vector machine-based audio environment classification model constructed by any of the foregoing embodiments or combinations of embodiments.

[0114] Figure 6 This is a schematic flowchart of an audio environment recognition method according to an embodiment of the present invention. See also... Figure 6 As shown, the audio environment recognition method may include at least the following steps S602 to S606.

[0115] Step S602: Acquire audio signals from the environment to be identified, and divide the audio signals into multiple signal frames.

[0116] Step S604: Preprocess each signal frame and extract the selected acoustic features from the preprocessed signal frames.

[0117] Step S606: Input the selected acoustic features of each signal frame into the audio environment classification model to obtain the environment category classification result of each signal frame.

[0118] This embodiment utilizes a trained audio environment classification model based on linear support vector machines to identify audio environment categories, thereby achieving high-precision identification of multiple audio environment categories.

[0119] The audio signal acquisition, framing, preprocessing, and acoustic feature extraction operations in steps S602 and S604 can be the same as the audio signal acquisition, framing, preprocessing, and acoustic feature extraction operations in the aforementioned audio environment classification model construction method.

[0120] In some embodiments, before inputting the extracted post-selected acoustic features into the audio environment classification model, the post-selected acoustic features can be standardized to reduce the impact of differences in the dimensions of different features on the classification results.

[0121] In some embodiments, the environmental category classification results are output in frame-level or short-time window-level format to reflect the current environmental state.

[0122] Figure 7 This is a schematic flowchart of an audio environment recognition method according to another embodiment of the present invention. See also Figure 7 As shown, in some embodiments, the audio environment recognition method may further include step S608: after obtaining the environment category classification results of multiple consecutive signal frames, performing time-domain stabilization processing on the environment category classification results of multiple consecutive signal frames.

[0123] Optionally, stabilization processing may include multi-frame voting decisions, sliding time window statistics, or smooth output of decision results.

[0124] By using continuous frame decision-making and stabilization processing, the frequent switching of environment categories between adjacent frames can be avoided, generating stable and reliable environment recognition results, which can serve as the control basis for subsequent audio processing or parameter adjustment of embedded audio devices (such as hearing aids).

[0125] In some embodiments, the audio environment recognition method may further include step S610: outputting the stabilized environment category classification result in the form of frame level or short window level, thereby achieving stable environment recognition output.

[0126] Based on the same technical concept, the present invention also provides an electronic device 700. Figure 8 This is a schematic structural block diagram of an electronic device 700 according to an embodiment of the present invention.

[0127] like Figure 8 As shown, the electronic device 700 may include a memory 720, a processor 710, and a machine-executable program 721 stored in the memory 720 and running on the processor 710. When the processor 710 executes the machine-executable program 721, it implements the audio environment recognition method of any of the above embodiments or combinations of embodiments.

[0128] In some embodiments, the electronic device 700 may be an embedded audio device.

[0129] In some specific embodiments, the embedded audio device may include hearing aids, smart headphones, or wearable audio devices, etc.

[0130] The technical solution of this invention can control the algorithm complexity while ensuring a high accuracy of environmental recognition. It is suitable for low-power, real-time embedded audio devices, and is especially suitable for hearing aid environmental classification applications.

[0131] Compared with the prior art, the technical solution of the present invention has at least the following beneficial effects: (1) It can achieve fine differentiation of various complex audio environments; (2) The computational complexity of the model is significantly reduced through feature combination optimization; (3) A linear kernel support vector machine is used, which is suitable for low-power embedded implementation; (4) Introduce data validity and numerical security mechanisms to improve system stability; (5) Provide a reliable environmental decision basis for the adaptive algorithms of hearing aids and smart audio devices.

[0132] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer program product for use by, or in conjunction with, instruction execution systems, apparatus or devices (such as computer-based systems, processor-included systems or other systems that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0133] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system.

[0134] The processor 710 can be a single-core processor, a multi-core processor, a computing cluster, or any other configuration. The memory 720 can include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0135] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in every case. Furthermore, the method may include additional operations. Within the scope of the technical concept provided by the method in this embodiment, additional variations can be made to the above method.

[0136] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for constructing an audio environment classification model based on linear support vector machines, comprising: Audio signals from various sample environments are collected, and each audio signal is divided into multiple signal frames. Each signal frame is preprocessed, and multidimensional acoustic features are extracted from the preprocessed signal frames. The contribution of each acoustic feature to environmental classification is evaluated, and the selected acoustic features are selected from the multidimensional acoustic features based on the contribution. Multiple feature combinations are generated based on the selected acoustic features, and the optimal feature combination is determined by evaluating the classification performance of each feature combination. Based on the preferred feature combination, a linear support vector machine classification model is constructed and trained to obtain an audio environment classification model based on linear support vector machine for multi-environment category recognition.

2. The method for constructing an audio environment classification model based on linear support vector machines according to claim 1, wherein, The steps for constructing and training a linear support vector machine classification model based on the preferred feature combination include: The feature vectors of the preferred feature combinations are standardized; A linear support vector machine classification model is constructed based on standardized feature vectors, wherein the support vector machine uses a linear kernel function; The linear support vector machine classification model is extended by a multi-class strategy to decompose the multi-class environment recognition problem into multiple binary sub-problems; The linear support vector machine classification model is trained using the labeled optimal feature combination as training samples, and the model performance is evaluated during the training process to determine the optimal model parameters; Save and solidify the preferred model parameters of the linear support vector machine classification model, the feature combination index used, and the statistical parameters required for feature vector standardization.

3. The method for constructing an audio environment classification model based on linear support vector machines according to claim 1, wherein, The steps of evaluating the contribution of each acoustic feature to environmental classification and selecting post-selection acoustic features from the multidimensional acoustic features based on the contribution include: Generate category labels for each acoustic feature, and form a training sample set based on the acoustic features and their category labels; A feature importance evaluation classification model is constructed by learning from the training sample set. The input and output of the feature importance evaluation classification model are the feature vector and category label of the acoustic feature, respectively. The contribution of each acoustic feature to the environmental classification results is calculated based on the change in model classification performance caused by feature perturbation. Acoustic features that meet the contribution requirements are selected as the post-selection acoustic features; The contribution requirements include: The contribution level is greater than or equal to the set contribution threshold; or The ranking of contributions is placed before the set ranking.

4. The method for constructing an audio environment classification model based on linear support vector machines according to claim 3, wherein, The step of calculating the contribution of each acoustic feature to the environmental classification result based on the change in model classification performance caused by feature perturbation includes: Obtain the original model accuracy with all acoustic features as input; Calculate the model accuracy after perturbing any acoustic feature; The difference between the accuracy of the original model and the accuracy of the perturbated model is calculated as the contribution of the acoustic feature to the environmental classification result.

5. The method for constructing an audio environment classification model based on linear support vector machines according to claim 1, wherein, The step of determining the preferred feature combination by evaluating the classification performance of each feature combination includes: Under the constraint of the size of the feature combination, a low-complexity performance evaluation method is used to quickly screen out candidate feature combinations that meet the classification performance requirements from the multiple sets of feature combinations. A high-reliability performance evaluation method is used to evaluate the classification accuracy of each candidate feature combination, and the optimal feature combination is determined based on the classification accuracy and computational cost of each candidate feature combination.

6. The method for constructing an audio environment classification model based on linear support vector machines according to claim 5, wherein, The low-complexity performance evaluation method includes low-fold cross-validation with fewer than 5 folds. or, The step of quickly selecting candidate feature combinations that meet the classification performance requirements from the multiple sets of feature combinations using a low-complexity performance evaluation method under the constraint of the feature combination size includes: For each combination of features, calculate the multiple class centers of each acoustic feature relative to the multiple types of sample environments; The inter-class distance and intra-class divergence of each feature combination are calculated based on the multiple class centers of each acoustic feature in each feature combination. The ratio of inter-class distance to intra-class divergence for each feature combination is used as an evaluation index for classification performance, and candidate feature combinations are selected based on the evaluation index.

7. The method for constructing an audio environment classification model based on linear support vector machines according to claim 5, wherein, The high reliability performance evaluation method includes K-fold cross-validation, where K is an integer greater than or equal to 5; The steps for evaluating the classification accuracy of each candidate feature combination using a high-reliability performance evaluation method include: For each candidate feature combination, the feature vector of the candidate feature combination is used as the training sample, and the training sample is divided into K parts; Based on the K training samples, the pre-set classification model is trained and tested in a K-fold loop to obtain the classification performance value of each fold. Calculate the K-fold average classification performance value of the candidate feature combination based on the classification performance value of each fold; The classification performance variance is calculated based on the classification performance value of each fold and the average classification performance value of K folds of the candidate feature combination, and is used as the stability index of the candidate feature combination. The classification accuracy of each candidate feature combination is evaluated based on its K-fold average classification performance value and stability index.

8. The method for constructing an audio environment classification model based on linear support vector machines according to claim 1, wherein, The multi-dimensional acoustic features extracted from each preprocessed signal frame constitute a frame-level feature vector. Before assessing the contribution of each acoustic feature to environmental classification, the construction method further includes: Detect whether there are abnormal feature values ​​in the feature vectors of each frame; If it exists, then remove or ignore that frame-level feature vector.

9. The method for constructing an audio environment classification model based on linear support vector machines according to any one of claims 1-8, wherein, The acoustic features include at least one of the following: energy-related features, spectral structure features, harmonic-related features, speech-related features, and music discrimination-related features; The energy-related features include noise level; The spectral structural features include spectral entropy and spectral centroid; The speech-related features include signal-to-noise ratio features; The music discrimination features include autocorrelation value, number of spectral peaks, and zero-crossing rate; The preprocessing includes at least one of DC component removal processing, filtering processing, energy normalization processing, and pre-emphasis processing. The multiple environment categories include various types such as quiet environment, quiet conversation environment, noisy environment, noisy conversation environment, and music environment.

10. An audio environment recognition method based on an audio environment classification model constructed according to any one of claims 1-9, comprising: The audio signal in the environment to be identified is collected and then divided into multiple signal frames. Each signal frame is preprocessed, and the selected acoustic features are extracted from the preprocessed signal frames. The selected acoustic features of each signal frame are input into the audio environment classification model to obtain the environment category classification result for each signal frame.

11. The audio environment recognition method according to claim 10, wherein, The audio environment recognition method further includes: After obtaining the environmental category classification results of multiple consecutive signal frames, the environmental category classification results of multiple consecutive signal frames are stabilized in the time domain. Output the stabilized environment category classification results in frame-level or short-window-level format; The stabilization process includes multi-frame voting decisions, sliding time window statistics, or smooth output of decision results.

12. An electronic device comprising a memory, a processor, and a machine-executable program stored in the memory and running on the processor, wherein the processor, when executing the machine-executable program, implements the audio environment recognition method according to claim 10 or 11.

13. The electronic device according to claim 12, wherein, The electronic device is an embedded audio device; The embedded audio device includes hearing aids, smart headphones, or wearable audio devices.

Images