Sound control method and device based on emotion perception, terminal equipment and storage medium

Abstract

The invention discloses a sound equipment control method and device based on emotion perception, terminal equipment and a storage medium, and the method comprises the steps: inputting a first offset, a second offset, a third offset, a rhythm feature vector, a spectrum feature vector and a tone quality feature vector into a preset emotion classification model, obtaining an emotion category and a confidence coefficient of the voice signal; if the confidence coefficient is lower than a preset threshold value, performing weighted correction on the tone quality feature vector according to a text semantic feature vector corresponding to the voice signal, inputting the tone quality feature vector into the emotion classification model again, and obtaining the emotion category and the confidence coefficient of the voice signal again; and according to the emotion type of the voice signal, adjusting operation parameters of the sound equipment. According to the invention, self-adaptive active adjustment from accurate emotion recognition to sound content, intonation and sound operation parameters is realized.

Description

Technical Field

[0001] This invention relates to the field of smart homes, and more particularly to a sound control method, device, terminal equipment, and storage medium based on emotion perception. Background Technology

[0002] With the development of artificial intelligence and Internet of Things technologies, smart speakers have become the core entry point for modern smart homes. Existing smart speakers (such as smart voice assistants) mainly possess speech recognition (ASR) and natural language processing (NLP) capabilities, enabling them to perform corresponding operations based on users' explicit voice commands (such as "turn on the air conditioner" or "play music").

[0003] Most existing speech emotion recognition technologies are based on general models trained on common datasets. However, there are significant differences in vocal habits and physiological structures among different users. For example, some users naturally have loud voices and speak quickly, which can easily be misidentified as "anger" by general models; the vocal cord vibrations of the elderly or users with specific voices are naturally accompanied by slight tremors, which can easily be misidentified as "sadness" or "fear." In other words, existing models struggle to distinguish between a user's "normal state" and genuine "emotional fluctuations," resulting in low recognition accuracy and a high likelihood of false triggers. Summary of the Invention

[0004] This invention provides an audio control method, device, terminal equipment, and storage medium based on emotion perception, to solve the technical problems in the prior art, such as low accuracy of emotion recognition and inability to identify fake emotions, caused by ignoring individual differences in user voices, lack of microscopic acoustic feature analysis, and lack of multimodal semantic verification mechanisms.

[0005] To achieve the above objectives, a first aspect of this application provides a sound control method based on emotion perception, comprising: The audio system continuously collects the current user's voice signal and performs frame segmentation and feature extraction on the voice signal to obtain prosodic feature vectors, spectral feature vectors, and sound quality feature vectors. The prosodic feature vector includes statistical features of fundamental frequency trajectory, energy intensity, speech rate, and pause duration. The spectral feature vector includes features of Mel-frequency cepstral coefficients and dynamic differences of cepstral coefficients. The sound quality feature vector includes physical features of fundamental frequency perturbation, amplitude perturbation, and harmonic noise ratio. Based on the speech signal, a voiceprint feature vector is extracted, and the voiceprint feature vector is matched with a preset user voiceprint database to determine the user identity of the current user. The emotional baseline parameters bound to the user's identity are retrieved, and a prosodic baseline vector, a spectral baseline vector, and a sound quality baseline vector are constructed based on the emotional baseline parameters. The emotional baseline parameters include the average fundamental frequency value, fundamental frequency standard deviation, average speech rate, and average energy value of the speech received by the user in a non-emotionally activated state. Calculate the first offset between the prosodic feature vector and the prosodic reference vector, the second offset between the spectral feature vector and the spectral reference vector, and the third offset between the phonological feature vector and the phonological reference vector, respectively. The first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector are input into a preset emotion classification model to obtain the emotion category and confidence level of the speech signal. If the confidence level is lower than a preset threshold, the phonological feature vector is weighted and corrected according to the text semantic feature vector corresponding to the speech signal. Then, the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector are input into the emotion classification model again to obtain the emotion category and confidence level of the speech signal. Adjust the operating parameters of the audio system based on the emotional category of the voice signal.

[0006] In one possible implementation of the first aspect, before extracting a voiceprint feature vector based on the speech signal and matching the voiceprint feature vector with a preset user voiceprint database to determine the user identity of the current user, the following specific steps are included: The speaker continuously identifies the text content of the speech signal based on the preset TinyBERT model, generating several text feature vectors in the time series; The identified text content is detected. If a text fragment containing control instructions is detected, the text feature vector corresponding to the fragment is determined to be the instruction text feature vector. Using the feature vector of the last instruction text as the dividing point, extract all non-instruction text feature vectors before the dividing point as historical context feature vectors, and extract all non-instruction text feature vectors after the dividing point as current context feature vectors. The historical context feature vector and the current context feature vector are weighted and concatenated using an attention mechanism to generate the text semantic feature vector.

[0007] In one possible implementation of the first aspect, the sentiment classification model employs a parallel convolutional recurrent neural network architecture, specifically including: A convolutional feature extraction layer is used to perform convolution operations on the spectral feature vector to extract frequency domain texture features; A bidirectional recurrent neural network layer is used to receive the output of the convolutional feature extraction layer and the prosodic feature vector to capture the emotional evolution dependency in the time series. An attention mechanism layer is used to adaptively adjust the feature weights at different time steps based on the magnitudes of the first offset, the second offset, and the third offset. The classification layer is used to output a probability distribution of the emotion category based on the weighted features.

[0008] In one possible implementation of the first aspect, the emotional baseline parameters further include: a fundamental frequency perturbation reference value and an amplitude perturbation reference value; The calculation process of the fundamental frequency perturbation reference value is as follows: extract the fundamental frequency period sequence of the user's emotionless speech, calculate the ratio of the absolute difference between the durations of adjacent periods to the average duration of the period, and obtain the single-sample local perturbation value; then calculate the arithmetic mean of all the single-sample local perturbation values ​​in the historical sample set to obtain the fundamental frequency perturbation reference value. The calculation process of the amplitude perturbation benchmark value is as follows: extract the pitch peak amplitude sequence of the user's emotionless speech, calculate the ratio of the absolute difference between adjacent amplitudes to the average amplitude, and obtain the single-sample local amplitude perturbation value; then calculate the arithmetic mean of all the single-sample local amplitude perturbation values ​​in the historical sample set to obtain the amplitude perturbation benchmark value.

[0009] In one possible implementation of the first aspect, calculating the third offset between the sound quality feature vector and the sound quality reference vector specifically includes: The real-time fundamental frequency perturbation value and the real-time amplitude perturbation value of the speech signal are calculated to obtain the sound quality feature vector; the first dimension component of the sound quality feature vector is the real-time fundamental frequency perturbation value, and the second dimension component of the sound quality feature vector is the real-time amplitude perturbation value; The difference between the sound quality feature vector and the sound quality reference vector is used as the third offset.

[0010] In one possible implementation of the first aspect, the weighted correction of the sound quality feature vector based on the text semantic feature vector corresponding to the speech signal specifically includes: The sound quality feature vector is mapped to the same dimensional space as the text semantic feature vector through a preset fully connected layer; Calculate the cosine similarity between the text semantic feature vector and the sound quality feature vector; If the cosine similarity is positive and less than the preset similarity threshold, the sound quality feature vector is extended in the original direction by half the projection value of the text semantic feature vector; If the cosine similarity is positive and greater than or equal to a preset similarity threshold, the projection value of the text semantic feature vector is extended in the original direction by the sound quality feature vector. If the cosine similarity is negative, it is determined that the emotional content of the speech signal conflicts with the text content, and the weight of the high-frequency components in the sound quality feature vector is increased.

[0011] In one possible implementation of the first aspect, adjusting the operating parameters of the audio system according to the emotion category of the speech signal specifically includes: Adjust the speech rate, tone, and timbre of the audio synthesized output; Based on the emotion category of the voice signal, the target media resource is matched in the preset emotion-content mapping table of the audio. Send linkage control commands to smart home devices connected to the speaker to adjust the ambient light color or ambient temperature.

[0012] A second aspect of this application provides an emotion-based audio control device, comprising: The acquisition module is used to continuously acquire the current user's voice signal and extract features to obtain prosodic feature vectors, spectral feature vectors, and sound quality feature vectors. The determination module is used to extract voiceprint feature vectors based on the speech signal, match the voiceprint feature vectors with a preset user voiceprint database, and determine the user identity of the current user. The retrieval module is used to retrieve the emotional baseline parameters bound to the user's identity, and construct a prosodic reference vector, a spectral reference vector, and a sound quality reference vector based on the emotional baseline parameters; the emotional baseline parameters include the average fundamental frequency value, fundamental frequency standard deviation, average speech rate, and average energy value of the speech received by the speaker when the user is in a non-emotionally activated state. The calculation module is used to calculate the first offset between the prosodic feature vector and the prosodic reference vector, the second offset between the spectral feature vector and the spectral reference vector, and the third offset between the phonological feature vector and the phonological reference vector, respectively. The recognition module is used to input the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector into a preset emotion classification model to obtain the emotion category and confidence level of the speech signal; if the confidence level is lower than a preset threshold, the phonological feature vector is weighted and corrected according to the text semantic feature vector corresponding to the speech signal, and then the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector are input into the emotion classification model again to obtain the emotion category and confidence level of the speech signal; The adjustment module is used to adjust the operating parameters of the audio system according to the emotional category of the voice signal.

[0013] A third aspect of this application provides a terminal device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement an emotion-based audio control method as described above.

[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an emotion-based sound control method as described above.

[0015] This invention first utilizes DSP technology to deconstruct speech in three dimensions: macroscopic prosody, mesoscopic spectrum, and microscopic sound quality (fundamental frequency / amplitude perturbation), enabling holographic capture of deep emotional features, including physiological tremors. Second, it identifies the user through voiceprint recognition and retrieves their emotional baseline in a calm state, calculating the offset between real-time features and the baseline. This mechanism establishes an evaluation system based on the individual's normal state, effectively eliminating interference from individual differences caused by naturally loud voices or fast speaking speeds, achieving accurate identification of each user. Third, during the model inference stage, if the initial confidence level is low, textual semantic vectors are introduced to weight and correct the sound quality features. This dual-modal verification mechanism uses explicit text to guide implicit acoustic features, effectively solving the identification challenges of complex emotions such as irony or restraint. Finally, the audio operating parameters are adjusted according to the final emotion category, realizing a closed loop of emotional interaction from accurate perception to proactive feedback.

[0016] In summary, this invention integrates macroscopic prosody with microscopic sound quality (fundamental frequency / amplitude perturbation) features to capture physiological emotional fluctuations that are difficult to fake; it uses voiceprints to lock in identity and eliminates misjudgments caused by individual voice differences by calculating the offset between real-time features and the user's "emotional baseline"; under low confidence, it uses text semantics to dynamically adjust acoustic weights to accurately identify complex contexts such as irony or restraint. Ultimately, it achieves everything from accurate emotion recognition to adaptive and proactive adjustment of audio content, tone, and audio operating parameters. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating an embodiment of the present invention that provides a sound control method based on emotion perception. Figure 2 This is a schematic diagram of the structure of an emotion-based audio control device provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] To resolve the above issues, please refer to [link / reference]. Figure 1An embodiment of the present invention provides a sound control method based on emotion perception, comprising: S10. The speaker continuously collects the current user's voice signal and performs frame segmentation and feature extraction on the voice signal to obtain prosodic feature vector, spectral feature vector, and sound quality feature vector. The prosodic feature vector includes statistical features of fundamental frequency trajectory, energy intensity, speech rate, and pause duration. The spectral feature vector includes features of Mel-frequency cepstral coefficients and dynamic differences of cepstral coefficients. The sound quality feature vector includes physical features of fundamental frequency perturbation, amplitude perturbation, and harmonic noise ratio.

[0020] S11. Extract voiceprint feature vectors based on the speech signal, match the voiceprint feature vectors with a preset user voiceprint database, and determine the user identity of the current user.

[0021] S12. Retrieve the emotional baseline parameters bound to the user's identity, and construct a prosodic reference vector, a spectral reference vector, and a sound quality reference vector based on the emotional baseline parameters; the emotional baseline parameters include the average fundamental frequency value, fundamental frequency standard deviation, average speech rate, and average energy value of the speech received by the user in a non-emotionally activated state.

[0022] S13. Calculate the first offset between the prosodic feature vector and the prosodic reference vector, the second offset between the spectral feature vector and the spectral reference vector, and the third offset between the phonological feature vector and the phonological reference vector, respectively.

[0023] S14. Input the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector into a preset emotion classification model to obtain the emotion category and confidence level of the speech signal; if the confidence level is lower than a preset threshold, then after weighting and correcting the phonological feature vector according to the text semantic feature vector corresponding to the speech signal, input the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector into the emotion classification model again to obtain the emotion category and confidence level of the speech signal.

[0024] S15. Adjust the operating parameters of the audio system according to the emotional category of the voice signal.

[0025] S10 is a process of deconstructing the multidimensional acoustic features of the speech signal received by the audio system. This step utilizes digital signal processing (DSP) technology to convert the continuous analog speech signal into a discrete feature vector that can be processed by a computer. The system does not simply extract volume, but deconstructs the speech from three dimensions: macroscopic dimension (prosodic features): calculating the fundamental frequency trajectory, energy, and speech rate to capture the rhythm and intensity of speech; mesoscopic dimension (spectral features): depicting the spectral envelope (i.e., timbre texture) of the sound through Mel-frequency cepstral coefficients (MFCCs) and their differences; and microscopic dimension (sound quality features): focusing on calculating fundamental frequency jitter and amplitude shimmer. Example: Even when a person tries to conceal their emotions, subtle tension in their vocal cord muscles can cause microsecond-level irregular vibrations in the vocal cord vibration cycle, which are captured by the jitter and shimmer parameters.

[0026] By extracting holographic features from three dimensions—macro-rhythm, meso-timbre texture, and micro-vocal cord physical stability—this step can comprehensively capture the user's externalized emotional features. In particular, by introducing micro-timbre features, the system is able to recognize deep physiological emotional reactions that are difficult for users to fake through subjective will (such as vocal cord tremors during extreme sadness or fear), thereby significantly improving the richness and authenticity of the features.

[0027] S11-S13 is an identity-based baseline differential calibration process: S11 uses voiceprint recognition technology to identify the current speaker (e.g., confirming whether it is a "father" or a "child"); S12 loads the user's "emotional baseline parameters" (i.e., the average value in a calm state) from the database; and S13 does not directly use the current feature absolute value, but instead calculates the offset. ,in, The offset represents the intensity of the emotion. Real-time feature values ​​extracted for S10; The baseline values ​​retrieved for S12 (prosodic baseline vector, spectral baseline vector, and phonological baseline vector). Example: If a user naturally speaks fast (high baseline speaking speed) and their current speaking speed is also fast, then... If the value is close to zero, it is considered calm; without a baseline for comparison, the general model may misjudge it as "anxiety".

[0028] The above steps establish a relative evaluation mechanism with the user's individual normal state as a reference. This step effectively eliminates recognition interference caused by the user's innate vocal habits (such as naturally loud voice or fast speech) or differences in physiological characteristics. By converting absolute feature values ​​into relative offset input models, the problem of misjudgment by general emotion models when facing different family members is solved, and adaptive and accurate recognition for different users is achieved.

[0029] S14 is the verification inference process: the offset and the original feature vector are input into the sentiment classification model (such as a neural network), and the output sentiment probability is calculated. When the confidence of the model's output is low (i.e., the model is confused), textual modalities are introduced for assistance. The system extracts the textual semantic feature vectors corresponding to the speech (e.g., recognizing "I'm fine" but with a negative context), adjusts the weights of the voice quality feature vectors using textual features (e.g., amplifying the weight of the voice tremor feature), and then performs secondary inference. The textual semantics act as a "contextual cue," helping the model focus on key physical features when acoustic features are ambiguous.

[0030] S14 introduces a text-acoustic dual-modal verification mechanism triggered by a confidence threshold. This step solves the problem of insufficient information in a single modality when dealing with complex emotions (such as irony, lies, and forbearance). By using explicit textual semantics to dynamically weight and guide implicit acoustic features, it effectively corrects the model's judgment bias in ambiguous contexts and significantly improves the system's inference robustness and accuracy in complex interactive scenarios.

[0031] S15 is the response execution process: mapping the final emotion category (such as "anger", "sadness", "happiness") output by S14 to a specific device control strategy table. Example: If "anger" is detected, the volume of the audio response is reduced and soothing music is played; if "happiness" is detected, the lighting is adjusted to a warm tone and the tone of the response is increased.

[0032] This step achieves an automated closed loop from perception and computation to physical feedback, enabling smart speakers to break through traditional command-based interaction and proactively provide differentiated content services and environmental adjustments based on the user's emotional state, thereby improving the intelligence level of human-computer interaction and the user's emotional experience.

[0033] In summary, the core logic of this solution is as follows: first, it captures deep physiological and emotional characteristics through multi-dimensional acoustic deconstruction; second, it eliminates individual differences by using personalized baseline differentiation; third, it corrects judgment deviations in complex situations through semantic-sound quality dual-modal verification when there is uncertainty; and finally, it adaptively adjusts the sound system operating parameters based on accurate emotional results, thereby achieving a high-precision, interference-resistant, and deeply understanding intelligent emotional interaction control.

[0034] For example, before extracting the voiceprint feature vector based on the speech signal, matching the voiceprint feature vector with a preset user voiceprint database, and determining the user identity of the current user, the process specifically includes: The speaker continuously identifies the text content of the speech signal based on the preset TinyBERT model, generating several text feature vectors in the time series.

[0035] The detected text content is detected. If a text fragment containing control instructions is detected, the text feature vector corresponding to the fragment is determined to be the instruction text feature vector.

[0036] Using the feature vector of the last instruction text as the dividing point, extract all non-instruction text feature vectors before the dividing point as historical context feature vectors, and extract all non-instruction text feature vectors after the dividing point as current context feature vectors.

[0037] The historical context feature vector and the current context feature vector are weighted and concatenated using an attention mechanism to generate the text semantic feature vector.

[0038] The speaker utilizes a lightweight pre-trained language model (such as TinyBERT) as a semantic encoder to transcribe and map the continuously input speech signal into a sequence of text feature vectors in a high-dimensional space in real time. Simultaneously, the system uses an intent recognition algorithm to detect and identify text feature vectors containing specific control functions within the vector sequence (e.g., identifying the vector corresponding to "play a song" in "feeling down, play a song"), and marking them as logical anchors in the semantic stream. By employing a lightweight model for real-time encoding and instruction anchoring at the device side, this step achieves the transformation of unstructured continuous speech streams into computer-processable structured semantic vectors with low power consumption and low latency. Precise instruction location not only provides a semantic foundation for subsequent sentiment analysis but also effectively avoids the privacy risks and network latency that may arise from cloud processing, ensuring real-time semantic understanding.

[0039] Then, using the identified "last instruction text feature vector" as a dividing point on the timeline, the continuous semantic stream is logically segmented: the vector sequence before the dividing point is extracted as the historical context feature vector (usually representing emotional setup or background reasons, such as "I'm too tired from work today"), and the vector sequence after the dividing point is extracted as the current context feature vector (usually representing supplementary explanations or immediate state, such as "I want light music"). Through this dynamic segmentation mechanism centered on core functional instructions, this step effectively decouples the user's long speech stream into two independent semantic fields: "background causes" and "immediate intent." This structured segmentation avoids information dilution and ambiguity caused by mixing long texts, enabling the system to clearly distinguish the semantic emphasis and emotional evolution logic of the user before and after issuing the instruction, providing structural support for refined understanding.

[0040] Finally, an attention mechanism is introduced to fuse the two types of contexts after segmentation, and to calculate the contribution weights of the historical context and the current context to the final semantic understanding.

[0041] Calculation logic and formulas: in, This is the final generated text semantic feature vector; and These are the historical and current context vectors obtained from the segmentation, respectively; and The attention weight coefficients are calculated by the model using the Softmax function, representing the relative importance of historical background information and current supplementary information, respectively.

[0042] By employing an attention mechanism to weightedly concatenate the dual temporal contexts, this step simulates the human cognitive process of "connecting with context," adaptively adjusting the weight of historical background information and current supplementary information in the final semantic representation. This not only solves the long-distance dependency problem in long texts but also ensures that the final generated feature vectors can accurately focus on the semantic segments most valuable for current emotion recognition (such as automatically ignoring invalid filler words and focusing on keywords that express emotions), significantly improving the robustness of the features.

[0043] This embodiment first utilizes a lightweight model to achieve low-power real-time semantic encoding and anchor the command location. Then, it structurally segments the continuous speech stream into historical context and current intent, using the command as the boundary. Finally, it adaptively fuses the semantic features of both through an attention mechanism. This process realizes the transformation from unstructured speech stream to highly robust semantic feature vectors, ensuring that the audio system can accurately grasp the user's core emotional intent when faced with long and difficult sentences or complex dialogues.

[0044] For example, the sentiment classification model employs a parallel convolutional recurrent neural network architecture, specifically including: A convolutional feature extraction layer is used to perform convolution operations on the spectral feature vector to extract frequency domain texture features; A bidirectional recurrent neural network layer is used to receive the output of the convolutional feature extraction layer and the prosodic feature vector to capture the emotional evolution dependency in the time series. An attention mechanism layer is used to adaptively adjust the feature weights at different time steps based on the magnitudes of the first offset, the second offset, and the third offset. The classification layer is used to output a probability distribution of the emotion category based on the weighted features.

[0045] This embodiment constructs a two-dimensional feature encoding architecture based on both spatial and temporal dimensions. First, the Convolutional Feature Extraction (CNN) layer treats the input spectral feature vector as a two-dimensional spectrogram, using a sliding scan operation with convolutional kernels to extract local texture features in the frequency domain (such as the roughness or harmonic structure of sound). Subsequently, a Bidirectional Recurrent Neural Network (Bi-RNN) layer receives the texture features extracted by the CNN and independent prosodic feature vectors in parallel. Through recursive operations in both forward and reverse temporal dimensions, it captures the emotional evolution of the speech signal over time. By combining the CNN's keen perception of frequency domain details with the recurrent neural network's ability to handle long-term temporal dependencies, this step achieves a three-dimensional extraction of deep features from the speech signal; it can capture both the instantaneous bursts of timbre texture (such as the roar of anger) and the logical fluctuations of emotion throughout the sentence (such as the transition from calm to excitement), thus significantly improving the model's ability to represent dynamic and complex emotions.

[0046] The attention mechanism layer introduces a special attention mechanism, the core of which is to use "offset" as a guiding factor for attention.

[0047] Calculation logic and formulas: ,in, For the first Feature weights for each time step; This is the sum of the magnitudes of the first, second, and third offsets corresponding to this time step; This is a mapping function.

[0048] The attention layer does not treat each frame of speech equally, but dynamically assigns weights based on the degree of deviation (offset magnitude) of the frame's features from the user's emotional baseline. If a speech feature deviates significantly from the usual baseline at a certain moment (large offset), the model will assign it extremely high weight. By adaptively adjusting feature weights using feature offsets, this step gives the model the ability to "discard falsehoods and retain true information"; it can automatically ignore invalid information close to the baseline (such as interjections, silences, or emotionless statements), forcing the model to focus on those "emotional outbursts" that significantly deviate from the user's normal state, thereby accurately capturing core emotional features in long speech containing a lot of redundant information.

[0049] Finally, a fully connected layer is used as the classifier, receiving the global feature vector after weighted fusion via an attention mechanism. The classification layer maps the high-dimensional feature space to a predefined emotion category space using a non-linear activation function (such as Softmax), outputting a probability distribution vector where the category with the highest probability is the judgment result. This layer achieves the final mapping from abstract high-dimensional features to specific emotion labels; by making decisions based on weighted salient features, the classification layer can output judgment results with confidence (probability values), providing not only the final emotion assessment but also a quantitative basis for subsequent system decisions on whether text verification is necessary.

[0050] The core logic of this embodiment lies in employing a parallel architecture of CNN and Bi-RNN to simultaneously capture the frequency domain quality and temporal logic of sound. Then, a baseline offset-driven attention mechanism is used to automatically focus the model on "abnormal frames" that significantly deviate from the user's normal range. Finally, high-precision probabilistic classification is achieved based on the weighted features of these key frames. This process ensures that the model can both see the details and grasp the key points, effectively improving its ability to capture subtle emotional fluctuations.

[0051] For example, the emotional baseline parameters also include: a fundamental frequency perturbation reference value and an amplitude perturbation reference value; The calculation process of the fundamental frequency perturbation reference value is as follows: extract the fundamental frequency period sequence of the user's emotionless speech, calculate the ratio of the absolute difference between the durations of adjacent periods to the average duration of the period, and obtain the single-sample local perturbation value; then calculate the arithmetic mean of all the single-sample local perturbation values ​​in the historical sample set to obtain the fundamental frequency perturbation reference value. The calculation process of the amplitude perturbation benchmark value is as follows: extract the pitch peak amplitude sequence of the user's emotionless speech, calculate the ratio of the absolute difference between adjacent amplitudes to the average amplitude, and obtain the single-sample local amplitude perturbation value; then calculate the arithmetic mean of all the single-sample local amplitude perturbation values ​​in the historical sample set to obtain the amplitude perturbation benchmark value.

[0052] This step aims to establish personalized norms for the stability of a user's vocal cord vibration. First, speech samples from the user in a calm state (i.e., emotionless speech) are identified. For each sample, a pitch period sequence (T, representing the time it takes for one vocal cord vibration) and a pitch peak amplitude sequence (A, representing the maximum intensity of vocal cord vibration) are extracted.

[0053] Local perturbation calculation: Calculate the relative volatility of adjacent periods or amplitudes.

[0054] Fundamental frequency perturbation formula logic: (in For the first The duration of each cycle, The duration of the next cycle, (This is the average duration).

[0055] Amplitude perturbation formula logic: (in For the first Peak amplitude of each cycle, (mean amplitude).

[0056] Global benchmarking: The arithmetic mean of all single-sample local perturbation values ​​in the historical sample set is calculated to finally determine the fundamental frequency perturbation benchmark value and amplitude perturbation benchmark value for the user.

[0057] By statistically analyzing the microscopic fluctuation constants of the frequency and amplitude of vocal cord vibration in a user's calm state, this step establishes a reference standard that can quantify the user's "physiological voice characteristics." This enables the system to effectively distinguish between congenital hoarseness / tremor caused by age, smoking, or physiological structure (manifested as a high baseline value) and temporary vocal cord micro-tremor caused by emotional excitement (manifested as a significant deviation above the baseline value). This avoids misjudging users with naturally hoarse voices as sad or fearful, greatly improving the recognition accuracy for special voice groups.

[0058] In summary, this embodiment utilizes historical calm voice data from users, delving into the microsecond level of vocal cord vibration cycles. By calculating the relative fluctuation ratio between adjacent cycles and amplitudes and taking the historical average, it quantifies the user's unique physiological perturbation baseline. This process transforms the abstract concept of "voice quality" into a measurable mathematical benchmark, providing a crucial calibration basis for subsequently distinguishing between "physiological tremor" and "emotional tremor."

[0059] For example, calculating the third offset between the sound quality feature vector and the sound quality reference vector specifically includes: The real-time fundamental frequency perturbation value and the real-time amplitude perturbation value of the speech signal are calculated to obtain the sound quality feature vector; the first dimension component of the sound quality feature vector is the real-time fundamental frequency perturbation value, and the second dimension component of the sound quality feature vector is the real-time amplitude perturbation value; The difference between the sound quality feature vector and the sound quality reference vector is used as the third offset.

[0060] This step first performs a physical analysis of the current speech signal, calculating the real-time fundamental frequency perturbation value (…). (characterizing the instantaneous instability of the current vocal cord vibration frequency) and real-time amplitude perturbation value ( This characterizes the instantaneous instability of the current vocal cord vibration amplitude, and uses these two as the first and second dimensional components respectively to construct a sound quality feature vector. Subsequently, the system performs a vector subtraction operation, subtracting the preset sound quality reference vector from the feature vector. (i.e., the perturbation baseline in the user's calm state), thus obtaining the third offset (i.e., the difference vector). ).

[0061] The above steps, by mapping independent physical micro-features to a unified vector space and calculating the difference relative to the individual baseline, realize the process of transforming "absolute physical quantities" into "relative emotional variables." This differential vectorization process not only preserves the independent dimensional information of frequency jitter and amplitude jitter, allowing the model to distinguish different types of emotions (such as fear focusing on frequency jitter and anger focusing on amplitude jitter), but more importantly, it effectively removes the user's original physiological voice features (such as naturally hoarse voice), ensuring that the model only focuses on the "additional" instability caused by emotional fluctuations, thereby eliminating the interference of individual physiological differences on recognition accuracy.

[0062] In summary, this embodiment structures the real-time captured microscopic physical stability of the vocal cords (fundamental frequency and amplitude perturbation) into a multidimensional vector, and extracts the "emotional deviation vector" by calculating the difference between the vector and the user's physiological baseline vector. This process transforms complex physical acoustic features into standardized relative changes, providing a mathematical basis for the system to accurately distinguish between "innate vocal traits" and "emotional vocal cord tremors".

[0063] For example, the step of weighting and correcting the sound quality feature vector based on the text semantic feature vector corresponding to the speech signal specifically includes: The sound quality feature vector is mapped to the same dimensional space as the text semantic feature vector through a preset fully connected layer; Calculate the cosine similarity between the text semantic feature vector and the sound quality feature vector; If the cosine similarity is positive and less than the preset similarity threshold, the sound quality feature vector is extended in the original direction by half the projection value of the text semantic feature vector; If the cosine similarity is positive and greater than or equal to a preset similarity threshold, the projection value of the text semantic feature vector is extended in the original direction by the sound quality feature vector. If the cosine similarity is negative, it is determined that the emotional content of the speech signal conflicts with the text content, and the weight of the high-frequency components in the sound quality feature vector is increased.

[0064] First, a fully connected layer is used as a feature projector to linearly map the low-dimensional audio quality feature vector to the same vector space as the high-dimensional text semantic feature vector, thus solving the problem of multimodal feature dimension mismatch. Then, the cosine similarity between the two vectors is calculated.

[0065] Formula logic: in, and These are the mapped audio quality vector and text vector, respectively; The angle between the two vectors in the feature space. This value quantifies the degree of consistency between the "speech perception" and the "text meaning" in terms of emotional polarity (1 represents complete consistency, -1 represents complete opposite).

[0066] By constructing a unified feature space and calculating the vector angle, this step achieves the mathematical alignment of speech physical features and text semantic features; this enables the system to quantitatively determine whether a user's words and actions are consistent, providing a precise mathematical basis for distinguishing between conventional expressions and complex emotions (such as irony and concealment).

[0067] Then, divergence processing is performed based on the sign and magnitude of the similarity: Positive enhancement (when consistent): When the similarity is positive, it is considered that the semantics and sound are consistent. The system extends the magnitude of the sound quality vector based on the projection value of the text vector onto the sound quality vector. If the similarity is high (greater than the threshold), the full projection is superimposed (strong confirmation); if the similarity is low, half projection is superimposed (weak confirmation). This is equivalent to using clear textual meaning to "amplify" weak sound features.

[0068] Negative Reconstruction (In Case of Conflict): When the similarity is negative, it is considered ironic or contradictory. The text is deemed unreliable, and the weight of the high-frequency components (corresponding to rapidly changing physical features such as fundamental frequency jitter and amplitude perturbation shimmer) in the sound quality feature vector is increased.

[0069] This embodiment implements a "semantic-guided acoustic enhancement mechanism". When words and actions are consistent, it uses explicit textual intent to enhance the salience of implicit acoustic features, solving the problem of missed detection caused by weak vocal emotions; when words and actions conflict (such as irony), it intelligently reduces the interference of text weights and forces the model to focus on physiological high-frequency vocal cord vibrations that are difficult to fake, thereby accurately restoring the user's true emotions that are masked by language.

[0070] For example, adjusting the operating parameters of the audio system according to the emotion category of the voice signal specifically includes: Adjust the speech rate, tone, and timbre of the audio synthesized output; Based on the emotion category of the voice signal, the target media resource is matched in the preset emotion-content mapping table of the audio. Send linkage control commands to smart home devices connected to the speaker to adjust the ambient light color or ambient temperature.

[0071] The speaker uses the identified emotion category as an index key to query a pre-set strategy mapping table in real time: On one hand, it directly modulates the underlying acoustic parameters of the text-to-speech (TTS) engine, changing the speech rate by stretching phoneme duration, changing the pitch by adjusting the fundamental frequency mean, and changing the timbre by switching vocoder parameters (e.g., automatically reducing speech rate and pitch when "sadness" is detected); on the other hand, it uses a metadata filtering algorithm to retrieve target resources with empathy tags (such as resonance) or compensatory tags (such as healing) from the media library. This achieves a qualitative leap in human-computer interaction from "mechanical response" to "empathic communication"; by enabling the speaker's feedback sound characteristics and playback content to "resonate" or "complementarily adjust" with the user's current mood, it not only eliminates the cold and rigid machine feel of traditional voice assistants, but also provides users with immediate psychological comfort and emotional guidance.

[0072] This embodiment uses a determined emotion category as a trigger signal to drive the speaker to execute a two-way adjustment strategy—internally adjusting its own voice form and content strategy, and externally coordinating with home appliances to change the physical environment to achieve "atmosphere healing." This process constitutes a complete emotional feedback loop, enabling the intelligent system to possess emotional care capabilities similar to those of humans.

[0073] Compared to existing technologies, this invention first utilizes DSP technology to deconstruct speech in three dimensions: macroscopic prosody, mesoscopic spectrum, and microscopic sound quality (fundamental frequency / amplitude perturbation), enabling holographic capture of deep emotional features, including physiological tremors. Secondly, it identifies the user through voiceprint recognition and retrieves their emotional baseline in a calm state, calculating the offset between real-time features and the baseline. This mechanism establishes an evaluation system based on individual norms, effectively eliminating interference from individual differences caused by naturally loud voices or fast speech rates, achieving accurate identification of each user. Thirdly, during the model inference stage, if the initial confidence level is low, textual semantic vectors are introduced to weighted correct the sound quality features. This dual-modal verification mechanism utilizes explicit text to guide implicit acoustic features, effectively solving the identification challenges of complex emotions such as irony or restraint. Finally, the audio operating parameters are adjusted according to the final emotion category, achieving a closed loop of emotional interaction from accurate perception to proactive feedback.

[0074] In summary, this invention integrates macroscopic prosody with microscopic sound quality (fundamental frequency / amplitude perturbation) features to capture physiological emotional fluctuations that are difficult to fake; it uses voiceprints to lock in identity and eliminates misjudgments caused by individual voice differences by calculating the offset between real-time features and the user's "emotional baseline"; under low confidence, it uses text semantics to dynamically adjust acoustic weights to accurately identify complex contexts such as irony or restraint. Ultimately, it achieves everything from accurate emotion recognition to adaptive and proactive adjustment of audio content, tone, and audio operating parameters.

[0075] See Figure 2One embodiment of this application provides an audio control device based on emotion perception, including: a data acquisition module 20, a determination module 21, a retrieval module 22, a calculation module 23, a recognition module 24, and an adjustment module 25.

[0076] The acquisition module 20 is used to continuously acquire the current user's voice signal and extract features to obtain prosodic feature vector, spectral feature vector and sound quality feature vector.

[0077] The determination module 21 is used to extract voiceprint feature vectors based on the speech signal, match the voiceprint feature vectors with a preset user voiceprint database, and determine the user identity of the current user.

[0078] The retrieval module 22 is used to retrieve the emotional baseline parameters bound to the user's identity, and construct a prosodic reference vector, a spectral reference vector, and a sound quality reference vector based on the emotional baseline parameters; the emotional baseline parameters include the average fundamental frequency value, fundamental frequency standard deviation, average speech rate, and average energy value of the speech received by the speaker when the user is in a non-emotionally activated state.

[0079] The calculation module 23 is used to calculate the first offset between the prosodic feature vector and the prosodic reference vector, the second offset between the spectral feature vector and the spectral reference vector, and the third offset between the phonological feature vector and the phonological reference vector, respectively.

[0080] The recognition module 24 is used to input the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector into a preset emotion classification model to obtain the emotion category and confidence level of the speech signal; if the confidence level is lower than a preset threshold, the phonological feature vector is weighted and corrected according to the text semantic feature vector corresponding to the speech signal, and then the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector are input into the emotion classification model again to obtain the emotion category and confidence level of the speech signal.

[0081] The adjustment module 25 is used to adjust the operating parameters of the audio system according to the emotional category of the voice signal.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the drug storage management device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0083] Compared to existing technologies, this invention first utilizes DSP technology to deconstruct speech in three dimensions: macroscopic prosody, mesoscopic spectrum, and microscopic sound quality (fundamental frequency / amplitude perturbation), enabling holographic capture of deep emotional features, including physiological tremors. Secondly, it identifies the user through voiceprint recognition and retrieves their emotional baseline in a calm state, calculating the offset between real-time features and the baseline. This mechanism establishes an evaluation system based on individual norms, effectively eliminating interference from individual differences caused by naturally loud voices or fast speech rates, achieving accurate identification of each user. Thirdly, during the model inference stage, if the initial confidence level is low, textual semantic vectors are introduced to weighted correct the sound quality features. This dual-modal verification mechanism utilizes explicit text to guide implicit acoustic features, effectively solving the identification challenges of complex emotions such as irony or restraint. Finally, the audio operating parameters are adjusted according to the final emotion category, achieving a closed loop of emotional interaction from accurate perception to proactive feedback.

[0084] In summary, this invention integrates macroscopic prosody with microscopic sound quality (fundamental frequency / amplitude perturbation) features to capture physiological emotional fluctuations that are difficult to fake; it uses voiceprints to lock in identity and eliminates misjudgments caused by individual voice differences by calculating the offset between real-time features and the user's "emotional baseline"; under low confidence, it uses text semantics to dynamically adjust acoustic weights to accurately identify complex contexts such as irony or restraint. Ultimately, it achieves everything from accurate emotion recognition to adaptive and proactive adjustment of audio content, tone, and audio operating parameters.

[0085] One embodiment of this application provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements an emotion-based audio control method as described above.

[0086] One embodiment of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an emotion-based sound control method as described above.

[0087] The computer device may be a smartphone, tablet, desktop computer, or cloud server, among other computing devices. This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the figures are merely examples of computer devices and do not constitute a limitation on the computer device. It may include more or fewer components than illustrated, or a combination of certain components, or different components, such as input / output devices, network access devices, etc.

[0088] The processor referred to can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0089] In some embodiments, the memory may be an internal storage unit of the computer device, such as a hard drive or RAM. In other embodiments, the memory may be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory may include both internal and external storage units of the computer device. The memory is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory can also be used to temporarily store data that has been output or will be output.

[0090] This application provides a computer program product that, when run on a computer device, enables the computer device to execute the steps described in the various method embodiments above.

[0091] In the several embodiments provided in this application, it will be understood that each block in the flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved.

[0092] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A sound control method based on emotion perception, characterized by, include: The audio system continuously collects the current user's voice signal and performs frame segmentation and feature extraction on the voice signal to obtain prosodic feature vectors, spectral feature vectors, and sound quality feature vectors. The prosodic feature vector includes statistical features of fundamental frequency trajectory, energy intensity, speech rate, and pause duration. The spectral feature vector includes features of Mel-frequency cepstral coefficients and dynamic differences of cepstral coefficients. The sound quality feature vector includes physical features of fundamental frequency perturbation, amplitude perturbation, and harmonic noise ratio. Based on the speech signal, a voiceprint feature vector is extracted, and the voiceprint feature vector is matched with a preset user voiceprint database to determine the user identity of the current user. The emotional baseline parameters bound to the user's identity are retrieved, and a prosodic baseline vector, a spectral baseline vector, and a sound quality baseline vector are constructed based on the emotional baseline parameters. The emotional baseline parameters include the average fundamental frequency value, fundamental frequency standard deviation, average speech rate, and average energy value of the speech received by the user in a non-emotionally activated state. Calculate the first offset between the prosodic feature vector and the prosodic reference vector, the second offset between the spectral feature vector and the spectral reference vector, and the third offset between the phonological feature vector and the phonological reference vector, respectively. The first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector are input into a preset emotion classification model to obtain the emotion category and confidence level of the speech signal. If the confidence level is lower than a preset threshold, the phonological feature vector is weighted and corrected according to the text semantic feature vector corresponding to the speech signal. Then, the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector are input into the emotion classification model again to obtain the emotion category and confidence level of the speech signal. Adjust the operating parameters of the audio system based on the emotional category of the voice signal.

2. The emotion-aware based sound control method of claim 1, wherein, Before the step of extracting a voiceprint feature vector based on the speech signal, matching the voiceprint feature vector with a preset user voiceprint database, and determining the user identity of the current user, specifically includes: The speaker continuously identifies the text content of the speech signal based on the preset TinyBERT model, generating several text feature vectors in the time series; The identified text content is detected. If a text fragment containing control instructions is detected, the text feature vector corresponding to the fragment is determined to be the instruction text feature vector. Using the feature vector of the last instruction text as the dividing point, extract all non-instruction text feature vectors before the dividing point as historical context feature vectors, and extract all non-instruction text feature vectors after the dividing point as current context feature vectors. The historical context feature vector and the current context feature vector are weighted and concatenated using an attention mechanism to generate the text semantic feature vector.

3. The emotion-aware based sound control method of claim 1, wherein, The sentiment classification model employs a parallel convolutional recurrent neural network architecture, specifically including: A convolutional feature extraction layer is used to perform convolution operations on the spectral feature vector to extract frequency domain texture features; A bidirectional recurrent neural network layer is used to receive the output of the convolutional feature extraction layer and the prosodic feature vector to capture the emotional evolution dependency in the time series. An attention mechanism layer is used to adaptively adjust the feature weights at different time steps based on the magnitudes of the first offset, the second offset, and the third offset. The classification layer is used to output a probability distribution of the emotion category based on the weighted features.

4. The emotion-aware based sound control method of claim 1, wherein, The emotional baseline parameters also include: fundamental frequency perturbation reference value and amplitude perturbation reference value; The calculation process of the fundamental frequency perturbation reference value is as follows: extract the fundamental frequency period sequence of the user's emotionless speech, calculate the ratio of the absolute difference between the durations of adjacent periods to the average duration of the period, and obtain the single-sample local perturbation value; then calculate the arithmetic mean of all the single-sample local perturbation values ​​in the historical sample set to obtain the fundamental frequency perturbation reference value. The calculation process of the amplitude perturbation benchmark value is as follows: extract the pitch peak amplitude sequence of the user's emotionless speech, calculate the ratio of the absolute difference between adjacent amplitudes to the average amplitude, and obtain the single-sample local amplitude perturbation value; then calculate the arithmetic mean of all the single-sample local amplitude perturbation values ​​in the historical sample set to obtain the amplitude perturbation benchmark value.

5. The emotion-aware based sound control method of claim 4, wherein, Calculating the third offset between the sound quality feature vector and the sound quality reference vector specifically includes: The real-time fundamental frequency perturbation value and the real-time amplitude perturbation value of the speech signal are calculated to obtain the sound quality feature vector; the first dimension component of the sound quality feature vector is the real-time fundamental frequency perturbation value, and the second dimension component of the sound quality feature vector is the real-time amplitude perturbation value; The difference between the sound quality feature vector and the sound quality reference vector is used as the third offset.

6. The emotion-aware based sound control method of claim 1, wherein, The step of weighting and correcting the sound quality feature vector based on the text semantic feature vector corresponding to the speech signal specifically includes: The sound quality feature vector is mapped to the same dimensional space as the text semantic feature vector through a preset fully connected layer; Calculate the cosine similarity between the text semantic feature vector and the sound quality feature vector; If the cosine similarity is positive and less than the preset similarity threshold, the sound quality feature vector is extended in the original direction by half the projection value of the text semantic feature vector; If the cosine similarity is positive and greater than or equal to a preset similarity threshold, the projection value of the text semantic feature vector is extended in the original direction by the sound quality feature vector. If the cosine similarity is negative, it is determined that the emotional content of the speech signal conflicts with the text content, and the weight of the high-frequency components in the sound quality feature vector is increased.

7. The emotion-aware based sound control method of claim 1, wherein, The step of adjusting the operating parameters of the audio system based on the emotion category of the voice signal specifically includes: Adjust the speech rate, tone, and timbre of the audio synthesized output; Based on the emotion category of the voice signal, the target media resource is matched in the preset emotion-content mapping table of the audio. Send linkage control commands to smart home devices connected to the speaker to adjust the ambient light color or ambient temperature.

8. A sound control device based on emotion perception, characterized in that, include: The acquisition module is used to continuously acquire the current user's voice signal and extract features to obtain prosodic feature vectors, spectral feature vectors, and sound quality feature vectors. The determination module is used to extract voiceprint feature vectors based on the speech signal, match the voiceprint feature vectors with a preset user voiceprint database, and determine the user identity of the current user. The retrieval module is used to retrieve the emotional baseline parameters bound to the user's identity, and construct a prosodic reference vector, a spectral reference vector, and a sound quality reference vector based on the emotional baseline parameters; the emotional baseline parameters include the average fundamental frequency value, fundamental frequency standard deviation, average speech rate, and average energy value of the speech received by the speaker when the user is in a non-emotionally activated state. The calculation module is used to calculate the first offset between the prosodic feature vector and the prosodic reference vector, the second offset between the spectral feature vector and the spectral reference vector, and the third offset between the phonological feature vector and the phonological reference vector, respectively. The recognition module is used to input the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector into a preset emotion classification model to obtain the emotion category and confidence level of the speech signal; if the confidence level is lower than a preset threshold, the phonological feature vector is weighted and corrected according to the text semantic feature vector corresponding to the speech signal, and then the first offset, the second offset, the third offset, the prosodic feature vector, the spectral feature vector, and the phonological feature vector are input into the emotion classification model again to obtain the emotion category and confidence level of the speech signal; The adjustment module is used to adjust the operating parameters of the audio system according to the emotional category of the voice signal.

9. A terminal device, comprising: The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements an emotion-based sound control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements an emotion-based sound control method as described in any one of claims 1 to 7.

Images