Natural emotional singing method and emotional singing audio processing method

Abstract

The application discloses a natural emotional singing method and an emotional singing audio processing method, and belongs to the singing information field, and aims to solve the problems of fragmented training elements and single feedback system of a traditional vocal music teaching method. The technical scheme comprises six training modules combined by action guidance, emotion awakening and sound area training, and a training system for improving singing emotional expression capability based on a deep learning model for real-time extraction of user singing audio emotional indexes and generation of dynamic visual feedback.

Description

Technical Field

[0001] This invention relates to the field of singing information, and in particular to a natural emotional singing method and an audio processing method for emotional singing. Background Technology

[0002] Vocal instruction, as an important branch of music education, aims to improve learners' singing skills and emotional expression. With the development of technology, traditional vocal teaching methods have gradually combined with digital technology, resulting in various singing training systems and audio processing tools. These systems typically focus on training pitch, rhythm, and vocal techniques, but often neglect the core role of emotional expression. Emotion is the soul of singing; how to effectively integrate emotion into the singing process has become a key challenge in the field of vocal instruction. Furthermore, for specific groups such as the middle-aged and elderly, their cognitive structure and learning characteristics require more intuitive and user-friendly teaching methods, where current technology has significant shortcomings.

[0003] In current technologies, vocal instruction largely relies on verbal guidance and demonstrations from teachers, with learners improving their skills through imitation. For example, some traditional teaching methods emphasize terminology and abstract instructions, such as "support your voice with breath" or "open your throat," concepts that are difficult for beginners, especially older learners, to intuitively understand and apply. Specifically, existing teaching programs often separate vocal techniques, emotional expression, and physical movement training, lacking a systematic and integrated design, making it difficult for learners to develop holistic singing abilities during training. Regarding technological assistance, existing singing training software typically focuses on pitch and rhythm correction, providing feedback through waveforms or spectrograms, but lacks dedicated training modules for emotional expression. In the field of audio processing, some systems can analyze audio features such as pitch, loudness, and spectrum, but rarely involve the extraction and evaluation of emotional indicators. Even when emotional analysis is available, it is often limited to text analysis or simple audio features, failing to incorporate deep learning models for multi-dimensional emotional vector comparison. Furthermore, existing feedback systems typically present results in numerical or graphical form, lacking intuitive multimodal interaction, such as dynamic visual feedback, making it difficult for users to perceive and adjust emotional expression in real time.

[0004] The inventors of this application have discovered the following technical problems with the above-mentioned technology:

[0005] 1. Existing vocal teaching methods rely too much on terminology and abstract instructions, and lack a comprehensive teaching system that organically integrates action guidance, emotional arousal, and vocal range training. As a result, middle-aged and elderly people find it difficult to understand and apply vocal techniques during the learning process, and the training effect is poor.

[0006] 2. Existing singing training systems lack effective feedback mechanisms for emotional expression. Users cannot intuitively perceive the deviation between their own emotional expression and the target emotion. Moreover, existing feedback systems are mostly presented in a single modality (such as audio or numerical values) and lack multimodal interaction (such as real-time dynamic visual feedback), which makes the training process abstract, difficult for users to adjust their emotional expression in real time, and results in low training efficiency. Summary of the Invention

[0007] This application provides an invention title to address the aforementioned technical problem.

[0008] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0009] On the one hand, an audio processing method for emotional singing is characterized by including:

[0010] Step S10: Obtain the lyrics and the target emotional state corresponding to the lyrics from the song database. The target emotional state includes a pre-labeled probability distribution of the target emotional type and a target emotional intensity value.

[0011] Step S20: Receive user singing audio data captured by the audio acquisition device, and convert the user singing audio data into a digital audio signal;

[0012] Step S30: Input the digital audio signal into a pre-trained deep learning model for feature extraction and generate extracted sentiment indicators, which include the probability distribution of user sentiment type and the user sentiment intensity value.

[0013] Step S40: Based on the target emotion type probability distribution, the target emotion intensity value, the user emotion type probability distribution, and the user emotion intensity value, calculate the normalized consistency score value;

[0014] Step S50: Generate dynamic visual feedback data based on the consistency score, wherein the dynamic visual feedback data includes a background color mapping layer and a particle animation layer;

[0015] Step S60: Synchronously output the text information of the lyrics and the dynamic visual feedback data on the graphical user interface.

[0016] On the other hand, a natural emotional singing training system includes the following steps:

[0017] Say goodbye to the plain voice training module. Through a combination of training methods such as the staircase method, the voice-warming punch, the surprise method, and the table-slapping method, establish basic pitch perception, vocal power, and preliminary emotional expression ability.

[0018] The Bright and Pleasant Sound Training Module activates the upper and middle vocal resonance areas through a combination of training methods including the O-shaped technique, the smiling brightening method, and the hand gesture method, thereby enhancing the brightness, forward projection, and emotional expressiveness of the voice.

[0019] The stereo resonance training module enhances the spatial extension and stereo layering of sound through a combination of training methods such as the ascending method, the water-holding method, and the rowing method.

[0020] The high-note singing training module establishes a scientific high-note vocal path through a combination of training methods such as yawning, questioning, sighing, and syllable drawing, enabling high notes to produce stable support and bright penetration.

[0021] The Emotionally Moving Training Module enhances the expressiveness and appeal of singing through a combination of techniques including spraying, physical movements, opera singing, and vocal embellishment, making the singing more sincere and vivid.

[0022] The "Learn One, Understand All" training module explores the vocal pathways of falsetto, mixed voice, head voice, breathy voice, and vibrato through a combination of training methods including private techniques, warm-up techniques, checkmark techniques, coquettish techniques, and ball-bouncing techniques, thus achieving a transition from basic vocalization to diverse singing styles.

[0023] The beneficial effects of this invention are as follows:

[0024] 1. To address the technical problem of traditional teaching relying too heavily on abstract terminology and the learning difficulties faced by middle-aged and elderly groups, this invention constructs six progressive training modules that integrate action guidance, emotional arousal, and vocal range training. Through concrete methods such as the "staircase method" and "vocal warm-up exercises," abstract vocal techniques are transformed into actionable movements, which is suitable for the cognitive characteristics of middle-aged and elderly people who "emphasize experience over theory." Tests have shown that the learning cycle for students is shortened, thus solving the problem of fragmented training elements.

[0025] 2. To address the problem of low training efficiency caused by the abstract and singular emotional feedback and lack of multimodal interaction in existing singing training systems, this application proposes a closed-loop technical solution of "teaching-learning-evaluation-correction". First, a CNN-LSTM hybrid deep learning model is used to extract the probability distribution of emotional type and emotional intensity values ​​of user audio in real time and accurately. Then, these are transformed into multidimensional vectors along with the target emotion. A quantified consistency score is calculated by combining cosine similarity and Gaussian kernel function. Finally, based on this score, real-time multimodal dynamic visual feedback is generated on the interface through color mapping background gradient, particle system floating animation, and dynamic text prompts. This solution transforms the extremely abstract vocal emotional deviation into an intuitive and immersive visual and text composite interaction, enabling users to perceive the gap between their own performance and the original song's meaning in real time and quickly adjust their singing state. It completely breaks down the understanding barrier of traditional numerical feedback, significantly reduces the training threshold for emotional expression, and thus greatly improves teaching and training efficiency. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the present invention;

[0027] Figure 2 A flowchart illustrating the training process for the natural and emotional singing method. Detailed Implementation

[0028] To make the content of this invention easier to understand, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings.

[0029] Example 1

[0030] The natural and emotional singing method includes the following steps:

[0031] The "Say Goodbye to Plain Voice" training module uses a combination of training methods such as the staircase method, the voice-warming punch, the surprise method, and the table-slapping method to solve problems such as inaccurate pitch, insufficient breath, harsh plain voice, and lack of rhythmic tension, and to establish basic pitch perception, vocal power, and preliminary emotional expression ability.

[0032] The Bright and Pleasant Sound Training Module uses a combination of O-shaped techniques, the Smiling Brightness Method, and hand gestures to improve issues such as muffled, unclear, and flat vocals, activate the upper and middle vocal resonance areas, and enhance the brightness, forward projection, and emotional expressiveness of the voice.

[0033] The stereo resonance training module, through a combination of training methods such as the ascending method, the water-holding method, and the rowing method, establishes a coordinated resonance system of the chest cavity, oral cavity, and head cavity, thereby enhancing the spatial extension and stereo layering of the sound.

[0034] The high-note singing training module establishes a scientific high-note vocal path through a combination of training methods such as yawning, questioning, sighing, and syllable drawing, enabling high notes to produce stable support and bright penetration.

[0035] The Emotionally Moving Training Module enhances the expressiveness and appeal of singing through a combination of techniques including spraying, physical movements, opera singing, and vocal embellishment, making the singing more sincere and vivid.

[0036] The "Learn One, Understand All" training module explores the vocal pathways of falsetto, mixed voice, head voice, breathy voice, and vibrato through a combination of training methods including private techniques, warm-up techniques, checkmark techniques, coquettish techniques, and ball-bouncing techniques, thus achieving a transition from basic vocalization to diverse singing styles.

[0037] In summary, this invention breaks away from the traditional teaching method that relies excessively on terminology and abstract instructions. It innovatively proposes 23 training methods, such as the "staircase method," "vocal opening punch," "table-slapping method," and "coquettish method," which integrate multiple channels such as actions, situations, and language to guide learners to directly understand the logic of vocalization and expression by simulating life behaviors or facial expressions. This greatly reduces the learning threshold and is especially suitable for the relatively conservative cognitive structure of the elderly.

[0038] Example 2

[0039] The "Say Goodbye to a Plain Voice" training module includes the staircase method, the voice-warming punch, the surprise method, and the table-slapping method.

[0040] The staircase method is applicable to sub-problems such as inaccurate melody and constant out-of-tune singing; the training objective is to strengthen pitch recognition through action simulation and improve melody stability and control; the operation steps are as follows:

[0041] 1. Guide students to understand the relationship between "the pitch of the melody" and "the verticality of space";

[0042] 2. Perform a simulated "climbing stairs" motion (such as raising your hands and rising your body) while singing scales upwards;

[0043] 3. Perform a simulated "going downstairs" action (such as lowering your hands and body) while singing scales downwards;

[0044] 4. Practice repeatedly to establish a "movement-pitch" mapping mechanism and strengthen pitch perception.

[0045] Teaching principle: Using movement to guide auditory cognition, constructing a "multi-sensory synchronous" training channel, suitable for pitch establishment in the initial learning stage.

[0046] The "Voice-Opening Fist" technique is suitable for addressing issues such as insufficient breath support and inability to sing loudly. Training objective: Through alternating striking and shouting, it aims to stimulate abdominal vocal power and establish awareness of breath support. Operation steps:

[0047] 1. The trainee clenches his fists and stands upright;

[0048] 2. Alternately strike your chest with force from left to right (e.g., cross-pat your shoulders or chest), engaging your core muscles;

[0049] 3. Shout out "Ha!" or other explosive sounds in rhythm, feeling your abdomen tighten instantly;

[0050] 4. Guide students to focus on the coordinated response between abdominal muscles and breath;

[0051] 5. Apply this feeling to melody practice to initially establish breath control and vocal support.

[0052] Teaching principle: Using movement to trigger abdominal pressure awareness and overcome the "cannot sing loudly" obstacle is an important step in transitioning to breath control training.

[0053] The surprise method applies to the following sub-problems: unpleasant unfiltered voice, lack of emotional expression.

[0054] Training objective: To activate upper respiratory resonance and emotional expressiveness by using expressions of surprise and breathing channels.

[0055] Operating steps:

[0056] 1. Guide trainees to simulate a state of surprise (eyes wide open, gasp for breath), opening the oral and nasal passages;

[0057] 2. Create a scenario such as "Oh my god! A classmate of ours won the Nobel Prize!" to evoke a natural emotional response;

[0058] 3. Continue to imitate saying "Me?", guiding the pronunciation to naturally rise in pitch;

[0059] 4. Transform this state of surprise into lyrics such as "I want to climb..." and incorporate it into the song's expression.

[0060] Teaching principle: By imitating the surprised reaction in social situations, facial muscle groups and breathing pathways are activated, thereby enhancing the brightness and emotional penetration of the voice.

[0061] The "clapping method" is applicable to the following sub-problems: inability to capture the essence of grassland music and lack of rhythmic tension; training objective: to enhance emphasis and emotional expression by combining key words with rhythmic body movements; operational steps:

[0062] 1. Analyze the emotional fluctuations of the song and identify the key words or emotional phrases that need to be emphasized;

[0063] 2. When singing, slap the table (thigh or tabletop) at key points to emphasize the notes;

[0064] 3. Encourage students to use movement to release rhythm and emotion, forming the unique "heroic tension" of grassland songs;

[0065] Teaching principle: The traditional "emphasis-emotion" theory is visualized into movement training to enhance middle-aged and elderly students' ability to perceive the combination of rhythm and emotion.

[0066] Example 3

[0067] The Bright and Pleasant Sound Training Module includes the O-shaped mantra, the smiling bright method, and the gesture method;

[0068] The O-shaped technique applies to the following sub-problem: a muffled and unclear voice; training goal: to guide students to develop the correct mouth shape and cavity structure, and optimize the sound wave propagation channel; operation steps:

[0069] 1. Guide students to imagine that they have an egg in their mouth, which will naturally form a round "O" shape.

[0070] 2. Maintain an O-shaped mouth position when producing sound, focusing resonance in the front of the oral cavity;

[0071] 3. When singing vowels and melodies, maintain oral cavity tension and shape to enhance brightness;

[0072] 4. Guide students to practice comparing the before and after to experience the changes in timbre.

[0073] Teaching principle: By using concrete imagery (including an egg) to activate facial muscles and oral cavity space, high-frequency resonance and vocal clarity are enhanced.

[0074] The "Smiling Brightness Method" is applicable to the sub-problem: how to sing brighter; training goal: to improve vocal structure by lifting the facial muscles, making the voice more penetrating; operation steps:

[0075] 1. Guide trainees to smile naturally, which will lift the facial muscles;

[0076] 2. Speak naturally while laughing, and experience the feeling of projecting sound forward;

[0077] 3. Practice singing with a smile, maintaining a joyful state and open vocal cords;

[0078] 4. Incorporate this state into the melody and lyrics to create a stable tone.

[0079] Teaching principle: Lifting the face can open the resonance channels between the nasal cavity and cheekbone, and a smiling state activates emotions, making the voice natural, bright, friendly and moving.

[0080] The gesture method is applicable to the sub-problem: how to sing with a sense of happiness; training goal: to guide the body to participate in emotional expression by linking emotional keywords with gestures; operation steps:

[0081] 1. Extract keywords from the lyrics that express happiness, such as "sunshine," "companionship," and "warmth."

[0082] 2. Guide trainees to use corresponding hand gestures to express keywords, such as "spreading arms", "lightly touching the chest", and "lifting upwards";

[0083] 3. Make hand gestures simultaneously while singing to synchronize the sound with the body's rhythm;

[0084] 4. Cultivate the coordination between emotions and actions through practicing complete sentences.

[0085] Teaching principle: Engaging the body through singing helps to project emotions, unify body rhythm with singing, and is suitable for senior students to improve their expressive abilities.

[0086] Example 4

[0087] The three-dimensional resonance training module includes methods such as the ascending method, the water-holding method, and the rowing method.

[0088] The "Ascending Hall Method" is applicable to the following sub-problem: insufficient chest resonance, resulting in a weak and unsupported voice; Training goal: To guide resonance downwards into the chest cavity through the vibrational sensation of the "woo" sound, thereby enhancing the thickness of the voice; Operation steps:

[0089] 1. The student purses their lips slightly and emits a long "woo" sound, with focused and stable vocalization;

[0090] 2. Guide them to focus their attention on their chest cavity and feel the vibration in the chest area;

[0091] 3. Incorporate this state into the melody and sing the "woo" sound, trying to maintain the resonance position unchanged;

[0092] 4. Switch to singing the lyrics, maintaining the support of chest resonance.

[0093] Teaching principle: Utilizing the closed-mouth "ooh" sound to create low-frequency resonance stimulates the chest cavity to participate in vocalization, resulting in a full and stable sound, suitable for training in the mid-to-low range.

[0094] The water-holding method is applicable to the following sub-problem: insufficient oral resonance, resulting in a muffled and unclear voice; Training goal: To relax the root of the tongue and raise the soft palate by simulating the state of holding water in the mouth, thereby expanding the upper cavity resonance space; Operation steps:

[0095] 1. Guide students to open their mouths easily, keeping their ear canals concave, simulating a watery state;

[0096] 2. Relax the base of your tongue, avoid pressing down on it, and at the same time lift your soft palate to open up the space at the top of your mouth;

[0097] 3. In this state, pronounce vowels, paying attention to controlling the concentration of the sound stream;

[0098] 4. Incorporate melody exercises to enhance students' control and perception of oral cavity structure.

[0099] Teaching principle: By using concrete imagery (holding water in the mouth), the perception of the oral cavity space is evoked, thereby improving the diffusion efficiency of sound waves in the upper cavity resonance area.

[0100] The rowing method is applicable to the following sub-problems: unclear resonance direction, lack of surround sound and spatial positioning; training goal: to guide breath and resonance path through body movements to improve the surround sound effect and spatial sense; operation steps:

[0101] 1. The student makes a rowing motion with both arms, and the body naturally rises and falls with the movement;

[0102] 2. Make a sound during the arm stroke, and feel the sound spread with the rhythm of your body;

[0103] 3. While singing the melody, row simultaneously to guide the breath along the back and head cavity;

[0104] 4. Enhance sound directionality while rowing and practice sound surround perception.

[0105] Teaching principle: Use dynamic movements to awaken the body's participation in the resonance path, allowing the sound to flow in three-dimensional space, breaking through the limitations of planar singing.

[0106] Example 5

[0107] The high-note singing training module includes the yawning method, the questioning method, the sighing method, and the elongated word method;

[0108] The yawning method applies to the sub-problem of opening the vocal tract; training objective: to open the pharynx and establish a vertical vocal tract by simulating a yawn; operational steps:

[0109] 1. Guide trainees to recall the natural state of yawning (mouth open, soft palate raised, pharynx relaxed);

[0110] 2. Maintain a half-yawning state to keep the pharyngeal cavity open;

[0111] 3. Feel the cool air flowing in the throat, and guide the student to try sounds such as "ha~";

[0112] 4. Apply this vocalization state to melody practice, especially the preparatory state before high notes.

[0113] Teaching principle: Yawning is a natural way to open the passage between the pharynx and the head cavity, which helps to eliminate throat congestion and provides a "breathing pathway" for high notes.

[0114] The questioning method is applicable to the sub-problem of finding the high note position; the training goal is to guide the high note vocal point upward by using the questioning tone of "Eh?" to establish head cavity vocalization perception; the operation steps are as follows:

[0115] 1. Trainees raise their eyebrows and widen their eyes, simulating a "questioning" facial expression;

[0116] 2. Make interjections such as "Eh?" or "Hmm?" and feel the sound coming from the top of your forehead;

[0117] 3. Pay attention to the point where the sound lands between your eyebrows or on the upper part of your forehead;

[0118] 4. Imitate this path in the high register of the melody to stabilize the sound positioning.

[0119] Teaching principle: By activating the natural "inquiry tone" through contextual simulation, the head cavity resonance area is guided to participate, making high notes easy and focused.

[0120] The sighing method is applicable to the sub-problem: finding the root of high notes (breath support); training goal: establishing coordination between deep breathing and breath release through the natural act of sighing; operational steps:

[0121] 1. Guide trainees to take a deep breath and sigh deeply to release physical tension;

[0122] 2. Add an "ah~" sound, making sure the sound falls naturally and the exhalation is steady;

[0123] 3. Strengthen the starting point and support of breath when sighing;

[0124] 4. Start the melody with a "sighing" opening phrase to establish a rhythmic sense of breathing before vocalization.

[0125] Teaching principle: Sighing is a natural way to release breath, which can open up the coordination mechanism between breathing and vocal cords, providing effective basic support for high notes.

[0126] The "pulling" method is applicable to the following sub-problem: maintaining a high pitch; training goal: strengthening the stability of the onset and the duration of the vocal tract through continuous vocalization exercises using the "pulling" sound; operation steps:

[0127] 1. Add a "la~" sound at the beginning of the high-pitched phrase in the melody to help find the correct sound placement;

[0128] 2. Guide the sound from the pharynx opened by the "yawning method" all the way up to the forehead, keeping it smooth;

[0129] 3. Pay attention to the stability and sustained force at the starting point of the "la" sound;

[0130] 4. Practice sentence patterns repeatedly to solidify the sense of support and directionality in high notes.

[0131] Teaching principle: By simulating the "extension and support" in vocalization through the "pulling" sound, the high-pitched channel control of breath, resonance and emotion is strengthened.

[0132] Example 6

[0133] The emotionally moving training module includes the spraying method, the body method, the opera singing method, and the embellishment method;

[0134] The "spraying" method is applicable to the following sub-problems: unclear pronunciation and flat emotional expression; training goal: to enhance the power and appeal of lyrics by combining explosive initial consonants with full, resonant final consonants; operation steps:

[0135] 1. Identify keywords in the lyrics (such as core emotional words like "love," "dream," and "home");

[0136] 2. Divide keywords into "initials" (open plosives) and "finals" (sustained endings);

[0137] 3. When pronouncing a word, the initial sound should be exhaled with a short, forceful burst of breath, while the final sound should be full and extended.

[0138] 4. Combine sentence-by-sentence exercises to achieve a gradual progression of emotions within the sentence.

[0139] Teaching principle: Imitating the "plosive pronunciation" form when emotions are emphasized in spoken language enhances syllable fluctuation and language expressiveness, and is suitable for training in emotional expression.

[0140] The physical method is applicable to the following sub-problems: stiff movements and lack of vocal fluidity; training goal: to guide the melodic line and body rhythm through hand gestures, enhancing vocal fluidity and emotional expression; operational steps:

[0141] 1. Guide students to use hand gestures to indicate spatial directions, such as "observing the sky and the earth" and "extending left and right";

[0142] 2. Encourage the use of actions such as drawing circles and parabolas to assist in the direction of the melody;

[0143] 3. Maintain synchronized hand gestures and natural body rhythm during the performance;

[0144] 4. Improve the consistency between movement, voice, and emotion during practice.

[0145] Teaching principle: Activate the "body-melody" correspondence mechanism to enable the voice to have spatial orientation and rhythmic variation, thereby enhancing the expressiveness and approachability of the performance.

[0146] The opera-style vocal technique addresses the following sub-problems: lack of distinctive timbre and performance flair; training objectives: to enrich vocal expression by drawing on resonance techniques and facial control from traditional opera; operational steps:

[0147] 1. Guide students to imitate the facial expressions of opera actors, such as "raising eyebrows, tucking cheeks, and raising the corners of the mouth";

[0148] 2. Practice the traditional operatic intonation of "Ah—" to open up the nasal resonance area;

[0149] 3. Focus on training the mouth control method of "exerting force from the corners of the mouth and projecting sound through the front teeth";

[0150] 4. Incorporate this singing technique into sentence-based singing to enhance expressive power.

[0151] Teaching principle: By using facial structure control methods developed in opera performances, the projection angle of sound waves and changes in timbre are enriched, which is suitable for middle-aged and elderly people to enhance their individual expression.

[0152] The embellishment method is applicable to the sub-problem of flat emotional delivery and lack of decorative details; training goal: to improve the dynamic expression and subtlety of song phrases by adding techniques such as emphasis, glissando, and embellishment; operation steps:

[0153] 1. Identify lyric paragraphs or sentence endings that can be polished;

[0154] 2. Try techniques such as "emphasis on the initial consonant," "glissando at the end of a consonant," and "embellishment and retraction" in vocalization;

[0155] 3. Practice sentence segmentation to refine pronunciation and ensure accuracy and emotional direction;

[0156] 4. Compare the differences between ordinary singing and embellished singing to enhance students' understanding.

[0157] Teaching principle: Ornaments are an effective way to enhance the level of singing. By using embellishment techniques, we can build emotional progression and phrasing beauty.

[0158] Example 7

[0159] The "Learn One, Understand All" training module includes private techniques, warm-hand techniques, checkmark techniques, coquettish techniques, and ball-bouncing techniques.

[0160] Sub-problem applicable to private phonation: Trying relaxed falsetto; Training goal: To guide the falsetto pathway through simulated whispering scenarios and establish a soft vocalization habit; Operation steps:

[0161] 1. Guide students to recall and imitate whispered tones (such as "Have you eaten?");

[0162] 2. Combine the "surprise method" of inhalation with the "smile brightening method" of adjusting the state of the cheek muscles as preparation;

[0163] 3. Gently say "Have you eaten?" to experience the relaxation of the glottis and the light, suspended state of the voice;

[0164] 4. Incorporate this state into the melody and try singing the softer passages in falsetto.

[0165] Teaching principle: By using natural vocalization behaviors in daily life to establish a "relaxation-falsetto-upper register" path, it helps to solve the problems of tension in high notes and throat exertion.

[0166] The warming-hands technique applies to the following sub-problem: attempting light, breathy sounds; training goal: perceiving the flow and resonance balance of breath through the physical action of "warming hands with breath"; operation steps:

[0167] 1. Students place their hands in front of their mouths, simulating the action of warming hands in winter;

[0168] 2. Make a "ha~" sound and feel the warm airflow and the soft tone;

[0169] 3. Keep your body relaxed and practice the vowel aspirated sounds repeatedly;

[0170] 4. Incorporate the melody and try to use your breath to guide your soft singing.

[0171] Teaching principle: By perceiving the "soft voice" state through kinesthetic and tactile senses, the accuracy of breath control is improved. It is suitable for soft lyrical pieces and breath voice training for middle-aged and elderly students.

[0172] The checkmark method is applicable to the following sub-problem: attempting a mixed voice with varying intensity; Training goal: using abdominal and back muscles and upward movements to shift the vocal fulcrum, thus improving vocal cord coordination in mixed voice; Operation steps:

[0173] 1. Inhale to prepare, actively engage your core muscles;

[0174] 2. Raise your arm in a "checkmark" motion to guide the pitch upward;

[0175] 3. When you make the sound "a~", the sound wave naturally rises;

[0176] 4. Practice repeatedly to develop a smooth transition path from chest voice to head voice.

[0177] Teaching principle: Combining movement and sound direction, guiding breath and glottis coordination, is an important bridge to transition to the high mixed voice range.

[0178] The "coquettish" method applies to the following sub-problem: attempting an ethereal head voice; Training goal: using a "coquettish" tone to open up the palate structure and frontal resonance, creating an ethereal and light vocal texture; Operation steps:

[0179] 1. Gently close your mouth and inhale while mimicking a coquettish tone (such as "Mmm~");

[0180] 2. Focus your voice on the area above your forehead;

[0181] 3. The voice should be soft and gentle, with rich head cavity overtones;

[0182] 4. Immerse yourself in the melody and sing ethereal and beautiful phrases.

[0183] Teaching principle: Imitating situational behaviors to mobilize body structure and resonance direction is an effective means of training head cavity timbre.

[0184] The ball-bouncing method is applicable to the following sub-problem: attempting a melodious vibrato; training objective: to perceive the rhythm of vibrato through movement rhythm, and to cultivate the ability to control the rhythm of vibrato and express emotions; operation steps:

[0185] 1. The student mimics the action of dribbling a ball by lowering their arm, maintaining a steady rhythm.

[0186] 2. Simultaneously pronounce "Ah~" and observe the vibration of the sound waves;

[0187] 3. By mimicking the vibration frequency of small air bubbles, a natural vibrato is created;

[0188] 4. Incorporate vibrato into melodic phrases, controlling the amplitude and start / end of the vibrato.

[0189] Teaching principle: Activating the rhythmic coordination between the vocal organs, abdominal pressure, and vocal cords through movement rhythm is an important way to train natural vibrato.

[0190] In summary, this invention breaks through the limitations of traditional teaching methods that rely excessively on terminology and abstract instructions. It innovatively proposes 23 training methods, including the "staircase method," "vocal warm-up punch," "table-slapping method," and "coquettish method," integrating multi-channel input such as action, context, and language. This guides learners to directly grasp vocal logic and expression paths through simulating life behaviors or facial expressions, significantly lowering the learning threshold, and is particularly suitable for the relatively conservative cognitive structure of the elderly. This method is the first to incorporate emotional arousal, keyword reinforcement, and contextual simulation into the core modules of vocal music teaching. Methods such as the "surprise method," "gesture method," "opera-style method," and "spraying method" construct a complete output system from cognition to action to emotional expression, filling the gap in existing teaching systems that lack operational paths and modular design for emotional training. Its six teaching modules progress from simple to complex, covering key stages from "saying goodbye to a plain voice" to "high note establishment," "emotional drive," and "vocal register exploration." These modules are transferable, decomposable, and superimposed, forming a complete progressive teaching process that effectively solves the shortcomings of traditional methods, such as fragmented training elements and a lack of systematicity. This method fully considers the learning characteristics of middle-aged and elderly people, who have "intuitive action experience but weak abstract understanding". It designs a variety of triple interaction methods of "body rhythm + sensory imagination + language guidance" to improve the perceptibility and controllability of vocal skills and realize the transformation from passive imitation to active expression.

[0191] The above six modules are executed sequentially according to the logic of "basic vocalization → timbre optimization → resonance expansion → high-note breakthrough → emotional enhancement → style transfer." Each preceding module provides technical support for the subsequent modules. The "Learn One, Master All" module serves as an advanced extension of emotional expression, transferring the basic vocalization and emotional expression abilities developed in the preceding modules to different singing styles through training in diverse vocal techniques. The six training modules are executed in order, and the next module can only be entered after the previous module's training is successfully completed.

[0192] To further enhance training effectiveness and achieve quantitative assessment and real-time guidance of emotional expression in singing, we propose an emotional singing audio processing method that combines modern audio processing and artificial intelligence technologies. This method can not only objectively analyze the singer's emotional state but also provide intuitive feedback through multimodal interaction, thus forming a closed loop with the training system in Example 1 to jointly construct a complete integrated system of "teaching-learning-assessment-correction".

[0193] Example 8

[0194] Step S10: Obtain the lyrics and the corresponding target emotional state from the song database. The target emotional state includes a pre-labeled probability distribution of the target emotional type and a target emotional intensity value. The song database is constructed as follows: First, collect song samples containing the original audio and corresponding lyrics, and invite at least five professional vocal teachers as annotators. Use an emotion wheel (such as the Plutchik model) to independently label the emotional type and intensity (1-5 points) of each song at multiple time slices. Second, perform consistency checks and averaging on the annotation results to generate the target emotional type probability distribution (normalized proportion of each emotional category) and target emotional intensity value (normalized to the 0-1 range) for each time point. Finally, associate the song ID, lyrics, timestamp, and corresponding target emotional vector (including probability distribution and intensity value) and store them in the database. Alternatively, use a deep learning-based music emotion recognition model (such as MusicBERT) to automatically analyze the original audio, generate a preliminary target emotional vector, and then have it reviewed and corrected by experts before being stored in the database.

[0195] The purpose of this step is to establish a baseline for the emotional assessment of the entire system. The principle is to pre-analyze the lyrics using natural language processing techniques (such as text sentiment analysis algorithms) to quantify the hidden emotional tendencies within them. The "target emotion type probability distribution" refers to the set of probabilities of a song containing multiple emotions (such as happiness, sadness, anger, and calmness); the "target emotion intensity value" refers to the intensity of the emotional expression, typically mapped to a value between 0 and 1. These two indicators together constitute a digital emotional scale, providing an absolute reference standard for subsequently evaluating whether a user's singing matches the original song's mood.

[0196] When a user selects to sing the classic song "The Moon Represents My Heart" in the system, the system retrieves the text information of the song from a local or cloud-based song database and obtains the pre-labeled target emotional state. The extracted target emotional type probability distribution is (sadness probability 0.7, happiness probability 0.3), and the target emotional intensity value is 0.8 (indicating that the song requires a high level of emotional engagement). Specifically, the 'target emotional state' stored in the song database is preferably configured and accessed as structured data in key-value pair format (such as JSON).

[0197] Step S20: Receive user singing audio data captured by the audio acquisition device, and convert the user singing audio data into a digital audio signal;

[0198] The purpose of this step is to acquire the user's physical sound and convert it into computer-readable low-level data. The principle is to use audio acquisition devices such as microphones to receive the continuous analog sound wave signal generated when the user sings, and then use an analog-to-digital converter (ADC) to perform high-frequency sampling and quantization of the continuous signal (e.g., a common 44.1kHz sampling rate), converting it into a discrete numerical array, i.e., the digital audio signal. Connecting this to the previous step, where step S10 acquired the target standard, this step is responsible for recording the actual acoustic data source to be tested.

[0199] The user begins singing the first line of "The Moon Represents My Heart" into the microphone connected to the system. The microphone captures continuous sound waves as the user's singing audio data, and the sound card in the system immediately converts it into discrete digital audio signals that can be processed by a computer.

[0200] Step S30: Input the digital audio signal into a pre-trained deep learning model for feature extraction and generate extracted sentiment indicators, which include the probability distribution of user sentiment type and the user sentiment intensity value.

[0201] The core of this invention lies in using artificial intelligence to analyze the emotional components of a user's authentic singing voice in real time. The principle involves inputting the digitized vocal features into a deep neural network pre-trained using a massive amount of human singing samples. The model, relying on its internal node weights, can automatically identify the emotional features implicit in acoustic parameters such as pitch, speech rate, breathiness, and vibrato. Following the digital audio signal acquired in step S20, the model automatically outputs the quantified probability distribution of the user's emotional type and the intensity value of the user's emotional state. The technical effect is that it completely eliminates the pain point of relying on subjective teacher evaluation in traditional teaching, achieving objective, real-time, and high-precision extraction of emotional features.

[0202] The system inputs the digital audio signal containing the user's singing into a pre-trained deep learning model. The model processes the signal within a very short delay and outputs the emotional index at that moment, wherein the probability distribution of the user's emotional type is (sadness probability 0.6, happiness probability 0.4), and the user's emotional intensity value is 0.7.

[0203] Step S40: Based on the target emotion type probability distribution, the target emotion intensity value, the user emotion type probability distribution, and the user emotion intensity value, calculate the normalized consistency score value;

[0204] The purpose of this step is to scientifically quantify the gap between the user's actual performance and the original song's target. The principle is to comprehensively compare the "direction" (whether the type matches) and "amplitude" (whether the intensity meets the standard) of the emotion. Using mathematical statistical algorithms, the probability distribution of the user's emotion type and the user's emotion intensity value generated in the previous step S30 are cross-compared with the benchmark data (the probability distribution of the target emotion type and the target emotion intensity value) obtained in step S10. The multi-dimensional differences are then calculated and compressed into a consistency score between 0 and 1. The innovation lies in this multi-feature dimensional fusion evaluation method, which provides unique and accurate driving data for the subsequent generation of multimodal visual feedback.

[0205] The system performs mathematical calculations on the extracted user emotional features (sadness 0.6, happiness 0.4, intensity 0.7) and the original song's target emotional features (sadness 0.7, happiness 0.3, intensity 0.8), and obtains a consistency score of 0.85, indicating that the user's singing emotion is very close to the original mood of "The Moon Represents My Heart".

[0206] Step S50: Generate dynamic visual feedback data based on the consistency score, wherein the dynamic visual feedback data includes a background color mapping layer and a particle animation layer;

[0207] The purpose of this step is to transform the highly abstract numerical scores into intuitive visuals that are easily understood by non-professionals. The principle is based on data visualization and computer graphics rendering technology. The consistency score calculated in step S40 is directly used as a control parameter for the graphics rendering engine, thereby driving dynamic changes in the visual attributes of image elements (such as warm and cool colors, particle quantity, and motion trajectory). The technical effect is a significant reduction in the cognitive threshold for middle-aged and elderly users or other beginners, allowing them to know whether they are singing correctly without having to read tedious charts, simply through intuitive sensory feedback.

[0208] After the system obtains the high consistency score of 0.85, it begins to generate the dynamic visual feedback data and issues an instruction to the graphics rendering engine: prepare to render a blue-tinted background color mapping layer representing good performance, and generate a set of particle animation layers that float smoothly upwards without any agitation.

[0209] Step S60: Synchronously output the text information of the lyrics and the dynamic visual feedback data on the graphical user interface.

[0210] The purpose of this step is to complete the final human-computer interaction loop and ensure the continuity of the user experience. The principle is to combine all processed text and graphic data and accurately render it on the user's visible screen according to a unified timeline. The text information of the lyrics obtained in step S10 and the dynamic visual feedback data generated in step S50 are layered, merged, and rendered, allowing the user to receive real-time emotional guidance from the system with zero latency while reading and singing along.

[0211] The system clearly scrolls and displays the text information of the lyrics of "The Moon Represents My Heart" in the center of the device screen (i.e., the graphical user interface). At the same time, a bluish background and stable floating particles are superimposed on the lyrics as dynamic visual feedback data. Users see a soft blue color and a stable image, intuitively knowing that their emotions are full and accurate, and continue to sing in the current state.

[0212] Furthermore, the step of inputting the digital audio signal into a pre-trained deep learning model for feature extraction and generating extracted sentiment indicators includes: performing frame segmentation and windowing processing on the digital audio signal to obtain framed audio signals; extracting Mel frequency cepstral coefficient features from the framed audio signals; and performing a short-time Fourier transform on the Mel frequency cepstral coefficient features to generate a Mel spectrogram, which is used as input data for subsequent neural network feature extraction.

[0213] This dependent claim details the underlying principles of front-end acoustic feature extraction in step S30. Since human sound is a long, continuous, and redundant waveform in the time domain, "framing and windowing" refers to dividing long audio segments into extremely short time fragments and eliminating waveform abrupt changes at the fragment edges. "Mel frequency cepstral coefficients (MFCC)" is currently the most mature acoustic feature in the field, best simulating the characteristics of human auditory perception. Converting it to a "Mel spectrogram" through "short-time Fourier transform" essentially transforms a one-dimensional audio band into two-dimensional image data reflecting frequency energy distribution. The technical effect is the removal of environmental noise and invalid information, providing the most refined and highest-quality input material for subsequent neural networks.

[0214] When processing the continuous digital audio signal of a user singing "The Moon Represents My Heart", the system first divides it into short segments of 25 milliseconds each and applies a Hamming window (frame-by-frame windowing processing). After extracting 40-dimensional MFCC features, it uses mathematical transformation to draw a two-dimensional Mel spectrogram that resembles a voiceprint fingerprint.

[0215] Furthermore, the pre-trained deep learning model includes a convolutional neural network and a long short-term memory network. The generation of the extracted sentiment indicators includes: inputting the Mel spectrogram into the convolutional neural network to perform spatial feature extraction to obtain a spatial feature sequence; inputting the spatial feature sequence into the long short-term memory network to perform time series dependency processing to output a final hidden state sequence; mapping the final hidden state sequence into a numerical result through a fully connected layer, and outputting the user sentiment type probability distribution and the user sentiment intensity value after processing by an activation function.

[0216] This dependent claim reveals the operating mechanism of the system's core AI model (i.e., the main inventive point of this invention). Following claim 2, since the "Mel spectrogram" has been converted into a two-dimensional image format, the "Convolutional Neural Network (CNN)" is extremely adept at extracting static details (such as spatial features during pronunciation) from image textures; however, singing is a time-lapse process, and emotions are continuous. Therefore, a "Long Short-Term Memory Network (LSTM)" is used to remember the sound features of the previous few seconds to infer the evolution of the current emotion (i.e., time-series dependence); finally, the "fully connected layer" acts as an intelligent classifier, flattening complex abstract features and mapping them into concrete numerical values ​​that are readable by humans. The technical effect is that this hybrid, interconnected network architecture perfectly balances the instantaneous bursts of audio with long-term emotional trends, significantly improving the stability and accuracy of emotional feature recognition.

[0217] The system feeds the Mel spectrogram representing the aria from "The Moon Represents My Heart" into the deep model. The convolutional neural network first identifies subtle breathy and labiodental sounds in the sound (spatial feature sequence). Then, the long short-term memory network combines the smooth transition features of the user's previous lyric to determine that the emotion is not broken (time series processing). Finally, the specific probability of sadness (0.6), probability of happiness (0.4), and intensity (0.7) are calculated through a fully connected layer. The specific internal architecture of the pre-trained deep learning model is further disclosed: the convolutional neural network (CNN) includes at least two two-dimensional convolutional layers. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function to extract local spatial features. The kernel size of the two-dimensional convolutional layer is preferably [3×3], and the stride is set to [1]. The long short-term memory network (LSTM) includes at least two hidden layers, with the number of hidden nodes in each layer configured as

[128] , to process the time series data after the feature map output by the CNN is flattened. The fully connected layer includes a feature mapping layer and an output layer. The output node responsible for outputting the 'user sentiment type probability distribution' uses the Softmax activation function to ensure that the sum of the multi-class probabilities is 1. The output node responsible for outputting the 'user sentiment intensity value' uses the Sigmoid activation function to stably map the intensity value to a normalized range of 0 to 1.

[0218] Furthermore, the step of calculating a normalized consistency score based on the target emotion type probability distribution, the target emotion intensity value, the user emotion type probability distribution, and the user emotion intensity value includes: constructing a target emotion vector containing the target emotion type probability distribution and the target emotion intensity value; constructing a user emotion vector containing the user emotion type probability distribution and the user emotion intensity value; and performing a dot product and modulus operation on the target emotion vector and the user emotion vector to output a cosine similarity score.

[0219] This dependent claim refines the algorithmic principle of comparing the "directional" differences in emotions in step S40. To make abstract human emotions calculable through formulas, the system packages them into "vectors" in a multi-dimensional space. The "target emotion vector" and the "user emotion vector" represent two geometric arrows projecting from the virtual multi-dimensional space. Calculating the "cosine similarity score" of these two arrows using classic vector operations essentially calculates the angle between them; the smaller the angle, the closer it is to 1, indicating a closer match in emotional direction. The technical effect is to elegantly reduce the extremely complex differences in the probability distributions of various emotions into a single-directional geometric similarity index.

[0220] The system packages the original song into a target emotion vector and the user's performance into a user emotion vector. Using the dot product formula, the cosine of the angle between these two multidimensional vectors is calculated, yielding a cosine similarity score of approximately 0.98. This indicates that the user's overall emotional direction (sadness / deep affection) when singing "The Moon Represents My Heart" is entirely correct.

[0221] Furthermore, the calculation of the normalized consistency score also includes: calculating the absolute difference between the target emotional intensity value and the user emotional intensity value; inputting the absolute difference value into a Gaussian kernel function configured with bandwidth parameters to convert it into an intensity difference score; obtaining a predefined weight table, and performing weighted merging processing on the cosine similarity score and the intensity difference score according to the extracted weight coefficients to generate the consistency score.

[0222] The dependent claim serves to supplement and address the impact of differences in emotional "amplitude," constituting a secondary inventive point of this invention. Simply assessing the consistency of emotional type is insufficient; if the original song requires emotional outburst (high intensity), and the user, while matching the type, sings weakly (low intensity), it also fails to meet the standard. The "absolute difference value" reflects the intuitive difference in intensity, while the principle of introducing the "Gaussian kernel function" is to achieve non-linear smoothing, allowing the system to have a high tolerance for small errors in vocal delivery, but drastically amplifying and deducting points for excessive deviations. Finally, a predefined weighting table is used to merge the similarity calculated in the aforementioned claims with the intensity difference calculated in this claim. The technical effect is the construction of a comprehensive emotional assessment dimension, making the machine's judgment closer to the subjective and discerning listening experience of vocal teachers.

[0223] The system calculates that the absolute difference between the original song's target intensity of 0.8 and the user's sung intensity of 0.7 is 0.1. Since sad songs have a high tolerance for small intensity errors, the system calls the configured Gaussian kernel function to smoothly convert this 0.1 difference into an extremely small intensity difference score of 0.05, and prepares to merge it with the previous cosine similarity score according to the predefined weight table.

[0224] Furthermore, the weighted merging of the cosine similarity score and the intensity difference score based on the extracted weight coefficients to generate the consistency score is performed as follows: obtaining the first and second weight coefficients set in the predefined weight table; and calculating according to the formula... The consistency score value is obtained by performing a linear merge process; wherein, The consistency score is... The first weighting coefficient, This is the second weighting coefficient. The cosine similarity score is... The intensity difference score is denoted as .

[0225] This dependent claim clarifies the ultimate mathematical calculation rule of the scoring fusion system. Its principle is to use a weighted linear combination method, changing the weight allocation to determine whether to prioritize "accuracy in grasping emotional type" (first weight) or "sufficient emotional intensity" (second weight) during the assessment. This can be dynamically invoked from a predefined weight table based on the specific song style. The technical effect is to ensure that the underlying scoring mechanism has strong interpretability and flexible configurability, avoiding the rigid "one-size-fits-all" scoring model of traditional systems.

[0226] For slow songs like "The Moon Represents My Heart," the system is designed to prioritize the emotional type of the song, therefore it retrieves... , Substituting into the calculations obtained in the previous steps... and By applying the formula The system quickly derives the final consistency score. The value is approximately 0.971, confirming that the user's current musical phrase expresses an excellent emotional quality.

[0227] Furthermore, generating dynamic visual feedback data based on the consistency score includes: establishing a linear mapping relationship between the consistency score and the hue parameter in the HSV color model; converting the currently obtained consistency score into a target hue parameter based on the linear mapping relationship; generating a brightness parameter based on the consistency score, and synthesizing the target hue parameter and the brightness parameter to generate the background color mapping layer.

[0228] This dependent claim details the specific execution logic of the background color change in step S50, which is a secondary inventive point of this invention. The HSV color model scientifically decomposes color into hue, saturation, and value. Establishing a "linear mapping relationship" means smoothly mapping the score value of 0-1 to a specific degree on the color wheel (for example, a low score corresponds to a reddish warning hue, and a high score corresponds to a bluish comfortable hue). The technical effect is that by using continuous mathematical mapping, abrupt changes and flickering of UI screen colors are avoided. Users can subtly perceive the quality of their performance simply by using peripheral vision to capture the gradual changes in the warmth and coolness and brightness of the screen, without being distracted.

[0229] The system substitutes the calculated high score of 0.971 into the HSV mapping relationship to calculate that the target hue parameter is in the cool hue range (such as 240 degrees pure blue). At the same time, the brightness parameter is increased to enhance the brightness contrast of the feedback. Finally, the rendering engine synthesizes a large area of ​​soft and bright blue background as the background color mapping layer of the interface.

[0230] Furthermore, the step of generating dynamic visual feedback data based on the consistency score also includes: initializing a particle system containing emotional expression elements and setting initial position attributes for the particle system; mapping the consistency score to the emissivity parameter of the particle system and the motion acceleration parameter of the emotional expression elements; extracting the emotion category with the highest value in the probability distribution of the user's emotion type, outputting the corresponding particle color data, and synthesizing the particle animation layer.

[0231] This dependent claim further enriches the visual expressiveness of step S50. Particle systems are a mature rendering technique in computer graphics used to simulate complex natural phenomena such as airflow, starlight, and water ripples. This system links the score value with "emissivity" (the number of particles) and "acceleration" (the degree of disorder in particle motion): a high score results in a large number of particles emerging and moving smoothly; a low score results in sparse and chaotic particles. The technical effect is to give the dry evaluation values ​​a dynamic presentation with "physical metaphors," greatly enhancing the immersiveness and sophistication of the feedback system.

[0232] Since the current score of 0.971 is at a high level, the system instructs the particle system to maintain an extremely high emissivity parameter and reduce the acceleration parameter to make it stable; at the same time, it detects that the dominant probability distribution at this time is sadness, which represents deep affection, so light blue is extracted as the color data of individual particles, and finally the particle animation layer is presented on the screen, in which countless light blue light spots are drifting upwards at a uniform speed, stably and abundantly.

[0233] Furthermore, the dynamic visual feedback data also includes additional text prompt data, and the processing method further includes: comparing the probability distribution of the target emotion type with the probability distribution of the user emotion type to extract the current emotion difference features; retrieving and outputting descriptive text content that matches the emotion difference features from a pre-established prompt library; and dynamically updating the display attributes of the descriptive text content and generating the text prompt data based on the value of the consistency score.

[0234] This dependent claim is a rational supplement to purely sensory visual perception (color and animation). While graphical feedback is intuitive, it cannot tell the user "where the mistake is." This step, by comparing the target features with the user's features again, accurately extracts "emotional difference features" (such as the user singing too loudly where it should be gentle). The "hint library" pre-stores professional guidance, similar to that of a vocal coach. The technical effect is that this invention not only "evaluates" but also directly "teaches," forming a positive error correction mechanism.

[0235] If a user's emotions are too intense and the singing of a certain part of "The Moon Represents My Heart" is out of sync, the system will compare and extract the emotional difference features that are too strong and have an abnormally high probability of happiness. Then, it will retrieve the descriptive text "Please relax your vocal tract and keep your voice smooth and continuous" from the prompt library and overlay it on the top of the screen in large red font (dynamically updated display attributes) as the text prompt data.

[0236] Furthermore, the step of synchronously outputting the text information of the lyrics and the dynamic visual feedback data on the graphical user interface includes: extracting the real-time timestamp information of the currently processed digital audio signal; configuring the rendering engine to draw the background color mapping layer as the overall background rendering layer on the graphical user interface; using the real-time timestamp information as a synchronization reference, highlighting the text information of the lyrics, and rendering the particle animation layer around the text.

[0237] This dependent claim clarifies the temporal alignment logic for the mixed display of various visual elements. Audiovisual synchronization is the lifeline of all music processing systems. Since deep learning models require computation time to extract features and calculate scores, if the visual feedback lags behind the accompaniment music, the user's singing will become increasingly chaotic. "Real-time timestamp information" acts like a built-in clock for the audio, and the rendering engine forcibly calibrates the background, lyrics, and particle animations, which have been processed at the underlying level, to the same time scale before uniformly displaying them. The technical effect is to ensure a tight closed loop in human-computer interaction, achieving extremely low latency synchronization between visual effects and auditory rhythm.

[0238] Furthermore, before applying the pre-trained deep learning model for feature extraction in step S30, the deep learning model needs to be pre-trained in a supervised manner. The specific training steps and parameter settings are as follows:

[0239] 1. Training Dataset Construction: A massive amount of human singing audio samples containing various emotional expressions were collected. The samples underwent acoustic preprocessing to extract Mel spectrograms. Simultaneously, professional vocal annotators manually labeled each audio sample, assigning corresponding emotion type labels (such as distribution vectors for emotions like happiness, sadness, and anger) and emotion intensity labels (scalars from 0 to 1), thus constructing a training dataset containing 'audio feature-emotion label' pairs.

[0240] 2. Loss Function Setting and Forward Propagation: The training dataset is input into the initial deep learning model, employing a multi-task learning architecture. For the sentiment classification task, the cross-entropy loss function is used to calculate the classification error between the predicted probability distribution and the true sentiment type label; for the sentiment intensity regression task, the mean squared error loss function is used to calculate the regression error between the predicted intensity and the true intensity.

[0241] 3. Backpropagation and parameter optimization: The total loss value is obtained by weighted summation of the classification error and regression error. The Adam optimizer is used to execute the backpropagation algorithm to update the network parameters of CNN and LSTM. The hyperparameters during training are set as follows: the initial learning rate is set to [0.001], the batch size is set to

[64] , the maximum number of training epochs is set to

[100] , and an early stopping mechanism is introduced. Training is stopped when the total loss on the validation set no longer decreases for

[10] consecutive epochs to prevent the model from overfitting. Finally, the weights are saved to obtain the pre-trained deep learning model.

[0242] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing audio of emotional singing, characterized in that, include: The lyrics and the target emotional states corresponding to the lyrics are obtained from the song database and are pre-labeled and stored. The target emotional states include the pre-labeled probability distribution of target emotional types and the target emotional intensity value. Receive audio data of a user singing captured by an audio acquisition device, and convert the user singing audio data into a digital audio signal; The digital audio signal is input into a pre-trained deep learning model for feature extraction, generating extracted sentiment indicators, which include the probability distribution of user sentiment type and the user sentiment intensity value. Based on the target emotion type probability distribution, the target emotion intensity value, the user emotion type probability distribution, and the user emotion intensity value, a normalized consistency score is calculated. Based on the consistency score, dynamic visual feedback data is generated, which includes a background color mapping layer and a particle animation layer. The text information of the lyrics and the dynamic visual feedback data are simultaneously output on the graphical user interface.

2. The emotional singing audio processing method according to claim 1, characterized in that, The step of inputting the digital audio signal into a pre-trained deep learning model for feature extraction and generating extracted sentiment indicators includes: The digital audio signal is subjected to frame segmentation and windowing processing to obtain framed audio signals; Extract Mel-frequency cepstral coefficient features from the framed audio signal; A short-time Fourier transform is performed on the Mel frequency cepstral coefficient features to generate a Mel spectrogram, which is used as input data for subsequent neural network feature extraction.

3. The emotional singing audio processing method according to claim 2, characterized in that, The pre-trained deep learning model includes a convolutional neural network and a long short-term memory network, and the generated and extracted sentiment indicators include: The Mel spectrogram is input into the convolutional neural network to perform spatial feature extraction, thereby obtaining a spatial feature sequence; The spatial feature sequence is input into the long short-term memory network to perform time series dependency processing, and the final hidden state sequence is output. The final hidden state sequence is mapped to a numerical result through a fully connected layer, and after processing by an activation function, the probability distribution of the user's sentiment type and the intensity value of the user's sentiment are output.

4. The emotional singing audio processing method according to claim 1, characterized in that, The normalized consistency score is calculated based on the target emotion type probability distribution, the target emotion intensity value, the user emotion type probability distribution, and the user emotion intensity value, including: Construct a target emotion vector that includes the probability distribution of the target emotion type and the target emotion intensity value; Construct a user emotion vector that includes the probability distribution of the user emotion type and the user emotion intensity value; Perform a dot product and modulus operation on the target sentiment vector and the user sentiment vector, and output a cosine similarity score.

5. The emotional singing audio processing method according to claim 4, characterized in that, The calculation of the normalized consistency score also includes: Calculate the absolute difference between the target emotional intensity value and the user's emotional intensity value; The absolute difference value is input into a Gaussian kernel function configured with a bandwidth parameter to be converted into an intensity difference score; Obtain a predefined weight table, and perform weighted merging of the cosine similarity score and the intensity difference score according to the extracted weight coefficients to generate the consistency score.

6. The natural and emotional singing method includes the following steps: Say goodbye to the plain voice training module. Through a combination of training methods such as the staircase method, the voice-warming punch, the surprise method, and the table-slapping method, establish basic pitch perception, vocal power, and preliminary emotional expression ability. The Bright and Pleasant Sound Training Module activates the upper and middle vocal resonance areas through a combination of training methods including the O-shaped technique, the smiling brightening method, and the hand gesture method, thereby enhancing the brightness, forward projection, and emotional expressiveness of the voice. The stereo resonance training module enhances the spatial extension and stereo layering of sound through a combination of training methods such as the ascending method, the water-holding method, and the rowing method. The high-note singing training module establishes a scientific high-note vocal path through a combination of training methods such as yawning, questioning, sighing, and syllable drawing, enabling high notes to produce stable support and bright penetration. The Emotionally Moving Training Module enhances the expressiveness and appeal of singing through a combination of techniques including spraying, physical movements, opera singing, and vocal embellishment, making the singing more sincere and vivid. The "Learn One, Understand All" training module explores the vocal pathways of falsetto, mixed voice, head voice, breathy voice, and vibrato through a combination of training methods including private techniques, warm-up techniques, checkmark techniques, coquettish techniques, and ball-bouncing techniques, thus achieving a transition from basic vocalization to diverse singing styles.

7. The natural emotional singing method according to claim 6, characterized in that, The steps for using the staircase method are as follows: Guide students to understand the relationship between "the pitch of the melody" and "the verticality of the space"; Perform a simulated "going upstairs" action while singing scales upwards; Perform a simulated "going downstairs" action while singing scales downwards; Repeated practice establishes a "movement-pitch" mapping mechanism, enhancing pitch perception.

8. The natural emotional singing method according to claim 6, characterized in that, Opening-up punch operation steps: The trainees clenched their fists and stood upright. Strike your chest forcefully, alternating between left and right, engaging your core muscles. Shout "Ha!" in rhythm, feeling your abdomen tighten instantly; Guide students to focus on the coordinated response between abdominal muscles and breath; Bring this feeling into melody practice to initially establish breath control and vocal support.

9. The natural emotional singing method according to claim 6, characterized in that, The steps for using the "smile bright" method are as follows: Guide trainees to smile naturally, which will lift the facial muscles. Speak naturally while laughing, and experience the feeling of projecting your sound forward; Practice singing with a smile, maintaining a joyful and open vocal cord posture; Incorporate this state into the melody and lyrics to create a stable tone.

10. The natural emotional singing method according to claim 6, characterized in that, Gesture method operation steps: Extract keywords from the lyrics that express happiness; Guide trainees to express keywords using corresponding gestures; Make hand gestures simultaneously while singing to synchronize your voice with your body rhythm; Practice using complete sentences to cultivate the coordination between emotions and actions.

Images