Learning guidance method and device, electronic device, and storage medium

Abstract

The embodiment of the application discloses a learning guidance method and device, electronic equipment and storage medium, and belongs to the technical field of intelligent education. The method comprises the following steps: determining a real-time emotion label of a user, wherein the real-time emotion label is used to indicate an emotional state of the user when the user uses the electronic equipment to learn pronunciation; determining a pronunciation score result of the user according to real-time voice data, wherein the pronunciation score result is used to indicate a correctness rate of the user when the user uses the electronic equipment to learn pronunciation; and outputting target guidance voice according to the real-time emotion label and the pronunciation score result. The application can provide a more personalized human-computer interaction experience for the user based on the emotion of the user in the learning process, so as to help the user relieve negative emotions in the learning process, and then provide effective learning guidance assistance for the user.

Description

Technical Field

[0001] This application relates to the field of intelligent education technology, and includes, but is not limited to, a learning guidance method and device, electronic device, and storage medium. Background Technology

[0002] With the rapid development of computer technology, more and more electronic devices have entered people's work and life, such as learning machines that can help users practice pronunciation.

[0003] In related technologies, common learning machines used for pronunciation practice typically have speech recognition and pronunciation scoring functions. Specifically, after the user begins practicing, the learning machine can acquire the user's voice in real time and compare it with a standard voice to determine whether the user's pronunciation is correct, and then score the user's learning progress.

[0004] However, the solutions in these technologies can only achieve mechanical scoring, making it difficult to provide users with a more human-computer interaction experience or help alleviate negative emotions such as stress and anxiety during the learning process. Therefore, this approach fails to provide effective learning guidance and assistance to users. Summary of the Invention

[0005] In view of this, the learning guidance method, apparatus, electronic device, and storage medium provided in this application can provide users with a more human-computer interaction experience based on their emotions during the learning process, thereby helping users alleviate negative emotions during the learning process and providing effective learning guidance assistance. The learning guidance method, apparatus, electronic device, and storage medium provided in this application are implemented as follows:

[0006] One aspect of this application provides a learning guidance method applied to an electronic device, the method comprising:

[0007] Determine the user's real-time emotion tag, which is used to indicate the user's emotional state when using the electronic device for pronunciation learning;

[0008] Based on real-time voice data, the pronunciation score of the user is determined, and the pronunciation score is used to indicate the accuracy of the user's pronunciation learning when using the electronic device;

[0009] The target guided speech is output based on the real-time emotion label and the pronunciation score result.

[0010] Optionally, the step of outputting the target guided speech based on the real-time emotion label and the pronunciation score result includes:

[0011] An initial guiding voice is generated based on the real-time emotion tag and the pronunciation score result;

[0012] Adjust the playback mode of the initial guiding voice to obtain and output the target guiding voice. The playback mode includes at least one of tone, speed, pitch, or volume when playing the voice.

[0013] Optionally, generating the initial guiding voice based on the real-time emotion tag and the pronunciation score result includes:

[0014] The real-time emotion tags, the pronunciation score results, and the preset template are fused together to obtain the initial guiding voice.

[0015] Optionally, the process of fusing the real-time emotion tag, the pronunciation score result, and the preset template to obtain the initial guiding voice includes:

[0016] The real-time emotion tag, the pronunciation score result, and the preset template are fused together to generate a first processing result that meets preset constraints. The preset constraints are used to make the text in the first processing result match the real-time emotion tag and / or the pronunciation score result.

[0017] The first processing result is converted into speech data to obtain the initial guidance speech.

[0018] Optionally, adjusting the playback mode of the initial guidance voice to obtain and output the target guidance voice includes:

[0019] The playback mode of the initial guiding voice is adjusted based on the real-time emotion tag to obtain and output the target guiding voice.

[0020] Optionally, the method further includes:

[0021] Virtual characters are displayed based on virtual reality technology, and the display parameters of the virtual characters are matched with the target guiding voice. The display parameters include posture and / or facial expressions.

[0022] Optionally, the display of virtual characters based on virtual reality technology includes:

[0023] The display parameters of the virtual character are adjusted based on the target-guided voice.

[0024] Optionally, determining the user's real-time emotion tag includes:

[0025] The user's real-time emotion label is determined based on the user's facial image.

[0026] Optionally, determining the user's real-time emotion tag based on the user's facial image includes:

[0027] Obtain facial information from the facial image, the facial information including facial movements and / or facial expressions;

[0028] The facial information is input into the target emotion recognition model to obtain the real-time emotion label.

[0029] Optionally, determining the user's pronunciation score based on real-time voice data includes:

[0030] The pronunciation score is obtained by comparing the real-time speech data with the standard speech data.

[0031] Optionally, the method further includes:

[0032] Identify the target speech in the real-time speech data and display the text content corresponding to the target speech, wherein the target speech is speech that differs from the standard speech data.

[0033] Another aspect of this application provides a learning guidance device applied to an electronic device, the device comprising:

[0034] The first determining module is used to determine the user's real-time emotion tag, which is used to indicate the user's emotional state when using the electronic device to learn pronunciation.

[0035] The second determining module is used to determine the user's pronunciation score result based on real-time voice data. The pronunciation score result is used to indicate the accuracy rate of the user when using the electronic device for pronunciation learning.

[0036] The output module is used to output the target guiding speech based on the real-time emotion label and the pronunciation score result.

[0037] The electronic device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0038] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.

[0039] The learning guidance method, apparatus, electronic device, and storage medium provided in this application determine the user's real-time emotion tag. Based on real-time speech data, they determine the user's pronunciation score. Based on the real-time emotion tag and the pronunciation score, they output target guidance speech.

[0040] Since users may experience varying emotional states when using an electronic device for pronunciation learning due to factors such as accuracy and environment, determining the real-time emotion tag allows for accurate identification of the user's emotional state during pronunciation learning. This enables the electronic device to interact with the user based on their emotional state. Pronunciation accuracy is a crucial indicator of a user's learning ability; therefore, obtaining the pronunciation score allows the electronic device to accurately determine the user's current learning ability and / or progress, enabling subsequent interaction. While the real-time emotion tag indicates the user's emotional state during pronunciation learning and the pronunciation score indicates accuracy, the target guidance voice is derived from both the real-time emotion tag and the pronunciation score. In other words, the target guidance voice is a synthesis of the user's emotional state and accuracy during pronunciation learning. In this way, when the electronic device outputs the target guidance voice, it can not only evaluate the user's learning results, but also provide emotional voice corresponding to the user's current emotional state to help the user alleviate negative emotions such as stress and anxiety during the learning process (or help the user maintain positive emotions such as happiness and surprise during the learning process) and guide the user to the next step of learning.

[0041] In this way, a more human-computer interaction experience can be provided to users based on their emotions during the learning process, which can help users alleviate negative emotions during the learning process and provide effective learning guidance and assistance, thereby at least partially solving the technical problems raised in the background technology. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart illustrating the first learning guidance method provided in this application embodiment;

[0044] Figure 2 A flowchart illustrating the second learning guidance method provided in this application embodiment;

[0045] Figure 3A flowchart illustrating the third learning guidance method provided in this application embodiment;

[0046] Figure 4 A flowchart illustrating the fourth learning guidance method provided in this application embodiment;

[0047] Figure 5 A flowchart illustrating the fifth learning guidance method provided in this application embodiment;

[0048] Figure 6 A schematic diagram of a virtual character provided for an embodiment of this application;

[0049] Figure 7 A flowchart of the sixth learning guidance method provided in the embodiments of this application;

[0050] Figure 8 This is a schematic diagram of the structure of a learning guidance device provided in an embodiment of this application;

[0051] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit the scope of this application.

[0054] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0055] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0056] In related technologies, common learning machines used for pronunciation practice typically have speech recognition and pronunciation scoring functions. Specifically, after the user begins practicing, the learning machine can acquire the user's voice in real time and compare it with a standard voice to determine whether the user's pronunciation is correct, and then score the user's learning progress.

[0057] However, the solutions in these technologies can only achieve mechanical scoring, making it difficult to provide users with a more human-computer interaction experience or help alleviate negative emotions such as stress and anxiety during the learning process. Therefore, this approach fails to provide effective learning guidance and assistance to users.

[0058] To address this, this application provides a learning guidance method that involves determining a user's real-time emotion tag; determining the user's pronunciation score based on real-time speech data; and outputting target guidance speech based on the real-time emotion tag and the pronunciation score. The real-time emotion tag indicates the user's emotional state while using the electronic device for pronunciation learning, and the pronunciation score indicates the user's accuracy rate while using the electronic device for pronunciation learning. This approach provides a more human-computer interaction experience based on the user's emotions during the learning process, helping to alleviate negative emotions and thus providing effective learning guidance assistance.

[0059] This application uses a learning guidance method applied in electronic devices as an example for illustration. However, it does not imply that this application's embodiments can only be applied to learning guidance in electronic devices.

[0060] Optionally, the electronic device may include, but is not limited to, a learning machine, a smartphone, a tablet computer, a laptop computer, etc. The functions implemented by this method can be achieved by the processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. Therefore, the electronic device includes at least a processor and a storage medium.

[0061] In this embodiment, the electronic device may also include a display screen for providing a display interface, an audio input device (such as a microphone) for collecting voice data, an image acquisition device (such as a camera) for collecting images, and any other possible device. This application embodiment does not limit this.

[0062] The learning guidance method provided in the embodiments of this application will be explained in detail below.

[0063] Figure 1 This is a flowchart illustrating a learning guidance method provided in this application. This method can be applied to the aforementioned electronic device, which may include any control unit and / or processor with functions such as processing, control, recognition, and computation. See also... Figure 1 This application provides a learning guidance method, which includes:

[0064] Step 110: Determine the user's real-time sentiment label.

[0065] In this embodiment, the real-time emotion tag is used to indicate the user's emotional state when using the electronic device to learn pronunciation at the current moment. For example, the real-time emotion tag may include any possible emotion such as confusion, tension, irritability, happiness, surprise, doubt, fear, disgust, embarrassment, contempt, helplessness, annoyance, and fear of difficulty. This embodiment of the application does not limit this.

[0066] In this embodiment, using the electronic device for pronunciation learning can mean: the electronic device plays standard audio data of a text, the user reads the text aloud repeatedly, and then the user's pronunciation is compared with the corresponding standard audio data to determine if the user's pronunciation is correct (i.e., the user learns pronunciation by repeating after the user); or the electronic device can mean: the user selects a text on the electronic device, the user begins to read the text aloud, and then the real-time audio data of the user's reading is compared with the corresponding standard audio data to determine if the user's pronunciation is correct (i.e., the user learns pronunciation by reading independently). This application embodiment does not limit this approach.

[0067] Optionally, the user's real-time emotion label can be determined in any possible way. For example, a facial image of the user while using the electronic device for pronunciation learning can be acquired, and then image recognition can be performed on the facial image to determine the user's real-time emotion label; alternatively, physiological information such as the user's heart rate while using the electronic device for pronunciation learning can be acquired, and then the user's real-time emotion label can be determined based on the physiological information such as the heart rate; alternatively, the user's speech while using the electronic device for pronunciation learning can be acquired, and the user's real-time emotion label can be determined based on the speech. This application does not limit this approach.

[0068] It is worth noting that since users may experience different emotional changes when using electronic devices for pronunciation learning due to various possible factors such as the accuracy of learning and the environment, determining the real-time emotion tag can accurately determine the user's emotional state when using the electronic device for pronunciation learning. This will facilitate the electronic device to interact with the user based on the user's emotional state in the future.

[0069] Step 120: Determine the user's pronunciation score based on real-time voice data.

[0070] In this embodiment, the real-time voice data refers to the voice produced by the user when reading text aloud while using the electronic device for pronunciation learning. In other words, the real-time voice data can indicate the user's actual pronunciation level.

[0071] In this embodiment, the pronunciation score can be obtained by comparing the real-time speech data with the aforementioned standard speech data. Specifically, this can be done by comparing whether each phoneme of the real-time speech data is the same as each phoneme of the standard speech data, or by comparing parameters such as the frequency domain characteristics and time domain characteristics of the real-time speech data and the standard speech data. This application embodiment does not limit this approach.

[0072] Among them, a phoneme can refer to the smallest unit of speech that is divided according to the natural attributes of speech. That is, a phoneme is the smallest unit or the smallest segment of speech that makes up a syllable.

[0073] Optionally, the pronunciation score is used to indicate the accuracy rate of the user's pronunciation learning using the electronic device. For example, if the real-time speech data and standard speech data corresponding to the same text are completely identical, then it can be determined that the user's accuracy rate in learning the pronunciation of this text using the electronic device is 100%; if the real-time speech data and standard speech data corresponding to the same text are completely different, then it can be determined that the user's accuracy rate in learning the pronunciation of this text using the electronic device is 0%. This application does not limit this aspect.

[0074] It is worth noting that, since the accuracy of pronunciation is an important indicator of a user's learning ability during the pronunciation learning process, after obtaining the pronunciation score, the electronic device can accurately determine the user's current learning ability and / or learning status, so that the electronic device can subsequently interact with the user based on the user's learning ability and / or learning status.

[0075] Step 130: Output the target guidance speech based on the real-time emotion label and the pronunciation score result.

[0076] Optionally, the target guidance speech can refer to a speech generated by the electronic device based on the real-time emotion tag and the pronunciation score result to guide the user to continue learning. Specifically, the target guidance speech can be speech with certain emotional characteristics, rather than mechanical, neutral speech.

[0077] In this embodiment, the target guiding voice can be made to have different emotional characteristics by adjusting parameters such as tone, speed, pitch, or volume. For example, if the target guiding voice is required to have the emotional characteristic of "severity," then the pitch can be adjusted to be higher, the speed to be faster, and the volume to be louder; if the target guiding voice is required to have the emotional characteristic of "gentleness," then the pitch can be adjusted to be lower, the speed to be slower, and the volume to be lower. This application embodiment does not limit this.

[0078] Specifically, outputting the target guidance voice can be achieved by outputting the target guidance voice through an audio playback device installed in the electronic device, or by sending the target guidance voice to any other device with an audio playback device that is directly or indirectly connected to the electronic device. This application does not limit this approach.

[0079] It's worth noting that while the real-time emotion tag indicates the user's emotional state during pronunciation learning using the electronic device, and the pronunciation score indicates the user's accuracy rate, the target guidance voice is derived from both the real-time emotion tag and the pronunciation score. In other words, the target guidance voice is a synthesis of the user's emotional state and accuracy rate during pronunciation learning. Thus, when the electronic device outputs the target guidance voice, it not only evaluates the user's learning progress but also provides emotionally resonant voice corresponding to the user's current emotional state. This helps alleviate negative emotions such as stress and anxiety during learning (or helps maintain positive emotions such as happiness and surprise) and guides the user to the next step of learning.

[0080] In this embodiment, a user's real-time emotion label is determined. Based on real-time speech data, the user's pronunciation score is determined. Target guidance speech is then output based on the real-time emotion label and the pronunciation score.

[0081] Since users may experience varying emotional states when using an electronic device for pronunciation learning due to factors such as accuracy and environment, determining the real-time emotion tag allows for accurate identification of the user's emotional state during pronunciation learning. This enables the electronic device to interact with the user based on their emotional state. Pronunciation accuracy is a crucial indicator of a user's learning ability; therefore, obtaining the pronunciation score allows the electronic device to accurately determine the user's current learning ability and / or progress, enabling subsequent interaction. While the real-time emotion tag indicates the user's emotional state during pronunciation learning and the pronunciation score indicates accuracy, the target guidance voice is derived from both the real-time emotion tag and the pronunciation score. In other words, the target guidance voice is a synthesis of the user's emotional state and accuracy during pronunciation learning. In this way, when the electronic device outputs the target guidance voice, it can not only evaluate the user's learning results, but also provide emotional voice corresponding to the user's current emotional state to help the user alleviate negative emotions such as stress and anxiety during the learning process (or help the user maintain positive emotions such as happiness and surprise during the learning process) and guide the user to the next step of learning.

[0082] In this way, a more human-computer interaction experience can be provided based on the user's emotions during the learning process, which can help alleviate negative emotions during the learning process and thus provide effective learning guidance and assistance.

[0083] In addition, because the learning guidance method provided in this application embodiment can analyze multiple dimensions of information such as the user's pronunciation assessment results and real-time emotion tags, the method can more comprehensively understand the user's learning status when using the electronic device to learn pronunciation. In this way, it can provide users with more accurate learning guidance to improve the effect of learning guidance.

[0084] The following explains how the learning guidance method provided in this application specifically generates and outputs the target guidance speech. In one possible implementation, see [link to relevant documentation]. Figure 2 Based on the real-time emotion label and the pronunciation score, the target guided speech is output, including:

[0085] Step 1301: Generate initial guidance speech based on the real-time emotion label and the pronunciation score result.

[0086] Optionally, the initial guiding speech may refer to neutral speech (i.e., mechanical speech) generated based on the real-time emotion label and the pronunciation score, without any emotional features.

[0087] Specifically, a script or text for interacting with the user can be determined based on the real-time emotion tag and the pronunciation score, and then the determined script or text can be converted into speech to obtain the initial guiding voice.

[0088] For example, if the user's real-time emotion label is determined to be "nervous" at the current moment, then a script or text with the content "Don't be nervous, let's practice English pronunciation easily together today" can be determined, and then this script or text can be converted into the initial guiding voice; if the user's real-time emotion label is determined to be "nervous" at the current moment, and the pronunciation score indicates that the user has made pronunciation errors during the pronunciation learning process, then a script or text with the content "You made a small mistake this time, but you've done great, try again" can be determined, and then this script or text can be converted into the initial guiding voice.

[0089] For example, if at the current moment, the user's real-time emotion label is determined to be "low" and the pronunciation score indicates that the user has made pronunciation errors during the pronunciation learning process, then a speech or text with the content "I see you are a little tired, do you want to take a break? We can try again, you can definitely do it" can be determined, and then this speech or text can be converted into the initial guiding voice.

[0090] For example, if at the current moment, the user's real-time emotion label is determined to be "confident" and the pronunciation score indicates that the user has a high accuracy rate in pronunciation during the pronunciation learning process, then a script or text with the content "You did very well! Now let's challenge ourselves with a more difficult sentence and see if you can complete it" can be determined, and then this script or text can be converted into the initial guiding voice.

[0091] It is understood that the above are merely examples provided to illustrate the generation of initial guiding speech (i.e., interactive dialogue or interactive text) based on the real-time emotion tag and the pronunciation score result. They do not represent that the learning guidance method provided in this application can only generate the initial guiding speech in this way, and the embodiments of this application do not limit this.

[0092] Step 1302: Adjust the playback mode of the initial guidance voice, obtain and output the target guidance voice.

[0093] Optionally, the playback method includes at least one of the following: tone of voice, speaking speed, pitch, or volume when playing the speech.

[0094] It is worth noting that since the initial guidance voice is obtained by converting the interactive dialogue or interactive text determined based on the real-time emotion label and the pronunciation score into audio, the emotional features of the initial guidance voice are not obvious enough, or the degree of matching with the current scene is low. Therefore, by adjusting the playback method of the initial guidance voice, it can be made to have certain emotional features, thus obtaining a target guidance voice with emotional features.

[0095] Specifically, adjusting the playback method of the initial guidance voice to obtain and output the target guidance voice includes:

[0096] Based on the real-time emotion tag, adjust the playback method of the initial guiding voice to obtain and output the target guiding voice.

[0097] For example, if the real-time emotion label is a negative emotion such as "confusion, tension, loss, frustration, irritability, doubt, fear, disgust, embarrassment, annoyance, fear of difficulty, or helplessness," then the playback mode of the initial guidance voice can be adjusted to obtain and output a target guidance voice with emotional characteristics such as "gentle" or "gentle." If the real-time emotion label is a negative emotion such as "contempt, pride, or laxity," then the playback mode of the initial guidance voice can be adjusted to obtain and output a target guidance voice with emotional characteristics such as "anger" or "sternness." If the real-time emotion label is a positive emotion such as "happiness, joy, surprise, or confidence," then the playback mode of the initial guidance voice can be adjusted to obtain and output a target guidance voice with emotional characteristics such as "positive" or "gentle." This application does not limit this specific approach.

[0098] For example, based on the above example, if the user's real-time emotion label is "nervous" at the current moment, the initial guiding voice content could be "Don't be nervous, let's practice English pronunciation easily together today" or "You made a small mistake this time, but you did great, try again." Specifically, by adjusting the tone, pitch, speed, and volume of the initial guiding voice to be more gentle, lower, slower, and lower (specifically by "making the fundamental frequency of the pitch lower and more stable, avoiding sharp fluctuations, and slightly lowering the pitch at the end of the sentence; making the overall volume of the sentence lower and the dynamic range smaller, avoiding sudden loudness changes; making the speed slower, the syllable boundaries clear, and the consonants soft"), a target guiding voice with emotional characteristics such as "gentle" and "gentle" can be obtained and output.

[0099] For example, based on the above example, if the user's real-time emotion label at the current moment is "frustrated, doubtful, annoyed, and fearful," and the initial guiding voice content is "It's okay, you've done great! We can try again, you can definitely do it." Specifically, this can be achieved by adjusting the tone of the initial guiding voice to be more cheerful, significantly raising the pitch, increasing the speaking speed, and increasing the volume (specifically, by "significantly raising the fundamental frequency of the pitch and expanding its fluctuation range to 1.2-1.5 times that of neutral speech, with an upward pitch at the end of the sentence; making the overall volume of the sentence higher, with a high dynamic range, making the speech sound brighter; increasing the speaking speed and shortening the pauses between syllables to 0.1-0.3 seconds, enhancing the coherence of the sentence"). This will result in a target guiding voice with emotional characteristics such as "positive and relaxed."

[0100] For example, based on the above example, if the user's real-time emotion label at the current moment is "contempt, pride, or laxity," and the initial guiding voice content is "Don't be too proud, there are more difficult sentences to come, concentrate," then the target guiding voice with emotional characteristics such as "severity" can be obtained and output by adjusting the tone of the initial guiding voice to be very urgent, the pitch to be higher, the speech rate to be faster, and the volume to be higher (specifically by "significantly raising and fluctuating the pitch, with the pitch at the end of the sentence often showing a sharp rise or fall to enhance the sense of intimidation; making the overall volume of the sentence increase sharply, with the spectral energy concentrated in the mid-to-high frequency range of 2-4kHz to enhance the sense of oppression; and shortening the pauses between syllables to less than 0.2 seconds, making the sentences appear rapid and disjointed").

[0101] It is worth noting that, in this embodiment, at least one parameter of the initial guidance voice—tone, speed, pitch, or volume—can be adjusted to give the initial guidance voice emotional characteristics corresponding to the user's real-time emotion tag, thereby obtaining and outputting the target guidance voice. This achieves the goal of providing a more human-computer interaction experience based on the user's emotions during the learning process.

[0102] In one possible implementation, see [link to relevant documentation]. Figure 3 Based on the real-time emotion tag and the pronunciation score, an initial guiding voice is generated, including:

[0103] Step 1303: The real-time emotion tag, the pronunciation score result, and the preset template are fused to obtain the initial guiding voice.

[0104] In this embodiment, the preset template can refer to some text templates or script templates set by relevant technical personnel according to actual needs. For example, the preset template can be in JSON format or any other possible format. Furthermore, the preset template can include various text templates corresponding to different real-time emotion tags.

[0105] For example, the preset template may include text templates corresponding to various negative emotions such as "Keep it up," "You've worked hard," "Don't be discouraged," and "Don't be arrogant," or text templates corresponding to various positive emotions such as "Great," "Very good," and "Keep trying harder challenges." Additionally, the preset template may include text templates corresponding to the pronunciation score result. For instance, the preset template may include text templates corresponding to a high accuracy rate in the user's pronunciation learning, such as "Well said," "Great speech," and "Very accurate," or text templates corresponding to a low accuracy rate in the user's pronunciation learning, such as "You made a small mistake this time, but you've done great," and "Don't be discouraged, try practicing a few more times." This application embodiment does not limit this.

[0106] Optionally, the real-time emotion tag, the pronunciation score, and the preset template can be fused using language processing techniques such as Natural Language Generation (NLG) and Natural Language Processing (NLP). For example, the real-time emotion tag and the pronunciation score can be input into an NLG language model built based on NLG technology. The NLG language model can then convert and parse the real-time emotion tag and the pronunciation score, and based on the processing results of the real-time emotion tag and the pronunciation score, various preset templates can be called to fill and combine them to obtain the initial guiding speech. This application does not limit this aspect.

[0107] Specifically, see Figure 4 The initial guiding voice is obtained by fusing the real-time emotion label, the pronunciation score, and the preset template, including:

[0108] Step 1304: The real-time emotion tag, the pronunciation score result, and the preset template are fused together to generate a first processing result that meets the preset constraints.

[0109] Optionally, the real-time emotion tag, the pronunciation score, and the preset template can be fused using NLG technology to generate a first processing result that meets the preset constraints.

[0110] Optionally, the preset constraint is used to ensure that the text in the first processing result matches the real-time emotion tag and / or the pronunciation score result.

[0111] In this embodiment, the preset constraints can be set by relevant technical personnel according to actual needs. For example, the preset constraints can be used to quantify the mapping relationship between emotional intensity values ​​and language intensity, dynamically adjust the emotional expression intensity of the generated text, and ensure that the output content matches the user's current emotional state; the preset constraints can also perform syntactic structure verification on the generated text, specifically including subject-verb-object integrity checks, double negation conversion, etc., to eliminate grammatical errors and improve text readability; the preset constraints can also be based on specialized dictionaries and filtering mechanisms in the education field to achieve sensitive word replacement, banned word blocking, and professional terminology adaptation (for example, by automatically replacing negative evaluation words through the preset constraints, "there are errors" can be replaced with "there is room for improvement," and encouraging sentence structures should be used as much as possible) to ensure that the generated content conforms to industry standards; the preset constraints can also control the diversity of generated results through historical record comparison and content duplication detection (such as limiting the frequency of use of the same preset template when generating continuously), avoiding mechanical repetition that leads to a decline in user experience. This application embodiment does not limit this aspect.

[0112] In this embodiment, the first processing result may refer to a piece of text or a speech obtained after combining, splicing, or otherwise processing multiple preset templates. That is, the first processing result may be in text format.

[0113] Step 1305: Convert the first processing result into speech data to obtain the initial guidance speech.

[0114] Optionally, the first processing result can be converted by calling a corresponding speech synthesis algorithm, such as a text-to-speech (TTS) algorithm. Specifically, in the process of converting the first processing result into speech data, a markup method such as Speech Synthesis Markup Language (SSML) can be used to guide the TTS algorithm to generate more natural and accurate speech output, so as to control the pronunciation, rhythm, emotion, and other parameters of the converted speech data, making the initial guiding speech synthesized by the electronic device as close as possible to human expression. This application does not limit this aspect.

[0115] It's worth noting that this allows the electronic device to dynamically generate personalized and targeted encouraging messages (i.e., the initial guiding voice) based on the user's emotional state and pronunciation. Furthermore, by subsequently adjusting the playback of this initial guiding voice based on the real-time emotion tag to obtain the target guiding voice, it ensures that the encouraging messages output by the electronic device are emotionally resonant and motivating, thereby improving the human-computer interaction experience and effectively stimulating the user's learning enthusiasm.

[0116] In one possible implementation, see [link to relevant documentation]. Figure 5 The method also includes:

[0117] Step 140: Display virtual characters based on virtual reality technology.

[0118] Optionally, the display parameters of the virtual character are matched with the target guiding voice, and the display parameters include posture and / or facial expressions.

[0119] Specifically, displaying virtual characters based on virtual reality technology includes adjusting the display parameters of the virtual character based on the target-guided voice.

[0120] For example, if the target guidance voice has an emotional tendency such as "gentle, mild, positive", the virtual character's facial expression can be set to smiling, blinking, or raising eyebrows, and the virtual character's gestures can be set to waving, hugging, and / or touching the head. Alternatively, if the target guidance voice has an emotional tendency such as "stern", the virtual character's facial expression can be set to frowning and / or pursing lips, and the virtual character's gestures can be set to crossing arms or slowly shaking the head.

[0121] For example, if the guiding voice includes words with meanings such as encouragement, praise, or commendation, the virtual character's facial expression can be set to a smile or a blink, and the virtual task's gesture can be set to a thumbs-up. Alternatively, if the guiding voice includes words with meanings such as a lesson or reprimand, the virtual character's facial expression can be set to a frown, and the virtual task's gesture can be set to crossed arms or a slow head shake. This application does not limit these specific actions.

[0122] For example, see Figure 6 , Figure 6 The diagram illustrates several possible virtual avatars. For example, the virtual avatar is displayed when the user turns on the electronic device. If the electronic device determines that the user's real-time emotion label is "nervous," then... Figure 6As shown in (a), the virtual character's facial expression can be set to a smile and its gesture to waving; at the same time, the text "Don't be nervous, let's practice English pronunciation easily together today" can be displayed, and the corresponding audio can be played at the same time.

[0123] For example, when a user begins using the electronic device to learn pronunciation, the device acquires the user's real-time speech data and determines the pronunciation score. If the user makes a pronunciation error or the pronunciation score is low, and the real-time emotion label is "nervous," then a target guidance voice with the emotional characteristic of "gentle" can be generated and output. The content of this target guidance voice could be something like, "You made a small mistake this time, but you did great! Try again!" Meanwhile, if... Figure 6 As shown in (b), the virtual character's facial expression can be set to a smile and its gesture to a thumbs-up; at the same time, the text corresponding to the target's guiding voice can also be displayed.

[0124] For example, if a user encounters pronunciation difficulties during the process of learning pronunciation using the electronic device and fails to pronounce successfully after multiple attempts, exhibiting a depressed expression and actions, then the real-time emotion label can be determined to be "depressed," and the accuracy indicated by the pronunciation score is low. Therefore, a target guiding voice with the emotional characteristic of "regret" can be generated and output. The content of this target guiding voice could be, "I see you're a little tired. Would you like to take a break? We can try again; you can definitely do it." Meanwhile, if... Figure 6 As shown in (c), the virtual character's facial expression can be set to a smile and its posture to an embrace; at the same time, text corresponding to the target's guiding voice can also be displayed.

[0125] For example, if a user begins using the electronic device to learn pronunciation, and after repeated practice, their pronunciation gradually becomes more accurate, displaying confident expressions and gestures, then the real-time emotion label can be determined to be "confident," and the accuracy rate indicated by the pronunciation score is high. Therefore, a target-guided voice with a "positive" emotional characteristic can be generated and output. This target-guided voice could say something like, "You did very well! Now let's challenge ourselves with a more difficult sentence and see if you can complete it!" However, if the electronic device's current display screen also shows other content, then the virtual character can be displayed in an area that does not obstruct other content. Figure 6 As shown in (d), the virtual character's facial expression can be set to a smile, and the virtual character's posture and movements can be hidden; at the same time, text corresponding to the target guidance voice can also be displayed.

[0126] It is worth noting that, in this embodiment, the display parameters of the virtual character can be adaptively adjusted according to the different target guidance voice outputs, so that the virtual character can exhibit expressions and / or actions that match the target guidance voice, the user's current real-time emotion tag, and / or pronunciation score results. This provides users with a more human-computer interaction experience, helping to alleviate negative emotions during the learning process and thus providing effective learning guidance assistance.

[0127] In one possible implementation, see [link to relevant documentation]. Figure 7 To determine the user's real-time sentiment tags, including:

[0128] Step 1101: Determine the user's real-time emotion label based on the user's facial image.

[0129] Optionally, the user's facial image can be acquired in real time by an image acquisition device installed in the electronic device, or it can be acquired by other devices or apparatuses connected to the electronic device and then sent to the electronic device. This application embodiment does not limit this.

[0130] Specifically, based on the user's facial image, the user's real-time emotion tag is determined, including:

[0131] Obtain the facial information from the facial image.

[0132] Optionally, the facial information includes facial movements and / or facial expressions.

[0133] For example, the facial movements may include any possible movements such as blinking frequency, raising eyebrows, frowning eyebrows, tightening facial muscles, relaxing facial muscles, closing lips, slightly trembling lips, raising or lowering the corners of the mouth. The facial expression can include expressions indicating confusion, such as "slightly narrowed eyes, as if deep in thought; the eyeballs may move slightly up, down, left, or right; eyebrows are gathered together, slightly furrowed; mouth slightly open, as if about to speak; lips may be slightly parted or pursed, attempting to pronounce a word"; rapid blinking, shifty gaze, raised eyebrows, slight trembling at the ends of the eyebrows, tightly closed lips, pale lips, or slightly open lips, trembling lips; sharp, impatient eyes, rapid eye movement, frequent side-eyes or rolling of the eyes, furrowed eyebrows, irregular trembling, lips pursed into a line, corners of the mouth turned down, clenched teeth, or slight exhalation"; and averted gaze, dull eyes, furrowed eyebrows, trembling and closed mouth, corners of the mouth turned down, indicating a fearful or hesitant expression. Additionally, the facial expression can include any other possible expressions. This application does not limit the scope of the embodiments.

[0134] The facial information is input into the target emotion recognition model to obtain the real-time emotion label.

[0135] In this embodiment, the target emotion recognition model can be pre-trained. The target emotion recognition model can be trained based on any possible algorithm such as convolutional neural network, long short-term memory network, multimodal large model analysis technology, etc. This application embodiment does not limit this.

[0136] In one possible implementation, the method further includes:

[0137] The initial emotion recognition model is trained at least once based on the sample training set to obtain the target emotion recognition model.

[0138] In this embodiment, the training set may include multiple different sample images and corresponding emotion labels for each sample image. Each sample image may be a facial image of a different person, and the emotion labels for each sample may generally be of the same type as the aforementioned real-time emotion labels.

[0139] Optionally, the initial emotion recognition model can refer to a target emotion recognition model that has not yet started training or has not been fully trained. Moreover, the initial emotion recognition model can also be used to predict the user's emotion label. The initial emotion recognition model and the target emotion recognition model have the same architecture, but the parameters of the initial emotion recognition model and the target emotion recognition model are different.

[0140] Understandably, this target emotion recognition model is the initial emotion recognition model after training. Furthermore, the performance of this target emotion recognition model in outputting user emotion labels is significantly improved compared to the initial emotion recognition model. In other words, by iteratively training and refining the initial emotion recognition model using this sample training set, the accuracy of the initial emotion recognition model in predicting user emotion labels can be improved. This, in turn, ensures the accuracy of the real-time emotion labels output by the target emotion recognition model.

[0141] Each training session includes:

[0142] Based on this sample training set, multiple sample images and corresponding sample emotion labels are obtained.

[0143] Feature extraction is performed on each sample image to obtain the image features corresponding to each sample image.

[0144] Optionally, feature extraction can be achieved by performing convolution and pooling operations on each sample image, or by inputting each sample image into an encoder-decoder structure, or by any other possible method. This application embodiment does not limit the specific methods used.

[0145] It is worth noting that after feature extraction from each sample image, the data dimensionality and redundant information can be reduced to obtain image features useful for training the initial emotion recognition model. This improves the efficiency and accuracy of training the initial emotion recognition model, and thus enhances the generalization performance of the initial emotion recognition model.

[0146] The initial emotion recognition model is trained based on the image features corresponding to each sample image and the sample emotion labels corresponding to each sample image.

[0147] It is important to understand that during the training of the initial emotion recognition model, the preset sample set is mainly used to train the parameters of the initial emotion recognition model. By using the preset sample set, the initial emotion recognition model can learn the mapping relationship between the input data and the output data (that is, the relationship between the image features of the facial image and the predicted real-time emotion label), thereby reducing the error when the initial emotion recognition model and the target emotion recognition model predict the corresponding emotion prediction information.

[0148] In one possible implementation, see [link to previous section] Figure 7 Based on real-time voice data, determine the user's pronunciation score, including:

[0149] Step 1201: Compare the real-time speech data with the standard speech data to obtain the pronunciation score result.

[0150] In this embodiment, the standard speech data can correspond to the same text as the real-time speech data. Furthermore, the standard speech data can indicate the correct and standard pronunciation of this text. Therefore, by comparing the real-time speech data with the standard speech data, it can be determined whether the user's pronunciation of this text is correct, and the specific accuracy rate can be determined.

[0151] In this embodiment, appropriate speech recognition technology can be used to convert the real-time speech data into a first phoneme sequence and the standard speech data into a second phoneme sequence, respectively. Each of the first and second phoneme sequences includes multiple phonemes arranged in temporal order. The first and second phoneme sequences are then aligned using a Dynamic Time Warping (DTW) algorithm to eliminate the influence of speech rate differences and ensure that phonemes corresponding to the same syllable match on the time axis. The similarity between the real-time speech data and the standard speech data is then calculated by comparing the spectral characteristics (i.e., pronunciation differences) between the two phonemes corresponding to each syllable in the first and second phoneme sequences. In this way, the accuracy rate of the real-time speech data and the pronunciation score can be accurately determined.

[0152] Alternatively, the similarity between the real-time speech data and the standard speech data can be determined by converting the real-time speech data into a first text, converting the standard speech data into a second text, and then comparing the differences between the first text and the second text. This application does not limit this approach.

[0153] As can be seen, this application allows for the determination of the user's pronunciation score through a variety of different methods, providing a degree of flexibility.

[0154] In one possible implementation, the method further includes:

[0155] Identify the target speech in the real-time speech data and display the corresponding text content.

[0156] Optionally, the target speech is speech that differs from standard speech data. That is, the target speech can refer to a speech segment in which the user makes pronunciation errors during pronunciation practice.

[0157] Specifically, the target speech can also be determined by comparing the real-time speech data with the standard speech data, but this application does not limit this method.

[0158] For example, see [link to previous article] Figure 6 ,like Figure 6As shown in (d), assuming the user's pronunciation practice exercise 1 is "ABCDEFGHIJKLMN", and the user's reading result determined based on the real-time speech data is also "ABCDEFGHIJKLMN", then it can be determined that the real-time speech data is completely identical to the standard speech data. Therefore, it can be determined that the target speech with a pronunciation score of 100 and no pronunciation errors can be identified. Furthermore, assuming the user's pronunciation practice exercise 2 is "nice to meet you", and the user's reading result determined based on the real-time speech data is also "nancy to meet you", then it can be determined that there is a difference between the real-time speech data and the standard speech data. Specifically, the "nancy" in the real-time speech data is different from the "nice" in the standard speech data. Therefore, it can be determined that the target speech with a pronunciation score of 80 and pronunciation errors can be identified as "nice".

[0159] from Figure 6 As can be seen from (d) in the diagram, in addition to displaying the text content corresponding to the target speech and the virtual character, the electronic device can also display any possible content such as the pronunciation question, the user's reading result corresponding to the real-time speech data, and the pronunciation score result. This application does not limit this aspect.

[0160] In another possible way, besides Figure 6 Besides the method shown in (d), the text content corresponding to the target speech can also be displayed in any other possible way. For example, the text content corresponding to the target speech that the user mispronounced can be highlighted or displayed in a special color in the pronunciation question. This application does not limit this.

[0161] As can be seen from the above, when a user uses this electronic device for pronunciation learning, the device can accurately and reliably display the text content on which the user makes pronunciation errors during pronunciation practice, so that the user can promptly identify and correct the errors. In this way, it can provide the user with effective learning guidance and assistance.

[0162] It should be noted that, in the learning guidance method provided in this application, in addition to outputting the target guidance voice based on the real-time emotion tag and the pronunciation score result, the target guidance voice can also be output based on the real-time emotion tag or based on the pronunciation score result. This application does not limit this aspect.

[0163] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0164] Based on the foregoing embodiments, this application provides a learning guidance device, which includes various modules and units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0165] Figure 8 This is a schematic diagram of the structure of a learning guidance device provided in an embodiment of this application. See also... Figure 9 The device includes:

[0166] The first determining module 201 is used to determine the user's real-time emotion tag.

[0167] Optionally, the real-time emotion tag is used to indicate the user's emotional state when using the electronic device for pronunciation learning.

[0168] The second determining module 202 is used to determine the pronunciation score of the user based on real-time voice data.

[0169] Optionally, the pronunciation score is used to indicate the accuracy of the user's pronunciation learning when using the electronic device.

[0170] Output module 203 is used to output target guidance speech based on the real-time emotion label and the pronunciation score result.

[0171] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0172] It should be noted that, in the embodiments of this application... Figure 8The module division shown in the learning guidance device is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or be integrated into one unit with two or more units. The integrated units can be implemented in hardware, as software functional units, or a combination of both.

[0173] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0174] This application provides an electronic device, which can be any possible device such as a learning machine, and its internal structure diagram can be as follows: Figure 9 As shown, the electronic device includes a processor 302, a memory, and a network interface 303 connected via a system bus 301. The processor 302 provides computing and control capabilities. The memory includes internal memory 3041 and a non-volatile storage medium 3042. The non-volatile storage medium 3042 stores an operating system, computer programs, and a database. The internal memory 3041 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium 3042. The database is used to store data. The network interface 303 is used to communicate with external terminals via a network connection. When the computer program is executed by the processor 302, it implements the aforementioned methods.

[0175] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0176] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0177] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0178] In one embodiment, the learning guidance device provided in this application can be implemented as a computer program, and the computer program can be implemented as follows: Figure 9 The device operates on the electronic device shown. The memory of the electronic device can store the various program modules that make up the above-described apparatus. The computer program composed of the various program modules causes the processor to execute the steps of the methods in the various embodiments of this application described in this specification.

[0179] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0180] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0181] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0182] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0184] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0185] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0186] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0187] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0188] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0189] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0190] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0191] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0192] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A learning guidance method, characterized in that, Applied to electronic devices, the method includes: Determine the user's real-time emotion tag, which is used to indicate the user's emotional state when using the electronic device for pronunciation learning; Based on real-time voice data, the pronunciation score of the user is determined, and the pronunciation score is used to indicate the accuracy of the user's pronunciation learning when using the electronic device; The target guided speech is output based on the real-time emotion label and the pronunciation score result.

2. The learning guidance method as described in claim 1, characterized in that, The step of outputting the target guided speech based on the real-time emotion tag and the pronunciation score result includes: An initial guiding voice is generated based on the real-time emotion tag and the pronunciation score result; Adjust the playback mode of the initial guiding voice to obtain and output the target guiding voice. The playback mode includes at least one of tone, speed, pitch, or volume when playing the voice.

3. The learning guidance method as described in claim 2, characterized in that, The step of generating initial guidance speech based on the real-time emotion tag and the pronunciation score result includes: The real-time emotion tags, the pronunciation score results, and the preset template are fused together to obtain the initial guiding voice.

4. The learning guidance method as described in claim 3, characterized in that, The process of fusing the real-time emotion tag, the pronunciation score result, and the preset template to obtain the initial guiding voice includes: The real-time emotion tag, the pronunciation score result, and the preset template are fused together to generate a first processing result that meets preset constraints. The preset constraints are used to make the text in the first processing result match the real-time emotion tag and / or the pronunciation score result. The first processing result is converted into speech data to obtain the initial guidance speech.

5. The learning guidance method as described in claim 2, characterized in that, The step of adjusting the playback mode of the initial guidance voice to obtain and output the target guidance voice includes: The playback mode of the initial guiding voice is adjusted based on the real-time emotion tag to obtain and output the target guiding voice.

6. The learning guidance method as described in any one of claims 1-5, characterized in that, The method further includes: Virtual characters are displayed based on virtual reality technology, and the display parameters of the virtual characters are matched with the target guiding voice. The display parameters include posture and / or facial expressions.

7. The learning guidance method as described in claim 6, characterized in that, The virtual character display based on virtual reality technology includes: The display parameters of the virtual character are adjusted based on the target-guided voice.

8. The learning guidance method as described in any one of claims 1-5, characterized in that, The determination of the user's real-time emotion tag includes: The user's real-time emotion label is determined based on the user's facial image.

9. The learning guidance method as described in claim 8, characterized in that, The step of determining the user's real-time emotion tag based on the user's facial image includes: Obtain facial information from the facial image, the facial information including facial movements and / or facial expressions; The facial information is input into the target emotion recognition model to obtain the real-time emotion label.

10. The learning guidance method according to any one of claims 1-5, characterized in that, The step of determining the user's pronunciation score based on real-time voice data includes: The pronunciation score is obtained by comparing the real-time speech data with the standard speech data.

11. The learning guidance method according to any one of claims 1-5, characterized in that, The method further includes: Identify the target speech in the real-time speech data and display the text content corresponding to the target speech, wherein the target speech is speech that differs from the standard speech data.

12. A learning guidance device, characterized in that, Applied to electronic devices, the device includes: The first determining module is used to determine the user's real-time emotion tag, which is used to indicate the user's emotional state when using the electronic device to learn pronunciation. The second determining module is used to determine the user's pronunciation score result based on real-time voice data. The pronunciation score result is used to indicate the accuracy rate of the user when using the electronic device for pronunciation learning. The output module is used to output the target guiding speech based on the real-time emotion label and the pronunciation score result.

13. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 11.

Images