Human-computer interaction method and device, electronic equipment and program product

Abstract

The application is suitable for the field of artificial intelligence technology, and provides a man-machine interaction method and device, electronic equipment and program product, comprising: in response to a question answering starting operation, determining a learning stage of a question answerer through a camera device of the electronic equipment; based on the learning stage, determining a question answering feedback strategy corresponding to the question answerer; in the question answering process, based on the question answering feedback strategy and the question answering information of the question answerer at different time, visualizing and displaying different virtual image feedback contents; the question answering information comprises at least one of a question answering result, a question answering correct number and facial information of the question answerer. The application determines the learning stage of the question answerer through camera recognition, and then matches the feedback strategy suitable for the learning stage, so that the feedback is more suitable for the cognitive level and learning needs of the question answerers at different stages, and the effectiveness and pertinence of the feedback are enhanced. Meanwhile, the application also dynamically generates corresponding virtual image visualized linkage feedback combined with real-time question answering information, and the interest and interactivity of learning are enhanced.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a human-computer interaction method, device, electronic device and program product. Background Technology

[0002] Currently, with the rapid development of computer technology, users can learn through the internet in electronic environments comprised of communication technology, computer technology, artificial intelligence, and multimedia technology (such as online education quiz systems). Taking online education quiz systems as an example, existing technologies typically only achieve human-computer interaction through simple text symbols (such as checkmarks or crosses), sound effects, or point rewards when users answer questions. This approach is not comprehensive enough and fails to meet actual needs. Summary of the Invention

[0003] This application provides a human-computer interaction method, device, electronic device, and program product to solve the problem that the existing technology simply achieves human-computer interaction through text symbols (such as checkmarks or crosses), sound effects, or point rewards, which is not comprehensive enough and cannot meet actual needs.

[0004] In a first aspect, embodiments of this application provide a human-computer interaction method, including: In response to the start of the question-answering process, the learning stage of the respondent is determined by the camera device of the electronic device; Based on the learning stage, determine the corresponding response feedback strategy for the respondent; During the quiz process, based on the quiz feedback strategy and the quiz information of the quizee at different times, different virtual image feedback content is visualized and displayed; the quiz information includes at least one of the following: quiz result, number of correct answers, and facial information of the quizee.

[0005] The beneficial effects of the embodiments in this application compared with the prior art are: This application provides a human-computer interaction method that, in response to a question-answering initiation operation, determines the learner's learning stage using an electronic device's camera. Based on the learning stage, a corresponding question-answering feedback strategy is determined. During the question-answering process, based on the feedback strategy and the learner's answer information at different times, different virtual avatar feedback content is visualized. The answer information includes at least one of the following: answer result, number of consecutive correct answers, and the learner's facial information. This application determines the learner's learning stage through camera recognition and then matches it with a corresponding feedback strategy, making the feedback more aligned with the learner's cognitive level and learning needs at each stage, thus enhancing the effectiveness and relevance of the feedback. Furthermore, this application combines multi-dimensional answer information such as correct / incorrect answers, consecutive correct answers, and facial expressions to dynamically switch virtual avatar feedback. It is not limited to a single correct / incorrect prompt; different encouragements can be given based on the number of consecutive correct answers, and different feedback can be given based on different emotions, making the feedback more humanized and more aligned with the real-time learning state, enhancing the fun and interactivity of learning.

[0006] In some embodiments, the answer information includes the number of consecutive correct answers; based on the answer feedback strategy and the answer information of the respondent at different times, different virtual avatar feedback content is visualized, including: The correct answer level of a respondent is determined based on the number of consecutive correct answers. From the existing virtual avatar action library, determine the action information that matches the correct answer level; the action information includes action type, action range, and action effects; The first feedback content is displayed visually; the first feedback content refers to the virtual image feedback content that matches the action information and the answer feedback strategy.

[0007] In the above embodiments, answer levels are divided based on the number of consecutive correct answers, quantifying the performance and making the feedback evaluation criteria clear and objective. Then, the different actions of virtual avatars are used to visually display the levels of correct answer feedback, conveying the results to students in an intuitive, vivid, and engaging way. This effectively enhances the fun and interactivity of learning and can effectively stimulate the learner's interest.

[0008] In some embodiments, the answer information includes the respondent's facial information; based on the answer feedback strategy and the respondent's answer information at different times, different virtual avatar feedback content is visualized, including: Facial information is analyzed to determine the respondent's current emotional state. The second feedback content is displayed visually; the second feedback content refers to the virtual avatar feedback content that matches the current emotion type and answering feedback strategy.

[0009] In the above embodiments, by combining facial information with real-time emotion recognition, the psychological state of the respondent can be accurately identified. Then, based on the feedback from the virtual avatar corresponding to the emotion, negative emotions can be specifically soothed, a positive mindset boosted, and the pressure of answering questions alleviated, making the interaction more human-centered. Simultaneously, by adjusting the guidance method according to the emotional state, it can effectively help maintain a stable learning state.

[0010] In some embodiments, the answer information further includes the answer result; the second feedback content is visualized, including: Based on the answers, determine the standard feedback format for the virtual avatar; The target feedback format is obtained by adjusting the standard feedback format based on the current emotion type. The adjusted second feedback content is displayed visually; the adjusted second feedback content refers to the virtual image feedback content that matches the target feedback format and answer feedback strategy.

[0011] In the above embodiments, basic feedback is set based on the answer results to make the evaluation fit the actual answer situation; then, the feedback form is fine-tuned according to the real-time emotional dynamics to enhance the interactivity.

[0012] In some embodiments, after visually displaying the adjusted second feedback content, the method further includes: When the current emotion type is negative and the answer is incorrect, a pairing protection operation is performed; the pairing protection operation is used to indicate that the number of consecutive correct answers for the respondent is not reset to zero.

[0013] In the above embodiments, retaining consecutive correct answers when the user answers incorrectly and experiences a negative emotional state can effectively prevent damage to their confidence in answering questions and also effectively protect their enthusiasm for answering questions, encouraging them to continue trying. Therefore, this embodiment can take into account both the actual answering situation and the user's psychological state, making the feedback mechanism more humane and helping to maintain a stable answering state.

[0014] In some embodiments, after visually displaying different virtual avatar feedback content, the method further includes: The number of times a participant achieved a higher correct answer level than the set level within the target time period; In response to reaching a set number of times, the target image of the virtual avatar is unlocked and displayed visually.

[0015] In the above embodiments, after a certain number of advanced question-answering levels are achieved, an exclusive image can be unlocked, thus establishing an advanced incentive system. This not only enhances the sense of accomplishment in learning but also effectively motivates students to maintain a high consecutive answering record.

[0016] In some embodiments, the method further includes: Update the virtual avatar action library set on electronic devices at set time intervals; And / or, Determine the current time type; time types include holiday types; In response to the time type being the aforementioned holiday type, the virtual avatar action library is updated based on the holiday type.

[0017] In the above embodiments, by regularly updating the action library, the interactive screen can be effectively kept from becoming monotonous and boring, thus avoiding aesthetic fatigue, and the feedback expression can be enriched, thereby maintaining the freshness of answering questions.

[0018] Secondly, embodiments of this application provide a human-computer interaction device, including: The stage determination unit is used to determine the learner's learning stage in response to the question-answering initiation operation, through the camera device of the electronic device; The strategy determination unit is used to determine the appropriate response feedback strategy for the respondent based on the learning stage. The visualization display unit is used to visualize different virtual image feedback content during the question-answering process, based on the question-answering feedback strategy and the question-answering information of the question-answerer at different times; the question-answering information includes at least one of the following: question-answering result, number of correct answers, and facial information of the question-answerer.

[0019] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the human-computer interaction method as described in any one of the first aspects above.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the human-computer interaction method as described in any one of the first aspects above.

[0021] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, enables the electronic device to execute the human-computer interaction method described in any of the first aspects above. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the implementation of a human-computer interaction method provided in an embodiment of this application; Figure 2 This is a flowchart illustrating the specific implementation of step S103 in a human-computer interaction method provided in an embodiment of this application; Figure 3 and Figure 3 (continued) This is a schematic diagram of the virtual image feedback content corresponding to different action information provided in an embodiment of this application; Figure 4 This is a flowchart illustrating the specific implementation of step S103 in a human-computer interaction method provided in another embodiment of this application; Figure 5 This is a schematic diagram of the virtual image feedback content corresponding to different emotion types provided in an embodiment of this application; Figure 6 This is a schematic diagram of the virtual image feedback content corresponding to different emotion types and different answer results provided in an embodiment of this application; Figure 7 This is a flowchart illustrating the implementation of a human-computer interaction method provided in another embodiment of this application; Figure 8 This is a schematic diagram of a virtual image provided in one embodiment of this application; Figure 9 This is a schematic diagram of the structure of a human-computer interaction device provided in an embodiment of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0024] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0025] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0028] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0029] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0030] In practical applications, with the rapid development of computer technology, users can learn through the internet in electronic environments comprised of communication technology, computer technology, artificial intelligence, and multimedia technology (such as online education quiz systems). Taking online education quiz systems as an example, the current feedback methods of online education quiz systems mainly include: text symbols (…). / The issues with sound effects, points rewards, and static IP character expression changes are as follows: The feedback is too simplistic: no matter how many questions are answered correctly, the feedback format is the same, lacking a sense of hierarchy and cumulative achievement. The virtual avatars are stiff and unnatural: they only make a binary distinction between "happy / sad" and the range of their movements is not linked to the quality of their answers. Ignoring the differences among respondents: Students of different ages and emotional states receive the same feedback, making younger students feel it is bland and older students feel it is childish; Unable to perceive student status: The system does not know who is answering questions, whether students are focused, or what their emotions are like, and the feedback is like "blind men touching an elephant".

[0031] Therefore, this application provides a human-computer interaction method that can solve the problems existing in the prior art.

[0032] Please see Figure 1 , Figure 1 This is a flowchart illustrating the implementation of a human-computer interaction method according to an embodiment of this application. In this embodiment, the executing entity of the human-computer interaction method is an electronic device. The electronic device may include a device deployed with an online education platform or learning application, such as a learning machine or learning platform.

[0033] It should be noted that in all the accompanying drawings provided in the embodiments of this application, the representation of the virtual images in the drawings is merely an example and not a limitation.

[0034] like Figure 1 As shown, a human-computer interaction method provided in one embodiment of this application may include S101~S103, which are described in detail below: In S101, in response to the question-answering initiation operation, the learning stage of the question-answerer is determined by the camera device of the electronic device.

[0035] It should be noted that the respondents specifically refer to users of electronic devices.

[0036] In one implementation of this application, the respondent can trigger the question-answering start operation by clicking the question-answering start button on the display interface of the electronic device or by initiating the question-answering operation through voice.

[0037] In this embodiment of the application, after the electronic device detects the above-mentioned question-answering initiation operation, it can determine that a question-answering initiation command has been detected. Therefore, the electronic device can formally enter the question-answering interaction process at this time.

[0038] Afterwards, the electronic device can access its built-in camera (such as a front-facing or rear-facing camera) to capture the face image of the person answering the question in real time.

[0039] Then, the electronic device can extract facial features from the aforementioned face image and determine the learner's learning stage based on the pre-stored correspondence between different facial features and different learning stages.

[0040] The learning stages include, but are not limited to: lower elementary grades (such as first, second and third grade), upper elementary grades (such as fourth, fifth and sixth grade), junior high school, and senior high school.

[0041] In S102, based on the learning stage, a response feedback strategy corresponding to the respondent is determined.

[0042] In this embodiment of the application, after determining the learning stage of the test taker, the electronic device can retrieve the feedback methods of different feedback dimensions corresponding to each learning stage that it has pre-stored, determine the feedback methods corresponding to the learning stage of the test taker in different feedback dimensions, and generate a test answering feedback strategy based on the feedback methods of different feedback dimensions.

[0043] The feedback dimensions include, but are not limited to: the virtual character's image style, action characteristics, language style, visual effects, and emotional tone.

[0044] In S103, during the question-answering process, based on the question-answering feedback strategy and the question-answering information of the question-answerer at different times, different virtual image feedback content is visualized and displayed; the question-answering information includes at least one of the following: question-answering result, number of correct answers, and facial information of the question-answerer.

[0045] In this embodiment of the application, during the question-answering process, the electronic device can continuously acquire at least one of the following question-answering information: the question-answering result, the number of correct answers, and the facial information of the question-answerer, as a basis for feedback of the virtual avatar.

[0046] The results of the questions include, but are not limited to, correct answer, incorrect answer, and no answer within the time limit.

[0047] Then, the electronic device can visualize different virtual avatar feedback content based on the answer feedback strategy and the answer information of the answerer at different times.

[0048] In some possible embodiments, the feedback content of the virtual avatar includes, but is not limited to, the virtual avatar's actions, expressions, special effects, and text content.

[0049] In this embodiment, when the electronic device detects that the respondent has answered a question correctly, it can control the virtual avatar to cheer, jump, give a thumbs up, and display reward effects.

[0050] When electronic devices detect that a user has answered a question incorrectly or is unable to answer, they can control a virtual avatar to provide gentle prompts or guide the user through a review.

[0051] When electronic devices detect that a test taker is in a bad mood, they can control a virtual avatar to comfort and encourage them, thus easing their anxiety while answering the questions.

[0052] In one embodiment of this application, when the answer information includes the number of consecutive correct answers, the electronic device can specifically achieve this through methods such as... Figure 2 Steps S201 to S203 shown implement step S103, as detailed below: In S201, the correct answer level of the respondent is determined based on the number of consecutive correct answers.

[0053] In this embodiment, when the answer information includes the number of consecutive correct answers, the electronic device can count the number of consecutive correct answers in real time after each correct answer by the answerer, and determine the correct answer level of the answerer based on the number of consecutive correct answers at this time.

[0054] It should be noted that the more consecutive correct answers a question receives, the higher the corresponding correct answer level.

[0055] It should be understood that when a respondent answers a question incorrectly, their consecutive correct answers are reset to zero, and the count restarts when they answer the next question correctly.

[0056] In this embodiment, when the electronic device detects that the number of consecutive correct answers by the respondent is less than or equal to a first number, the electronic device can determine that the respondent's correct answer level is the first level; when the electronic device detects that the number of consecutive correct answers by the respondent is greater than the first number and less than or equal to a second number, the electronic device can determine that the respondent's correct answer level is the second level; when the electronic device detects that the number of consecutive correct answers by the respondent is greater than the second number, the electronic device can determine that the respondent's correct answer level is the third level.

[0057] The first level is lower than the second level, and the second level is lower than the third level.

[0058] The first quantity can be 2, and the second quantity can be 4.

[0059] In S202, the action information that matches the correct answer level is determined from the existing virtual avatar action library; the action information includes action type, action range, and action effects.

[0060] In this embodiment, after determining the correct answer level for the respondent, the electronic device can identify the matching action information from the pre-built virtual avatar action library. This action information includes the action type, action amplitude, and action effects.

[0061] It should be noted that the higher the correct answer level, the greater the range of motion.

[0062] Specifically, when the electronic device detects that the correct answer level of the respondent is the first level, it can determine that the corresponding action information is a small-amplitude limb movement; when the electronic device detects that the correct answer level of the respondent is the second level, it can determine that the corresponding action information is a medium-amplitude limb movement; and when the electronic device detects that the correct answer level of the respondent is the third level, it can determine that the corresponding action information is a large-amplitude limb movement.

[0063] Among them, small-amplitude body movements refer to encouraging elements placed in a small area around the virtual avatar's body, expressing basic affirmation and encouragement (such as "Great!").

[0064] Medium-amplitude body movements specifically refer to the expanded range of motion of the virtual character, which includes elements of jumping or waving, and expresses obvious happiness and approval (such as "Amazing!").

[0065] Large-scale physical movements specifically refer to virtual characters whose movements can cover the entire screen and include elements such as jumping, spinning, and special effects, while expressing strong surprise and admiration (such as "So cool!").

[0066] In S203, the first feedback content is displayed visually; the first feedback content refers to the virtual image feedback content that matches the action information and the answer feedback strategy.

[0067] In this embodiment, after the electronic device determines the virtual image feedback content that matches the above-mentioned action information and answer feedback strategy, namely the first feedback content, it can visually display the first feedback content on the current answer interface.

[0068] For example, please refer to Figure 3 and Figure 3 (continued) , Figure 3 and Figure 3 (continued) This is a schematic diagram illustrating the virtual avatar feedback content corresponding to different action information provided in an embodiment of this application. For example... Figure 3 As shown in (a), after the respondent completes the questions on the answer interface that require judging the correctness of the picture, when the electronic device detects that the respondent has 1 consecutive correct answers, since the virtual image's motion information is a small-amplitude limb movement at this time, the electronic device displays the virtual image's head in the lower left corner of the answer interface, outputs the phrase "Great!" around its head, and displays five-pointed star images of different sizes.

[0069] like Figure 3 As shown in (b), after the respondent completes the picture-based questions displayed on the answer interface and judges whether the answers are correct or not, when the electronic device detects that the respondent has 3 consecutive correct answers, since the virtual image's motion information is a medium-amplitude limb movement at this time, the electronic device displays the upper body of the virtual image in the lower left corner of the answer interface, outputs the statements "Amazing!" and "3 consecutive correct answers!" on the right side of the upper body, and displays petal images of different sizes.

[0070] like Figure 3 (continued) As shown in (c), after the respondent completes the questions displayed on the answer interface, which require judging the correctness of the picture, the electronic device detects that the respondent has 5 consecutive correct answers. Since the virtual image's action information is a large-scale limb movement at this time, the electronic device displays the virtual image's entire body rushing upwards in the entire area of ​​the answer interface, and outputs the statements "So cool!" and "5 consecutive correct answers!" around its body.

[0071] Combining steps S201-S203, this embodiment categorizes answer levels based on the number of consecutive correct answers, quantifying performance and making the feedback evaluation criteria clear and objective. Subsequently, different actions of virtual avatars are used to visually display the levels of correct answer feedback, conveying the results to students in an intuitive, vivid, and engaging way. This effectively enhances the fun and interactivity of learning and can effectively stimulate students' learning interest.

[0072] In another embodiment of this application, when the answer information includes the respondent's facial information, the electronic device can specifically utilize methods such as... Figure 4 Steps S301 to S302 shown implement step S103, as detailed below: In S301, facial information is analyzed to determine the respondent's current emotional type.

[0073] In S302, the second feedback content is visualized; the second feedback content refers to the virtual avatar feedback content that matches the current emotion type and answer feedback strategy.

[0074] In this embodiment, the electronic device can use its front-facing camera to collect the facial information of the respondent in real time during each question, and input the facial information into a trained emotion recognition model for analysis to determine the respondent's current emotion type.

[0075] Among them, facial information can be a human face image.

[0076] Emotion types include, but are not limited to: positive emotions (such as happiness), negative emotions (such as frustration / depression), confusion, fatigue, etc.

[0077] It should be noted that the emotion recognition model can be obtained by training a pre-built deep learning model based on a preset sample set. Each sample in the preset sample set includes facial information and the corresponding emotion type. When training the pre-built deep learning model, the facial information from each sample is used as the input, and the corresponding emotion type is used as the output. Through training, the deep learning model can learn the correspondence between all possible facial information and emotion types, and the trained deep learning model becomes the emotion recognition model.

[0078] In this embodiment, after determining the respondent's current emotion type, the electronic device can, based on a defined response feedback strategy and in conjunction with emotion matching rules corresponding to that current emotion type, retrieve virtual avatar materials, gestures, textual descriptions, or dynamic effects that match the current emotion type to generate virtual avatar feedback content—the second feedback content—that matches the current emotion type and the response feedback strategy. The electronic device can then visually display this second feedback content on the current response interface.

[0079] For example, please refer to Figure 5 , Figure 5 This is a schematic diagram illustrating the virtual avatar feedback content corresponding to different emotion types provided in an embodiment of this application. For example... Figure 5 As shown in (a), when the respondent is answering a word completion question displayed on the answer interface, if the electronic device detects that the respondent's current emotion type is confused, the electronic device can display a virtual avatar in the lower left corner of the answer interface and output the help statement "Hey! Need a hint?" around it.

[0080] like Figure 5 As shown in (b), during the question-and-answer process, when the electronic device detects that the current emotional type of the respondent is fatigue, the electronic device can directly display a virtual image of the respondent lying down on the entire question-and-answer interface, and display a rest countdown in real time below it, such as 00:04:21.

[0081] Combining steps S301-S302, this embodiment uses facial information to recognize real-time emotions, accurately identifying the respondent's psychological state. Then, based on the feedback from the corresponding virtual avatar matched to the emotion, it can specifically soothe negative emotions, boost a positive mindset, and alleviate the pressure of answering questions, making the interaction more human-centered. Simultaneously, adjusting the guidance method according to emotional state can effectively help maintain a stable learning state.

[0082] In another embodiment of this application, when the answer information also includes the answer result, the electronic device can specifically implement step S302 according to the following steps, detailed below: Based on the answers, determine the standard feedback format for the virtual avatar; The target feedback format is obtained by adjusting the standard feedback format based on the current emotion type. The adjusted second feedback content is displayed visually; the adjusted second feedback content refers to the virtual image feedback content that matches the target feedback format and answer feedback strategy.

[0083] In this embodiment, after determining the answer result of the question, the electronic device can determine the basic actions, expressions, text, etc. of the corresponding virtual character based on the answer result, and obtain the standard feedback form of the virtual character.

[0084] Afterwards, the electronic device can continue to adapt and adjust the standard feedback format based on the current emotion type corresponding to the respondent's answer, thus obtaining the target feedback format.

[0085] For example, when a user answers a question incorrectly and feels down, the serious prompts from the virtual avatar can be toned down; when a user answers a question correctly and feels happy, the animation of the virtual avatar can be extended for a set time and additional special effects can be added. The set time can be determined according to actual needs and is not limited here. For example, the set time could be 0.5 seconds.

[0086] In this embodiment, after the electronic device determines the virtual avatar feedback content that matches the target feedback format and answer feedback strategy, i.e., the adjusted second feedback content, it can visually display the adjusted second feedback content on the answer interface. Figure 6 This is a schematic diagram illustrating the virtual avatar feedback content corresponding to different emotion types and different answer results provided in an embodiment of this application. For example... Figure 6 As shown in (a), after the respondent completes the multiple-choice questions displayed on the answer interface, when the electronic device detects that the respondent has answered correctly and the current emotion type is happy, the electronic device can display a virtual image in the middle of the answer interface, extend the animation duration of the virtual image, and add colored bar effects around the virtual image.

[0087] like Figure 6 As shown in (b), after the respondent completes a question with incomplete words displayed on the answer interface, when the electronic device detects that the respondent has answered incorrectly and the current mood type is low, the electronic device can display a virtual image in the lower left corner of the answer interface and output the comforting statement "Let's review the wrong questions again!"

[0088] In the above embodiments, basic feedback is set based on the answer results to make the evaluation fit the actual answer situation; then, the feedback form is fine-tuned according to the real-time emotional dynamics to enhance the interactivity.

[0089] In another embodiment of this application, after the electronic device displays the adjusted second feedback content visually, it can further perform a pair-match protection operation when it detects that the current emotion type is negative and the answer result is incorrect. The pair-match protection operation represents the operation of not resetting the number of consecutive correct answers for the respondent to zero.

[0090] In this embodiment, retaining consecutive correct answers when the user answers incorrectly and experiences a negative emotional state effectively prevents damage to their confidence and protects their enthusiasm for answering questions, encouraging them to continue trying. Therefore, this embodiment balances the actual answering experience with the user's psychological state, making the feedback mechanism more humane and helping to maintain a stable answering state.

[0091] As can be seen from the above, the human-computer interaction method provided in this application, in response to a question-answering initiation operation, determines the learner's learning stage through the camera device of an electronic device; based on the learning stage, it determines a corresponding question-answering feedback strategy; during the question-answering process, based on the question-answering feedback strategy and the answerer's answering information at different times, it visualizes different virtual image feedback content; the answering information includes at least one of the following: answering result, number of consecutive correct answers, and facial information of the answerer. This application determines the learner's learning stage through camera recognition and then matches it with a corresponding feedback strategy, making the feedback more aligned with the learner's cognitive level and learning needs at each stage, thus enhancing the effectiveness and relevance of the feedback. Simultaneously, this application also combines multi-dimensional answering information such as correct / incorrect answers, number of consecutive correct answers, and facial expressions to dynamically switch virtual image feedback, not limited to a single correct / incorrect prompt. It can provide different encouragement based on the number of consecutive correct answers and different feedback based on different emotions, making the feedback more humanized and more aligned with the real-time learning state, enhancing the fun and interactivity of learning.

[0092] Please see Figure 7 , Figure 7 This is a flowchart illustrating the implementation of a human-computer interaction method according to another embodiment of this application. Compared to... Figure 1 In a corresponding embodiment, this embodiment may further include S401~S402 after S103, as detailed below: In S401, the number of times a respondent achieves a higher correct answer level than the set level within the target time period is counted.

[0093] It should be noted that the target time period can be determined according to actual needs, and there are no restrictions here.

[0094] In this embodiment, the electronic device can record the answer results of the respondent in the target time period one question at a time, so as to obtain the number of consecutive correct answers of the respondent in the target time period and determine the correct answer level corresponding to the number of consecutive correct answers each time.

[0095] Afterwards, the electronic device can accumulate and count the number of times the respondent reaches a set level. The set level can be determined according to actual needs and is not limited here. For example, the set level could be the third level.

[0096] In S402, in response to reaching a set number of times, the target image of the virtual avatar is unlocked and visualized.

[0097] In this embodiment, after the electronic device detects that the number of times the achievement has reached a set number, it can unlock and visually display the target image of the virtual image associated with the achievement corresponding to the number of times the achievement has been reached (such as the image of "Winning Streak God").

[0098] As can be seen from the above, the human-computer interaction method provided in this embodiment can unlock an exclusive image after a certain number of times the advanced question-answering level is achieved, thus building an advanced incentive system. This not only enhances the sense of accomplishment in learning, but also effectively motivates students to maintain a high consecutive answering record.

[0099] In one embodiment of this application, the electronic device can also set the image of the "God of Wrong Questions" for the virtual avatar, and can unlock the image of the "God of Wrong Questions" after detecting that the respondent has completed the re-practice of the questions he answered incorrectly in the historical time period and answered them correctly.

[0100] For example, please refer to Figure 8 , Figure 8 This is a schematic diagram of a virtual image provided in one embodiment of this application. For example... Figure 8 As shown, the electronic device can visualize the image of the "God of Mistakes" on the interface of incorrect questions, and the image of the God of Mistakes carries the special effects of ribbons and glowing five-pointed stars.

[0101] In another embodiment of this application, the electronic device may further perform the following steps, detailed below: The virtual avatar action library of electronic devices is updated at set time intervals.

[0102] In this embodiment, to avoid aesthetic fatigue among test takers regarding the virtual avatar's appearance, the electronic device can periodically update its virtual avatar action library at set time intervals, such as adding new actions and / or voices. The set time interval can be determined according to actual needs and is not limited here.

[0103] In another embodiment of this application, the electronic device may further perform the following steps, detailed below: Determine the current time type; time types include holiday types; In response to the time type being the aforementioned holiday type, the virtual avatar action library is updated based on the holiday type.

[0104] In this embodiment, the electronic device can determine the time type of the current time, and when it detects that the time type of the current time is a festival type, it can update the virtual character action library based on the festival type, that is, add virtual character-specific actions and special effects during the festival, such as the Spring Festival greeting action and the Mid-Autumn Festival moon-viewing action.

[0105] It should be noted that the above-mentioned holiday types include, but are not limited to, statutory holidays and marketing days defined by merchants (such as anniversaries).

[0106] As can be seen from the above, the human-computer interaction method provided in this embodiment can effectively avoid monotonous and boring interactive screens by regularly updating the action library, thereby avoiding aesthetic fatigue and enriching the feedback expression forms, thus maintaining the freshness of answering questions.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0108] Corresponding to the human-computer interaction method described in the above embodiments, Figure 9 A schematic diagram of a human-computer interaction device according to an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown. (Refer to...) Figure 9 The human-computer interaction device 500 includes: a stage determination unit 51, a strategy determination unit 52, and a visualization display unit 53. Wherein: The stage determination unit 51 is used to determine the learning stage of the respondent in response to the question-answering initiation operation by using the camera device of the electronic device.

[0109] The strategy determination unit 52 is used to determine the corresponding response feedback strategy for the respondent based on the learning stage.

[0110] The visualization display unit 53 is used to visualize different virtual image feedback content during the question-answering process, based on the question-answering feedback strategy and the question-answering information of the question-answerer at different times; the question-answering information includes at least one of the following: question-answering result, number of correct answers, and facial information of the question-answerer.

[0111] In one embodiment of this application, the answer information includes the number of consecutive correct answers; the visualization display unit 53 specifically includes: a gear position determination unit, an action determination unit, and a first display unit. Wherein: The grade determination unit is used to determine the correct grade of a candidate's answer based on the number of consecutive correct answers.

[0112] The action determination unit is used to determine the action information that matches the correct answer level from the built virtual character action library; the action information includes action type, action range, and action effects.

[0113] The first display unit is used to visually display the first feedback content; the first feedback content refers to the virtual image feedback content that matches the action information and the answer feedback strategy.

[0114] In one embodiment of this application, the answer information includes the respondent's facial information; the visualization display unit 53 specifically includes: an analysis unit and a second display unit. Wherein: The analysis unit is used to analyze facial information to determine the respondent's current emotional state.

[0115] The second display unit is used to visually display the second feedback content; the second feedback content refers to the virtual image feedback content that matches the current emotion type and answering feedback strategy.

[0116] In one embodiment of this application, the answer information further includes the answer result; the second display unit specifically includes: a format determination unit, an adjustment unit, and a third display unit. Wherein: The format determination unit is used to determine the standard feedback format of the virtual avatar based on the answer results.

[0117] The adjustment unit is used to adjust the standard feedback format based on the current emotion type to obtain the target feedback format.

[0118] The third display unit is used to visually display the adjusted second feedback content; the adjusted second feedback content refers to the virtual image feedback content that matches the answer feedback strategy described in the target feedback format.

[0119] In one embodiment of this application, the human-computer interaction device 500 further includes an operation execution unit.

[0120] The operation execution unit is used to perform a pair protection operation when the current emotion type is negative and the answer result is incorrect; the pair protection operation is used to indicate that the number of consecutive correct answers of the respondent is not cleared.

[0121] In one embodiment of this application, the human-computer interaction device 500 further includes: a statistics unit and an unlocking unit. Wherein: The statistics unit is used to count the number of times a respondent achieves a correct answer level higher than the set level within the target time period.

[0122] The unlocking unit is used to unlock and visually display the target image of the virtual avatar in response to reaching a set number of attempts.

[0123] In one embodiment of this application, the human-computer interaction device 500 further includes: a first updating unit and / or a type determining unit and a second updating unit. Wherein: The first update unit is used to update the virtual avatar action library of electronic devices based on a set time interval.

[0124] The type determination unit is used to determine the time type of the current time; the time type includes the holiday type.

[0125] The second update unit is used to update the virtual character action library based on the festival type in response to the time type being the festival type.

[0126] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0127] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0128] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device 6 of this embodiment includes: at least one processor 60 ( Figure 10 (Only one is shown in the diagram), memory 61, and computer program 62 stored in said memory 61 and executable on said at least one processor 60, wherein the processor 60 executes said computer program 62 to implement the steps in any of the above-described human-computer interaction method embodiments.

[0129] The electronic device 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0130] The processor 60 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0131] In some embodiments, the memory 61 may be an internal storage unit of the electronic device 6, such as the RAM of the electronic device 6. In other embodiments, the memory 61 may be an external storage device of the electronic device 6, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the electronic device 6. Furthermore, the memory 61 may include both internal and external storage units of the electronic device 6. The memory 61 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0132] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0133] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0134] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to an electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0135] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0136] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A human-machine interaction method, characterized in that, include: In response to the start of the question-answering process, the learning stage of the respondent is determined by the camera device of the electronic device; Based on the learning stage, determine the corresponding response feedback strategy for the respondent; During the question-answering process, based on the question-answering feedback strategy and the question-answering information of the respondent at different times, different virtual image feedback content is visualized and displayed; the question-answering information includes at least one of the following: question answering result, number of correct answers, and facial information of the respondent.

2. The human-machine interaction method of claim 1, wherein, The answer information includes the number of consecutive correct answers; the visualization of different virtual avatar feedback content based on the answer feedback strategy and the answerer's answer information at different times includes: Based on the number of consecutive correct answers, the correct answer level of the respondent is determined; From the existing virtual avatar action library, determine the action information that matches the correct answer level; the action information includes action type, action range, and action effects; The first feedback content is displayed visually; the first feedback content refers to the virtual image feedback content that matches the action information and the answer feedback strategy.

3. The human-machine interaction method of claim 1, wherein, The answer information includes the respondent's facial information; the visualization of different virtual avatar feedback content based on the answer feedback strategy and the respondent's answer information at different times includes: The facial information is analyzed to determine the respondent's current emotion type; The second feedback content is displayed visually; the second feedback content refers to the virtual image feedback content that matches the current emotion type and the answer feedback strategy.

4. The human-machine interaction method of claim 3, wherein, The answer information also includes the answer results; the visualization of the second feedback content includes: Based on the answers, determine the standard feedback format for the virtual avatar; The standard feedback format is adjusted based on the current emotion type to obtain the target feedback format; The adjusted second feedback content is displayed visually; the adjusted second feedback content refers to the virtual image feedback content that matches the target feedback format and the answer feedback strategy.

5. The human-machine interaction method of claim 4, wherein, Following the second feedback content after the visualization adjustment, the following is also included: When the current emotion type is negative and the answer result is incorrect, a pairing protection operation is performed; the pairing protection operation is used to indicate that the number of consecutive correct answers of the respondent is not reset to zero.

6. The human-machine interaction method of claim 1, wherein, After visually displaying different virtual avatar feedback content, the method further includes: The number of times the respondents achieved a higher correct answer level than the set level within the target time period is counted. In response to the achievement of a set number of times, the target image of the virtual avatar is unlocked and visualized.

7. The human-machine interaction method according to any one of claims 1 to 6, characterized in that, The method further includes: The virtual avatar action library set by the electronic device is updated at set time intervals; And / or, Determine the time type of the current time; the time type includes holiday types; In response to the time type being the festival type, the virtual avatar action library is updated based on the festival type.

8. A human-machine interaction device, characterized in that, include: The stage determination unit is used to determine the learner's learning stage in response to the question-answering initiation operation, through the camera device of the electronic device; The strategy determination unit is used to determine the answer feedback strategy corresponding to the respondent based on the learning stage. A visualization display unit is used to visualize different virtual image feedback content during the question-answering process, based on the question-answering feedback strategy and the question-answering information of the respondent at different times; the question-answering information includes at least one of the following: question-answering result, number of correct answers, and facial information of the respondent.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the human-computer interaction method as described in any one of claims 1 to 7.

10. A computer program product, characterised in that, It includes a computer program that, when run, implements the human-computer interaction method as described in any one of claims 1 to 7.

Images