Autistic child emotional learning auxiliary method and system based on generative artificial intelligence

Abstract

The invention discloses an autistic child emotional learning assistance method based on generative artificial intelligence. The method comprises the steps of obtaining user information; setting an interaction initial parameter according to the to-be-learned emotion of the user; and according to the user information, constructing an interaction model under a large language model framework, wherein the interaction model comprises a context generation module, an interaction module, an expression extension module, a feedback generation module and an evaluation module. The invention further provides an autistic child emotion learning auxiliary system. According to the method provided by the invention, more personalized and diversified auxiliary learning contents can be provided, so that the learning enthusiasm of the user is improved, and the emotional learning effect is further improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer-assisted learning, and in particular relates to a generative artificial intelligence-based emotional learning assistance method and system for children with autism. Background Art

[0002] Autism spectrum disorder (ASD) is a widespread neurodevelopmental disorder characterized by deficits in social interaction and communication, as well as repetitive, stereotyped behaviors. Among them, patients with high-functioning autism (HFA) have normal intelligence, but due to their impairments in emotion recognition and expression, they have difficulty establishing satisfactory social relationships. This challenge is particularly daunting for children with high-functioning autism attending mainstream schools. They are often considered indifferent to others because of their lack of emotional expression and inability to recognize the emotions of others through facial expressions. In order to support the social communication of these children, it is crucial to understand their emotional challenges and provide them with appropriate help. Currently, various social and emotional interventions for children with high-functioning autism, including social skills training (SST), cognitive behavioral therapy (CBT), and social story intervention, usually rely on local human therapists, resulting in a large demand for human resources and low accessibility.

[0003] Patent document CN117137485A discloses an emotion recognition and expression evaluation robot and evaluation method, which includes a guidance and teaching stage: an expression-based emotion recognition stage, a situation-based emotion expression recognition stage; and an evaluation stage: a situation emotion recognition stage, an emotion expression imitation stage, and a self-emotion recognition stage. The device has an interactive and immersive experience, which is helpful for children's intervention and evaluation interaction.

[0004] Patent document CN112331305A discloses a computer-assisted emotional and social rehabilitation training system for children with autism, including an identity recognition training module, which is used to train children with autism based on set identity recognition training topics and collect identity recognition feedback information, and determine the training content, training difficulty and training frequency of the sight training module; a sight training module, which is used to train children with autism based on set sight training topics and collect sight feedback information, and determine the training content, difficulty and frequency of the expression training module; an expression training module, which is used to train children with autism based on set expression training topics and collect expression feedback information, and determine the training content, difficulty and frequency of the social scene emotion training module; a social scene emotion training module, which is used to train children with autism based on set social scene emotion training topics and collect feedback information, and determine the comprehensive training content and training frequency.

[0005] However, the emotion training system in the above invention only provides fixed emotion training content and feedback based on predefined rules, while ignoring the individual differences between different autistic children, which makes it impossible to conduct targeted emotion training according to children's preferences and needs. In addition, the training content they provide lacks diversity, which is not conducive to the development of transfer and generalization skills of autistic children, making it difficult for autistic children to apply the emotional skills learned in this system to other situations. The above two limitations reduce the effectiveness of the current invention in supporting emotional learning of high-functioning autistic children. Summary of the invention

[0006] The purpose of the present invention is to provide an emotional learning assistance method and device for autistic children based on generative artificial intelligence. The method can provide more personalized and diverse auxiliary learning content to enhance the user's learning enthusiasm and further improve the emotional learning effect.

[0007] In order to achieve the first objective of the present invention, the following technical solution is provided: a method for assisting emotional learning of children with autism based on generative artificial intelligence, comprising:

[0008] Obtain user information, including user personal information and preferences;

[0009] Set initial interaction parameters according to the user's emotions to be learned, including occurrence scene parameters and event materials;

[0010] Building an interaction model under the framework of a large language model according to the user information, wherein the interaction model includes a context generation module, an interaction module, an expression expansion module, a feedback generation module, and an evaluation module;

[0011] The context generation module obtains keywords in the initial interaction parameters according to a preset first prompt word template to generate a first-person interaction text;

[0012] The interaction module performs audio and text output according to the generated first-person interaction text, generates corresponding emotion recognition options, and collects user interaction text input by the user;

[0013] The expression expansion module extracts keywords from the user interaction text according to a preset second prompt word template to generate an emotional prompt text;

[0014] The feedback generation module outputs positive feedback description text or negative feedback description text according to the selection result of the emotion recognition option and the emotion prompt text;

[0015] The evaluation result evaluates and records the user's emotional learning situation based on multiple rounds of feedback description texts generated by the feedback generation module.

[0016] The present invention uses the actual experience of users as material to construct corresponding interactive texts, and uses pre-constructed prompt word templates to quickly obtain keywords about emotional content in various texts, thereby further generating corresponding interactive texts and evaluation results.

[0017] Specifically, the preference settings include favorite food and activities, and expression style.

[0018] Specifically, the event material is based on the user's recent experience and the emotional information involved.

[0019] Specifically, the context generation module obtains keywords in the initial interaction parameters through the first prompt word template, and uses a sentence structure that meets the language standard to create a personalized first-person interaction text.

[0020] Specifically, there are multiple categories of the first-person interactive texts, and each first-person interactive text contains a unique emotion.

[0021] Specifically, the expression expansion module extracts keywords from the user interaction text through the second prompt word template, and expands and enriches the user interaction text according to the dialogue context to obtain the emotion prompt text.

[0022] Specifically, the feedback generation module is based on a pre-built analysis prompt word template, and the analysis prompt word template contains "predefined results" to obtain keywords in the emotional prompt text to generate feedback description text. When the emotional recognition option is selected incorrectly, negative feedback description text is output, and when the emotional recognition option is selected correctly, positive feedback description text is output. The predefined results are preset emotional types.

[0023] Specifically, the positive feedback description text includes praise text when the selected emotion is correct;

[0024] The negative feedback explanation text includes comforting text when the emotion selection is wrong, and explanation text about the correct emotion selection.

[0025] In order to achieve the second purpose of the present invention, the following scheme is provided: an emotion learning assistance system for children with autism, which is implemented by the above-mentioned emotion learning assistance method for children with autism based on generative artificial intelligence, and its specific execution process is as follows:

[0026] The initial interaction parameters are input into the interaction model, and the user is allowed to interact with the interaction model to obtain the final emotional learning evaluation.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] Construct corresponding emotional interaction texts based on the user's own historical events to enhance the user's sense of involvement, which is conducive to the development of transfer and generalization skills of autistic children. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart of a generative artificial intelligence-based method for assisting emotional learning for children with autism provided in this embodiment;

[0030] Figure 2 The setting process of the emotional learning auxiliary system for autistic children provided in this embodiment;

[0031] Figure 3 A schematic diagram of the interaction process of the emotional learning assistance system for children with autism provided in this embodiment;

[0032] Figure 4 A schematic diagram of the feedback process of the emotion learning assistance system for children with autism provided in this embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0034] A generative artificial intelligence-based emotional learning assistance method for children with autism provided in this embodiment includes:

[0035] Obtain user information, including user personal information and preferences;

[0036] Set initial interaction parameters according to the user's emotions to be learned, including occurrence scene parameters and event materials;

[0037] Building an interaction model under the framework of a large language model according to the user information, wherein the interaction model includes a context generation module, an interaction module, an expression expansion module, a feedback generation module, and an evaluation module;

[0038] The context generation module obtains keywords in the initial interaction parameters according to a preset first prompt word template to generate a first-person interaction text;

[0039] The interaction module performs audio and text output according to the generated first-person interaction text, generates corresponding emotion recognition options, and collects user interaction text input by the user;

[0040] The expression expansion module extracts keywords from the user interaction text according to a preset second prompt word template to generate an emotional prompt text;

[0041] The feedback generation module outputs positive feedback description text or negative feedback description text according to the selection result of the emotion recognition option and the emotion prompt text;

[0042] The evaluation result evaluates and records the user's emotional learning situation based on multiple rounds of feedback description texts generated by the feedback generation module.

[0043] Furthermore, if Figure 1 As shown, the method provided in this embodiment is specifically divided into four steps: setting, dialogue, summary and evaluation.

[0044] During the setup phase, the system collects the child's personal information, including the child's name, age, preferred food or activities, personal expression style, emotions that need to be trained, dialogue scenarios that need to be trained, and the child's recent experiences. In addition, the system records the character customization parameters entered by the child, including texture, appearance, and color.

[0045] During the dialogue phase, a large language model is used to create a specific dialogue context based on the information collected during the setup phase.

[0046] Specifically, the child's interests, favorite foods or activities, personal expression style, recent experiences, emotions involved in the conversation, and the place where the conversation takes place are added to the prompt words and input into the large language model to generate the conversation plot. In the prompt words, the system lets the large language model play the protagonist of the conversation to generate content. In order to ensure that the generated conversation content is suitable for children, the system emphasizes the specific age of the child in the prompt words; in order to ensure that the generated content is easy for children to understand, the system emphasizes in the prompt words that "the story should be simple enough, and use more short sentences and straightforward vocabulary, without too many tasks and complex plots"; in order to further improve the quality of the generated content, the system also adds appropriate story examples to the prompt words. The generated content includes three rounds of dialogue, corresponding to the three different emotions in the settings, and a summary.

[0047] The generated content includes three rounds of dialogue, corresponding to the three different emotions in the settings, and a summary. After the user selects the story scene, the context generation module in the model will generate all the content at once, and use the symbol "###" to separate each part of the content. Each part of the content will be automatically identified and displayed to the user one by one.

[0048] In addition, in order to make the interface more attractive to child users, the Stablediffusion model is used to generate cartoon character images based on the character customization parameters input by the children, and to generate dialogue background images based on the dialogue context.

[0049] To ensure the effectiveness of the training, the conversations generated by the system follow the following structure: Each conversation begins with the character greeting the child and asking if they are willing to communicate. The model then interacts with the child in three rounds following Gottman's emotional coaching method, and each interaction includes narration, questioning, answering, expansion, and feedback.

[0050] Narrative: The cartoon character verbally shares his experience with the child, with emoticons reflecting his emotions. The structure of the narrative follows the principles and guidelines of social stories, and it includes two to three descriptive sentences, an opinion sentence, and an instruction sentence, which are also included in the prompt words input to the large language model.

[0051] Question and answer: After the narration, the character asks two questions to train children to recognize and respond to other people's emotions. The first question requires children to identify the character's emotions based on the dialogue content and the character's facial expressions. Children need to choose the corresponding emoticon to complete this task. The second question requires children to respond to the character's expression through voice input using social skills such as praise, inquiry, and comment.

[0052] Extension: Before submitting the answer, the expression extension module expands the child's brief, fragmented input into emotionally rich expressions based on the conversation context, teaching the child how to respond to others' emotions meaningfully and considerately. Specifically, the expression extension module adds the content of the current conversation and the child's words to the prompt words to allow the large language model to generate extended content, ensuring that the generated content is relevant to the conversation content. In addition, the system prompts the large language model to add more emotional expressions to the extended content.

[0053] Feedback: The feedback generation module simulates two types of emotional feedback based on the answers submitted by children to help children understand how their expressions affect others' emotions. For correct emotion recognition, the system provides a star and positive verbal feedback. For incorrect emotion recognition and irrelevant or offensive emotional expressions, the system provides negative verbal feedback and explains the expected response type.

[0054] In the summary phase, after three rounds of dialogue interaction, children are invited to select their own emojis and share their thoughts on the dialogue to guide them to identify their own emotions. The system will then expand on their responses and provide feedback to teach children how to express their emotions effectively. The personal emojis are generated by the Stablediffusion model in the system based on the child's photos, including a series of emojis of the child under different emotions.

[0055] During the evaluation phase, the evaluation result module records the child's performance in the conversation on a table and adjusts the training difficulty level based on this table to ensure the effectiveness of the training. When the child's emotion recognition accuracy reaches 100% for three consecutive days, the evaluation result module will automatically increase the difficulty of the child's emotion practice. Specifically, the system will provide more complex emotions for children to recognize, including gratitude, regret and shame.

[0056] In addition, the context generation module, expression expansion module, and feedback generation module, each of which consists of prompt words that describe its function. Each module receives input and embeds it into a prompt word template, which is then passed to the API of the large language model (LLM) to obtain the returned result. Each module consists of two parts of prompts: SYSTEM and USER, which serve as different role inputs of the LLM. The SYSTEM part outlines the general tasks of the LLM and assigns roles. The USER part provides details and content for each subtask. For example, in the context generator, the SYSTEM part defines the role of the LLM as a storyteller, while the USER part specifies the requirements of each story. The slot {} represents the content that can be replaced and is a real-time variable.

[0057] In order to better illustrate the content of the above prompt word template, this embodiment provides the following content for illustration.

[0058] The prompt word template for the context generation module uses a sentence structure that conforms to language standards to create a personalized dialogue situation. The part about SYSTEM is: Let's play a role-playing game. You are {ai_name}, you are an {ai_char}, and my name is {user_name}. You have to tell me a story that happened to you. Your story needs to guide me and inspire me. Here is some information about me: {user_info}.

[0059] The part about USER is: Please remember that you are {ai_name}, the protagonist of the story. Please use "I" instead of "{ai_name}". Use "you" instead of "{user_name}". You have to tell me a story that happened to you. In this story, your mood changes are "{mood1}->{mood2}->{mood3}". According to these three mood changes, the story is divided into three parts, namely "Part 1: {mood1}; Part 2: {mood2}; Part 3: {mood3}". The beginning of each of the three parts of the story is "###Part 1:", "###Part 2:", "###Part 3:". More specifically, the plot of each part should only contain the corresponding only one mood, do not contain other moods, and do not have any language outside the plot. The story needs to be simple enough, use simple short sentences, and the language similar to children's picture books is simple and easy to understand. Remember: do not have too many characters, do not have complicated plots, and do not have too many twists and turns.

[0060] Finally, start with "###Part 4", summarize the story, explain the truth or life experience or way of getting along with others, what can I learn from it? What can I do in the future when I encounter a situation?

[0061] The prompt word template for the expression extension module expands and enriches the brief expressions of autistic children according to the context of the conversation. The part about SYSTEM is: Now you are needed to assist the speech-impaired people to express themselves. I will give them a passage that they hear and their answers after hearing it.

[0062] The USER section is: Based on the user's answer, please provide an optimized response that takes into account the user's context without changing the original meaning. You need to emphasize emotional expression and rich language. What you heard: {story}, and the user's reply: {message}, only the optimized user response is required. If you understand, please give an answer.

[0063] The prompt word template for the feedback generation module is mainly used to evaluate the feedback of children with autism and provide appropriate responses. In this example, by including "predefined results" in the template, LLM can determine the accuracy of the response. Then, different measures are implemented to motivate or clarify, depending on the accuracy of the response selected by the child with autism spectrum disorder.

[0064] The part about SYSTEM reads: “Please remember that you are {ai_name}, the protagonist of the story, and “I” is {user_name}”. Please provide your response based on the following requirements.

[0065] The part about USER is: After listening to the story, {user_name} guessed that your mood at that time was {front_msg}, and {user_name} wants to say to you: {user_words}. Now, please analyze whether {user_name}'s guess matches your mood at that time {front_mood} as described in the story. In addition, after hearing what {user_name} said, please share your mood. If you feel that {user_name}'s answer is inconsistent with yours, please tell him / her the reason behind his / her mood at that time and what kind of response you expect. If you feel that {user_name}'s answer is consistent with your true mood at that time, please give {user_name} some encouragement. I know that you are {ai_name}, a close friend of {user_name}, please provide a concise answer of about 50 words. You should answer with a positive, inclusive and friendly attitude.

[0066] This embodiment also provides an autistic child emotion learning assistance system, which is implemented by the autistic child emotion learning assistance method based on generative artificial intelligence provided in the above embodiment, and its specific execution process is as follows:

[0067] The initial interaction parameters are input into the interaction model, and the user is allowed to interact with the interaction model to obtain the final emotional learning evaluation.

[0068] like Figure 2 As shown in the figure, it is the process of setting up the stage. Figure 2 (a) is used to select the emotions and scenes for training in the dialogue and input the child’s recent experiences.

[0069] exist Figure 2 (b) Enter the child's information, preferences and language habits.

[0070] exist Figure 2 In (c), children select three parameters to customize the character.

[0071] like Figure 3 The interactive process is shown in FIG. 1 : wherein children can select the emotion option they think is correct in the interface, and the emotion option includes a cartoon pattern and a corresponding emotion name.

[0072] In addition, based on the children's input voice during the interaction process, keywords are extracted and the content is expanded.

[0073] like Figure 4 As shown, it is a schematic diagram of the feedback process. Figure 4 (a) is the settlement reward interface after selecting the correct emotion. Figure 4(b) The interface for the positive feedback text after selecting the correct emotion. Figure 4 (c) Negative feedback description text after selecting the wrong emotion, which is used for error correction guidance in emotion recognition.

[0074] In order to better illustrate the technical effect of the method provided in this embodiment, a 22-day test involving 6 high-functioning autistic children showed the following beneficial effects:

[0075] The behavior of six high-functioning autistic children during the 10-day training process was recorded, and the results showed that they exhibited five main behaviors that can be considered as active participation in training, including imitating the facial expressions, behaviors, and language of the characters in the system, being attracted to specific vocabulary or specific functions involved in the system, empathizing with the characters, and sharing their experience of using the system with others.

[0076] In the pretest before the start of training, the initial emotion recognition accuracy of high-functioning autistic children was recorded. Correspondingly, the posttest after the training recorded the emotion recognition accuracy of high-functioning autistic children after training. The paired sample t-test was performed on the pretest and posttest data, and the results showed that the emotion recognition accuracy of high-functioning autistic children increased significantly from pre-test (M = 4.833, SD = 1.722) to post-test (M = 10.333, SD = 1.366) (t(5) = 8.883, p = 0.0003). The variance analysis of the data of 10 days of training confirmed the significant positive effect of training duration on emotion recognition accuracy (F(9, 47) = 3.368, p = 0.002, partial η2 = 0.411). Overall, the above results show that the continuous use of this system can improve the emotion recognition ability of high-functioning autistic children.

[0077] The number of emotion representations for each child in the pre-test and post-test was calculated and a paired t-test was performed. The results showed that there was a significant difference (t(5)=2.954, p=0.032) in the number of emotion representations before the test (M=29.833, SD=6.706) and after the test (M=44.500, SD=8.432), which indicated that the emotional richness in children's emotional expressions was enhanced after training. The mean cosine distance between the expression and the content of the conversation for each child in the pre-test and post-test was calculated and a paired t-test was performed. The results showed that the mean cosine distance between the expression and the content of the conversation of children was significantly improved (t(5)=-3.750, p=0.013) from the pre-test (M=0.787, SD=0.058) to the post-test (M=0.925, SD=0.062). This means that after training, children paid more attention to the emotions of others and adjusted their expressions according to different contexts. Overall, the above results show that the present system can improve the emotional expression ability of high-functioning children with autism.

[0078] Interviews with parents five days after the training showed that children's ability to recognize and express emotions in daily life had been improved to a certain extent and lastingly, including becoming more sensitive to the emotional changes of others, being more willing to express their own emotions, and applying the skills learned in the training in their lives.

[0079] In addition, the terms "upper", "lower", "inner", "outer", "front", "rear" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps described in these embodiments do not limit the scope of the present invention.

[0080] Of course, the above description is only a specific embodiment of the present invention and is not intended to limit the scope of implementation of the present invention. All equivalent changes or modifications made according to the structure, characteristics and principles described in the patent application scope of the present invention should be included in the patent application scope of the present invention.

[0081] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A generative artificial intelligence-based emotional learning assistance method for children with autism, characterized in that: include: Obtain user information, including user personal information and preferences; Set initial interaction parameters according to the user's emotions to be learned, including occurrence scene parameters and event materials; Building an interaction model under the framework of a large language model according to the user information, wherein the interaction model includes a context generation module, an interaction module, an expression expansion module, a feedback generation module, and an evaluation module; The context generation module obtains keywords in the initial interaction parameters according to a preset first prompt word template to generate a first-person interaction text; The interaction module performs audio and text output according to the generated first-person interaction text, generates corresponding emotion recognition options, and collects user interaction text input by the user; The expression expansion module extracts keywords from the user interaction text according to a preset second prompt word template to generate an emotional prompt text; The feedback generation module outputs positive feedback description text or negative feedback description text according to the selection result of the emotion recognition option and the emotion prompt text; The evaluation result evaluates and records the user's emotional learning situation based on multiple rounds of feedback description texts generated by the feedback generation module.

2. The method for assisting emotional learning for children with autism based on generative artificial intelligence according to claim 1, characterized in that: The preferences include favorite foods and activities, as well as expression styles.

3. The method for assisting emotional learning for children with autism based on generative artificial intelligence according to claim 1, characterized in that: The event material is based on the user's recent experience and the emotional information involved.

4. The method for assisting emotional learning for children with autism based on generative artificial intelligence according to claim 1, characterized in that: The context generation module obtains keywords in the initial interaction parameters through the first prompt word template, and uses a sentence structure that meets language standards to create a personalized first-person interaction text.

5. The method for assisting emotional learning for children with autism based on generative artificial intelligence according to claim 1 or 4, characterized in that: The first-person interaction texts have multiple categories, and each first-person interaction text contains a unique emotion.

6. The method for assisting emotional learning for children with autism based on generative artificial intelligence according to claim 1, characterized in that: The expression expansion module extracts keywords from the user interaction text through the second prompt word template, and expands and enriches the user interaction text according to the dialogue context to obtain the emotional prompt text.

7. The method for assisting emotional learning for children with autism based on generative artificial intelligence according to claim 1, characterized in that: The feedback generation module is based on a pre-built analysis prompt word template, which contains "predefined results" to obtain keywords in the emotional prompt text to generate feedback description text. When the emotional recognition option is selected incorrectly, negative feedback description text is output. When the emotional recognition option is selected correctly, positive feedback description text is output. The predefined result is a preset emotional type.

8. The method for assisting emotional learning for children with autism based on generative artificial intelligence according to claim 1 or 7, characterized in that: The positive feedback description text includes praise text when the selected emotion is correct; The negative feedback explanation text includes comforting text when the emotion selection is wrong, and explanation text about the correct emotion selection.

9. An emotional learning assistance system for children with autism, characterized in that: The method for assisting children with autism to learn emotions based on generative artificial intelligence is implemented by any one of claims 1 to 8, and the specific implementation process is as follows: The initial interaction parameters are input into the interaction model, and the user is allowed to interact with the interaction model to obtain the final emotional learning evaluation.

Images