A learning method and system based on developmental toys for children

Abstract

This application relates to the field of computer technology, and more particularly to a learning method and system based on a child-developmental toy. The method includes: acquiring voice information input by a child through an AI toy, and identifying the child's emotional state during voice input; converting the voice information into text information, detecting grammatical errors and pronunciation deviations based on the text and voice information, and guiding the child to correct these errors through real-time feedback to obtain corrected text information; integrating the corrected text information into a pre-set knowledge base to store personalized knowledge; searching the knowledge base for matching information based on the text information, and outputting relevant text; and using TTS technology to play and visualize the relevant text based on the created or cloned voice tone, emotional state, and stored personalized knowledge. This application helps provide children with a more efficient and accurate learning experience.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a learning method and system based on a developmental children's toy. Background Technology

[0002] Learning methods based on interactive toys are an educational approach that combines play and learning. They aim to stimulate children's interest and initiative in learning through the interactivity and fun of the toys. This approach not only focuses on children's knowledge acquisition but also emphasizes the development of their cognitive abilities, emotional growth, and social skills. In the context of constantly evolving educational philosophies, interactive toys are gradually becoming an important tool for early childhood education. Their core lies in using gamified design to enable children to learn while playing and grow while learning.

[0003] However, with the continuous updating of educational philosophies and the development of technology, the limitations of traditional toys in terms of educational function have gradually become apparent. Traditional toys are often limited to one-way interaction or pre-set content, lacking real-time feedback, personalized adaptation, and emotional interaction capabilities, making it difficult to provide modern children with a more efficient and precise learning experience. Summary of the Invention

[0004] Therefore, it is necessary to provide a learning method and system based on developmental children's toys that can provide children with an efficient and accurate learning experience, addressing the aforementioned technical issues.

[0005] Firstly, this application provides a learning method based on developmental children's toys, the method comprising:

[0006] The AI ​​toy acquires the child's voice input and identifies the child's emotional state during the voice input based on the voice information.

[0007] The speech information is converted into text information. Based on the text information and the speech information, grammatical errors and pronunciation deviations are detected. Based on the detected grammatical errors and pronunciation deviations, the child is guided to correct them through real-time feedback to obtain the corrected text information.

[0008] The corrected text information is integrated into a preset knowledge base to store personalized knowledge. The knowledge base is built based on the RAG architecture.

[0009] Based on the text information, the knowledge base is searched to find matching information, and the relevant text is output.

[0010] Based on the created or cloned timbre, the emotional state, and stored personalized knowledge, the relevant text is played and visualized using TTS technology.

[0011] In one embodiment, the step of playing and visually displaying the relevant text via TTS technology, based on the created or cloned timbre, the emotional state, and stored personalized knowledge, includes:

[0012] Based on stored personalized knowledge, children's interests, preferences, and learning levels can be obtained;

[0013] Based on the described emotional state, determine whether to adjust the interaction method;

[0014] If not, then based on the created or cloned timbre, the emotional state, and the child's interests and learning level, the relevant text is played and visualized via TTS technology.

[0015] In one embodiment, after the step of playing and visually displaying the relevant text via TTS technology based on the created or cloned timbre, the emotional state, and the child's interests and learning level, the method further includes:

[0016] Based on stored personalized knowledge, children's learning behaviors are obtained, and a child profile is constructed based on the learning behaviors, the child's interests and learning level.

[0017] Based on the child's profile, a personalized learning path is generated, and corresponding learning resources are recommended.

[0018] The personalized learning path and the learning resources are dynamically adjusted based on the child's learning progress with the learning resources.

[0019] In one embodiment, after generating a personalized learning path and recommending corresponding learning resources based on the child's profile, the method further includes:

[0020] Children are motivated to complete the learning resources through leaderboards or team challenge modes.

[0021] In one embodiment, the step of converting the corrected speech information into text information and integrating the text information into a preset knowledge base to store personalized knowledge includes:

[0022] The corrected speech information is converted into text information, and the text information is cleaned and noise is removed to obtain the processed text information.

[0023] Extract key information from the processed text information;

[0024] The key information is expressed in a preset form and integrated into a preset knowledge base to store personalized knowledge.

[0025] In one embodiment, before retrieving the knowledge base based on the text information, finding matching information, and outputting relevant text, the method further includes:

[0026] Obtain the knowledge document and store the knowledge document in the knowledge base;

[0027] Based on the RAG architecture, FastGPT is used to process the content of knowledge documents and divide the knowledge into different text fragments.

[0028] The text fragments are converted into vectors using the vector retrieval model M3E to enable semantic similarity search.

[0029] In one embodiment, the step of retrieving the knowledge base based on the text information, finding matching information, and outputting relevant text includes:

[0030] Generate a query vector based on the text information;

[0031] In the embedding space of the knowledge base, the vector most similar to the query vector is found by an approximate nearest neighbor search algorithm, and the corresponding text fragment is extracted as context.

[0032] Based on the context and the text information, the relevant text is output using the ChatGLM3 generation model.

[0033] In one embodiment, the process of converting the speech information into text information, detecting grammatical errors and pronunciation deviations based on the text information and the speech information, and guiding the child to correct these errors through real-time feedback based on the detected grammatical errors and pronunciation deviations, results in corrected text information including:

[0034] The speech information is converted into text information. Based on the text information, an NLP model is used to detect the type of syntax error and output the error confidence and correction candidates.

[0035] Based on the syntax error type, the error confidence level, and the correction candidates, combined with the syntax error difference score, the severity of the syntax error is quantified, so as to prioritize the feedback of critical errors according to the severity of the syntax error.

[0036] Based on the spoken information, pronunciation deviations are detected;

[0037] Based on the detected grammatical errors and pronunciation deviations, and considering the severity of the grammatical errors, the child is guided to correct them through real-time feedback, resulting in corrected text information.

[0038] In one embodiment, the step of guiding the child to correct grammatical errors and pronunciation deviations based on the detected grammatical errors and their severity, through real-time feedback, to obtain the corrected text information includes:

[0039] To obtain the child's age and language development stage;

[0040] Select a feedback strategy based on the child's age and language development stage;

[0041] Based on the feedback strategy, detected grammatical errors and pronunciation deviations, and considering the severity of the grammatical errors, children are guided to correct their mistakes through immediate feedback, resulting in corrected text information.

[0042] Secondly, this application also provides a learning device based on a developmental children's toy. The device includes:

[0043] An emotion state recognition module is used to acquire voice information input by children through AI toys, and to identify the emotional state of children when they input voice information.

[0044] The voice information correction module is used to convert the voice information into text information, detect grammatical errors and pronunciation deviations based on the text information and the voice information, and guide the child to correct the grammatical errors and pronunciation deviations through real-time feedback to obtain the corrected text information.

[0045] The text information integration module is used to integrate the corrected text information into a preset knowledge base to store personalized knowledge. The knowledge base is built based on the RAG architecture.

[0046] The information retrieval module is used to retrieve the knowledge base based on the text information, find matching information, and output relevant text.

[0047] The text playback and display module is used to play and visualize the relevant text via TTS technology, based on the created or cloned timbre, the emotional state, and stored personalized knowledge.

[0048] In summary, this application includes the following beneficial technical effects:

[0049] By acquiring children's voice input through AI toys and recognizing their emotional states, it helps to better understand children's learning status and psychological reactions, thereby adjusting the interaction method. By detecting grammatical errors and pronunciation deviations and guiding children to correct them through real-time feedback, it helps to improve children's language expression ability and language accuracy. The knowledge base based on the RAG architecture supports multi-layered structured management, enabling rapid retrieval and matching of information, and outputting it in combination with children's personalized knowledge. This not only improves learning efficiency but also enhances the relevance and practicality of learning content. By using TTS technology to play relevant text information aloud and combining it with visual display, the learning process becomes more vivid and intuitive, helping to increase children's learning interest and participation, thus providing children with a more efficient and accurate learning experience. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a learning method based on a developmental children's toy in one embodiment;

[0051] Figure 2 This is a flowchart illustrating a learning method based on a nurturing children's toy in another embodiment;

[0052] Figure 3 This is a structural block diagram of a learning device based on a developmental children's toy in one embodiment. Detailed Implementation

[0053] This invention provides a learning method and system based on developmental children's toys.

[0054] The embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0055] In the description of the embodiments disclosed in this invention, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0056] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1One embodiment of the learning method based on developmental children's toys in this invention includes:

[0057] The S100 uses AI toys to acquire voice information input by children and identifies their emotional state during voice input based on that information.

[0058] Specifically, children input information via voice input through AI toys, including their questions. The system then performs emotion assessment on the voice, identifying the child's emotional state (such as positive or negative) during the input and adjusting subsequent interactions accordingly. This process is based on voice emotion recognition technology, which infers the speaker's emotional state by analyzing acoustic features (such as pitch, speech rate, and intensity) in the voice signal. For example, angry speech typically has a high pitch and fast speech rate, while sad speech may be lower, slower, and with drawn-out sounds. Furthermore, voice emotion recognition can also describe emotions using dimensional models, such as pleasure and activation levels, thus capturing emotional states more comprehensively. After emotion assessment, subsequent interactions are adjusted based on the identified emotional state. For instance, if the system identifies a child as "moderately sad," it might choose to talk to the child, ask about the reasons, or play their favorite story to alleviate their emotions. This emotion-based interaction is closer to children's psychological needs, enhancing the naturalness and friendliness of human-computer interaction.

[0059] In addition to voice recognition and emotion analysis, AI toys can also combine other modal information, such as vision and touch, to more comprehensively understand children's emotional state. For example, AI toys can capture children's facial expressions, tone of voice and movements through cameras or sensors, and combine this with tactile sensations of temperature changes to analyze their emotional state.

[0060] In this embodiment, when children input information via voice, the system analyzes the acoustic features in the voice to identify their emotional state and adjusts the interaction method accordingly, thereby achieving more natural and effective communication.

[0061] The S200 converts speech information into text information. Based on the text and speech information, it detects grammatical errors and pronunciation deviations. Based on the detected grammatical errors and pronunciation deviations, it guides children to correct them through real-time feedback, resulting in corrected text information.

[0062] Specifically, combining Automatic Speech-to-Text (ASR) and Natural Language Processing (NLP) technologies enables real-time grammatical error detection of children's speech input. ASR technology converts children's speech information into text information, its core being the extraction of features from sound signals and mapping them to text form. Furthermore, ASR typically incorporates deep learning models (such as Transformer or wav2vec 2.0) to improve recognition accuracy. NLP technology performs grammatical and semantic analysis on the generated text. After speech-to-text conversion, NLP models can identify grammatical errors (such as subject-verb disagreement, tense errors, and missing punctuation). Additionally, NLP can combine syntactic analysis and semantic reasoning to determine whether children's expressions conform to linguistic norms. Pronunciation deviation detection usually relies on speech models and pronunciation databases. By comparing the similarity between children's pronunciation and standard pronunciation, the system can determine if pronunciation deviations exist. Upon detecting grammatical errors or pronunciation deviations, the system needs to provide immediate feedback to guide children in correction (such as repeated practice of incorrect vocabulary).

[0063] In this embodiment, real-time monitoring and correction of children's speech can be achieved through speech recognition, grammatical error detection, pronunciation deviation detection, and instant feedback.

[0064] S300 integrates the corrected text information into a preset knowledge base to store personalized knowledge.

[0065] Specifically, traditional knowledge bases suffer from shortcomings in content updates, semantic understanding, and interactive capabilities, making it difficult to meet the demands of modern personalized and intelligent services. Therefore, a knowledge base based on the RAG (Retrieval-Augmented Generation) architecture is adopted. RAG is an artificial intelligence architecture that combines information retrieval with text generation, aiming to improve the accuracy, controllability, and knowledge richness of text generation models. By integrating the knowledge base and the generative model, the RAG architecture enables the system to dynamically retrieve the most relevant information to answer user questions and generate answers based on this information, rather than relying solely on static knowledge in the training data. This ability based on information retrieval and text generation provides efficient and accurate solutions for fields such as intelligent question answering and knowledge management. Through the RAG architecture, information entered by children each time can be automatically integrated into the knowledge base without retraining the generative model, and the personalized knowledge stored in the knowledge base can be updated in real time.

[0066] In this embodiment, a knowledge base is built based on the RAG architecture, which enables the knowledge base to have good scalability and can automatically integrate the corrected text information into the preset knowledge base, providing support for children's personalized learning.

[0067] S400 retrieves the knowledge base based on the text information, finds matching information, and outputs the relevant text.

[0068] Specifically, a knowledge base was built based on the RAG architecture, which integrates information retrieval and text generation. Under the RAG architecture, the knowledge base is searched to find the text fragment that best matches the text information. This matched text fragment, along with the text information, is then combined as input to the generative model. The generative model generates the final output based on the input information, thereby improving the relevance and accuracy of the output text. Furthermore, the relevant output text can be trimmed or formatted to ensure its conciseness and readability.

[0069] In this embodiment, the most relevant document fragments are retrieved from the knowledge base using the RAG architecture, and combined with the generative model, the accuracy and timeliness of the output text are significantly improved.

[0070] The S500, based on created or cloned timbres, emotional states, and stored personalized knowledge, uses TTS technology to play and visualize relevant text.

[0071] Specifically, creating a new timbre or cloning an existing one—for example, cloning a parent's voice to enhance emotional connection with the child—first requires collecting high-quality audio samples of the target voice. These samples can be recordings of everyday conversations, readings, or singing, typically at least 10 minutes long, to ensure the model can capture various vocal features. Next, the collected audio samples are processed, including noise removal and segmentation, and vocal features are extracted from the processed samples. These features include timbre, intonation, and rhythm. Finally, based on the extracted features, a neural network model capable of generating similar voices is trained. This process usually uses deep learning techniques, such as recurrent neural networks, long short-term memory networks, or Transformers. Inputting text into the trained neural network model converts the text into synthesized speech similar to the original voice. When creating a timbre, a preset timbre is first selected as the base. After selecting the base timbre, the user enters edit mode, where various parameters can be adjusted to create new timbres. After creating a voice or cloning an existing voice, and combining it with the child's current emotional state and stored personalized knowledge, the relevant text can be played aloud using TTS technology, while also displaying the text.

[0072] In this embodiment, the TTS technology, which combines timbre, emotional state, and personalized knowledge, not only breaks through the limitations of traditional TTS, but also promotes the upgrade of interaction from "information transmission" to "emotional resonance", enhancing the realism and naturalness of the interaction.

[0073] In one embodiment, such as Figure 2 As shown, S500 includes:

[0074] S520, based on stored personalized knowledge, obtains children's interests, preferences, and learning levels;

[0075] S540 determines whether to adjust the interaction method based on the emotional state;

[0076] S560, if not, then based on the created or cloned timbre, emotional state, and the child's interests and learning level, it uses TTS technology to play and visualize the relevant text.

[0077] Specifically, based on the relevant text matched to the retrieved knowledge base, and combined with the child's interests and learning level, the system dynamically generates explanations suitable for the child's comprehension. For example, when a child asks about a scientific concept, the system retrieves relevant text from the knowledge base and generates a simple and easy-to-understand explanation based on the child's age and cognitive level. This personalized learning approach not only improves learning efficiency but also enhances the child's learning interest and motivation. Through emotion recognition, the system can identify the child's emotional state during voice input and determine whether to adjust the interaction method accordingly. For example, if a child appears sad, the system might choose to talk to the child, ask about the reasons, or play a story the child likes to alleviate their emotions. When no adjustment to the interaction method is needed, the tone and rhythm of the voice are adjusted based on the created or cloned timbre and the child's emotional state. For example, a steady tone is used when the child is confused, or a brisk rhythm is added when the child is happy, thereby enhancing the interactive experience. This addition of emotional intelligence makes the learning process more human and helps to establish an emotional connection between the child and the learning content, thereby improving learning effectiveness and satisfaction. Then, based on the generated explanation and combined with the adjusted timbre, the audio is broadcast, making the content more vivid, natural, and in line with the child's auditory habits.

[0078] In this embodiment, by using TTS technology and combining it with children's emotional state, interests, and learning levels, an efficient, fun, and emotionally supportive learning environment is provided for children. This not only improves learning efficiency but also enhances children's learning interest and self-confidence.

[0079] In one embodiment, after playing and visually displaying relevant text via TTS technology, based on the created or cloned timbre, emotional state, and the child's interests and learning level, the method further includes:

[0080] Based on stored personalized knowledge, children's learning behaviors are obtained, and a child profile is constructed based on these behaviors, as well as the child's interests, preferences, and learning level. Based on the child profile, a personalized learning path is generated, and corresponding learning resources are recommended. Based on the child's learning progress with the learning resources, the personalized learning path and learning resources are dynamically adjusted.

[0081] Specifically, based on children's learning behaviors, interests, and learning levels, a child profile is constructed, and a personalized learning path is generated accordingly. The system recommends the most suitable learning resources; for example, for children with weak foundations, the system recommends more basic exercises and supplementary materials, while for children with advanced learning abilities, more challenging questions and extended reading materials can be recommended. Furthermore, the system can construct a knowledge graph of children's learning. This knowledge graph provides connections and hierarchical structures between knowledge points, allowing personalized recommendations to offer more targeted and hierarchical learning resources. Simultaneously, the knowledge graph reveals the distribution of knowledge points in incorrect learning questions, enabling the recommended learning resources to be categorized according to these knowledge points. This knowledge graph-based recommendation method helps children master basic knowledge while gradually transitioning to more complex problems, thus forming a "growth-oriented knowledge path." Based on children's learning progress, the system can dynamically adjust personalized learning paths and resources to ensure that children are always on the learning path that best suits them. For example, when a child makes significant progress in a certain knowledge point, the system will automatically reduce the recommendation of simple questions for that knowledge point and add some more challenging questions to encourage the child to continue to improve. This dynamic adjustment mechanism makes the learning process more flexible and efficient, avoids the "one-size-fits-all" teaching method, and thus better meets the learning needs of different children.

[0082] In this embodiment, personalized knowledge recommendations are supported. The difficulty of knowledge is adjusted according to the child's interaction history to form a "growth-oriented knowledge path". This not only helps children adapt to changes in their own level during the learning process, but also stimulates their learning interest and initiative, thereby achieving more efficient learning results.

[0083] In one embodiment, after generating a personalized learning path based on the child's profile and recommending relevant learning resources, the method further includes:

[0084] Children are motivated to complete their learning through leaderboards or team challenge modes.

[0085] Specifically, the leaderboard converts learning outcomes into quantifiable points and displays rankings within the group, thereby stimulating children's competitive spirit and sense of accomplishment. Children earn points upon completing learning resources, and can receive rewards after accumulating a certain number of points; this mechanism enhances children's learning interest. The team challenge mode further enhances the fun and interactivity of learning through teamwork and competition. Children are required to complete lessons with friends to earn points, thus motivating each other. This model not only increases children's engagement with learning but also strengthens the social aspect of learning.

[0086] In one embodiment, the corrected speech information is converted into text information, and the text information is integrated into a preset knowledge base to store personalized knowledge, including:

[0087] The corrected speech information is converted into text information, and the text information is cleaned and noise is removed to obtain processed text information; key information is extracted from the processed text information; the key information is expressed in a preset form and integrated into a preset knowledge base to store personalized knowledge.

[0088] Specifically, the first step is to convert speech information into text information. This process is typically achieved through speech recognition technology, which analyzes the characteristics of the speech signal and converts it into corresponding text content. After converting speech information into text information, the text information needs to be cleaned and preprocessed to remove noise and irrelevant information. Text information cleaning mainly removes irrelevant symbols, redundant spaces, special characters, etc., and segments the text into words or phrases to facilitate subsequent semantic analysis, as well as removing some common and meaningless words. After the text information is processed, key information needs to be extracted from the text information. Key information extraction is the core step in text information processing, and its goal is to identify representative information, such as keywords and topics, from a large amount of text. The extracted key information needs to be expressed in a pre-defined form and integrated into a pre-defined knowledge base to store personalized knowledge. When integrating into the knowledge base, a database interaction approach can be used to store key information in the database and interact with other databases to achieve data reading and writing. In addition, graph models or knowledge graph technologies can be used to integrate and display the extracted information.

[0089] In this embodiment, the processing from voice information to text information is realized, followed by the extraction of key information and the integration of the knowledge base, ultimately achieving the storage and application of personalized knowledge.

[0090] In one embodiment, before retrieving the knowledge base based on the text information, finding matching information, and outputting the relevant text, the method further includes:

[0091] The system acquires knowledge documents and stores them in a knowledge base. Based on the RAG architecture, it processes the knowledge document content using FastGPT to segment the knowledge into different text fragments. The text fragments are then converted into vectors using the vector retrieval model M3E for semantic similarity search.

[0092] FastGPT is a knowledge base question-answering system based on a large language model, providing out-of-the-box data processing and model calling capabilities. It can train AI models by importing documents or existing question-answering, enabling the models to answer questions in an interactive dialogue manner based on the documents. The M3E model is a multilingual, multimodal semantic embedding model that can map text content to a unified high-dimensional vector space. Its core advantages lie in the accuracy of semantic expression and compatibility with multilingual and multimodal content, giving it superior performance in complex knowledge retrieval scenarios.

[0093] Specifically, firstly, a knowledge base is built based on the RAG architecture, storing knowledge documents, which can be PDF files. Then, FastGPT processes the knowledge document content, segmenting the knowledge into different text fragments, such as question-and-answer splitting and text segmentation. These fragments are then vectorized and stored in a knowledge base supporting vector retrieval. The M3E model is used as the text vector retrieval model and integrated into the FastGPT framework. Through the FastGPT framework's model management interface, the M3E model is registered as a global vectorization service for system retrieval. The vector retrieval model M3E converts text fragments into vectors for subsequent semantic similarity searches.

[0094] In this embodiment, based on the RAG architecture, the RAG question-answering system was quickly and easily built using the components and tools provided by the FastGPT framework. This enabled data management and querying, and the M3E model was used to convert text fragments into vectors for semantic similarity search. This better adapts to the diverse content needs of children and enables rapid response to children's dialogues.

[0095] In one embodiment, based on text information, the knowledge base is searched to find matching information, and relevant text is output, including:

[0096] Based on the text information, a query vector is generated; in the embedding space of the knowledge base, the vector most similar to the query vector is found through an approximate nearest neighbor search algorithm, and the corresponding text fragment is extracted as context; based on the context and text information, the relevant text is output through the generation model ChatGLM3.

[0097] The ChatGLM3 model is one of the core components of the knowledge base, responsible for natural language question answering and context generation, providing high-quality generation capabilities for intelligent question answering. The model is stored in JSON format and dynamically loaded into the system through a JSON file. The JSON file contains the model path and configuration parameters, enabling efficient loading and flexible invocation of the model.

[0098] Specifically, after a child inputs speech, the speech-to-text information is obtained. The generation model ChatGLM3 generates a retrieval command based on the text information and converts the retrieval command into a vector using a vectorization model, resulting in a query vector. An approximate nearest neighbor search algorithm is used to find the vector most similar to the query vector, and the corresponding text fragment is obtained to determine the most relevant content. This approximate nearest neighbor search algorithm can efficiently find approximate nearest neighbors in large-scale, high-dimensional datasets without requiring time-consuming precise calculations. The most similar text fragment is then fed back to the generation model ChatGLM3 to output the relevant text.

[0099] In this embodiment, efficient task retrieval and processing are achieved through a vectorized model and an approximate nearest neighbor search algorithm, thereby improving the system's response speed and accuracy. Based on the RAG architecture, combined with the FastGPT framework, the generative model ChatGLM3, and the vector retrieval model M3E, automated content organization and dynamic question answering are realized.

[0100] In one embodiment, speech information is converted into text information. Based on the text and speech information, grammatical errors and pronunciation deviations are detected. Based on the detected grammatical errors and pronunciation deviations, the child is guided to correct them through real-time feedback. The corrected text information includes:

[0101] The process involves converting speech information into text information, using an NLP model to detect grammatical error types, and outputting error confidence and correction candidates. Based on the grammatical error type, error confidence, and correction candidates, and combined with a grammatical error difference score, the severity of grammatical errors is quantified, prioritizing feedback on critical errors according to their severity. Pronunciation deviations are detected based on the speech information. Based on the detected grammatical errors and pronunciation deviations, and combined with the severity of the grammatical errors, the process provides real-time feedback to guide children in correction, resulting in corrected text information.

[0102] Among them, the syntax error difference score is an indicator used to quantify the severity of syntax errors in text. It assesses the severity of errors by calculating the difference between the input text and the standard text. Error confidence represents the degree of certainty of the system regarding the detected error. The higher the confidence, the more reliable the system's judgment of the error. Correction candidates are possible correction schemes generated by the system based on the syntax error type and context.

[0103] Specifically, the speech information is converted into text information and input into an NLP model. The model analyzes the text information to determine if it conforms to correct grammatical rules. If the analysis fails, a grammatical error is identified. After detecting a grammatical error, the model further classifies the error type into low-level syntactic errors, high-level syntactic errors, and semantic errors. After grammatical error detection and classification, the model generates correction suggestions (correction candidates) and error confidence scores. Based on the correction candidates, a standard text information is obtained. Combined with the grammatical error difference score, the difference between the standard text information and the input text information is calculated. Based on this difference and the grammatical error type, the error severity is quantified to prioritize critical errors, improving the accuracy and efficiency of error correction. Different types of grammatical errors have different impacts on text quality; for example, sentence structure errors may have a greater impact on overall comprehension than spelling errors. Simultaneously, pronunciation deviations are detected based on the speech information. Then, based on the detected grammatical errors and pronunciation deviations, combined with the error severity, children are guided to correct their errors through immediate feedback according to importance, resulting in corrected text information.

[0104] In this embodiment, the syntax error difference score provides a basis for classifying the severity of errors by quantifying the differences in syntax errors. Combined with syntax error types and a priority feedback mechanism, it can effectively prioritize the handling of critical errors and improve error correction efficiency.

[0105] In one embodiment, based on the detected grammatical errors and pronunciation deviations, and considering the severity of the grammatical errors, the child is guided to correct them through immediate feedback, resulting in corrected text information including:

[0106] Obtain the child's age and language development stage; select a feedback strategy based on the child's age and language development stage; based on the feedback strategy, detected grammatical errors and pronunciation deviations, and the degree of grammatical errors, guide the child to correct them through immediate feedback, and obtain the corrected text information.

[0107] Specifically, children's language development exhibits distinct stages, with significant differences in language comprehension and expression abilities among children of different ages. Therefore, appropriate feedback should be provided based on each child's developmental stage. Children employ different feedback strategies during language comprehension, which gradually evolve into more complex understanding methods as they grow older. For example, for 2-3 year olds, questioning and demonstration can help them understand the meaning of sentences; while for 3-6 year olds, they can be encouraged to deepen their understanding through questioning and discussion. In providing immediate feedback, different targeted feedback strategies are employed based on the child's age and language development stage to guide them in correcting errors, thereby improving the efficiency of error correction.

[0108] In this embodiment, by understanding the child's age and language development stage, feedback strategies can be selected more effectively to help the child make continuous progress in language development.

[0109] In one embodiment, such as Figure 3 As shown, a learning device based on a developmental children's toy is provided, including: an emotion state recognition module 10, a voice information correction module 20, a text information integration module 30, an information retrieval module 40, and a text playback and display module 50, wherein:

[0110] The emotion state recognition module 10 is used to acquire the voice information input by the child through the AI ​​toy, and to identify the child's emotional state when inputting the voice based on the voice information.

[0111] The voice information correction module 20 is used to convert voice information into text information, detect grammatical errors and pronunciation deviations based on the text information and voice information, and guide children to correct the grammatical errors and pronunciation deviations through real-time feedback to obtain the corrected text information.

[0112] The text information integration module 30 is used to integrate the corrected text information into a preset knowledge base to store personalized knowledge. The knowledge base is built based on the RAG architecture.

[0113] The information retrieval module 40 is used to search the knowledge base based on text information, find matching information, and output relevant text.

[0114] The text playback and display module 50 is used to play and visualize relevant text via TTS technology, based on the created or cloned timbre, emotional state, and stored personalized knowledge.

[0115] In one embodiment, the text playback and display module obtains the child's interests and learning level based on stored personalized knowledge; determines whether to adjust the interaction method based on the emotional state; if not, it plays and displays the relevant text via TTS technology based on the created or cloned timbre, emotional state, and the child's interests and learning level.

[0116] In one embodiment, the learning device based on the developmental children's toy further includes a learning resource recommendation module, which is used to acquire children's learning behavior based on stored personalized knowledge, and construct a child profile based on the learning behavior, children's interests and learning level; generate a personalized learning path based on the child profile, and recommend corresponding learning resources; and dynamically adjust the personalized learning path and learning resources based on the child's learning progress.

[0117] In one embodiment, the learning device based on the nurturing children's toy further includes an incentive module for motivating children to complete the learning of learning resources through a points leaderboard or team challenge mode.

[0118] In one embodiment, the text information integration module 30 is further configured to convert the corrected speech information into text information, clean and remove noise from the text information to obtain processed text information; extract key information from the processed text information; express the key information in a preset form and integrate it into a preset knowledge base to store personalized knowledge.

[0119] In one embodiment, the learning device based on the developmental children's toy further includes a vector transformation module for acquiring knowledge documents and storing them in a knowledge base; based on the RAG architecture, the knowledge document content is processed by FastGPT to segment the knowledge into different text fragments; and the text fragments are converted into vectors by the vector retrieval model M3E for semantic similarity search.

[0120] In one embodiment, the information retrieval module 40 is further configured to generate a query vector based on text information; in the embedding space of the knowledge base, the vector most similar to the query vector is found through an approximate nearest neighbor search algorithm, and the corresponding text fragment is extracted as context; based on the context and text information, the relevant text is output through the generation model ChatGLM3.

[0121] The modules in the aforementioned learning device based on a developmental toy for children can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0122] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A learning method based on developmental toys for children, characterized in that, include: The AI ​​toy acquires the child's voice input and identifies the child's emotional state during the voice input based on the voice information. The speech information is converted into text information. Based on the text information and the speech information, grammatical errors and pronunciation deviations are detected. Based on the detected grammatical errors and pronunciation deviations, the child is guided to correct them through real-time feedback to obtain the corrected text information. The corrected text information is integrated into a preset knowledge base to store personalized knowledge. The knowledge base is built based on the RAG architecture. Based on the text information, the knowledge base is searched to find matching information, and the relevant text is output. Based on the created or cloned timbre, the emotional state, and stored personalized knowledge, the relevant text is played and visualized using TTS technology. The step of using TTS technology to play and visualize the relevant text based on the created or cloned timbre, the emotional state, and stored personalized knowledge includes: Based on stored personalized knowledge, children's interests, preferences, and learning levels can be obtained; Based on the described emotional state, determine whether to adjust the interaction method; If not, then based on the created or cloned timbre, the emotional state, and the child's interests and learning level, the relevant text is played and visualized using TTS technology. The process involves converting the speech information into text information, detecting grammatical errors and pronunciation deviations based on the text information and the speech information, and guiding the child to correct these errors through real-time feedback. The resulting corrected text information includes: The speech information is converted into text information. Based on the text information, an NLP model is used to detect the type of syntax error and output the error confidence and correction candidates. Based on the syntax error type, the error confidence level, and the correction candidates, combined with the syntax error difference score, the severity of the syntax error is quantified, so as to prioritize the feedback of critical errors according to the severity of the syntax error. Based on the spoken information, pronunciation deviations are detected; Based on the detected grammatical errors and pronunciation deviations, and considering the severity of the grammatical errors, the child is guided to correct them through real-time feedback, resulting in corrected text information.

2. The learning method based on a developmental children's toy according to claim 1, characterized in that, After the text is played and visualized using TTS technology based on the created or cloned timbre, the emotional state, and the child's interests and learning level, the process further includes: Based on stored personalized knowledge, children's learning behaviors are obtained, and a child profile is constructed based on the learning behaviors, the child's interests and learning level. Based on the child's profile, a personalized learning path is generated, and corresponding learning resources are recommended. The personalized learning path and the learning resources are dynamically adjusted based on the child's learning progress with the learning resources.

3. A learning method based on a developmental children's toy according to claim 2, characterized in that, After generating a personalized learning path and recommending corresponding learning resources based on the child's profile, the process also includes: Children are motivated to complete the learning resources through leaderboards or team challenge modes.

4. A learning method based on a developmental children's toy according to claim 1, characterized in that, Before retrieving the knowledge base based on the text information, finding matching information, and outputting relevant text, the method further includes: Obtain the knowledge document and store the knowledge document in the knowledge base; Based on the RAG architecture, FastGPT is used to process the content of knowledge documents and divide the knowledge into different text fragments. The text fragments are converted into vectors using the vector retrieval model M3E to perform semantic similarity search.

5. A learning method based on a developmental children's toy according to claim 1, characterized in that, The step of retrieving the knowledge base based on the text information, finding matching information, and outputting relevant text includes: Generate a query vector based on the text information; In the embedding space of the knowledge base, the vector most similar to the query vector is found by an approximate nearest neighbor search algorithm, and the corresponding text fragment is extracted as context. Based on the context and the text information, the relevant text is output using the ChatGLM3 generation model.

6. A learning method based on a developmental children's toy according to claim 1, characterized in that, The process involves guiding children to correct grammatical errors and pronunciation deviations based on the detected grammatical errors and their severity, through real-time feedback. The corrected text information includes: To obtain the child's age and language development stage; Select a feedback strategy based on the child's age and language development stage; Based on the feedback strategy, detected grammatical errors and pronunciation deviations, and considering the severity of the grammatical errors, children are guided to correct their mistakes through immediate feedback, resulting in corrected text information.

7. A learning device based on a developmental children's toy, characterized in that, include: An emotion state recognition module is used to acquire voice information input by children through AI toys, and to identify the emotional state of children when they input voice information. The voice information correction module is used to convert the voice information into text information, detect grammatical errors and pronunciation deviations based on the text information and the voice information, and guide the child to correct the grammatical errors and pronunciation deviations through real-time feedback to obtain the corrected text information. The text information integration module is used to integrate the corrected text information into a preset knowledge base to store personalized knowledge. The knowledge base is built based on the RAG architecture. The information retrieval module is used to retrieve the knowledge base based on the text information, find matching information, and output relevant text. The text playback and display module is used to play and visualize the relevant text by voice using TTS technology, based on the created or cloned timbre, the emotional state, and the stored personalized knowledge. The text playback and display module is also used to obtain the child's interests and learning level based on the stored personalized knowledge; determine whether to adjust the interaction method based on the emotional state; if not, then play and visualize the relevant text by voice using TTS technology based on the created or cloned timbre, the emotional state, and the child's interests and learning level. The speech information correction module is also used to convert the speech information into text information, and based on the text information, to detect the syntax error type through an NLP model and output the error confidence and correction candidates. Based on the syntax error type, the error confidence level, and the correction candidates, combined with the syntax error difference score, the severity of the syntax error is quantified, so as to prioritize the feedback of critical errors according to the severity of the syntax error. Based on the spoken information, pronunciation deviations are detected; based on the detected grammatical errors and pronunciation deviations, and combined with the degree of the grammatical errors, the child is guided to correct them through real-time feedback, resulting in corrected text information.

Images