Intelligent electronic equipment for cultivating interest of children

Abstract

The invention discloses an intelligent electronic device for cultivating interest of children, which comprises a sensor module for acquiring a real-time video stream; the voice module is used for identifying and detecting the voice data, understanding semantics, generating voice output responding to the semantics and realizing voice interaction; the action detection module is used for carrying out behavior detection on the image data to obtain child behavior data; the action control module is used for generating interactive actions, adjusting equipment behaviors and completing action feedback according to the voice input by the child and the behavior data; the interest analysis module is used for carrying out behavior classification and generating a visual data report and an interest map according to the child behavior and interactive action data; the evaluation feedback module is used for recording learning data of the children, giving evaluation and providing suggestions; the parent supervision module is used for acquiring the visual data report and providing a parent monitoring function; and the core control module is used for processing data and scheduling each module to execute corresponding tasks. The method can effectively help children to cultivate and develop interests.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent electronic devices and relates to an intelligent electronic device for cultivating children's interests. Background Art

[0002] With the development of technology, electronic devices are increasingly used in children's home education. At the same time, with the rise of artificial intelligence, people can use electronic products more conveniently, and the development of artificial intelligence (AI) technology in fields such as behavior detection, emotion recognition, and speech interaction has been relatively mature, with many pre-trained open-source models. However, less research focuses on developing children's interests and providing companionship.

[0003] In modern families, parents are busy with work and it is difficult to spare extra time to learn and then teach and tutor the content that children are interested in. More often, they can only accompany and participate. However, children need emotional support and interaction, especially when their parents are busy.

[0004] Most existing children's educational devices focus on knowledge imparting and lack attention to the cultivation and development of children's interests. They usually cannot monitor children's interest activity status, sitting postures, attention and other behaviors in real time, and it is difficult to help parents effectively supervise their children's behavior habits.

[0005] Based on the popularization of AI technology: AI technologies such as computer vision and natural language processing have been mature and can be applied to the cultivation and development of children's interests. In addition, the popularization of intelligent devices: the cost of intelligent hardware such as cameras, microphones, and sensors has been reduced, providing a basis for the development of intelligent electronic companions.

[0006] Therefore, it is of great significance and broad prospects to develop an electronic companion that can detect behaviors for personalized interaction and help children cultivate and develop their interests. Summary of the Invention

[0007] To solve the above technical problems existing in the prior art, the present invention provides an intelligent electronic device for cultivating children's interests, which can detect behaviors for interaction and help children cultivate and develop their interests. The specific technical solution is as follows: An intelligent electronic device for cultivating children's interests, comprising: A sensor module, which acquires a real-time video stream and converts an analog signal into a digital signal; A voice module, which recognizes and detects the voice data of the video stream, understands the semantics and generates a voice output in response to the semantics to achieve voice interaction; An action detection module, which performs behavior detection on the image data of the video stream to obtain children's behavior data; The action control module generates interactive actions based on the voice input and behavior data of children, adjusts the device behavior, and completes action feedback; The interest analysis module classifies behaviors and generates a visual data report and an interest map based on voice semantics, children's behaviors, and interactive action data; The evaluation and feedback module records children's learning data, gives evaluations, and provides suggestions; The parent supervision module obtains the visual data report and provides a parent monitoring function; The core control module receives data from the sensor module, the interest analysis module, the evaluation and feedback module, and the parent supervision module, integrates and processes the data, and schedules the voice module, the action detection module, and the action control module to execute interactive behavior tasks.

[0008] Furthermore, the hardware part of the core control module includes a main control chip, a memory, a communication interface, and a sensor interface that support multi-task parallel processing. The software part uses a micro-control method to control the electronic device and run the AI model, uses a machine learning model for children's interest analysis and behavior prediction, uses a natural language processing model for voice conversations and sentiment analysis, and identifies children's facial expressions through a convolutional neural network to judge their emotional states.

[0009] Furthermore, the sensor module includes an accelerometer, a gyroscope, a microphone, and a camera.

[0010] Furthermore, the voice module understands the semantics and generates a voice output that responds to the semantics, specifically including the following steps: Step1: Convert the voice signal to text: Use the Mel Frequency Cepstral Coefficient (MFCC) voice recognition feature extraction method to extract the feature vector m, and then use the deep learning model Transformer for the conversion from voice to text T; Input: Voice signal s(t), converted to: MFCC feature vector m; Output: Text T; Step2: Understand the child's intention and generate a response. Use the Transformer model for natural language processing. Its core is the self-attention mechanism. The self-attention formula: , where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the vector; Step3: Convert the text to a voice signal for voice synthesis and then output.

[0011] Furthermore, the action detection module performs behavior detection on the image data, specifically including the following steps: Step1: Use a deep learning model to perform pose detection on the child pose data captured by the camera, and predict the positions of the human key points in the image; Input: Image I; Output: Key point coordinates (x i , y i ), where i represents the i-th key point; Step2: Judge the pose according to the relative positions of the key points; Step3: Use a large model to detect specific objects, and transform the object detection problem into a regression problem to directly predict the bounding box and class probability, which is used to detect whether the child is safe during the interest activity; Input: Image I; Output: Bounding box (x, y, w, h) and class probability p; Loss function: , where is the number of grids, B is the number of bounding boxes for each grid, indicates whether the j-th bounding box in the i-th grid contains the target; Step4: Estimate the distance between the user and the screen through the camera and the size of the known object. The distance formula: , where D is the distance, f is the focal length of the camera, W is the actual width of the object, and p is the pixel width of the object in the image.

[0012] Furthermore, the action control module supports a predefined action library or dynamically generated actions.

[0013] Furthermore, the interest analysis module generates a visual data report, which specifically includes the following steps: Step1: Statistically calculate the average value, maximum value, minimum value, and standard deviation of the learning duration, and analyze the trend of the behavior data; Step2: Use a decision tree classification model to classify the behavior, including excellent, good, and bad, and use a regression model to predict the future behavior trend of the child; The decision tree classification selects the best splitting point through information gain: , where is the entropy of the dataset D; Linear regression minimizes the loss function: ; Step3: Use the visualization tool Matplotlib to generate a bar chart of the learning duration, a line chart of the sitting posture angle, and a pie chart of the behavior classification.

[0014] Furthermore, the parental supervision module provides a mobile application for viewing the child's interest development, learning progress, as well as the child's behavior data and interaction data on the mobile device.

[0015] Furthermore, it also includes a display module that provides an interactive interface to display the interest map.

[0016] Furthermore, it also includes a network module that supports Wifi and Bluetooth connections.

[0017] Beneficial effects: The electronic companion of the present invention can help children cultivate and develop interests; it can detect and evaluate children's behaviors, and can timely remind or encourage children to better develop their interests and hobbies and form good habits; compared with traditional electronic devices that help children learn, it is more intelligent, can record the interactions between children and the electronic companion, analyze the children's movements and emotions after detection, and is more personalized, which can help parents better understand their children's preferences and interests; based on the present invention, many in-depth expansion studies can be carried out, such as not only targeting children's interests and hobbies, but also developing to children's classroom learning, after-school tutoring, etc., with strong function expandability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a system module block diagram of an intelligent electronic device for children's interest cultivation according to an embodiment of the present invention; Figure 2 is a schematic diagram of data interaction among the voice module, the core control module, and the parental supervision module according to an embodiment of the present invention; Figure 3 is a schematic diagram of interaction between a person and the device according to an embodiment of the present invention; Figure 4 is a screen display module instance scene diagram of a person learning to draw in this embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] In order to make the objectives, technical solutions, and technical effects of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings of the specification and embodiments.

[0020] As Figure 1 and Figure 2 shown, this embodiment discloses an intelligent electronic device for children's interest cultivation, which can be divided into two subsystems, namely, a behavior detection function and a guidance function, according to the functions of the system: by integrating a high-precision motion sensor, it can capture the child's movements and sounds in real time, including the posture and amplitude of hand movements and the utensils used, so as to provide action guidance and safety guardianship for the child. The guidance function includes voice interaction and voice guidance.

[0021] Specifically, the device includes: a core control module, a sensor module, a voice module, an action detection module, an action control module, an interest analysis module, an evaluation and feedback module, a parent supervision module, a display module, and a network module.

[0022] The sensor module integrates various sensors such as an accelerometer, a gyroscope, a microphone, and a camera, detects and collects sounds and action images, obtains a real-time video stream, monitors the child's sitting posture, attention, distance from the screen and other behaviors in real time, converts analog signals into digital signals, and transmits them to the core control module for analysis and processing. It uses computer vision technology to detect the child's action postures, detect interfering objects such as mobile phones, and use a classification and recognition model to detect dangerous items such as scissors.

[0023] The core control module stores data locally or in the cloud, mainly for data integration and processing, task scheduling, real-time feedback, and system monitoring. Specifically, it receives data from the sensor module, interest analysis module, evaluation and feedback module, and parent supervision module, and performs integration and processing. According to the results of processing and analysis, it schedules each module to execute corresponding tasks, controls the voice module to output voice for intelligent conversation with the child, controls the action control module to dynamically adjust the device's behavior output, interacts with the child in real time, and monitors the device's motion state to ensure that each module works properly.

[0024] The hardware part of the core control module includes a main control chip, a memory, a communication interface, a sensor interface, etc. that support multi-task parallel processing. The main control chip uses a high-performance, low-power architecture processor with powerful computing capabilities to quickly process behavior detection data, analyze the standardization and correctness of the child's actions, and immediately give improvement suggestions and evaluations to ensure the smooth operation of the system.

[0025] The software part receives the original voice and image data from the sensor module through the sensor interface and communication interface, then cleans, denoises, and formats the original data to ensure data quality, and then performs interest analysis to generate or update the child's interest map. On this basis, according to historical data and the current state, it predicts the child's next behavior or need, schedules the corresponding module tasks, and adjusts the device's behavior according to the child's real-time feedback, such as facial expression changes and voice content.

[0026] The core control module uses a micro-control method to control the electronic device and run the AI model, relying on a variety of algorithms and models to achieve intelligent functions. It uses a machine learning model for interest analysis and behavior prediction, training the model based on historical data; uses a natural language processing model for voice conversation and sentiment analysis; and uses a convolutional neural network to recognize the child's facial expressions and judge their emotional state.

[0027] The action detection module performs real-time action detection on the acquired video images to provide the behavior data of the child, specifically including the following steps: Step1: For the child pose data captured by the camera, use a deep learning model to perform pose detection, predict the positions of the human key points in the image, such as shoulders, arms, knees, etc., and judge the child's pose; Input: Image I; Output: Key point coordinates (x i , y i ), where i represents the i-th key point; Step2: Judge the pose according to the relative positions of the key points. For example, when practicing dancing, judge whether the heights of the arm and leg movements are in place. If the height is higher or lower than the threshold, it is considered that the movement is not in place enough; Step3: Use a large model to detect specific objects, such as scissors, craft knives, etc., and transform the object detection problem into a regression problem, directly predicting the bounding box and class probability, which is used to detect whether the child is safe when performing interest activities such as paper-cutting; Input: Image I; Output: Bounding box (x, y, w, h) and class probability p; Loss function: , where is the number of grids, B is the number of bounding boxes in each grid, indicates whether the j-th bounding box in the i-th grid contains the target; Step4: Estimate the distance between the user and the screen through the camera and the size of the known object. The distance formula: , where D is the distance, f is the focal length of the camera, W is the actual width of the object, and p is the pixel width of the object in the image.

[0028] The voice module better interacts and gives feedback by recognizing the child's voice. First, convert the voice input collected by the microphone array of the sensor module into text, then use a natural language model to process the voice signal, understand the child's intention and generate an appropriate response, and finally convert the text into voice output. The child's mood, such as happy or frustrated, can be detected by recognizing the tone and intonation of the child's voice, and then appropriate feedback, such as playing music or telling jokes, can be provided according to the emotional state to give emotional company and support and achieve an intelligent conversation with the child. The specific processes of converting voice input into text and text into voice output are as follows: Step1. Convert the speech signal into text: Use Mel Frequency Cepstral Coefficients (MFCC), a commonly used method for speech recognition feature extraction, to extract the feature vector m, and then use the deep learning model Transformer to convert the speech into text T.

[0029] Input: Speech signal s(t); Convert to: MFCC feature vector m.

[0030] Output: Text T Step2: Understand the child's intention and generate a response. Use the Transformer model for natural language processing. The core of it is the self-attention mechanism. The self-attention formula: , where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the vector.

[0031] Step3: Convert the text into a speech signal for speech synthesis and then output.

[0032] This module uses speech recognition technology to understand the child's speech input and speech synthesis technology to generate the electronic partner's speech feedback.

[0033] The said action control module controls the actions of the device of the present invention according to the child's input speech or emotional state, generates the required actions, such as giving origami tutorials, piano sheet music, giving emotional responses, etc. At the same time, according to the child's interest preferences and usage habits, it intelligently adjusts the interaction content and methods to provide personalized companion services. The implementation of the action feedback of this module mainly includes the following steps: Step1: Predesign the actions of the electronic partner, such as dancing, waving, nodding, and store them in the action library; Step2: Generate actions according to the child's instructions or environmental status through rules; Step3: Control the electronic partner to execute actions through hardware, and can give the child some simple demonstrations.

[0034] The said action control module supports a predefined action library or dynamically generates actions.

[0035] After the said interest analysis module records the child's behavior and the interaction data with the electronic partner, it statistically analyzes the recorded data, such as average value, trend analysis, uses a machine learning model to classify or predict the behavior, and finally visualizes the data to generate reports, such as bar charts, line charts, pie charts, and provides them for parents to view on the mobile phone APP. Specifically, it includes: Step 1: Statistically calculate the mean, maximum, minimum, standard deviation, and other statistics of the learning duration, and analyze the trends in the behavioral data, such as changes in the learning duration.

[0036] Step 2: Use a decision tree classification model to classify behaviors, such as excellent, good, and poor, and use a regression model to predict the future behavioral trends of the child.

[0037] The decision tree classification selects the best splitting point through information gain: , where is the entropy of the dataset D; Linear regression minimizes the loss function: ; Step 3: Conduct data visualization, and use the visualization tool Matplotlib to generate bar charts of the learning duration, line charts of the sitting posture angles, pie charts of the behavior classifications, etc.

[0038] The evaluation and feedback module can automatically compare answers or standard actions and give evaluations, providing instant feedback. At the same time, it provides tutorials to help children better understand and master the key points, associate errors with relevant knowledge points, and provide targeted improvement suggestions for real-time correction. It also records the learning data of children for analysis and feedback, helping children comprehensively cultivate and develop their interests, discover deficiencies and make improvements. At the same time, through interactive feedback and emotional incentives, it stimulates children's motivation to persevere and sense of participation, encourages children to be positive, and ultimately improves the learning effect and realizes interest guidance.

[0039] The parent supervision module provides a parent monitoring function, displaying the result reports generated by the data analysis tool. Parents can view the children's interest development and learning progress, as well as the children's behavioral data, such as learning duration, sitting posture, attention, distance from the screen, etc., and interactive data, such as voice interaction content and emotional state, through a mobile application.

[0040] The display module contains a high-definition display screen, providing a personalized interest map for children's learning. Children can selectively watch the content they are interested in. This module can well display the expressions, actions, interactive content of characters, as well as information related to cultivating interests, such as practice arrangements and duration prompts, knowledge point explanation screens, etc. It integrates a high-fidelity speaker and an audio amplifier to ensure the quality of the output sound, enabling children to clearly hear the voice responses of the electronic companions, key point explanations, skill analyses, etc. At the same time, it supports Bluetooth connection to external speakers to provide a better audio experience.

[0041] The display module is specifically responsible for the presentation and interaction of visual information, mainly acting on information presentation, interaction interfaces, practice assistance, and parental supervision, etc. It displays images, videos, texts, etc. on the screen to help children more intuitively understand and learn knowledge, while enhancing the interactive experience. It displays learning content, prompt information, dialogue texts, etc. through text. It displays pictures, charts, paintings, etc. through images. It plays teaching videos, animations, interactive content, etc. through videos.

[0042] The network module supports Wifi and Bluetooth connections, connects to the home network through Wifi to achieve data synchronization, obtain online learning resources, and remote communication with the parent mobile APP.

[0043] Such as Figure 3 As shown below, taking the voice interaction experiment between a child and an electronic buddy as an example, the application method of the present invention will be specifically described.

[0044] The child is using the electronic buddy to learn how to dance a dance.

[0045] Voice collection: The child asks, "How do I dance this dance?" Voice recognition: The voice module converts the voice into the text "How do I dance this dance?" Semantic understanding: The voice module recognizes the child's intention as "hoping to obtain a dance tutorial and teaching".

[0046] Knowledge base query: Retrieve relevant information from the knowledge base and generate a reply text "Here is the complete video and breakdown action teaching video of this dance that I found. I hope it will be helpful to you." Voice synthesis: The voice module converts the reply text into voice.

[0047] Voice output: Play the voice through the speaker.

[0048] Below, taking the experiment of a child using the electronic buddy to learn origami as an example, the application method of the present invention will be specifically described.

[0049] Information presentation: The display module shows the origami tutorial video and step-by-step instructions.

[0050] Interaction interface: The child selects papers of different colors and sizes according to their preferences.

[0051] Learning assistance: The electronic buddy analyzes whether the sequence of the child's origami steps is correct by detecting the child's hand movements and provides improvement suggestions.

[0052] Interest guidance: Recommend relevant origami materials and tutorials according to the child's interests.

[0053] Emotional support: After the child completes an origami work, the display module plays a celebration animation and shows the text "You're amazing!"

[0054] As Figure 4 shown below, take the interactive correction experiment after the electronic buddy gives an evaluation feedback when the child is painting as an example to specifically illustrate the application method of the present invention.

[0055] Real-time painting analysis: When the child is painting on the tablet, the system analyzes elements such as the painting content, lines, proportions, and color combinations in real time. Suppose the child is drawing a cat. The system identifies features such as the outline, ears, and tail of the cat and then detects whether the lines are smooth and whether the proportions are coordinated.

[0056] Evaluation feedback: After the painting is completed, the system evaluates and gives feedback on the child's work in dimensions such as whether the lines are smooth and coherent, whether the proportions of each part of the object are reasonable, whether the color use is coordinated, creativity, and uniqueness. Then, through text feedback, such as "Your lines are very smooth, but the proportion of the cat's ears can be adjusted." At the same time, visual feedback is given: mark the areas that need improvement on the painting (such as circling the ears in red).

[0057] If it is detected that the child has not moved the pen for a long time or the child asks how to draw through voice, a video is played at this time to show how to correctly draw the cat's ears. Finally, after the child finishes drawing, the system gives a score: line smoothness 8 / 10, proportion accuracy 6 / 10, color combination 9 / 10, creative expression 7 / 10. The feedback content is: "Your color combination is great! But the cat's ears can be drawn a little bigger, which will be cuter!" Interest motivation: Maintain the child's learning interest through an achievement system and positive feedback. For example, rewards (such as badges, points) are obtained after completing specific tasks. When the child completes a complete painting of a cat, they get a "Little Painter" badge.

[0058] The above is only the preferred implementation case of the present invention and does not impose any form of limitation on the present invention. Although the implementation process of the present invention has been described in detail above, for those familiar with the field, they can still modify the technical solutions recorded in the foregoing examples or make equivalent replacements for some of the technical features. Any modifications, equivalent replacements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intelligent electronic device for children's interest cultivation, characterized in that, It includes: A sensor module that acquires a real-time video stream; A voice module that performs recognition and detection on the voice data of the video stream, understands the semantics, and generates a voice output with response semantics to achieve voice interaction; An action detection module that performs behavior detection on the image data of the video stream to obtain children's behavior data; An action control module that generates interaction actions according to the children's input voice and behavior data, adjusts the device behavior, and completes action feedback; An interest analysis module that classifies behaviors based on children's behavior and interaction action data and generates a visual data report and an interest map; An evaluation feedback module that records children's learning data, gives evaluations, and provides suggestions; A parent supervision module that acquires the visual data report and provides a parent monitoring function; A core control module that receives and processes the data of the sensor module, interest analysis module, evaluation feedback module, and parent supervision module, and schedules the voice module, action detection module, and action control module to execute interactive behavior tasks.

2. The intelligent electronic device for children's interest cultivation according to claim 1, characterized in that, The hardware part of the core control module includes a main control chip, a memory, a communication interface, and a sensor interface that support multi-task parallel processing. The software part uses a micro-control method to control the electronic device and run the AI model, uses a machine learning model for children's interest analysis and behavior prediction, uses a natural language processing model for voice dialogue and sentiment analysis, and identifies children's facial expressions through a convolutional neural network to judge their emotional states.

3. The intelligent electronic device for children's interest cultivation according to claim 1, characterized in that, The sensor module includes an accelerometer, a gyroscope, a microphone, and a camera.

4. The intelligent electronic device for children's interest cultivation according to claim 1, wherein The voice module understands the semantics and generates a voice output with response semantics, specifically including the following steps: Step1: Convert the voice signal to text: Use the Mel Frequency Cepstral Coefficient (MFCC) voice recognition feature extraction method to extract the feature vector m, and then use the deep learning model Transformer for the conversion from voice to text T; Input: Voice signal s(t), converted to: MFCC feature vector m; Output: Text T; Step2: Understand the child's intention and generate a response, use the Transformer model for natural language processing, the core of which is the self-attention mechanism, and the self-attention formula: ， where Q is the query matrix, K is the key matrix, V is the value matrix, and d k is the dimension of the vector; Step3: Convert the text to a voice signal for voice synthesis and then output.

5. The intelligent electronic device for children's interest cultivation according to claim 1, characterized in that, The action detection module performs behavior detection on the image data, specifically including the following steps: Step1: Use a deep learning model to perform pose detection on the child's pose data captured by the camera to predict the positions of the human key points in the image; Input: Image I; Output: Key point coordinates (x i , y i ), where i represents the i-th key point; Step2: Judge the pose according to the relative positions of the key points; Step3: Use a large model to detect specific objects and transform the object detection problem into a regression problem to directly predict the bounding box and class probability, which is used to detect whether children are safe during interest activities; Input: Image I; Output: Bounding box (x, y, w, h) and class probability p; Loss function: ， Among them, is the number of grids, B is the number of bounding boxes for each grid, indicates whether the j-th bounding box of the i-th grid contains an object; Step4: Estimate the distance between the user and the screen through the camera and the size of the known object. The distance formula is: , where D is the distance, f is the focal length of the camera, W is the actual width of the object, and p is the pixel width of the object in the image.

6. The intelligent electronic device for children's interest cultivation according to claim 1, characterized in that, The action control module supports a predefined action library or dynamic action generation.

7. The intelligent electronic device for children's interest cultivation according to claim 1, characterized in that, The interest analysis module generates a visual data report, which specifically includes the following steps: Step1: Statistically calculate the average, maximum, minimum, and standard deviation of the learning duration, and analyze the trend of the behavior data; Step2: Use a decision tree classification model to classify behaviors, including excellent, good, and bad, and use a regression model to predict the future behavior trend of children; The decision tree classification selects the best splitting point through information gain: ， Among them, is the entropy of the data set D; Linear regression minimizes the loss function: ； Step3: Use the visualization tool Matplotlib to generate a bar chart of the learning duration, a line chart of the sitting posture angle, and a pie chart of the behavior classification.

8. The intelligent electronic device for children's interest cultivation according to claim 1, characterized in that, The parent supervision module provides a mobile application for viewing the interest development and learning progress of children, as well as the behavior data and interaction data of children on the mobile device.

9. The intelligent electronic device for cultivating children's interests according to claim 1, wherein, It also includes a display module that provides an interactive interface to display the interest map.

10. The intelligent electronic device for children's interest cultivation according to claim 1, wherein It also includes a network module that supports Wifi and Bluetooth connections.

Images