An ai companion growing method and system based on child behavior data

Abstract

This invention discloses an AI-assisted growth method and system based on children's behavioral data. It uses multimodal sensors to sense environmental safety and collect children's behavioral data, and uses IMU compensation to ensure the accuracy of behavioral data. Then, the unstructured raw data is transformed into structured behavioral tags. Combined with children's daily routines and historical profiles, it accurately identifies children's real-time status and dynamically updates the profiles incrementally. This forms a closed-loop processing from proactive perception, status recognition, profile generation, multimodal execution strategy to feedback optimization. It can proactively adjust the interaction mode according to children's real-time emotions, concentration and long-term development characteristics, and provide safe and highly personalized AI-assisted growth services.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and robotics, specifically to an AI-assisted growth method and system based on children's behavioral data. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, companion robots are gradually entering the home and becoming important partners in children's growth. Existing children's companion robots mainly use preset fixed programs or simple rule engines to perform interactive tasks, such as playing preset stories after detecting sound or engaging in fixed question-and-answer interactions at specific time periods. These systems typically lack the ability to deeply perceive the individual behavioral characteristics of children, failing to capture subtle changes in children's emotions, fluctuations in concentration, or shifts in interest in real time. Furthermore, existing solutions often directly use raw audio and video data for cloud processing or simple matching, lacking a graded protection mechanism for children's privacy and failing to dynamically adjust companionship strategies based on children's long-term growth trajectory. This results in rigid interactive content, low personalization, and an inability to truly adapt to the psychological and behavioral needs of children at different developmental stages.

[0003] Based on the above situation, the main technical problem with existing technologies is that existing child companionship systems lack the ability to perceive and analyze children's multi-dimensional behavioral data in a refined and structured manner, and are unable to construct a dynamically evolving profile of children's growth. This results in a single and passive companionship strategy, making it difficult to achieve truly personalized and proactive companionship and long-term adaptive growth guidance. Summary of the Invention

[0004] This invention aims to provide an AI-assisted growth method and system based on children's behavioral data, which solves the problem of rigid companionship strategies caused by inaccurate data perception and static and lagging profiles in the existing technology.

[0005] To achieve the above objectives, the technical solution adopted by this invention is: an AI-assisted growth method based on children's behavioral data, comprising: S1: Sensing the presence of children in the home environment and the safety of the surrounding environment through multimodal sensors; S2: In response to the detection of a child's presence and a safe environment, collect the child's multimodal behavior data and use the IMU to perform robot motion compensation on the multimodal behavior data; S3: Convert motion-compensated multimodal behavioral data into structured behavioral labels; S4: Based on behavioral tags, preset routines, and the child's historical developmental profile, identify the child's current real-time status; S5: Read the historical growth profile and incrementally update the historical growth profile in conjunction with the real-time status to generate an updated growth profile; Among them, the robot's companionship strategy, task form, interaction intensity and family routine adaptation method are adjusted in reverse based on the updated growth profile. The corresponding interaction is executed and the feedback data of the child is collected and fed back to the growth profile to complete the iterative closed loop, so as to realize the functions of the robot to understand the child in the long term, actively approach the child and dynamically adapt to the child's state.

[0006] Preferably, in S1, the multimodal sensor includes a forward-looking depth camera, a voice module, a DTOF array, a rear-looking ultrasonic radar, and a cliff detection sensor. Perceiving whether the surrounding environment is safe includes using forward-looking depth cameras, area array DTOF, rear-looking ultrasonic radar, and cliff detection sensors for near-range obstacle avoidance, rearward obstacle detection, and fall risk detection.

[0007] Preferably, in S2, the collection of children's multimodal behavioral data includes: The child's position, posture, and range of motion are captured using a forward-looking depth camera and a high-definition head camera. Children's facial expressions are captured using a high-definition camera on their heads; Children's voices are collected through a voice module; Children's touch behavior is collected using head touch sensors.

[0008] Preferably, in S2, the robot performs motion compensation on the multimodal behavior data in conjunction with the IMU, including: using the pitch angle, roll angle, yaw angle and acceleration information provided by the IMU to perform coordinate compensation on the spatial data collected by the forward-looking depth camera and the head high-definition camera.

[0009] Preferably, in S4, when multiple real-time states are satisfied simultaneously, arbitration is carried out in the following order: security first, privacy first, appeasement first, learning second, and entertainment third. The safety priority includes: when cliff detection is triggered, backward obstacles are too close, robot posture is abnormal, or the distance between the child and the robot is lower than the safety threshold, all non-safety-related strategies are suspended, and obstacle avoidance or retreat actions are prioritized.

[0010] Preferably, in S5, the historical growth profile includes at least the learning focus index, active exploration index, language expression index, physical activity index, emotional stability index, task persistence index, social intimacy index, interest preference index, and daily routine index. The incremental update specifically includes: converting the behavior tags in this interaction into profile update events, and updating the relevant profile metrics with weights based on the event type, duration, frequency of occurrence, confidence level, and task context.

[0011] Preferably, the method further includes: Generate companionship strategies based on real-time status, updated child development profiles, parental rules, and safety constraints; The companionship strategy is executed through quadrupedal movement, head display, voice, light, touch-activated lighting, or projection. During the process, children's reaction time, completion rate, facial expression changes, voice feedback, touch behavior, and sustained participation are collected as feedback data, which are then written into the updated growth profile.

[0012] Preferably, the multimodal behavior data is collected using a unified timestamp and the events are sliced ​​according to short windows of 0.5 to 3 seconds, task windows of 5 to 30 minutes, and long windows of days, weeks, and months.

[0013] Preferably, the method further includes privacy-level processing of children's behavioral data, which includes primary data, secondary data, and tertiary data; The primary data includes raw video, raw audio, images of children's faces, and images of the home space, which are processed locally by default. Secondary data includes the structured behavioral tags and the real-time status, used to generate the historical growth profile locally; The three-level data includes growth summaries, task statistics, and trend reports, which can be synchronized to the parents' devices.

[0014] On the other hand, this invention proposes an AI-assisted growth system based on children's behavioral data, comprising: The environmental sensing module is used to detect whether a child is present in the home environment and whether the surrounding environment is safe through multimodal sensors; The behavior acquisition module is used to collect multimodal behavior data of children when the environment is safe, and to perform motion compensation for the robot itself in conjunction with the IMU; The label generation module is used to convert motion-compensated multimodal behavioral data into structured behavioral labels; The status recognition module is used to identify the child's current real-time status based on the behavior tags, preset daily routines, and the child's historical growth profile. The profile management module is used to read the historical growth profile and incrementally update the historical growth profile in combination with the real-time status to generate an updated growth profile. Among them, the robot's companionship strategy, task form, interaction intensity and family routine adaptation method are adjusted in reverse based on the updated growth profile. The corresponding interaction is executed and the feedback data of the child is collected and fed back to the growth profile to complete the iterative closed loop, so as to realize the functions of the robot to understand the child in the long term, actively approach the child and dynamically adapt to the child's state.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention uses multimodal sensors to perceive environmental safety and collect children's behavioral data. It utilizes IMU compensation to ensure the accuracy of the behavioral data, and then transforms the unstructured raw data into structured behavioral tags. Combining children's daily routines and historical profiles, it accurately identifies children's real-time status and dynamically updates the profiles incrementally. This forms a closed-loop processing from proactive perception, status recognition, profile generation, multimodal execution strategy to feedback optimization. It enables the system to proactively adjust interaction methods based on children's real-time emotions, concentration, and long-term developmental characteristics, providing safe and highly personalized AI companionship and growth services. Attached Figure Description

[0016] Figure 1 This is a flowchart of the AI-assisted growth method based on children's behavioral data according to the present invention; Figure 2 This is a block diagram of the AI-assisted growth system based on children's behavioral data, as described in this invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a specific posture. If the specific posture changes, the directional indicators will also change accordingly.

[0019] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the use of "and / or" or "and / or" throughout the text includes three parallel solutions. For example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0020] Please see Figure 1 This invention provides an AI-assisted growth method based on children's behavioral data, comprising: S1: Sensing the presence of children in the home environment and the safety of the surrounding environment through multimodal sensors; the multimodal sensors include a forward-looking depth camera, a voice module, a DTOF array, a rear-view ultrasonic radar, and a cliff detection sensor; sensing the safety of the surrounding environment includes near-field obstacle avoidance, rearward obstacle detection, and fall risk detection through the forward-looking depth camera, DTOF array, rear-view ultrasonic radar, and cliff detection sensor.

[0021] Multimodal behavioral data is collected using a unified timestamp and segmented into event windows of 0.5 to 3 seconds, 5 to 30 minutes, and long-term windows of days, weeks, and months. For image jitter and posture changes during quadruped robot movement, the system uses pitch, roll, yaw, and acceleration information provided by the IMU to compensate for posture changes in the visual recognition results, preventing misjudgments of a child's line of sight, distance, or range of motion when the robot nods, turns, or moves.

[0022] By integrating a combination of multimodal sensors, the system achieves comprehensive environmental perception and precise obstacle avoidance, effectively ensuring children's interactive safety. At the same time, by adopting a unified timestamp mechanism and combining short-term, task-based, and long-term multi-dimensional event slicing strategies, it not only eliminates data deviations caused by the robot's own motion interference, but also realizes full-scale data fusion from millisecond-level motion capture to long-term behavioral trend analysis, providing a reliable and continuous time-series data foundation for building a high-precision dynamic growth profile.

[0023] The forward-looking depth camera is mainly used for spatial reconstruction, collecting children's body posture, relative distance, direction of activity, upper body movements, as well as obstacles and interactive areas in the home space. It can support children to follow, avoid obstacles, learn posture recognition, and make preliminary judgments on the projection area.

[0024] Area array DTOF is used for scenarios such as near-range obstacle avoidance, binocular failure compensation, black object detection, glass detection compensation, strong light environment compensation, and emergency braking.

[0025] Binocular vision failure compensation method: When the binocular camera cannot match the depth due to low texture, dark light, black objects or strong reflective scenes, DTOF can provide absolute distance measurement per pixel.

[0026] Black object detection method: Black or low-reflectivity objects are poorly detected by ordinary RGB or binocular depth detection, while DTOF obtains distance by measuring time of flight and does not rely on reflective textures. Therefore, the robot can still accurately detect and avoid black floors, furniture or toys.

[0027] Glass detection compensation method: Transparent or semi-transparent glass can cause binocular / depth cameras to malfunction. DTOF can capture time difference signals through partial reflection of the glass to generate effective distance data, preventing the robot from colliding with glass doors or screens and ensuring its safe approach or timely stop.

[0028] Strong light environment compensation method: Binocular or structured light is easily interfered with under strong light. DTOF uses active infrared emission and time ranging technology to filter out ambient light interference and ensure the navigation safety of the robot when it is outdoors or near a window.

[0029] Emergency stop mode: When the DTOF detects that the distance threshold is lower than the safe distance, it immediately sends an emergency stop command to the motion controller.

[0030] Rear-view ultrasonic radar is used for detecting obstacles behind a robot when it is moving backward, turning around, yielding, or waiting safely.

[0031] Cliff detection sensors are used to detect steps, stairs, edges, or height differences to prevent robots from falling as children follow or approach them.

[0032] S2: In response to sensing the presence of a child and that the environment is safe, collect the child's multimodal behavior data and combine it with the IMU to perform robot motion compensation on the multimodal behavior data; the multimodal behavior data includes the child's position, posture, voice, facial expression, touch, and range of motion; the collection of the child's multimodal behavior data includes: collecting the child's position, posture, and range of motion through a forward-looking depth camera and a high-definition head camera, collecting the child's facial expression through a high-definition head camera, collecting the child's voice through a voice module, and collecting the child's touch behavior through a head touch sensor.

[0033] The specific method of using IMU to compensate for the robot's own motion based on multimodal behavioral data is as follows: using the pitch angle, roll angle, yaw angle and acceleration information provided by the IMU, coordinate compensation is performed on the spatial data collected by the forward-looking depth camera and the head high-definition camera to avoid misjudgment of the child's position, distance or range of motion due to the robot's body tilting or shaking.

[0034] By fusing IMU posture and acceleration data to perform real-time motion compensation on multimodal behavioral data, spatial errors introduced by the robot's own movement, tilting, or shaking are effectively eliminated, significantly improving the accuracy and robustness of position localization, motion amplitude recognition, and facial expression capture. Combined with the collaborative acquisition of multi-source sensors such as depth cameras, high-definition cameras, and voice modules, accurate perception of children's position, posture, voice, facial expressions, and touch behavior across all dimensions is achieved, providing a high-fidelity, interference-free, high-quality behavioral data foundation for subsequent state recognition and profile updates.

[0035] The head-mounted high-definition camera is used for children's facial expressions, gaze direction, gestures, object interactions, and fine-grained behavior recognition. It also supports taking photos, video calls, and projection correction, and can work in a time-sharing mode in conjunction with a projection module.

[0036] An IMU (Integrated Measurement Unit) is used for robot posture estimation, motion state determination, visual data posture compensation, and motion stabilization control. It can acquire the three-axis angular velocity, three-axis acceleration, and posture change information of the quadruped robot body. The system uses the IMU to calculate the body's pitch angle, roll angle, yaw angle, and acceleration changes in real time, and integrates them with joint motion state, foot contact state, and visual depth data to serve robot posture estimation, motion state determination, visual perception compensation, and motion stabilization control.

[0037] Specifically, when the robot walks, turns, nods, projects its image downwards, or is disturbed by external forces, the IMU outputs the changes in the robot's posture in real time. The control system uses these changes to perform coordinate compensation on the spatial data collected by the forward-looking depth camera, high-definition camera, and DTOF, avoiding deviations in the recognition results of the child's position, obstacle distance, and projection area due to the robot's tilt or shaking. At the same time, the motion control unit uses the IMU feedback to determine whether the robot is in a state of forward tilt, backward tilt, side tilt, sudden stop, fall, or being picked up, and dynamically adjusts its gait, center of gravity, joint output, and range of motion.

[0038] When performing tasks such as companionship, approaching children, waiting, yielding, providing touch-sensitive feedback, or projection, the IMU is also used to constrain the robot's motion stability. When the system detects that the attitude angle or angular velocity exceeds a preset threshold, it will reduce the movement speed, pause the projection action, or trigger safety protection to ensure the robot's movement safety and interaction stability in the home environment.

[0039] The voice module includes a microphone array and speakers for voice wake-up, sound source localization, speech recognition, voice emotion judgment, task broadcasting, and reassurance feedback.

[0040] The head touch sensor is used to recognize a child’s stroking, patting or proactive intimate interaction and trigger head light, facial expression, voice or motion feedback.

[0041] S3: Convert motion-compensated multimodal behavioral data into structured behavioral labels; By converting motion-compensated raw multimodal data into structured behavioral labels, redundancy and noise in unstructured data are effectively eliminated, achieving accurate mapping from complex sensory information to standardized semantic features; This not only significantly reduces the computational complexity and misjudgment rate of subsequent state recognition algorithms, but also enables children's behavioral patterns (such as emotions, concentration, and interaction intentions) to be efficiently understood and quantified by machines, laying a solid data foundation for building interpretable and computable dynamic growth profiles.

[0042] Behavioral tags include, but are not limited to: child appearing, child leaving, child approaching the robot, child moving away from the robot, child looking at the robot, child looking at the projection area, child looking down, child lying on the table, child waving, child pointing, child repeating, child asking questions, child remaining silent for a long time, child answering incorrectly for a long time, child touching their head, child hitting the robot, child being depressed, child being excited, child refusing the task, and child completing the task.

[0043] Visual tags are generated jointly by a forward-looking depth camera and a high-resolution head camera. The forward-looking depth camera is mainly used for recognizing human presence, distance, orientation, body posture, and range of motion; the high-resolution head camera is mainly used for recognizing facial expressions, gaze, gestures, and fine-grained interactions. Voice tags are generated by a voice module, including calls, responses, repeats, speech rate, volume, tone of voice, and crying sounds. Touch tags are generated by a head touch sensor, including light touches, sustained touches, continuous taps, and stroking.

[0044] In scenarios with low computing power or privacy restrictions, the system can retain only behavioral tags and statistical features, without saving the original video and audio.

[0045] S4: Based on behavioral tags, preset routines, and the child's historical growth profile, identify the child's current real-time state; when multiple real-time states are met simultaneously, arbitration is conducted in the following order: safety priority, privacy priority, comfort priority, learning second, and entertainment third; wherein, the safety priority includes: when cliff detection is triggered, backward obstacles are too close, robot posture is abnormal, or the distance between the child and the robot is lower than the safety threshold, all non-safety-related strategy execution is suspended, and obstacle avoidance or retreat actions are prioritized.

[0046] By integrating behavioral tags, daily routines, and historical profiles, the system achieves accurate real-time identification of children's status and establishes a multi-level dynamic arbitration mechanism that prioritizes safety, privacy, comfort, learning, and entertainment. This ensures that in complex scenarios, the system prioritizes the safety of children and the environment (such as automatic obstacle avoidance and posture correction) while also protecting privacy and providing emotional comfort. This strategy effectively solves the problem of rigid decision-making in multi-state conflicts, enabling the robot to adaptively adjust interaction priorities according to the urgency of the situation. Thus, while providing personalized growth companionship, it builds a highly reliable, explainable, and ethically compliant proactive safety protection system.

[0047] S5: Read the historical growth profile and incrementally update the historical growth profile in combination with the real-time status to generate an updated growth profile. Based on the updated growth profile, adjust the robot's companionship strategy, task form, interaction intensity, and family routine adaptation method in reverse. Execute the corresponding interaction and collect the child's feedback data to return to the growth profile to complete the iterative closed loop, so as to realize the robot's long-term understanding of children, proactive approach to children, and dynamic adaptation to children's status.

[0048] A historical growth profile includes at least the following indices: learning focus index, proactive exploration index, language expression index, physical activity index, emotional stability index, task persistence index, social intimacy index, interest preference index, and daily routine index. Incremental updates specifically include: converting the behavioral tags in this interaction into profile update events, and weighting and updating the relevant profile metrics based on the event type, duration, frequency of occurrence, confidence level, and task context.

[0049] By constructing a closed-loop incremental update mechanism that combines behavioral tags, profile events, and weighted indicators, the growth profile has transitioned from static recording to dynamic evolution. This solution can quantify and correct indices across nine dimensions, including learning focus and emotional stability, in real time based on the type, frequency, and context of interactive events. This effectively solves the problems of lagging updates and coarse granularity in traditional profiles, ensuring that profile data accurately reflects children's current ability changes and growth trends, thereby providing a basis for continuously optimized personalized interaction strategies for the robot.

[0050] The above method also includes: generating a companionship strategy based on real-time status, updated growth profile, parental rules, and safety constraints; executing the companionship strategy through quadrupedal movement, head display, voice, light, touch-activated lighting, or projection; and during execution, collecting the child's reaction time, completion rate, facial expression changes, voice feedback, touch behavior, and sustained participation as feedback data, and writing the feedback data into the updated growth profile.

[0051] This closed-loop mechanism enables dynamic optimization of the entire process, from strategy generation and multimodal execution to feedback perception and profile iteration. Through multi-channel collaboration, including quadrupedal movement, visual interaction, and tactile projection, it significantly improves the appropriateness of the companionship strategy and children's participation. At the same time, real-time data such as reaction time, completion rate, and emotional feedback are fed back to the growth profile, which not only verifies the effectiveness of the strategy and quantifies the child's immediate response characteristics, but also drives the profile to continuously evolve. This ensures that the robot can adaptively adjust its subsequent behavior based on the latest feedback, thereby achieving truly personalized, highly interactive, and self-learning intelligent companionship.

[0052] The above method also includes privacy-graded processing of children's behavioral data, which includes primary data, secondary data, and tertiary data. Primary data includes raw video, raw audio, children's facial images, and family space images, which are processed locally by default. Secondary data includes the structured behavioral tags and the real-time status, used to generate the historical growth profile locally. Tertiary data includes growth summaries, task statistics, and trend reports, which can be synchronized to the parents' end.

[0053] By constructing a three-tiered privacy-classifying processing mechanism—localizing raw data, privatizing feature data, and cloudifying summary data—the risk of privacy leakage is effectively avoided while ensuring that core sensitive information such as children's faces and family spaces (Level 1) do not leave the local area. At the same time, it enables the layered utilization of high-value behavioral features (Level 2) and growth insights (Level 3), which not only meets parents' needs for personalized services but also complies with the principle of minimum necessity and data compliance requirements, thus building a trustworthy, secure, and explainable data governance framework for smart companion devices.

[0054] On the other hand, this invention proposes an AI-assisted growth system based on children's behavioral data, such as... Figure 2 As shown, it includes: The environmental sensing module is used to detect whether a child is present in the home environment and whether the surrounding environment is safe through multimodal sensors; The behavior acquisition module is used to collect multimodal behavior data of children when the environment is safe, and to perform motion compensation for the robot itself in conjunction with the IMU; The label generation module is used to convert motion-compensated multimodal behavioral data into structured behavioral labels; The status recognition module is used to identify the child's current real-time status based on the behavior tags, preset daily routines, and the child's historical growth profile. The profile management module is used to read the historical growth profile and incrementally update the historical growth profile in combination with the real-time status to generate an updated growth profile. Based on the updated growth profile, the robot's companionship strategy, task format, interaction intensity, and family routine adaptation method are adjusted in reverse. Corresponding interactions are executed and child feedback data is collected and fed back to the growth profile to complete the iterative closed loop, so as to realize the functions of the robot to understand children in the long term, actively approach children, and dynamically adapt to children's status.

[0055] Furthermore, this embodiment also involves a projection module, which can be used for English reading aloud, programming enlightenment, motor coordination, emotional expression, and parent-child collaborative tasks.

[0056] The projection task generation process includes: identifying the projectable area, determining the child's location, selecting the task type, adjusting the projection angle and focal length, projecting the task content, collecting the child's feedback, and writing a growth profile. The projectable area can be identified by a forward-looking depth camera, a high-definition camera, or a TOF board integrated into the projection module.

[0057] When there is interference between the head-mounted high-definition camera and the projection field of view, the system can adopt a time-sharing working mode, that is, first turn on the camera to identify the child's position and the projection area, and then turn on the projection to perform the task. During the task, feedback is collected by voice, DTOF or intermittent visual frames.

[0058] Children's feedback on the projected task can be identified through voice repetition, eye contact, gesture pointing, body movement, approaching the projection area, and task completion results.

[0059] The projection task adaptation scoring algorithm is as follows: The system selects the most suitable projection task from a set of candidate tasks based on the child's real-time status, developmental profile, parental rules, and safety constraints. Candidate tasks may include English reading aloud tasks, introductory programming tasks, motor coordination tasks, emotion expression tasks, and parent-child collaboration tasks.

[0060] For any candidate task (T_i), the system calculates its task fit score: ; in, This represents the fit score for the i-th type of task; This indicates the degree of interest matching between children and this type of task; This indicates the child's current attentional engagement level; This indicates the degree of match between the child's current ability level and the difficulty of the task; This indicates the value of such tasks in improving children's weaknesses; F indicates the level of support for the task from parental rules or preferences; R indicates the level of child fatigue or resistance; R indicates the current environment or safety risks associated with the robot's movement. to This is a weighting factor that can be adjusted based on the child's age, family rules, and historical usage data.

[0061] The system selects the task that meets the safety requirements and has the highest score as the current projection task: When the safety risk R exceeds the preset threshold, or when the child is in a non-disturbing state, the system will not generate a projection learning task, but will switch to a wait, soothe, or exit strategy.

[0062] The algorithm for dynamically adjusting the difficulty of projection tasks is as follows: The system adjusts the difficulty of the next task based on the performance of the current task. The formula for updating task difficulty is: ; in, Indicates the difficulty of the current task; Indicates the difficulty of the next task; This indicates the quality of the task completion; This indicates the expected quality threshold. Indicates the level of children's participation; Indicates the degree of fatigue, resistance, or low mood in children; , , This is the adjustment coefficient; This indicates that the difficulty of the task will be limited to... to Within the range.

[0063] When children complete tasks with high quality, participate actively, and show no obvious fatigue, the system increases the difficulty of the task; when children fail repeatedly, their attention decreases, or they become depressed, the system reduces the difficulty of the task or shortens the task duration.

[0064] The evaluation algorithm for the English reading aloud task is as follows: In English reading aloud tasks, the projection module displays words, pictures, or sentences, while the speech module captures the child's pronunciation. The system calculates an English reading aloud score based on pronunciation accuracy, completeness, rhythm matching, and the child's willingness to actively follow along. ; in, Indicates the quality of completion of the English reading aloud task; Indicates pronunciation accuracy; Indicates the completeness of the repetition; Indicates the degree of matching between speech rate and rhythm; This indicates the child's level of enthusiasm for actively following along or making repeated attempts; to These are the weighting coefficients.

[0065] The pronunciation accuracy can be calculated based on the edit distance between the target phoneme sequence and the phoneme sequence obtained from the child's speech recognition: ; in, Represents the standard pronunciation phoneme sequence; Indicates the sequence of phonemes that children read aloud; Indicates edit distance; Indicates the sequence length.

[0066] The evaluation algorithm for programming introductory tasks is as follows: In introductory programming tasks, the projection module displays directional arrows, flowcharts, or action instructions. Children arrange these instructions via voice, gestures, or touch to guide the robot through the designated actions. The system calculates a score based on the correctness of the instruction sequence, path accessibility, and task completion. ; in, L represents the quality of completion of the programming enlightenment task; G represents the degree of matching between the sequence of instructions arranged by the child and the target sequence of instructions; V represents the path reachability determined based on the home map and robot safety constraints; and V represents whether the robot reaches the target or completes the specified action after execution. to These are the weighting coefficients.

[0067] The instruction sequence matching degree can be calculated using the longest common subsequence: ; in, This represents the sequence of instructions input by the child. Indicates the target instruction sequence; This represents the length of the longest common subsequence.

[0068] The algorithm for evaluating motion coordination tasks is as follows: In motor coordination tasks, the projection module projects footprints, direction points, rhythm points, or movement cues onto the ground or wall. The system uses a forward-looking depth camera, a high-definition camera, and a DTOF array to identify the child's performance. Motor coordination scores can be expressed as: ; in, Indicates the quality of completion of a motion coordination task; This indicates the degree of matching between the child's movement position and the projected target position; This indicates the degree to which the child's movement rhythm matches the task rhythm; This indicates the degree to which the safe distance between children and robots / obstacles is met during the activity; to These are the weighting coefficients.

[0069] The position matching degree can be expressed as: ; Where N represents the number of target action points; This represents the distance between the child's actual action point and the j-th projected target point; This indicates the maximum permissible deviation distance.

[0070] The evaluation algorithm for the emotion expression task is as follows: In the emotion expression task, the projection module displays emoji cards, colors, story scenes, or emotion options. Children express their current emotions through voice, gestures, or pointing. The system calculates an emotion expression score based on the consistency between the child's choices, their voice, and their facial expressions. ; in, Indicates the quality of completion of the emotion expression task; Indicate the emotion cards or projection options selected by the child; This indicates the results of voice emotion recognition; This indicates the facial expression recognition result; This indicates whether the child is willing to describe the reason in words; to These are the weighting coefficients.

[0071] This rating is not used to simply judge right or wrong, but to determine whether a child is willing to express emotions, and serves as a basis for subsequent soothing strategies and updates to the child's developmental profile.

[0072] The evaluation algorithm for parent-child collaborative tasks is as follows: In parent-child collaborative tasks, the projection module displays tasks that require joint completion by children and parents, such as taking turns answering questions, completing a story together, cooperative physical activities, or parent-child games. The system can calculate a score based on the level of participation, completeness of each round, and degree of collaborative completion. ; in, Indicates the quality of completion of parent-child collaborative tasks; Indicates child participation; Indicates parental involvement; Indicates the completeness of parent-child interaction rounds; Indicates the completion rate of collaborative tasks; to These are the weighting coefficients.

[0073] The above task evaluation results can be uniformly recorded as It also inputs the growth profile update unit and the task difficulty dynamic adjustment unit to update indicators such as children's learning focus, interest preferences, task persistence, motor activity, emotional expression ability and close interaction.

[0074] Furthermore, this system primarily utilizes a quadruped robot for family-based educational companionship. The robot itself is preferably equipped with a quadrupedal locomotion mechanism, a head-mounted interactive structure, a voice module, a visual perception module, a safety perception module, a lighting system, and a body-based computing and control unit. Its core functionality does not rely on a single sensor, but rather on the fusion of multiple sensors within a home setting to achieve a closed-loop system for child status recognition and development.

[0075] The lighting system is used to indicate the robot's status, provide feedback on children's touch behavior, reward learning, create a soothing atmosphere, indicate low battery status, and support multimodal emotional expression.

[0076] Furthermore, the system provided by the present invention includes at least a multimodal behavior data acquisition unit, a data preprocessing and synchronization unit, a behavior tag generation unit, a child real-time status recognition unit, a growth profile generation unit, a companionship strategy generation unit, a multimodal interaction execution unit, a feedback evaluation unit, a parent-side collaboration unit, and a data classification and privacy protection unit.

[0077] Specifically, the multimodal behavior data acquisition unit is used to collect data on children's movements, expressions, speech, distance, touch, location, and task feedback.

[0078] The data preprocessing and synchronization unit is responsible for time alignment, noise reduction, attitude compensation, event slicing, and anomaly filtering of data from different sensors. The specific algorithms are as follows: The data preprocessing and synchronization unit employs a sliding window synchronization algorithm based on a unified timestamp, a multi-source data denoising algorithm based on median filtering / Kalman filtering, a spatial coordinate compensation algorithm based on IMU attitude calculation, a behavior data slicing algorithm based on trigger events, and an anomaly filtering algorithm based on confidence and multi-source consistency to uniformly process multi-sensor data. This algorithm combination enables sensor data with different sampling frequencies, spatial coordinate systems, and noise characteristics to form stable and reliable children's behavior data frames, providing a foundation for subsequent state recognition, growth profile generation, and companionship strategy arbitration.

[0079] The behavior label generation unit is used to convert raw perceptual data into structured behavior labels, which serve as input for subsequent state recognition and profile updates.

[0080] The specific conversion algorithm is as follows: The behavior tag generation unit is responsible for converting the raw sensor data collected by multimodal sensors into structured behavior tags, providing standardized input for child status recognition, growth profile updates, and companionship strategy generation.

[0081] Specifically, this unit employs a behavior label generation algorithm that integrates rule-based thresholding, feature extraction, machine learning classification, and multimodal confidence. First, it performs time synchronization and preprocessing on data from the forward-looking depth camera, high-definition camera, area array DTOF, voice module, touch sensor, IMU, rear-view ultrasonic radar, and cliff detection sensor. Then, it extracts features from the sensor data, including the child's distance, orientation, body posture, head orientation, gaze direction, voice content, voice energy, tone changes, touch location, touch duration, and robot posture changes. Finally, combining a pre-defined rule model and classification model, these features are converted into structured behavior labels such as "child approaching the robot," "child moving away from the robot," "child looking at the robot," "child looking at the projection area," "child calling out to the robot," "child touching the robot's head," "child leaving the task area," "child completing an answer," "child's attention decreasing," "child feeling depressed," and "child in motion."

[0082] In one implementation, the system uses a rule-based threshold algorithm to generate tags for simple and highly deterministic behaviors. For example, when the distance between a child and the robot is less than a preset threshold and the duration exceeds a set time, a tag "child approaches robot" is generated; when the head touch sensor detects a touch signal and the duration exceeds a preset threshold, a tag "child touches robot head" is generated; when the voice module recognizes a wake word or robot name, a tag "child calls robot" is generated; and when a child leaves the recognition area during a task and the duration exceeds a preset time, a tag "child leaves task area" is generated.

[0083] In another implementation, the system generates labels using a feature classification algorithm for complex behaviors such as posture, facial expressions, attention, and emotions. This feature classification algorithm can be implemented using one or more techniques selected from human keypoint recognition, gaze direction estimation, speech emotion recognition, action sequence classification, lightweight neural network classifiers, decision trees, support vector machines, or temporal models. The system outputs corresponding behavioral labels and confidence scores based on the child's posture changes, gaze duration, speech tone changes, interaction response duration, and task engagement.

[0084] Furthermore, the behavior label generation unit employs a multimodal confidence fusion algorithm to comprehensively evaluate the output results of multiple sensors. When different sensors consistently judge the same behavior, the confidence level of the corresponding behavior label is increased. When some sensors are affected by factors such as occlusion, strong light, noise, excessive distance, or communication abnormalities, the system reduces the weight of abnormal sensors and prioritizes the results from sensors with higher confidence levels. For example, if the high-definition camera cannot stably identify a child's gaze, the system can combine the human orientation data from the forward-looking depth camera with the direction of the voice source to generate a "child-focused robot" label. When the forward-looking depth camera is affected by black objects or low-texture environments, the system can combine the distance data from the area array DTOF to determine whether to assign an approach or obstacle avoidance label.

[0085] The behavioral tags include at least the behavior type, occurrence time, duration, confidence level, associated sensors, spatial location, and task context information. In this way, the system can transform continuous, heterogeneous, and unstructured raw sensory data into computable, storable, and traceable structured behavioral tags, providing a unified data foundation for subsequent child status recognition, growth profile updates, and companionship strategy arbitration.

[0086] The child real-time status recognition unit is used to determine whether the child is currently in a suitable state for learning, companionship, exercise, comforting, or non-interference based on the child's current behavior tags, historical growth profile, family scene information, parent rules, and robot safety status, and outputs the recognition results to the companionship strategy generation unit.

[0087] Specifically, the child's real-time state recognition unit employs a real-time state determination algorithm based on multimodal behavior label scoring, state priority arbitration, and finite state machine switching. The system first receives structured behavior labels output by the behavior label generation unit, such as a child approaching the robot, looking at the robot, looking at the projection area, actively calling out to the robot, touching the robot's head, leaving the task area, answering questions, prolonged silence, low mood, and high activity level. Then, combining the child's developmental profile, family routines, current time period, the child's spatial location, and robot safety constraints, it calculates state scores for different candidate states.

[0088] In one implementation, the system sets five candidate states: learning state, companionship state, exercise state, soothing state, and non-disturbing state, and establishes corresponding state scoring functions for each: Learning status rating: When a child is at a desk, in the living room study area, or in an area where a projector is available, and exhibits behaviors such as looking at the robot or the projection area, providing positive voice feedback, maintaining a stable posture, and having an attention span that reaches a preset threshold, and the current time matches the study period set by the parents, the system will increase the learning status score. If the system detects that the child frequently leaves the room, looks down at the desk, has a significant delay in responding, is in a low mood, or the parents have set the child to "do not disturb," the learning status score will decrease.

[0089] Companionship Status Rating: When a child actively calls out to the robot, approaches the robot, touches the robot's head, spends a long time alone with it, and does not show obvious fatigue or resistance during non-study periods, the system increases the companionship status score; if the child is focused on studying, sleeping, eating, or communicating with parents, the companionship status score decreases.

[0090] Motion status score: When a child is in a safe and open area such as the living room, and is in a standing or walking position, and the robot confirms that the surrounding space meets the conditions for safe movement through a front-view depth camera, area array DTOF, rear-view ultrasonic radar, and cliff detection, the system increases the movement status score; if obstacles, cliff risks, children being too close, or the robot's posture being unstable are detected, the movement status score decreases.

[0091] Reassurance Status Rating: When the system detects behaviors such as crying, low mood in the voice, prolonged silence, continuous task failure, moving away from the task area, turning away from the robot, lying down with the head down, or repeatedly touching the robot, it increases the soothing status score. When the soothing status score is higher than the preset threshold, the system will prioritize stopping the learning task and switch to low-stimulation companionship, soft lighting effects, voice comfort, or send a reminder to the parents.

[0092] Non-disturbing status rating: When a child is sleeping, eating, in a do-not-disturb period or private area set by the parents, is focused on doing homework, communicating with parents, or when the current environment is not suitable for the robot to intervene, the system increases the do-not-disturb state score. The do-not-disturb state has a high priority, and when its score exceeds the threshold, the robot remains in a waiting, exiting, or silent state.

[0093] Furthermore, the system employs a priority arbitration mechanism to decide on candidate states. The priorities are set as follows: safety protection takes precedence over non-interference, non-interference takes precedence over soothing, soothing takes precedence over learning, and learning takes precedence over normal companionship. The movement state must simultaneously meet the conditions of child suitability and environmental safety. Specifically, if a child exhibits both learning and soothing characteristics, the system will prioritize the soothing state; if the child's state is suitable for movement, but a cliff, obstacle, or abnormal robot posture is detected, the system will not enter the movement state but will switch to a waiting or safety protection state.

[0094] To avoid frequent state transitions, the system also sets state holding time and switching thresholds. The system will only switch to a candidate state if the score of the candidate state continuously exceeds the preset threshold and the duration reaches the preset standard. If the current state score fluctuates for a short period of time, the system will maintain the original state or enter the observation state to prevent the robot from frequently interrupting the child.

[0095] The real-time status recognition results for children should include at least the current status type, status confidence level, triggering reason, associated behavior label, and suggested execution method. For example, the system can output "Current status is learnable, confidence level 0.82, triggering reason is looking at the projection area, stable posture, and positive voice feedback"; or it can output "Current status is needing comforting, confidence level 0.87, triggering reason is prolonged silence, continuous failure to answer, and low voice emotion".

[0096] The growth profile generation unit is responsible for statistical analysis, weighting, trend analysis, and capability modeling of long-term behavioral tags.

[0097] The companionship strategy generation unit combines real-time status, growth profile, parental rules, and safety constraints to generate robot execution strategies.

[0098] The multimodal interaction execution unit controls quadrupedal movement, voice, display, lighting, touch-activated lighting, projection, and optional motion mechanism execution strategies. For the head touch sensor, the system can provide feedback such as touch-activated lighting, continuous touch-activated lighting, continuous tapping to switch expressions or play positive voice messages; this feedback is not just a simple lighting effect, but a component of intimate behavior tags, which will be incorporated into the social intimacy index calculation and companionship strategy generation process.

[0099] The lighting system can indicate various robot states, such as standby, learning, reward, soothing, low battery, obstacle avoidance, touch feedback, and a low-brightness mode before bedtime. The brightness, flashing frequency, and color changes of the lights need to be matched to the child's state: the brightness and speed of change should be reduced in soothing mode, while the brightness can be increased briefly in reward mode.

[0100] Voice output and displayed facial expressions need to be synchronized with the task strategy. For example, display a positive expression and give encouragement when the child completes the task; reduce the speaking speed and reduce continuous questions when the child feels frustrated; and only keep the necessary status lights on when the child is in a non-disturbing state, without actively broadcasting.

[0101] The feedback and evaluation unit is used to collect children's responses to the strategy, including task completion, reaction time, emotional changes, and continued participation.

[0102] The parent-side collaboration unit provides features such as growth summary, exception reminders, rule configuration, privacy authorization, and family schedule settings.

[0103] The data classification and privacy protection unit performs classification processing on raw data, behavioral tags, growth summaries, and anonymized optimized data.

[0104] Specifically, the strategy generation method is as follows: The robot's computing unit generates the most suitable companionship strategy based on behavioral tags, the child's real-time status, developmental profile, parental rules, and safety constraints. The system first determines whether the environment in which the robot and the child are located meets safety requirements. If safety conditions are met, the system determines the strategy type based on the child's current state, including learning, companionship, exercise, soothing, or non-interference. Then, combining the child's developmental profile with their interests, weaknesses, task persistence, emotional stability, and level of close interaction, the system determines the specific task content, difficulty, duration, and interaction method. When the child's state is suitable for learning, the system prioritizes generating a learning guidance strategy; when the child actively approaches, calls out to, or touches the robot, a companionship or close interaction strategy is generated; when the child's activity level is high and the environment is safe, an exercise interaction strategy is generated; when the child is depressed, resistant, or experiences repeated failures, a soothing strategy is generated; when the child is in a non-interference period, a private area, sleeping, or focused state, a waiting, silent, or exit strategy is generated. The strategy generates results including task type, execution content, execution duration, difficulty level, voice feedback method, light feedback method, robot position, movement method, and whether parental reminders are required. The strategy is dynamically adjusted based on the child's feedback during execution.

[0105] The core closed-loop process of this invention is as follows: multimodal data acquisition, preprocessing and synchronization, behavior tag generation, real-time status recognition, growth profile update, strategy generation, multimodal execution, and child feedback collection. This closed loop is not a one-time question-and-answer process, but a long-term iterative process—the more frequently the system is used, the more it can align with children's interests, growth pace, and family life patterns.

[0106] The growth profile update unit is responsible for incrementally updating the child's growth profile based on behavioral tags, real-time status recognition results, and task feedback. The growth profile covers indicators such as learning focus, interest preferences, task persistence, emotional stability, motor activity, close interaction, regularity of daily routines, and knowledge mastery. For example, the learning focus index can be calculated based on the percentage of time the child spends looking at the task area, the number of times they leave the task area, the duration of continuous answering, the frequency of irrelevant actions, and the number of task interruptions; the active exploration index can be calculated based on the number of times the child actively asks questions, actively chooses tasks, tries new tasks, and completes tasks without parental prompts; the social intimacy index can be calculated based on dimensions such as actively calling the robot, touching the robot's head, approaching the robot, saying goodbye to the robot, and actively sharing behaviors. The growth profile is updated incrementally: after each task, the system writes information such as task type, start time, duration, completion rate, changes in the child's status, and feedback behavior into the profile; for abnormal data such as excessive environmental noise causing unreliable speech recognition, the system will reduce the weight of that data to avoid erroneous updates.

[0107] This unit adopts an incremental update algorithm based on event weight, time decay and trend correction: the system converts the behavior tags in each interaction into profile update events, and updates the relevant profile indicators with weights according to the event type, duration, frequency of occurrence, confidence level and task context.

[0108] For example, when a child continuously gazes at the projection area, actively answers questions, and completes tasks, the system will improve their learning focus and knowledge acquisition; when a child actively selects a certain type of content or asks questions, their interest in the corresponding area will be strengthened; when a child actively approaches, calls out to, or touches the robot's head, the level of close interaction will be increased; and when a child experiences consecutive failures, prolonged silence, leaves the task area, or becomes depressed, the system will reduce the recommendation intensity of the current task and increase the trigger weight of soothing strategies.

[0109] State recognition can be achieved through a weighted scoring method. Taking a learnable state as an example, its score consists of scores for stable posture, focused gaze, stable emotions, alignment with daily routine, and historical acceptance, while deducting scores for fatigue, resistance, environmental noise, and parental privacy. When a state score exceeds a threshold and meets safety constraints, the system will enter the corresponding strategy execution process.

[0110] If multiple conditions are met simultaneously, the system will arbitrate according to the following priority order: "safety first, privacy first, comfort first, learning second, entertainment last." For example, if a child is crying, even if the learning plan has not been completed, the system will stop the current learning task and enter the comforting process or send a reminder to the parents.

[0111] The system employs a time decay process for historical profiles, giving higher weight to recent behaviors than earlier ones. A single abnormal behavior serves only as a basis for short-term correction and does not directly overwrite long-term profiles. When similar behaviors occur repeatedly, their trend weight is increased. Through this mechanism, the child's developmental profile can be continuously and dynamically updated as the system is used, and this updated profile can be used to adjust subsequent task content, difficulty, duration, and companionship strategies.

[0112] The companionship strategy is not solely determined by course recommendations, but rather by a strategy arbitrator jointly inputting the child's real-time status, developmental profile, parental rules, and safety constraints. The arbitrator then outputs task parameters and execution control instructions. When the child is in a state requiring soothing or undisturbed interaction, the system will prioritize suspending the learning task and switch to a soothing, waiting, reduced interaction intensity, or active exit mode.

[0113] The strategy arbitrator's function is to comprehensively determine the companionship strategy that the robot should currently execute based on the child's real-time status, developmental profile, parental rules, and safety constraints. This module employs multi-factor weighted scoring, priority arbitration, and safety pre-constraint algorithms.

[0114] The specific method is as follows: The system first assesses safety conditions, including cliff risks, close-range obstacles, robot posture, battery status, and the distance to the child. If a safety risk exists, the system prioritizes stopping, waiting, retreating, or safety protection strategies. Once the safety conditions are met, the system arbitrates strategies based on the child's real-time status: If the child is in a non-disturbing state, the robot remains silent, waits, or exits; if the child needs comforting, the learning task is paused, and low-stimulation voice, soft lighting effects, or quiet companionship are used; if the child is in a learning-ready state, the system determines the learning content, task difficulty, and interaction duration based on the child's developmental profile; if the child is in a movement-ready state and the environment is safe, a movement companionship task is generated; if the child actively approaches, calls out to, or touches the robot, the system executes response, companionship, or close interaction strategies.

[0115] The strategy arbitrator outputs results including strategy type, task content, execution method, task difficulty, duration, reminder intensity, robot positioning, light feedback, voice feedback, and action feedback. During execution, the system recalculates strategy priorities based on the child's feedback, dynamically switching, pausing, or terminating the current task as necessary.

[0116] The application scenarios are as follows: 1. Proactive companionship and short-term learning after school After school, the child enters the living room. The robot detects the child's presence using its forward-facing depth camera and recognizes the child's voice prompt, "Kai Kai." After reviewing the child's developmental profile, the system detects a recent high level of interest in English animal vocabulary, but advises against continuous learning sessions exceeding three minutes. The robot then moves to a safe distance slightly in front of the child, displays a positive expression, and initiates a short English reading task. After the child reads along, the voice module assesses completion, while the high-definition head camera simultaneously detects whether the child is looking at the robot or the task screen. If the child performs well, the system awards a light and records their active learning behavior; if the child's attention wanes, the system stops adding tasks and switches to a relaxed conversation.

[0117] For proactive positioning, the primary sources are forward-looking depth cameras, high-definition head cameras, sound source localization from the voice module, and home maps. UWB is an optional module that can be used for assisted positioning in scenarios involving cross-room navigation, occlusion, or loss of vision. LiDAR is only used for SLAM navigation and mapping and is not a mandatory component of this solution.

[0118] The robot determines the target companion position based on the child's state: when the child is studying, it will preferentially be positioned to the side front or side rear to avoid blocking the desk, wall or passageway; when in companionship mode, it can maintain a natural distance of 0.8 to 1.5 meters; when the child is in a low mood and actively touches or calls, the robot can slowly approach to a closer safe distance; when the robot moves quickly or approaches the robot's feet, the robot will actively back away or move to the side.

[0119] Safety constraints are achieved through a combination of a forward-looking depth camera, a DTOF array, a rear-looking ultrasonic radar, cliff detection, and an IMU. In any situation, if cliff detection is triggered, a rearward obstacle is too close, there is an abnormal posture, or the child's distance is below the safety threshold, motor control takes precedence over learning and play strategies.

[0120] 2. Providing comfort and companionship when children are feeling down. If a child sits in the corner of the sofa for an extended period with their head down and no verbal interaction, the system uses a forward-facing depth camera to identify their stationary posture, a high-definition head camera to recognize their facial expression while looking down, and a voice module to detect prolonged silence, thus determining that the child needs comforting. At this point, the robot does not initiate a learning task but instead moves slowly to its side and front, dims the lights, and gently asks if the child needs companionship. When the child touches the robot's head, the head touch sensor triggers a soft light, the system records this interaction, and sends an emotional summary to the parent's device if necessary.

[0121] 3. Head touch light and intimacy growth portrait After a child completes a task, they touch the robot's head. When the head's touch sensor detects continuous touch, the lighting system gradually brightens and displays a reward effect, while the voice module simultaneously outputs encouraging words. The system marks this event as "proactive and intimate interaction after task completion," increasing the social intimacy index and the weight of positive feedback for the task. In subsequent similar tasks, the system prioritizes touch-activated lights and voice encouragement as feedback methods.

[0122] 4. Active following and safe retreat of all four legs When a child moves slowly in the living room, the robot estimates the child's relative distance and direction using a forward-looking depth camera, maintaining a natural following distance of approximately 1 to 1.5 meters. If the rear-view ultrasonic radar detects an obstacle behind it, or if cliff detection identifies the edge of the ground, the robot immediately stops moving backward or adjusts its path. If a child quickly approaches the robot's feet, the robot reduces its movement or actively backs away to avoid a collision.

[0123] 5. Projection Learning Task The system first uses a forward-looking depth camera and a DTOF array to identify the distance to the wall or tabletop, then adjusts the projected content according to the child's position. If the head-mounted high-definition camera and the projection module cannot work stably at the same time, the system uses a time-sharing mode, first identifying the child's position and task area, and then turning on the projection. The child completes the task by repeating aloud, pointing with gestures, or moving their body, and the feedback results are synchronized to the child's growth profile.

[0124] 6. Methods for emotional reassurance and parental collaboration When the system recognizes that a child is in a state of depression, irritability, crying, repeated failures, silence, or actively seeking comfort, the robot immediately enters the emotional soothing process: suspending the learning task, reducing the brightness of the lights and the sound of voice stimulation, and using short sentences, slow speech, and low-motion movements to accompany the child.

[0125] If a child touches the robot's head, the system triggers a soft light and voice response, and records the interaction. If the child continues to cry, call for help, or is in a dangerous area, the system sends a summary alert to the parent. The alert prioritizes an event summary rather than the original audio and video, such as "The robot has paused its learning task and entered companionship mode after detecting continuous low mood in the child," and can receive further instructions from the parent.

[0126] The parent app supports setting family routines, do-not-disturb periods, privacy zones, learning goals, task preferences, notification thresholds, and whether to allow cloud summary synchronization.

[0127] The above description is merely an exemplary embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention specification and drawings under the technical concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A method for AI-assisted growth based on children's behavioral data, characterized in that, include: S1: Sensing the presence of children in the home environment and the safety of the surrounding environment through multimodal sensors; S2: In response to the detection of a child's presence and a safe environment, collect the child's multimodal behavior data and use the IMU to perform robot motion compensation on the multimodal behavior data; S3: Convert motion-compensated multimodal behavioral data into structured behavioral labels; S4: Based on behavioral tags, preset routines, and the child's historical developmental profile, identify the child's current real-time status; S5: Read the historical growth profile and incrementally update the historical growth profile in conjunction with the real-time status to generate an updated growth profile; Among them, the robot's companionship strategy, task form, interaction intensity and family routine adaptation method are adjusted in reverse based on the updated growth profile. The corresponding interaction is executed and the feedback data of the child is collected and fed back to the growth profile to complete the iterative closed loop, so as to realize the functions of the robot to understand the child in the long term, actively approach the child and dynamically adapt to the child's state.

2. The AI-assisted growth method based on children's behavioral data according to claim 1, characterized in that, In S1, the multimodal sensors include a forward-looking depth camera, a voice module, a DTOF array, a rear-looking ultrasonic radar, and a cliff detection sensor. Perceiving whether the surrounding environment is safe includes using forward-looking depth cameras, area array DTOF, rear-looking ultrasonic radar, and cliff detection sensors for near-range obstacle avoidance, rearward obstacle detection, and fall risk detection.

3. The AI-assisted growth method based on children's behavioral data according to claim 2, characterized in that, In S2, the collection of children's multimodal behavioral data includes: The child's position, posture, and range of motion are captured using a forward-looking depth camera and a high-definition head camera. Children's facial expressions are captured using a high-definition camera on their heads; Children's voices are collected through a voice module; Children's touch behavior is collected using head touch sensors.

4. The AI-assisted growth method based on children's behavioral data according to claim 1, characterized in that, In S2, the robot performs motion compensation on multimodal behavior data by combining IMU, including: using pitch angle, roll angle, yaw angle and acceleration information provided by IMU to perform coordinate compensation on spatial data collected by forward depth camera and head high-definition camera.

5. The AI-assisted growth method based on children's behavioral data according to claim 1, characterized in that, The method further includes: Generate companionship strategies based on real-time status, updated child development profiles, parental rules, and safety constraints; The companionship strategy is executed through quadrupedal movement, head display, voice, light, touch-activated lighting, or projection. During the process, children's reaction time, completion rate, facial expression changes, voice feedback, touch behavior, and sustained participation are collected as feedback data, which are then written into the updated growth profile.

6. The AI-assisted growth method based on children's behavioral data according to claim 1, characterized in that, The multimodal behavioral data was collected using a unified timestamp and the events were sliced ​​according to short windows of 0.5 to 3 seconds, task windows of 5 to 30 minutes, and long windows of days, weeks, and months.

7. The AI-assisted growth method based on children's behavioral data according to claim 1, characterized in that, The method also includes privacy-level processing of children's behavioral data, which includes primary data, secondary data and tertiary data; The primary data includes raw video, raw audio, images of children's faces, and images of the home space, which are processed locally by default. Secondary data includes the structured behavioral tags and the real-time status, used to generate the historical growth profile locally; The three-level data includes growth summaries, task statistics, and trend reports, which can be synchronized to the parents' devices.

8. An AI-powered companionship and growth system based on children's behavioral data for implementing the method as described in any one of claims 1-7, characterized in that, include: The environmental sensing module is used to detect whether a child is present in the home environment and whether the surrounding environment is safe through multimodal sensors; The behavior acquisition module is used to collect multimodal behavior data of children when the environment is safe, and to perform motion compensation for the robot itself in conjunction with the IMU; The label generation module is used to convert motion-compensated multimodal behavioral data into structured behavioral labels; The status recognition module is used to identify the child's current real-time status based on the behavior tags, preset daily routines, and the child's historical growth profile. The profile management module is used to read the historical growth profile and incrementally update the historical growth profile in combination with the real-time status to generate an updated growth profile. Among them, the robot's companionship strategy, task form, interaction intensity and family routine adaptation method are adjusted in reverse based on the updated growth profile. The corresponding interaction is executed and the feedback data of the child is collected and fed back to the growth profile to complete the iterative closed loop, so as to realize the functions of the robot to understand the child in the long term, actively approach the child and dynamically adapt to the child's state.

Images