A projection interactive teaching system and method based on a mobile robot

Abstract

The application discloses a projection interactive teaching system and method based on a mobile robot, solves the problem that traditional equipment is limited in space and cannot follow the movement of children by means of the robot moving unit autonomously navigating and actively seeking users or moving to a specified position, collects depth information by means of a TOF module and automatically calculates an inclination angle, generates a perspective transformation matrix to correct a projection image, realizes clear projection at any angle and overcomes the trapezoidal distortion problem that is easily caused by fixed projection, collects facial and upper limb images in real time by means of a multi-modal perception system, accurately identifies expression states and gesture actions, links these features with a projection task, so that the robot can dynamically adjust subsequent teaching content or robot actions according to real-time feedback of children, and finally generates a personalized learning path by recording learning behavior data of users, and realizes the change from one-way infusion to active adaptation.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to a projection-based interactive teaching system and method using a mobile robot. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, educational robots have been gradually applied to early childhood education, interactive teaching, and entertainment. Existing educational robots or smart projection devices (such as desktop learning machines and fixed projectors) primarily use static displays or simple voice-based question-and-answer modes for interaction. These existing technologies typically fix the display terminal to a desktop, playing preset animations or videos on the screen, or only possessing basic voice recognition capabilities to answer children's questions. While some high-end devices incorporate cameras for facial recognition, these are often limited to simple identity verification or facial expression feedback, lacking in-depth motion capture and real-time interactive capabilities.

[0003] Furthermore, the projection range of existing devices is limited to a fixed location, making it impossible to dynamically adjust the projection area according to the child's activity range. This results in children struggling to achieve an immersive visual experience while moving around. At the same time, existing technologies often lack in-depth analysis of children's behavioral data, failing to generate personalized learning paths in real time based on children's answers, reaction speed, and attention levels. This leads to rigid teaching content, making it difficult to stimulate children's learning interest and easily causing them to forget what they've learned or become bored. Summary of the Invention

[0004] This invention aims to provide a projection interactive teaching system and method based on mobile robots, in order to solve the problems that existing projection interactive teaching devices are mostly fixed or passively responsive, lacking the ability to project dynamically and the multimodal real-time perception and linkage mechanism, resulting in a lack of immersion in the teaching scene, and the inability to dynamically adjust the teaching content based on children's real-time behavioral data (such as facial expressions and gestures) to form a personalized active learning loop.

[0005] To achieve the above objectives, the technical solution adopted by this invention is: a projection-based interactive teaching method using a mobile robot, comprising: S1: The robot navigates autonomously using its mobile unit and actively seeks out users or moves to a designated location to stop according to a preset strategy. S2: When the robot moving unit reaches the target position, it controls the TOF module in the optical-mechanical projection module to collect the depth information of the projection surface, and calculates the tilt angle of the projection surface relative to the projection optical axis based on the depth information collected by the TOF module. S3: Generate a perspective transformation matrix based on the calculated tilt angle, and use the perspective transformation matrix to correct the projected image, then output the corrected image to the optomechanical module for projection display; S4: Acquire facial and upper limb images of children through the visual camera in the multimodal perception system, and use facial key point localization and hand key point extraction algorithms to identify children's facial expressions and gestures. S5: Link the recognized facial expressions and gestures with the current projection task, and adjust the subsequent teaching content or robot actions through the interactive control system based on the linkage results. At the same time, record the user's learning behavior data to generate a personalized learning path.

[0006] Preferably, in S1, actively finding the user is achieved through a UWB module, which includes a UWB positioning module installed on the robot and a UWB positioning module installed on the indoor ceiling or wall area, and the user wears a tag with a built-in battery and charging IC. The label can be one of the following: a bracelet, card, keychain, backpack, or clothing.

[0007] Preferably, in S2, the TOF module uses structured light to perform automatic keystone correction of the projected content, specifically including: selecting four corner or uniformly distributed sampling points to collect the depth of the projection plane, calculating the offset of each corner point, and updating the correction matrix in real time to support fine-tuning of the module or screen position.

[0008] Preferably, in S4, the judgment of the facial expression state is based on the analysis of changes in key facial areas, which is achieved by calculating the change in the corners of the mouth and the aspect ratio (EAR) of the eyes; The recognition of the gestures is achieved by extracting the wrist, fingertips, and knuckles, and calculating the angular relationship between the wrist, fingertips, and knuckles to identify click, wave, drag, or selection actions.

[0009] Preferably, in S4, the multimodal perception system further includes a binocular camera, a DTOF sensor, an ultrasonic radar, and a cliff sensor; The binocular camera is used for navigation, obstacle avoidance, and depth perception, while the cliff sensor is installed near the soles of the feet to monitor changes in height in real time to prevent falls.

[0010] Preferably, in S5, the main control terminal of the interactive control system includes a SOC chip, a serializer chip, and an MCU; The SOC chip is used for multimodal data processing, AI inference and control decision-making, while the MCU is used for low-latency execution of servo motion, fine-tuning of projected position and attitude control.

[0011] Preferably, in S5, the recording of user learning behavior data includes recording the user's answer accuracy, weaknesses, and answer response speed, and generating a real-time learning report that is fed back to the mobile APP.

[0012] Preferably, the method further includes cloud collaboration, specifically including: The recorded data is synchronized to the cloud for AI model training, learning effect evaluation, and learning content optimization, and user privacy is guaranteed through data encryption, access and permission control, or local AI closed-loop methods.

[0013] Preferably, the optical engine projection module further includes a heat dissipation system, which consists of copper pipes, heat dissipation fins and a fan, and adopts a combination of active and passive heat dissipation methods.

[0014] On the other hand, this invention proposes a projection-based interactive teaching system using a mobile robot, comprising: The robot's mobile unit has at least 4 degrees of freedom of movement, enabling autonomous navigation and obstacle avoidance, and is equipped with a UWB module for actively seeking out or following users. An optical-mechanical projection module includes an optical-mechanical module and a TOF module. The TOF module is used to acquire depth information of the projection surface and perform automatic keystone correction and automatic focusing. The multimodal perception system includes a high-definition visual camera, a binocular camera, a DTOF sensor, an ultrasonic radar, a cliff sensor, and an audio module, used to collect the user's facial features, gestures, and environmental depth information. The interactive control system includes a main control terminal and a head-mounted display terminal. The main control terminal includes a SOC chip, a serializer chip, and an MCU, which are used to process multimodal data, run AI inference, and control the output of the projected image and the robot's movements. A power management system is used to provide power management and distribution for mobile robots, projection modules, and multimodal sensors; The teaching closed-loop module is used to store users' behavior, learning, portrait and voice data, and generate personalized learning paths based on the data, and adjust the teaching difficulty in real time. The cloud collaboration module is used to synchronize data to the cloud for AI model training and content optimization, and to implement privacy protection measures.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention solves the problems of limited space and inability to follow children in traditional devices by enabling the robot's mobile unit to autonomously navigate and actively seek out users or move to designated locations. It utilizes a Time-of-Flight (TOF) module to collect depth information and automatically calculate tilt angles, generating a perspective transformation matrix to correct projected images, achieving clear projection at any angle and overcoming the trapezoidal distortion problem common in fixed projections. A multimodal perception system collects facial and upper limb images in real time, accurately identifying facial expressions and gestures, and linking these features with the projection task. This allows the robot to dynamically adjust subsequent teaching content or robot actions based on children's real-time feedback (such as confused expressions or incorrect gestures). Finally, by recording users' learning behavior data, a personalized learning path is generated, realizing a shift from one-way instruction to proactive adaptation. Attached Figure Description

[0016] Figure 1 The flowchart shows the projection-based interactive teaching method of the present invention based on a mobile robot. Figure 2 This is a block diagram of the projection interactive teaching system based on a mobile robot according to the present invention; Figure 3 This is a hardware architecture diagram of the present invention; Figure 4 This is a schematic diagram of the mobile robot structure of the present invention; Figure 5 This is a schematic diagram illustrating the motion capture principle of the present invention. Figure 6 This is a flowchart illustrating the learning process of this invention; Figure 7 This is a flowchart illustrating the educational closed-loop process of the present invention. Detailed Implementation

[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0018] like Figure 1 As shown, this invention proposes a projection-based interactive teaching method using a mobile robot, comprising: S1: The robot navigates autonomously via its mobile unit and actively seeks out users or stops at a designated location according to a preset strategy. Actively seeking out users is achieved through a UWB module, which includes a UWB positioning module installed on the robot and another UWB positioning module installed on the indoor ceiling or wall. The user wears a tag with a built-in battery and charging IC. The tag can be a wristband, card, keychain, backpack, or clothing.

[0019] The high-precision positioning and tag recognition achieved through the UWB module enable the robot to accurately perceive the real-time location of the user (child) and actively follow their movement, effectively solving the problem that traditional fixed equipment cannot adapt to the range of children's free movement. At the same time, the solution supports various portable wearing forms (such as wristbands, clothing, etc.), significantly improving the flexibility of teaching scenarios and the convenience of user experience.

[0020] S2: When the robot moving unit reaches the target position, the TOF module in the control optomechanical projection module collects the depth information of the projection surface and calculates the tilt angle of the projection surface relative to the projection optical axis based on the depth information collected by the TOF module; the TOF module uses structured light to perform automatic trapezoidal correction of the projected content, specifically including: selecting four corner or uniformly distributed sampling points to collect the depth of the projection plane, calculating the offset of each corner point, and updating the correction matrix in real time to support fine-tuning of the module or screen position.

[0021] By utilizing a TOF module combined with structured light technology for depth perception and automatic keystone correction, it can adapt to any tilt angle of the wall or ground and changes in projection distance in real time, eliminating image distortion without manual intervention. At the same time, through the acquisition of four corner sampling points and the real-time matrix update mechanism, it supports fine-tuning of the projection position, ensuring that the projected image is always clear, square and sharp at the edges in different scenarios, significantly improving the visual quality and automation of the projection display.

[0022] Furthermore, the optical engine projection module also includes a heat dissipation system, which consists of copper pipes, heat sink fins, and a fan, employing a combination of active and passive cooling methods. By using a dual active and passive cooling mechanism combining copper pipes, heat sink fins, and a fan, the heat accumulation problem of high-power optical engine modules during long-term continuous projection operations is effectively solved.

[0023] S3: Generate a perspective transformation matrix based on the calculated tilt angle, and use the perspective transformation matrix to correct the projected image. Output the corrected image to the optomechanical module for projection display. By generating the perspective transformation matrix in real time to dynamically correct the image, trapezoidal distortion of the image caused by the tilt of the projection surface or changes in the robot's posture can be automatically eliminated. This allows the projected image to present a standard rectangular shape at any angle, with clear and sharp edges, significantly improving the visual experience. At the same time, it achieves adaptive projection correction without manual intervention.

[0024] S4: The visual camera in the multimodal perception system acquires facial and upper limb images of children, and uses facial key point localization and hand key point extraction algorithms to identify children's facial expressions and gestures. The judgment of facial expressions is based on the analysis of changes in key facial regions, which is achieved by calculating the change in the corner of the mouth and the aspect ratio (EAR) of the eyes. The recognition of gestures is achieved by extracting the wrist, fingertips, and knuckles, and calculating the angular relationship between the wrist, fingertips, and knuckles to identify click, wave, drag, or selection actions.

[0025] Furthermore, the multimodal perception system also includes a binocular camera, a DTOF sensor, an ultrasonic radar, and a cliff sensor; wherein, the binocular camera is used for navigation, obstacle avoidance, and depth perception, and the cliff sensor is installed near the soles of the feet to monitor changes in height in real time to prevent falls.

[0026] By integrating facial and hand key point analysis, accurate real-time recognition of children's facial expressions (such as joy, anger, sorrow, and happiness) and gestures (clicking, dragging, etc.) is achieved, creating a natural and smooth human-computer interaction experience. At the same time, by combining a multi-dimensional perception network of binocular cameras, DTOF, ultrasound, and cliff sensors, the robot's navigation and obstacle avoidance capabilities and fall prevention safety in complex environments are greatly improved, and comprehensive monitoring and immediate response to children's behavior in dynamic scenarios are ensured.

[0027] S5: Link the recognized facial expressions and gestures with the current projection task, and adjust the subsequent teaching content or robot actions through the interactive control system based on the linkage results. At the same time, record the user's learning behavior data to generate a personalized learning path.

[0028] Specifically, the main control unit of the interactive control system includes a SOC chip, a serializer chip, and an MCU; wherein, the SOC chip is used for multimodal data processing, AI inference and control decision-making, and the MCU is used for low-latency execution of servo motor movement, fine-tuning of projected position and attitude control.

[0029] The system records user learning behavior data, including the user's answer accuracy, weaknesses, and response speed, and generates real-time learning reports that are fed back to the mobile app.

[0030] Through real-time interaction of facial expressions and gestures, the teaching content is adaptively adjusted and the robot's movements are precisely responded to, significantly enhancing the fun and immersion of the interaction. Combining the high computing power of SOC's AI inference with the low latency of MCU's motion control ensures the smoothness of the system's response. At the same time, based on multi-dimensional data collection of answer accuracy, weaknesses, and reaction speed, personalized learning paths and real-time feedback reports can be automatically generated, effectively supporting a precise teaching model that is tailored to individual needs.

[0031] The above methods also include cloud collaboration, specifically: synchronizing recorded data to the cloud for AI model training, learning effect evaluation, and learning content optimization, and ensuring user privacy through data encryption, access and permission control, or local AI closed-loop methods.

[0032] Through cloud-based collaboration, massive amounts of data are used to continuously iterate and optimize AI models and teaching content, enabling dynamic evaluation of learning outcomes and intelligent upgrading of teaching resources; at the same time, data encryption, access control, and local closed-loop mechanisms are combined.

[0033] On the other hand, this invention proposes a projection-based interactive teaching system using a mobile robot, such as... Figures 2-7 As shown, it includes: The robot mobile unit has at least 4 degrees of freedom of movement, is capable of autonomous navigation and obstacle avoidance, and is equipped with a UWB module for actively finding or following users; its movement mode can be wheeled, legged, wheeled, etc.

[0034] The optical engine projection module includes an optical engine module 2, a Time-of-Flight (TOF) module 1, and a heat dissipation system. The TOF module is used to collect depth information of the projection surface and perform automatic keystone correction and autofocus. The optical engine module is mainly responsible for projecting the educational content, teaching content, game content, training content, and other displayable content such as movies, animations, navigation, and translations. The heat dissipation system consists of two main cooling methods: active cooling and passive cooling. The system comprises copper pipes, heat sink fins, and a fan (active cooling).

[0035] The automatic trapezoidal correction is implemented as follows: Acquire projection plane depth: The TOF board acquires projection plane depth information, and sampling points can be selected at the four corners or evenly distributed.

[0036] 1. Calculate trapezoidal distortion: Based on the depth of the four corners and the position of the module, calculate the tilt angle of the projection surface relative to the projection optical axis to obtain the offset of each corner point. The specific calculation formula is as follows: Assuming the projected optical axis is along the Z-axis, the trapezoidal distortion angle is calculated using the angle θ between the plane normal vector n=(A,B,C) and the optical axis vector v=(0,0,1): θ is the tilt angle of the plane relative to the projection optical axis. Based on θ, the required correction offset of each corner point in the pixel or projection coordinate system can be calculated.

[0037] Specific calculation of the projection plane: The TOF board collects the depths Z1, Z2, Z3, and Z4 at the four corners of the projection plane, corresponding to the upper left (X1, Y1), upper right (X2, Y2), lower left (X3, Y3), and lower right (X4, Y4), respectively. The projection plane can be represented by the plane equation Ax + By + Cz + D = 0. Using the four known points (Xi, Yi, Zi), the least squares method is used to fit the plane, and the plane parameters A, B, C, and D can be obtained. .

[0038] 2. Generate the correction matrix: Based on the projected pixel coordinates and the physical coordinates measured by Time-of-Flight (TOF), a perspective transformation matrix is ​​generated. This matrix is ​​used to correct the projected image and is output to the projection optical system. The specific formula is as follows: Given the pixel coordinates of the original projected rectangle as (x1, y1), ..., (x4, y4); and the coordinates of the actual rectangle after projection onto the inclined plane as (x1′, y1′), ..., (x4′, y4′); calculate using the perspective transformation matrix H:

[0039] By applying H to the projected image, trapezoidal correction can be achieved.

[0040] 3. Output Corrected Image: The corrected image is output to the projection optical module through the projection TOF board, supporting real-time updates and fine-tuning of the module or screen position.

[0041] The multimodal perception system includes a high-definition visual camera 3 (RGB), a binocular camera 5 (RGBD), a DTOF sensor 4, an ultrasonic radar, a cliff sensor 6, UWB (optional), an audio module, etc., used to collect the user's facial features, gestures, and environmental depth information. A combination of ultrasonic radar, DTOF sensor, and binocular camera is used to generate a map of surrounding obstacles; cliff sensor is used to detect elevation differences and ensure safety; UWB provides global position reference. Among them, the high-definition vision camera is mainly responsible for taking pictures, recording videos, facial recognition, and expression recognition. Including gesture capture and other functions. The system uses a high-definition head-mounted visual camera to capture images of children's faces and upper limbs, and utilizes face detection, key point localization, and feature extraction algorithms to achieve related functions: In expression recognition, the system calculates expression features (such as changes in the corners of the mouth, the degree of eye opening and closing, etc., where the degree of eye opening and closing is quantified by the eye aspect ratio EAR); in the identity recognition stage, the face is first extracted based on the key point region. The feature vector (F is the facial feature vector, fn is the feature value of the corresponding region) is then normalized to reduce the influence of lighting and angle. Finally, the current feature vector (Fa) is compared with local or cloud-based features. Cosine similarity is calculated from the features (Fb) in the library. The comparison is performed, and when the similarity S is greater than the recognition threshold T, the child is identified as the same person. The specific process for keypoint localization is as follows: First, a high-definition visual camera acquires an RGB image I(x,y) (x and y are pixel coordinates, and I(x,y) represents the image pixel value); then, a face detection algorithm is used to obtain the face region B=(x,y,w,h) (x and y are the coordinates of the upper left corner of the face bounding box, and w and h are the width and height of the face bounding box); next, keypoints Pi=(xi,yi) are extracted from the face region (Pi represents the i-th keypoint, and xi and yi are the coordinates of the keypoint). The keypoint localization model f and its parameters θ are generated through a regression function. Furthermore, the system determines facial expression by analyzing changes in key facial regions; it also detects and extracts keypoints (Hi=(xi,yi) of the wrist, fingertips, and knuckles) and recognizes gestures such as clicking, waving, dragging, and selecting based on the angular relationships between these keypoints. The joint angle θ is calculated from key points A, B, and C. Gesture capture is achieved by combining deep data fusion, and the recognition results are linked with projection tasks, voice interaction, and robot action feedback. Finally, multimodal fusion (R=g(F,H,V), where F is facial features, H is gesture features, V is voice features, and R is the final interaction result) meets the needs of children's educational scenarios.

[0042] The binocular cameras are primarily responsible for navigation, obstacle avoidance, depth perception, pose estimation, and near-field scene understanding. The system acquires a depth map of the space in front through the chest-mounted binocular cameras, combines this with near-field ranging information supplemented by DTOF sensors, obstacle data detected by ultrasonic radar, ground height abrupt changes monitored by cliff sensors, and IMU and joint state data to achieve robot pose estimation. The control system fuses this data to generate obstacle distances, human position, projection area status, and near-field scene semantic information, thereby completing tasks such as obstacle avoidance, depth perception, pose estimation, and near-field educational interaction scene understanding. Combined with TOF and projection, it enables real-time localization and motion recognition for projection-interactive tasks.

[0043] DTOF sensors are primarily used for navigation, positioning, and obstacle avoidance when binocular cameras are ineffective, typically employing an 8×8 or larger area array TOF. Ultrasonic radar mainly assists robots in autonomous navigation, preventing collisions with walls or obstacles and ensuring safe movement. Cliff sensors for four-legged robots are installed near the soles of each leg to monitor height changes in real time, providing fall prevention and terrain boundary protection. Their working principle is as follows: a built-in transmitting module emits a 940nm infrared laser downwards; a ranging module captures the laser reflected from the ground using a SPAD chip, accurately calculating the sensor's actual distance from the ground using the laser's round-trip time; the system presets a safe ground clearance threshold of 8-15mm; when the measured distance exceeds the threshold, it is determined to be suspended in mid-air or a cliff; finally, the distance data is transmitted to the CMS via I2C or other communication methods, which then controls the robot's movements.

[0044] The audio module is used for functions such as sound pickup and sound source localization, supporting voice interaction; some mobile robots are equipped with LiDAR for navigation and mapping; UWB modules can also be configured for indoor positioning.

[0045] The interactive control system includes a main control terminal and a head-mounted display terminal. The main control terminal includes a SOC chip (composed of RK3588, ORINNX, etc. and their peripheral circuits), a serializer chip (SerDes architecture), and an MCU, which are used to process multimodal data, run AI inference, and control the output of the projected image and the robot's movements. The SOC primarily handles multimodal data processing, AI inference and control decision-making, image / video processing and encoding, and communication with the serializer chip. Multimodal data processing is used for real-time fusion and preprocessing of camera, binocular, voice, and spatial information. The AI ​​inference module runs large-model inference and educational task logic, including projection content generation, learning task guidance, gamified learning strategy formulation, and behavioral decision-making. Image / video processing and encoding are responsible for encoding, decoding, and rendering of head-mounted camera video and projection content, while controlling the output of the projection screen to ensure low latency and high frame rate. If a SerDes architecture is adopted, the SOC also needs to transmit head-mounted display video, projection screen, and control signals through communication with the serializer to ensure synchronization between projection display and movement. The MCU is mainly responsible for low-latency control execution, such as servo movement, fine-tuning of projection position, attitude control, and sensor sampling.

[0046] A power management system is used to manage and distribute power for mobile robots, as well as to supply power to projection modules, multimodal sensors, and other components. The teaching closed-loop module is used to store users' behavior, learning, portrait and voice data, and generate personalized learning paths based on the data, and adjust the teaching difficulty in real time. The teaching closed-loop module mainly consists of a software system and storage. From the moment a child first comes into contact with the robot, the robot records all the child's behavior, learning, appearance, voice and other data and stores them on the robot itself. Based on the child's behavior data, it generates a personalized learning path, adjusts the teaching difficulty and task content in real time, records the learning effect, and feeds the effect back to the parent's mobile APP.

[0047] The cloud collaboration module is used to synchronize data to the cloud for AI model training and content optimization, and to implement privacy protection measures.

[0048] In summary, this invention enables the autonomous movement of robots to dynamically present educational content, including entertainment content, in space. It can also adjust the projection position and content in real time in any space according to the child's position or by adjusting the head movement angle and the robot's flexible rotation, breaking through the limitations of traditional fixed terminals.

[0049] The system can generate personalized tasks based on real-time learning behavior and posture assessments, integrating reward mechanisms and growth paths to enable children to learn in an immersive and captivating way, gain a corresponding sense of accomplishment, and help the robot grow through their own abilities, achieving mutual growth and forming long-term dependence.

[0050] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A projection-based interactive teaching method using a mobile robot, characterized in that, include: S1: The robot navigates autonomously using its mobile unit and actively seeks out users or moves to a designated location to stop according to a preset strategy. S2: When the robot moving unit reaches the target position, it controls the TOF module in the optical-mechanical projection module to collect the depth information of the projection surface, and calculates the tilt angle of the projection surface relative to the projection optical axis based on the depth information collected by the TOF module. S3: Generate a perspective transformation matrix based on the calculated tilt angle, and use the perspective transformation matrix to correct the projected image, then output the corrected image to the optomechanical module for projection display; S4: Acquire facial and upper limb images of children through the visual camera in the multimodal perception system, and use facial key point localization and hand key point extraction algorithms to identify children's facial expressions and gestures. S5: Link the recognized facial expressions and gestures with the current projection task, and adjust the subsequent teaching content or robot actions through the interactive control system based on the linkage results. At the same time, record the user's learning behavior data to generate a personalized learning path.

2. The projection-based interactive teaching method based on a mobile robot according to claim 1, characterized in that, In S1, actively finding users is achieved through a UWB module, which includes a UWB positioning module installed on the robot and a UWB positioning module installed on the indoor ceiling or wall area, and the user wears a tag with a built-in battery and charging IC. The label can be one of the following: a bracelet, card, keychain, backpack, or clothing.

3. The projection-based interactive teaching method according to claim 1, characterized in that, In S2, the TOF module uses structured light to perform automatic keystone correction of the projected content, specifically including: selecting four corner or uniformly distributed sampling points to collect the depth of the projection plane, calculating the offset of each corner point, and updating the correction matrix in real time to support fine-tuning of the module or screen position.

4. The projection-based interactive teaching method according to claim 1, characterized in that, In S4, the judgment of the facial expression state is based on the analysis of changes in key facial areas, which is achieved by calculating the change in the corners of the mouth and the aspect ratio (EAR) of the eyes. The recognition of the gestures is achieved by extracting the wrist, fingertips, and knuckles, and calculating the angular relationship between the wrist, fingertips, and knuckles to identify click, wave, drag, or selection actions.

5. The projection-based interactive teaching method according to claim 1, characterized in that, In S4, the multimodal perception system also includes a binocular camera, a DTOF sensor, an ultrasonic radar, and a cliff sensor; The binocular camera is used for navigation, obstacle avoidance, and depth perception, while the cliff sensor is installed near the soles of the feet to monitor changes in height in real time to prevent falls.

6. The projection-based interactive teaching method based on a mobile robot according to claim 1, characterized in that, In S5, the main control terminal of the interactive control system includes a SOC chip, a serializer chip, and an MCU; The SOC chip is used for multimodal data processing, AI inference and control decision-making, while the MCU is used for low-latency execution of servo motion, fine-tuning of projected position and attitude control.

7. The projection-based interactive teaching method according to claim 1, characterized in that, In S5, the data recorded on the user's learning behavior includes recording the user's answer accuracy, weaknesses, and answer response speed, and generating a real-time learning report that is fed back to the mobile APP.

8. The projection-based interactive teaching method according to claim 1, characterized in that, The method also includes cloud collaboration, specifically including: The recorded data is synchronized to the cloud for AI model training, learning effect evaluation, and learning content optimization, and user privacy is guaranteed through data encryption, access and permission control, or local AI closed-loop methods.

9. The projection-based interactive teaching method according to claim 1, characterized in that, The optical engine projection module also includes a heat dissipation system, which consists of copper pipes, heat dissipation fins and a fan, and adopts a combination of active and passive heat dissipation methods.

10. A mobile robot-based projection interactive teaching system for implementing the method as described in any one of claims 1-9, characterized in that, include: The robot's mobile unit has at least 4 degrees of freedom of movement, enabling autonomous navigation and obstacle avoidance, and is equipped with a UWB module for actively seeking out or following users. An optical-mechanical projection module includes an optical-mechanical module and a TOF module. The TOF module is used to acquire depth information of the projection surface and perform automatic keystone correction and automatic focusing. The multimodal perception system includes a high-definition visual camera, a binocular camera, a DTOF sensor, an ultrasonic radar, a cliff sensor, and an audio module, used to collect the user's facial features, gestures, and environmental depth information. The interactive control system includes a main control terminal and a head-mounted display terminal. The main control terminal includes a SOC chip, a serializer chip, and an MCU, which are used to process multimodal data, run AI inference, and control the output of the projected image and the robot's movements. A power management system is used to provide power management and distribution for mobile robots, projection modules, and multimodal sensors; The teaching closed-loop module is used to store users' behavior, learning, portrait and voice data, and generate personalized learning paths based on the data, and adjust the teaching difficulty in real time. The cloud collaboration module is used to synchronize data to the cloud for AI model training and content optimization, and to implement privacy protection measures.

Images