Desktop self-balancing education programming robot based on AI interaction

Abstract

The invention discloses a desktop self-balancing education programming robot based on AI interaction. The system comprises the following modules: A, a visual perception module; b, a voice module; d, data transmission; e, an attitude sensing unit; f, a balance control unit; and G, a motion execution unit. The method has the advantages that natural language instructions, visual programming card recognition and automatic logic generation are fused, human-like communication type programming interaction is achieved, the learning threshold is remarkably lowered, and the method is suitable for being used by all-age users, especially low-age users; through real-time visual identification of a specific programming card and dynamic combination with a voice instruction / programming task, a physical-digital fusion teaching mode which can be flexibly configured and is highly situational is created, and immersion and interactivity of learning are greatly enhanced.

Description

Technical Field

[0001] This invention relates to the field of intelligent educational robot technology, and in particular to a desktop self-balancing educational programming robot based on AI interaction. Background Technology

[0002] Current educational robots on the market still have certain limitations in terms of intelligent interaction, teaching adaptability, user experience, and motor skills. First, existing robots mostly rely on traditional programming commands as their primary interaction method, lacking natural language understanding capabilities, offering limited human-computer interaction, and having a high learning threshold, thus restricting the user experience for non-professional users, especially young children. Second, teaching models often depend on fixed programming software or preset interfaces, lacking support for multimodal inputs such as speech and image recognition, making it difficult to meet the intelligent perception and interaction needs in dynamic teaching scenarios. Furthermore, existing programming-based learning processes are often mechanical and lack engagement, failing to effectively stimulate learning interest and sustained participation. Additionally, some educational robots use tracked or fixed chassis structures, resulting in weak autonomous movement capabilities and a lack of autonomous balance and highly flexible movement, limiting their adaptability and interactive experience in complex learning environments. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a desktop self-balancing educational programming robot based on AI interaction that features natural interaction, multimodal fusion, high flexibility, AI closed-loop guidance, and intelligent scalability.

[0004] To address the aforementioned technical problems, the AI-interactive desktop self-balancing educational programming robot of the present invention includes the following modules: A. Visual perception module: Uses a camera to capture, locate and scan icon cards on the desktop in real time, while also taking into account user gesture recognition; a. Icon recognition: For the two specific instruction icons, left turn and straight ahead, the following process is performed: a1. Icon edge and contour positioning: After the camera captures the image, the rectangular boundary of the card is first identified by the edge detection algorithm to eliminate the interference of the desktop background and determine the effective area where the left turn and straight icons are located; then the contour extraction technology is used to capture the contour features of the core graphic of the icon and form a digital coordinate matrix of the graphic contour. a2. Icon feature matching and instruction determination: Call the standard template in the preset instruction icon feature library, and compare the real-time collected icon outline with the standard template through the template matching algorithm. When the matching degree exceeds 95%, the type of the currently scanned icon can be determined. a3. Digital Conversion of Command Information: After determining the icon type, the module will further extract the parameter information associated with the icon, extract the parameter values ​​through image text recognition technology, and finally integrate the icon type and parameter information into a standardized digital signal; B. Voice module: Uses a microphone array and voice recognition unit to capture user voice commands; C. AI Intent Understanding and Logic Generation Unit: It can be flexibly deployed on local edge computing modules or cloud servers to receive natural language input from the voice module and icon command input from the visual perception module. Through the deep integration of deep learning and semantic analysis technologies, it can understand the user's programming intent and dynamically transform it into a standardized programming logic structure and executable code. c1. In-depth analysis of multimodal input information: ① Voice input parsing: First, the audio data transmitted by the voice module is preprocessed. Environmental noise is filtered out by noise reduction algorithm. Then, the audio signal is converted into text information using a speech-to-text model based on the Transformer architecture. Subsequently, a pre-trained natural language processing model is called to perform semantic word segmentation, part-of-speech tagging and intent extraction on the text. The parsing results are marked as speech semantic data, and the semantic relationships are recorded simultaneously. ② Visual icon instruction parsing: Receive the icon digitization signal transmitted by the visual perception module, and match and verify it with the preset icon instruction feature library through the convolutional neural network model to further confirm the instruction type and parameter accuracy corresponding to the icon; at the same time, combine the icon placement sequence recorded by the visual sensor to determine the temporal logical relationship between icon instructions, and mark the parsing result as visual icon data. c2. Dynamic fusion of multimodal information: The unit constructs a multimodal information fusion model through an attention mechanism to associate and integrate speech semantic data and visual icon data in real time. When both the visual perception module and the speech module are received simultaneously, the fusion model will identify the relationship between speech and actions in the visuals through semantic similarity calculation and temporal logic matching. The order of the visual icons is the order of action execution, and finally the fusion forms a unified programming intent. c3. Programming Logic Structure and Executable Code Generation: Through an end-to-end transformation process of intent-logic-code, the program realizes the transformation of natural semantics and visual symbols into programming results; based on the fused programming intent, a deep learning sequence generation model is invoked to automatically construct a standardized programming logic structure. D. Data transmission: Through skeletal key point detection technology, preset gestures are identified, gesture actions are converted into control signals, and the icon type and parameter information are integrated into standardized digital signals and gesture control signals, which are transmitted in real time to the AI ​​intent understanding and logic generation unit as the core data of the visual dimension of multimodal input, providing visual instruction support for subsequent programming logic generation; E. Attitude sensing unit: integrates a high-precision gyroscope and accelerometer to monitor the robot's attitude tilt angle, angular velocity and acceleration in real time; F. Balance control unit: Adopts incremental PID control algorithm, reduces the proportional coefficient in traditional PID algorithm, increases the integral coefficient, sets the derivative coefficient, and introduces tilt angle threshold protection mechanism. When the robot's forward and backward tilt angle exceeds the critical value, it automatically triggers emergency gain adjustment of PID algorithm. G. Motion Execution Unit: Includes drive motors and controllers, which accurately execute movement commands from the balance control unit or the AI ​​intent understanding and logic generation unit to achieve flexible, smooth and stable multi-degree-of-freedom motion.

[0005] In step A, the standard outline templates of left turn and straight go icons are pre-stored in the instruction icon feature library.

[0006] The specific calculation process of the real-time balance control unit in step F is as follows: First, taking the robot's vertical standing posture as the target balance posture, the difference between the real-time tilt angle θ transmitted by the posture sensing unit and the target angle 0° is used as the input deviation value e = θ - 0° for the PID algorithm; subsequently, the PID controller calculates the output control quantity U based on the deviation value, using the following formula: U = Kp×e + Ki×∫e dt + Kd×de / dt Where Kp is the proportional coefficient, Ki is the integral coefficient, Kd is the differential coefficient, ∫e dt is the integral value of the deviation, and de / dt is the rate of change of the deviation; Finally, the control quantity U is converted into speed adjustment commands and torque adjustment commands for the drive wheels. When the robot tilts forward (θ is a positive value), the forward speed of the two drive wheels is increased to drive the robot forward, while the motor output torque is increased to counteract the forward tilt inertia. When the robot tilts backward (θ is a negative value), the reverse speed of the two drive wheels is increased to drive the robot backward, and the torque is adjusted synchronously to match the backward tilt force. If the robot needs to turn (i.e., the gyroscope detects the angular velocity around the vertical axis), the speed difference between the two drive wheels is calculated on the basis of balance control to ensure that balance is maintained during turning. The entire calculation process achieves real-time dynamic adjustment.

[0007] The movement instructions in step G come from two sources: I. From the real-time balance control unit: When the robot needs to correct its balance by moving due to tilting, the balance control unit will convert the calculated speed adjustment command into a specific motor control signal and transmit it to the motion execution unit. II. From the AI ​​Intent Understanding and Logic Generation Unit: When the user inputs programming intent via voice or icons, the executable code generated by the AI ​​unit is converted into standardized movement instructions and transmitted to the motion execution unit via serial communication. After receiving the instructions, the motor controller precisely adjusts the voltage and current of the motor through PWM technology, controls the drive wheels to run at the instructed speed and direction, and simultaneously provides real-time feedback on the motor's operating status to the balance control unit, forming a closed-loop control to ensure that the motion process is flexible, smooth, and stable, and to achieve multi-degree-of-freedom movements. H. Status Feedback and Communication Module: Includes an RGB ambient light and a Bluetooth / Wi-Fi module. The RGB ambient light is located on the front of the robot and provides intuitive feedback on the system status through different colors and flashing modes. The Bluetooth / Wi-Fi module supports connection with mobile phones, tablets, computers, or wireless controllers to realize remote command input, programming logic transmission, teaching trajectory control, and robot status monitoring.

[0008] Advantages of this invention: 1. A natural and low-barrier intelligent programming interaction system: It integrates natural language instructions, visual programming card recognition and automatic logic generation to achieve human-like communication-style programming interaction, significantly reducing the learning threshold and making it suitable for users of all ages, especially young children. 2. Dynamic and contextualized teaching capabilities: By recognizing specific programming cards in real time and dynamically combining them with voice commands / programming tasks, a flexible and highly contextualized physical-digital fusion teaching model is created, greatly enhancing the immersion and interactivity of learning; 3. Revolutionary motion interaction experience: The high-performance two-wheeled self-balancing technology is systematically applied to the desktop educational robot. The flexible, stable and expressive dynamic motion capability it brings not only solves the problem of clumsy movement of traditional educational robots, but also presents the complex control principle in an intuitive and fun way, significantly improving user participation and learning interest. 4. Closed-loop intelligent learning assistance: The built-in AI voice assistant can provide real-time, intelligent conversational guidance and feedback throughout the entire programming process, forming an effective learning loop and improving learning efficiency and depth of understanding. 5. Multimodal synergy: The high degree of synergy and complementarity between voice, vision, touch (screen), light feedback and dynamic motion multimodal technologies jointly constructs an intelligent learning system with high interaction efficiency, rich information transmission and pleasant user experience; 6. High flexibility and scalability: Dual-mode programming support and wireless communication capabilities enable the system to adapt to the needs of different user groups and learning stages, and have good scalability potential. Attached Figure Description

[0009] Figure 1This is a programming principle diagram of the first application of the AI-interactive desktop self-balancing educational programming robot of the present invention. Figure 2 This is a schematic diagram of the programming principle of the second application of the desktop self-balancing educational programming robot based on AI interaction of the present invention. Detailed Implementation

[0010] The following detailed description of the AI-interactive desktop self-balancing educational programming robot of the present invention, in conjunction with the accompanying drawings and specific embodiments, provides further insight. Example

[0011] The AI-interactive desktop self-balancing educational programming robot includes the following modules: 1. Multimodal Interaction and AI Understanding Subsystem: (1) Natural language speech interaction module: includes a microphone array (supporting 360° far-field sound pickup) and a speech recognition unit; this module captures user voice commands (such as "make the robot turn 90 degrees to the right"); (2) Visual perception module: It uses a high-definition industrial-grade camera as the core hardware, and is equipped with 1080P resolution and 30 frames / second real-time acquisition parameters. It can locate and scan the icons and cards on the desktop, and also take into account the user gesture recognition function, providing visual data support for multimodal input. I. Icon Recognition Process: For the two specific instruction icons, left turn and straight ahead, the module completes the recognition process through three steps: "feature extraction - matching verification - parameter conversion". Step 1: Positioning the icon edges and outlines After the camera captures the image, it first uses an edge detection algorithm (such as the Canny operator) to identify the rectangular boundary of the card, eliminates desktop background interference, and determines the effective area where the left turn and straight icons are located. Then, it uses contour extraction technology to capture the contour features of the core graphic of the icon—the left turn icon (such as a graphic with a "left arrow + turning arc") and the straight icon (such as a "horizontal right straight arrow"), forming a digital coordinate matrix of the graphic contour. Step 2: Icon Feature Matching and Command Determination The system calls a pre-defined "instruction icon feature library," which stores standard outline templates for left-turn and straight-ahead icons (such as the arrow angle and turning radius parameters for left-turn icons, and the arrow length and line thickness parameters for straight-ahead icons). Using a template matching algorithm (such as an SSD-based object detection model), the system compares the real-time captured icon outlines with the standard templates. When the matching degree exceeds 95%, the system can accurately determine the type of icon being scanned (left-turn or straight-ahead). Step 3: Digital Conversion of Instruction Information After determining the icon type, the module further extracts the parameter information associated with the icon (if the icon is labeled with parameters such as "moving distance" or "turning angle", such as "90°" next to the left turn icon and "10 cm" next to the straight icon), extracts the parameter values ​​through image text recognition (OCR) technology, and finally integrates "icon type + parameter information" into standardized digital signals (such as "left turn instruction -90°" and "straight instruction -10 cm"). II. Accessibility Functions and Data Transmission: In addition to icon recognition, the module retains gesture recognition functionality: through skeletal keypoint detection technology, it recognizes preset gestures such as "start scanning" and "switch icons," converting the gestures into control signals such as "start recognition" and "stop recognition." These icon command signals and gesture control signals are transmitted in real time to the AI ​​intent understanding and logic generation unit, serving as core visual data for multimodal input and providing precise visual instruction support for subsequent programming logic generation (such as the logical combination of "turn 90° left and then go straight for 10 centimeters").

[0012] (3) AI Intent Understanding and Logic Generation Unit: This unit can be flexibly deployed on local edge computing modules or cloud servers. Its core function is to receive natural language input from the voice module and icon instruction input from the visual perception module (such as left turn icon and straight icon). Through the deep integration of deep learning and semantic analysis technology, it can accurately understand the user's programming intent and dynamically transform it into a standardized programming logic structure and executable code.

[0013] I. In-depth analysis of multimodal input information Voice input parsing: First, the audio data transmitted by the voice module is preprocessed. Environmental noise is filtered out using noise reduction algorithms (such as wavelet thresholding). Then, a speech-to-text (ASR) model based on the Transformer architecture is used to accurately convert the audio signal into text information (such as "loop 5 times" and "advance 10 cm each time"). Subsequently, a pre-trained natural language processing (NLP) model (such as the BERT fine-tuning model) is called to perform semantic segmentation, part-of-speech tagging, and intent extraction on the text. For example, "operation type: loop, loop parameter: 5 times" is extracted from "loop 5 times", and "action type: straight, action parameter: 10 cm" is extracted from "advance 10 cm each time". The parsing results are labeled as "voice semantic data", and the semantic relationships are recorded simultaneously (such as "loop operation includes straight action"). Visual icon instruction parsing: The system receives digitized icon signals (e.g., "left turn instruction -90°" and "straight ahead instruction -10 cm") transmitted from the visual perception module. These signals are then matched against a pre-set "icon instruction feature library" using a convolutional neural network (CNN) model to further confirm the instruction type and parameter accuracy corresponding to the icon (e.g., verifying whether the 90° turning parameter of the left turn icon conforms to the standard instruction range). Simultaneously, by combining the icon placement sequence recorded by the visual sensor (e.g., scanning the straight ahead icon first, then the left turn icon), the system determines the temporal logical relationship between icon instructions (e.g., "straight ahead first, then left turn"), and marks the parsing results as "visual icon data."

[0014] II. Dynamic fusion of multimodal information The unit constructs a multimodal information fusion model through an attention mechanism to associate and integrate "voice semantic data" and "visual icon data" in real time. For example, when simultaneously receiving "straight ahead -10 cm" and "turn left -90°" instructions from the visual perception module and "repeat this sequence 3 times" from the voice module, the fusion model will identify the association between "repeat 3 times" in the voice and "straight ahead + turn left" actions in the vision through semantic similarity calculation and temporal logic matching. It also identifies that there is a "repeat containing action sequence" relationship between the "repeat 3 times" in the voice and the "straight ahead + turn left" actions in the vision. Furthermore, the visual icon placement order "straight ahead first, then turn left" is the action execution order. Finally, the fusion forms a unified programming intent: "repeat 3 times to execute the action sequence 'straight ahead 10 cm → turn left 90°'". III. Programming Logic Structure and Executable Code Generation This is the core innovation of the unit. Through an end-to-end transformation process of "intent-logic-code", it realizes the transformation of natural semantics and visual symbols into programming results: Programming logic structure generation: Based on the fused programming intent, a deep learning sequence generation model (such as the LSTM sequence model) is invoked to automatically construct a standardized programming logic structure. For example, for a loop intent, a complete loop structure is generated, including "loop initialization (e.g., i=0), loop condition (e.g., i<3), loop body (go straight for 10 cm → turn left 90°), loop iteration (e.g., i=i+1)".

[0015] (4) AI voice assistant module: Based on the output of the above understanding unit, it generates natural language voice feedback in real time to explain programming logic, answer user questions, provide operation guidance (such as "Okay, I will move forward 10 times in a loop, 5 centimeters each time"), and supports multi-language switching; the display screen is used to synchronously display the generated code, AI dialogue text, dynamic teaching animation and robot status information.

[0016] 2. Self-balancing motion control subsystem: (1) Two-wheel self-balancing wheel chassis: The chassis adopts a two-wheel symmetrical stable structure as the core mobile platform of the robot. The overall design follows the principle of "central center of gravity and balanced force" to meet the stable movement requirements of desktop educational robots.

[0017] (2) Attitude sensing unit: integrates high-precision gyroscope and accelerometer to monitor the robot's attitude tilt angle, angular velocity and acceleration in real time.

[0018] This unit integrates a high-precision MEMS gyroscope and a three-axis accelerometer into a single chip (model: LSM6DS0TR). This chip features low power consumption and high stability, and its sampling frequency can be flexibly adjusted to 100Hz (to meet the posture monitoring needs of desktop robots). It can collect posture data of the robot in three-dimensional space in real time. The gyroscope module is responsible for monitoring the angular velocity (unit: ° / s) of the robot around the vertical axis (steering axis) and the forward and backward tilting axis. The measurement range supports ±250° / s to ±2000° / s adjustable. For desktop robots with slight tilting scenarios, the default range is set to ±500° / s, which can accurately capture the rate of change of subtle angles when the robot tilts or turns. The accelerometer module monitors the linear acceleration (unit: m / s²) of the robot in the horizontal direction (forward / backward) and the vertical direction (slight up and down bumps). The range supports ±2g to ±16g adjustable, with the default range of ±4g. It can sensitively identify the bump amplitude of the desktop within 2mm, and help judge the robot's motion status and the flatness of the desktop. In addition, the LSM6DS0TR chip has a built-in temperature compensation function, which can counteract the interference of desktop ambient temperature changes (such as the normal indoor temperature difference of 15℃~35℃) on attitude data and ensure data accuracy. The chip connects to the real-time balance control unit through the I2C communication protocol, and the transmission delay is controlled within 10ms. The acquired attitude tilt angle (such as forward and backward tilt angle ±5°), angular velocity, and acceleration data can be synchronized in real time, providing a high-frequency, accurate and stable data source for balance control.

[0019] (3) Real-time balance control unit: The core innovation of this unit lies in adapting the incremental PID control algorithm (proportional-integral-derivative control) to the desktop educational robot scenario. The specific application and calculation process are as follows: Algorithm Application: Considering the characteristics of lightweight desktop robots (total weight ≤500g), low load, and minimal desktop vibration (≤2mm), the parameters of the traditional PID algorithm are optimized—the proportional coefficient is reduced (P=2.5) to avoid excessive oscillation during balance adjustment, the integral coefficient is increased (I=0.8) to quickly offset the continuous tilt deviation caused by slight desktop vibrations, and the derivative coefficient is set (D=0.3) to suppress the impact of sudden attitude changes. Simultaneously, a "tilt angle threshold protection mechanism" is introduced: when the robot's forward / backward tilt angle exceeds ±8° (approaching the imbalance threshold), the emergency gain adjustment of the PID algorithm is automatically triggered, temporarily increasing the proportional coefficient to 3.2 to quickly correct the attitude.

[0020] The calculation process for drive wheel speed and torque is as follows: First, with the robot's "vertical standing (tilt angle 0°)" as the target balance posture, the difference between the real-time tilt angle (denoted as θ) transmitted by the posture sensing unit and the target angle (0°) is used as the input deviation value of the PID algorithm (e=θ-0°); subsequently, the PID controller calculates the output control quantity (U) based on the deviation value, using the following formula: U=Kp×e+Ki×∫edt+Kd×de / dt (Where Kp is the proportional coefficient, Ki is the integral coefficient, Kd is the derivative coefficient, ∫edt is the integral value of the deviation, and de / dt is the rate of change of the deviation); Finally, the control quantity U is converted into speed adjustment commands and torque adjustment commands for the drive wheels—when the robot tilts forward (θ is a positive value), it is calculated that "the forward speed of the two drive wheels is increased (driving the robot forward), and the output torque of the motor is increased to counteract the forward tilting inertia"; when the robot tilts backward (θ is a negative value), it is calculated that "the reverse speed of the two drive wheels is increased (driving the robot backward), and the torque is adjusted synchronously to match the backward tilting force"; if the robot needs to turn (the gyroscope monitors the angular velocity around the vertical axis), then on the basis of balance control, the "speed difference between the two drive wheels" is additionally calculated (e.g., when turning left, the speed of the left drive wheel is reduced by 20%, and the speed of the right drive wheel is increased by 20%) to ensure that balance is maintained when turning. The entire calculation process is executed once every 10ms to achieve real-time dynamic adjustment.

[0021] (4) Motion Execution Unit: Includes a drive motor and controller, which precisely executes movement commands (such as forward, backward, turning, and rotation) from the balance control unit or AI logic generation unit to achieve flexible, smooth, and stable multi-degree-of-freedom motion. The movement commands it receives come from two sources: I. The first type comes from the real-time balance control unit: When the robot needs to correct its balance by moving due to posture tilt (such as needing to move forward when tilting forward and needing to move backward when tilting backward), the balance control unit will convert the calculated speed adjustment command into a specific motor control signal (such as "positive PWM duty cycle 60%" corresponding to a forward speed of 180rpm) and transmit it to the motion execution unit. II. The second type originates from the AI ​​intent understanding and logic generation unit: When the user inputs programming intent via voice or icons (such as "move forward 10 cm" or "turn left 90°"), the executable code generated by the AI ​​unit is converted into standardized movement commands (such as "move forward command: motor rotates forward for 2 seconds (corresponding to a distance of 10 cm)" or "turn left command: left motor rotates in reverse for 0.5 seconds, right motor rotates forward for 0.5 seconds"), which are then transmitted to the motion execution unit via serial communication. Upon receiving the commands, the motor controller precisely adjusts the motor's voltage and current using PWM technology, controlling the drive wheels to rotate at the commanded speed and direction. Simultaneously, it provides real-time feedback on the motor's operating status (such as actual speed and current value) to the balance control unit, forming a closed-loop control system. This ensures flexible, smooth, and stable movement, enabling multi-degree-of-freedom actions such as forward, backward, turning, and rotation.

[0022] 3. Status feedback and communication module: (1) RGB ambient light: Located on the front of the robot, it provides intuitive feedback on the system status (such as standby, execution, programming success, error warning) through different colors and flashing modes.

[0023] (2) Bluetooth / Wi-Fi module: Supports connection with mobile phones, tablets, computers or wireless controllers to realize remote command input, programming logic transmission, teaching trajectory control and robot status monitoring.

[0024] System collaborative workflow: (1) Users issue commands or display specific programming cards through natural language voice.

[0025] (2) The voice interaction module captures voice, and the visual perception module recognizes cards.

[0026] (3) Information input AI intent understanding and logic generation unit. This unit dynamically integrates multimodal input, understands the overall intent, and generates structured programming logic and executable code. This process is one of the core technological innovations.

[0027] (4) The generated movement commands (such as direction, distance, speed) are sent to the self-balancing motion control subsystem, and the attitude sensing unit monitors the robot's attitude in real time.

[0028] (5) The real-time balance control unit combines the movement command and the current posture data to dynamically calculate and output the drive wheel control command to ensure that it maintains self-balance when performing any action. The application of this dynamic balance capability in desktop educational robots is another core technological innovation.

[0029] (6) The motion execution unit drives the wheels to complete precise movements. At the same time, the AI ​​voice assistant module generates and broadcasts voice feedback based on the current status (understanding intent, code generation, execution in progress, execution result) to provide teaching guidance.

[0030] (7) The display screen synchronously displays relevant information. The RGB ambient light reflects the system status in real time, and users can remotely interact and extend control via Bluetooth / Wi-Fi (such as mobile app programming and gamepad remote control teaching).

[0031] Application Scenario 1: Children can control the robot's movement by using voice commands such as "turn 90 degrees to the right" and adjust the wheel speed by viewing cards for "speed adjustment". The robot responds in real time and executes the actions, with AI voice explaining the programming principles during the process. like Figure 1 As shown, the implementation principle of this scenario is as follows: the robot receives voice commands through a microphone array, recognizes card commands through a camera, automatically generates programming programs through AI analysis and executes corresponding actions. During the execution of actions, AI generates programming logic dialogues that children can understand and presents them in the form of voice and text. Application Scenario 2: Children can control the robot's trajectory using a mobile phone or wireless controller (the robot's camera can transmit images in real time). They can input programming logic commands through their mobile phone or controller. Once completed, the robot will automatically move according to the input programming commands and the taught trajectory, allowing children to participate in programming lessons while having fun. like Figure 2 As shown, the implementation principle of this scenario is as follows: the mobile phone remotely controls the robot via Bluetooth or WiFi. The robot moves by relying on two-wheeled motors and a gyroscope. Programming instructions are inserted through wireless transmission, and the programming program is automatically generated by AI analysis based on the robot's current trajectory. After the remote control teaching is completed, the robot executes the corresponding actions according to the predetermined control trajectory and preset programming instructions.

Claims

1. An AI interaction-based desktop self-balancing educational programming robot, characterized in that, Comprise the following modules: A. Visual perception module: use the camera to collect, locate and scan the icon cards on the table in real time, while taking into account the user's gesture recognition; a. Identify the icon: for the two specific instruction icons of left turn and straight ahead, the following process is carried out: a1. Icon edge and contour positioning: after the camera collects the image, first identify the rectangular boundary of the card through the edge detection algorithm, exclude the interference of the desktop background, determine the effective area where the left turn and straight ahead icons are located; then through the contour extraction technology, capture the contour features of the icon core graphics, form the digital coordinate matrix of the graphic contour; a2. Icon feature matching and instruction determination: call the standard template in the preset instruction icon feature library, compare the real-time collected icon contour with the standard template through the template matching algorithm, when the matching degree exceeds 95%, the icon type of the current scanning can be determined; a3. Instruction information digital transformation: after determining the icon type, the module will further extract the parameter information associated with the icon, extract the parameter value through image text recognition technology, and finally integrate the icon type and parameter information into standardized digital signals; B. Voice module: use a microphone array and a voice recognition unit to capture user voice commands; C. AI intent understanding and logic generation unit: can be flexibly deployed in local edge computing module or cloud server, used to receive natural language input from voice module and icon instruction input from visual perception module, through the deep integration of deep learning and semantic analysis technology, understand the user's programming intent, and dynamically convert it into standardized programming logic structure and executable code; c1. Deep analysis of multi-modal input information: ① Voice input analysis: first, pre-process the audio data transmitted by the voice module, filter environmental noise through noise reduction algorithm, then use the speech-to-text model based on Transformer architecture to convert audio signals into text information; then, call the pre-trained natural language processing model to perform semantic segmentation, part-of-speech tagging and intent extraction on the text, and mark the analysis results as voice semantic data, and record the association between semantics; ② Visual icon instruction analysis: receive the icon digital signal transmitted by the visual perception module, match and verify through the convolutional neural network model and the preset icon instruction feature library, further confirm the instruction type and parameter accuracy corresponding to the icon; at the same time, determine the timing logic relationship between icon instructions combined with the timing sequence of icon placement recorded by the visual sensor, mark the analysis results as visual icon data; c2. Dynamic fusion of multi-modal information: the unit constructs a multi-modal information fusion model through attention mechanism, and associates and integrates the voice semantic data and visual icon data in real time; when receiving the visual perception module and the voice module at the same time, the fusion model will identify the association between actions in voice and vision through semantic similarity calculation and timing logic matching, and the order of visual icon placement is the order of action execution, finally form a unified programming intent. c3. Programming logic structure and executable code generation: through the end-to-end conversion process of intent-logic-code, the natural semantics and visual symbols are landed to the programming results; based on the fused programming intent, the deep learning sequence generation model is called to automatically build the standardized programming logic structure; D. Data transmission: through the skeleton key point detection technology, the preset gestures are recognized, the gesture actions are converted into control signals, the icon types and parameter information are integrated into standardized digital signals and gesture control signals, and are transmitted to the AI intent understanding and logic generation unit in real time as the visual dimension core data of the multi-modal input, providing visual instruction support for subsequent programming logic generation; E. Posture sensing unit: integrating high-precision gyroscopes and accelerometers, the posture inclination angle, angular velocity and acceleration of the robot are monitored in real time; F. Balance control unit: using the incremental PID control algorithm, the proportional coefficient in the traditional PID algorithm is reduced, the integral coefficient is improved, the differential coefficient is set, and the inclination threshold protection mechanism is introduced, when the robot is tilted forward or backward beyond the critical value, the emergency gain adjustment of the PID algorithm is automatically triggered; G. Motion execution unit: containing drive motors and controllers, the moving instructions from the balance control unit or the AI intent understanding and logic generation unit are accurately executed to realize flexible, smooth and stable multi-degree-of-freedom motion.

2. The AI interaction based desktop self-balancing educational programming robot according to claim 1, wherein: The standard contour templates of the left turn and straight ahead icons in the instruction icon feature library in step A are pre-stored.

3. The AI interaction based self-balancing educational programming robot for desktops as claimed in claim 1, wherein: The specific calculation process of the real-time balance control unit in step F is as follows: first, taking the vertical standing of the robot as the target balance posture, the difference between the real-time inclination angle θ transmitted by the posture sensing unit and the target angle 0° is taken as the input bias value e = θ - 0° of the PID algorithm; then, the PID controller calculates the output control quantity U according to the bias value, the formula is: U = Kp × e + Ki × ∫e dt + Kd × de / dt Wherein, Kp is the proportional coefficient, Ki is the integral coefficient, Kd is the differential coefficient, ∫e dt is the bias integral value, de / dt is the bias change rate; Finally, the control quantity U is converted into the speed adjustment instruction and torque adjustment instruction of the drive wheel, when the robot tilts forward, i.e. θ is positive, the positive speed of the two drive wheels is calculated to drive the robot forward, and the output torque of the motor is increased to offset the forward inertia; when the robot tilts backward, i.e. θ is negative, the reverse speed of the two drive wheels is calculated to drive the robot backward, and the torque is adjusted synchronously to match the backward force; if the robot needs to turn, i.e. the gyro monitors the angular velocity around the vertical axis, the speed difference of the two drive wheels is calculated additionally based on the balance control to ensure that the balance is maintained during turning, and the whole calculation process realizes real-time dynamic adjustment.

4. The AI interaction based self-balancing educational programming robot for desktops as claimed in claim 1, wherein: The moving instruction sources in step G are divided into two categories: I. From the real-time balance control unit: when the robot needs to correct the balance by moving due to posture inclination, the balance control unit will convert the calculated speed adjustment instruction into specific motor control signals and transmit them to the motion execution unit; II. From AI intent understanding and logic generation unit: when the user inputs the programming intent through voice or icon, the executable code generated by the AI unit is converted into standardized mobile instructions, which are transmitted to the motion execution unit through serial communication; after receiving the instructions, the motor controller accurately adjusts the voltage and current of the motor through PWM technology to control the driving wheel to run at the instructed speed and steering, while real-time feedback of the motor running state to the balance control unit is formed to ensure flexible, smooth and stable movement process, realizing multi-degree-of-freedom action.

5. The AI interaction based self-balancing educational programming robot for desktops as claimed in claim 1, wherein: H. State feedback and communication module: including RGB atmosphere lamp and Bluetooth / Wi-Fi module, the RGB atmosphere lamp is arranged on the front side of the robot, which intuitively feedbacks the system state through different colors and flashing modes; the Bluetooth / Wi-Fi module supports connection with mobile phone, tablet computer, computer or wireless handle, realizing remote instruction input, programming logic transmission, demonstration trajectory control and robot state monitoring.

Images