Interactive AI teaching feedback system based on behavior data analysis

Abstract

The invention discloses an interactive AI teaching feedback system based on behavior data analysis, and relates to the technical field of artificial intelligence education. Comprising a hardware acquisition module, a data analysis module, a teaching guide module and a feedback execution module. The system is connected with various sensors through the Raspberry Pi core controller to collect student operation data, processes the data by using the DFCNN + Transform speech recognition unit and the Tiny YOLO target detection unit, and performs correlation analysis on operation result data and behavior process data through the time sequence behavior analysis unit. And based on an analysis result, the teaching guide module dynamically generates a personalized teaching strategy, and the feedback execution module provides immersive teaching feedback through a multi-mode cooperation mode of Scratch visual prompt, intelligent car action feedback and voice synthesis. According to the invention, deep analysis of learning behaviors, intelligent teaching guidance and multi-mode collaborative feedback are realized, and the effect of artificial intelligence practical teaching and the learning experience of students are effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence education technology, and in particular relates to an interactive AI teaching feedback system based on behavioral data analysis. Background Technology

[0002] With the popularization of artificial intelligence education in primary and secondary schools, existing AI teaching systems have gradually integrated hardware peripherals, graphical programming environments, and AI recognition modules. Most mainstream systems currently use hardware such as Raspberry Pi as the core controller, paired with peripherals such as cameras and microphones, and combined with the Scratch graphical programming environment. They integrate core technologies such as the DFCNN+Transformer speech recognition model and the Tiny YOLO object detection algorithm, achieving basic functions such as voice command response and real-time object detection.

[0003] However, existing technologies still have significant drawbacks: The data utilization is superficial: it only collects and processes operation result data (such as speech recognition results and target detection results), lacks in-depth correlation analysis of behavioral process data (such as operation time, error retry frequency, and module switching trajectory), and cannot fully depict the students' learning status. The feedback mechanism is too simple: the interactive feedback is often limited to a single modality (such as only voice prompts or only interface prompts), and there is a lack of effective coordination between multimodal feedback, resulting in a stiff interactive experience and insufficient immersion. Instructional guidance is rigid: The teaching content and path are usually pre-set, lacking the ability to dynamically adjust based on real-time student behavior data, as well as the ability to intelligently link across knowledge points and generate tasks; Insufficient system coordination: The hardware, algorithms, and application layers are tightly coupled, resulting in poor scalability. Furthermore, the real-time performance of key modules (such as speech recognition and object detection) needs further improvement to ensure the smoothness of the teaching process.

[0004] These shortcomings limit the further improvement of the personalization level and teaching effectiveness of AI teaching systems. Therefore, there is an urgent need in this field for an interactive AI teaching system that can deeply integrate behavioral data analysis and provide intelligent guidance and collaborative feedback. Summary of the Invention

[0005] The purpose of this invention is to overcome the aforementioned deficiencies of the prior art and provide an interactive AI teaching feedback system based on behavioral data analysis. This system aims to generate personalized teaching guidance strategies through in-depth correlation analysis of students' multi-dimensional behavioral data, and to enhance the effectiveness, relevance, and student engagement of teaching through multimodal collaborative interactive feedback.

[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: This invention provides an interactive AI teaching feedback system based on behavioral data analysis, comprising: The hardware acquisition module includes a control unit based on a Raspberry Pi, and connected to it a camera, microphone, and an inertial sensor integrated into the smart car for collecting student operation data. The data analysis module integrates a DFCNN+Transformer speech recognition unit for processing voice commands and a Tiny YOLO-based target detection unit for processing visual operation data. It also includes a temporal behavior analysis unit for performing correlation analysis on operation result data from the speech recognition unit and the target detection unit, as well as behavior process data consisting of operation duration and error retry frequency. The teaching guidance module is used to dynamically generate personalized teaching guidance strategies for Scratch programming or smart car operation practice based on the correlation analysis results of the temporal behavior analysis unit. The feedback execution module is used to execute the teaching guidance strategy and to perform multimodal interaction by integrating the following methods: providing visual prompts based on analysis results in the Scratch graphical programming environment, driving the smart car to perform specific actions that represent the correctness of the operation or provide demonstrations, and generating speech synthesis prompts that coordinate with the visual prompts and the smart car's actions.

[0007] Preferably, the timing behavior analysis unit adopts a sliding window mechanism with a window duration of 5-10 seconds to perform real-time timing correlation analysis on the module switching sequence of students in the Scratch programming environment and the control instruction sequence in the operation of the smart car, so as to identify operating habits and cognitive impairment patterns.

[0008] Preferably, the teaching guidance module includes a step-by-step error operation guidance unit, a difficult function assistance unit, and a personalized task push unit, which are used to generate step-by-step operation guidance animations, usage skill examples, and targeted practice tasks in specific scenarios.

[0009] Preferably, in the feedback execution module, the specific actions of the smart car include preset light flashing patterns, turning actions, or travel trajectories, in order to provide immediate and concrete physical feedback on the student's operation results.

[0010] Preferably, the data analysis module uses the Tengine inference framework to accelerate the forward inference of the DFCNN+Transformer speech recognition unit and the Tiny YOLO-based object detection unit, so that the end-to-end latency of speech recognition and object detection is less than 100 milliseconds, in order to meet the real-time requirements of teaching interaction.

[0011] Preferably, the system adopts a hybrid storage architecture, using a local SD card to cache high-frequency behavioral data collected in real time, and using a cloud-based distributed database to store structured user information, course content, and unstructured operation logs; wherein, the cloud-based database provides historical data support for the time-series behavioral analysis unit.

[0012] Preferably, the system also includes a teacher interaction interface, which generates a heatmap of the overall learning behavior of the class and a report on the behavior trajectory of individual students based on the output of the time-series behavior analysis unit, and allows teachers to manually trigger, adjust or override strategies automatically generated by the teaching guidance module based on the report.

[0013] Preferably, the teaching guidance module also integrates a knowledge graph unit, which semantically associates Scratch programming knowledge points with intelligent vehicle operation skills, and dynamically generates interdisciplinary composite teaching tasks based on the association analysis results of the temporal behavior analysis unit.

[0014] Preferably, the feedback execution module is equipped with a multimodal synchronization unit. The multimodal synchronization unit uses timestamp alignment technology to ensure that Scratch visual prompts, smart car actions and voice synthesis prompts achieve millisecond-level coordinated output, so as to provide a coherent and immersive interactive feedback experience.

[0015] Preferably, the hardware acquisition module is further provided with a peripheral management unit for unified management of the driver interfaces of the camera, microphone and smart car, and supports plug-and-play and data synchronization of the peripherals in the Scratch programming environment.

[0016] The present invention has the following beneficial effects: This invention achieves precise assessment of learning status and personalized teaching guidance through deep correlation analysis of student operation result data and behavioral process data. The system innovatively integrates knowledge graph technology, enabling dynamic generation of composite teaching tasks across Scratch programming and intelligent vehicle operation, effectively promoting the cultivation of students' comprehensive abilities. Through a multimodal synchronization unit, it achieves millisecond-level collaborative feedback of visual prompts, hardware actions, and speech synthesis, greatly enhancing the coherence and immersion of teaching interaction. The Tengine inference framework is used to accelerate the core AI algorithm, ensuring real-time response performance in complex teaching scenarios. Simultaneously, the hardware abstraction layer design and hybrid storage architecture provide the system with excellent scalability and maintainability. Overall, this system not only significantly improves the effectiveness and efficiency of AI practical teaching but also effectively stimulates students' learning interest and innovation capabilities through intelligent data analysis and a natural and smooth interactive experience.

[0017] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a block diagram of the overall architecture of the system of the present invention; Figure 2 This is a flowchart illustrating the data analysis and teaching guidance process of the system of this invention; Figure 3 This is a semantic association diagram of the knowledge graph unit of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Please see Figure 1-3 As shown, an interactive AI teaching feedback system based on behavioral data analysis is centered on capturing learning behavior data from all dimensions through a hardware acquisition module. After being processed by a data analysis module, personalized strategies are generated by a teaching guidance module, and finally, multimodal collaborative feedback is achieved through a feedback execution module. Specifically, it includes the following modules: Hardware acquisition module Core Components: The core control unit is a Raspberry Pi, connected to peripherals for collecting different types of data. Camera: A 1080P high-definition USB camera with a frame rate of 30fps is used to collect visual operation data such as images of the student's Scratch programming interface and images of the target detection scene of the smart car. Microphone: A high-fidelity USB microphone with a sampling rate of 16kHz is used to collect students' voice commands (such as "start the car" and "recognize objects"). Inertial sensor: integrated into the smart car, it collects the car's motion status data and the student's operation data of controlling the car; Peripheral Management Unit: Built-in unified driver interface management program, which supports automatic identification of device type and loading of adapter driver when cameras, microphones and smart cars are connected via USB or GPIO interface, realizing plug-and-play in the Scratch programming environment without manual parameter configuration; at the same time, it synchronizes the peripheral data acquisition frequency to ensure data timing consistency.

[0022] Data Analysis Module Core function: Receives multi-source data output from the hardware acquisition module and performs deep correlation analysis through three types of units: The DFCNN+Transformer speech recognition unit uses a deep convolutional neural network (DFCNN) to extract speech features (such as Mel frequency cepstral coefficients) and combines them with a Transformer encoder to capture the temporal dependencies of speech, thereby enabling the recognition of student speech commands (such as "go forward", "stop", "recognize apple"). Tiny YOLO-based target detection unit: Using the Tiny YOLOv4 algorithm, it takes image data collected by a camera as input, detects target objects (such as obstacles, designated recognition objects) in the teaching scene of the smart car, and outputs the target coordinates and category; Temporal Behavior Analysis Unit: Employing a sliding window mechanism with configurable window duration, this unit correlates the "operation result data" output by the speech recognition unit and the object detection unit with the "behavioral process data" (operation duration, error retry frequency, Scratch module switching sequence, and vehicle control command sequence) collected by the hardware. For example, when a student connects the "Motion Module" and the "Control Module" in Scratch, the operation time (from clicking the module to the connection being completed) is recorded; if the connection fails (such as a module interface mismatch), the number of retries and the retry interval are recorded. By using time-series modeling, we can identify students' operating habits (such as whether they frequently skip basic modules and directly use complex functions) and cognitive impairment patterns (such as the repeated occurrence of certain types of programming logic errors).

[0023] Accelerated inference: The Tengine forward inference framework is used to optimize the operators of the DFCNN+Transformer model and Tiny YOLO model, reducing the computational load of the models and lowering the end-to-end latency of speech recognition and object detection, thus meeting the real-time requirements of teaching interaction.

[0024] Teaching guidance module Core Functionality: Based on the correlation analysis results of the temporal behavior analysis units, it generates personalized teaching guidance strategies adapted to Scratch programming or smart car operation, including four core units: Error Operation Step-by-Step Guidance Unit: A preset error threshold is set. When the timing analysis identifies that a student has triggered the threshold, a step-by-step operation guidance animation is generated within the Scratch environment. For example, when a student repeatedly connects the "Loop Module" and the "Action Module" incorrectly, the animation demonstrates the steps of "clicking the Loop Module → selecting the number of loops → dragging it above the Action Module → confirming the connection" in sequence. Difficult Function Assistance Unit: Preset dwell time threshold. When it is detected that a student has stayed for too long in a specific Scratch programming module or car control task, it will automatically push usage tips for that function. For example, push a graphic tutorial on "how to record the distance traveled by the car through variables" plus a short operation video. Personalized task push unit: Based on the combined results of object detection and speech recognition, it identifies knowledge gaps and pushes targeted exercises from the task library. For example, for "weak shape recognition", it pushes the task of "the car identifies three shapes and classifies them for parking", and for "instruction confusion", it pushes the task of "voice control of the car to drive along a specified route". Knowledge Graph Unit: Construct a semantic association graph between Scratch programming knowledge points (such as loops, variables, and conditional statements) and intelligent car operation skills (such as obstacle avoidance, target tracking, and voice control). For example, the "Scratch loop module" is associated with "continuous obstacle avoidance of the car". After students master the loop module, an interdisciplinary composite task (integrating programming logic and mechanical control) is automatically generated to "achieve the car travel along a fixed route through loops".

[0025] Feedback Execution Module Core Functionality: Executes strategies generated by the instructional guidance module, enabling interaction with students through multimodal collaborative feedback, and includes three core units: Scratch Visual Hint Unit: Generates feedback content within the Scratch graphical programming interface: for example, a green checkmark and progress bar are displayed when the operation is correct (e.g., "Loop module mastered, progress +10%"); a red hint box is displayed when the operation is incorrect, pointing to the error location and indicating the reason (e.g., "Module interface mismatch, please select 'Control' class interface"). Intelligent vehicle motion feedback unit: Through the inertial sensors of the hardware acquisition module and the vehicle drive, it controls the vehicle to perform specific actions and characterizes the operation results. Correct operation: The car's lights are constantly green, demonstrating the correct action; Operational error: The car's lights are flashing red, triggering a warning action; Task guidance: The car travels along a preset trajectory, demonstrating the correct operating path; Voice synthesis prompt unit: Using voice synthesis technology, it generates voice prompts that coordinate with visual prompts and the car's actions. For example, when the car performs an obstacle avoidance demonstration, it simultaneously announces "Please note: There is an obstacle ahead, you need to turn left to avoid it"; when a student completes a programming task, it announces "Congratulations! You have successfully completed the loop control task. Next, try setting variables." Multimodal synchronization: A multimodal synchronization unit is provided, which uses timestamp alignment technology (assigning unified timestamps to visual prompts, car actions, and voice prompts) to ensure that the output latency difference of the three types of feedback is ≤10 milliseconds, providing a coherent and immersive interactive experience.

[0026] Data storage and teacher interaction module Hybrid storage architecture: adopting a "local cache + cloud storage" model. Local storage: Cache high-frequency behavioral data collected in real time via an external SD card connected to the Raspberry Pi to avoid data loss due to network interruption; Cloud storage: It adopts a distributed database of MySQL + MongoDB. MySQL stores structured data (user information, course content, error threshold parameters), and MongoDB stores unstructured data (operation logs, tutorial animations, voice prompt files); cloud data provides historical data support for the time-series behavior analysis unit (such as comparing the changes in students' operation efficiency this week and last week).

[0027] Teacher interaction interface: Provides a web-based management backend, generating two types of core reports based on time-series behavior analysis results: Class-wide learning behavior heatmap: Visually displays the frequently missed modules and average operation time distribution of students in the class, helping teachers to identify common problems in the class; Individual student behavior trajectory report: records the operation sequence, knowledge gaps, and task completion progress of a single student; teachers can manually trigger / adjust / override teaching guidance strategies through the interface: for example, adjust the "error threshold" from 3 times to 2 times to meet the special needs of a student, or manually push specific practice tasks.

[0028] A specific operational flow diagram of the system in this invention is shown below: System operation flow (taking "intelligent car target tracking teaching" as an example) Data collection phase: After logging into the system, students select the "Car Target Tracking" task, and the hardware acquisition module is activated: the camera captures an image of the front of the car (including the target object "orange"), the microphone captures the student's voice command "identify the orange and track it", and the inertial sensor collects the car's initial position data; The peripheral management unit automatically identifies all devices, synchronizes the data acquisition frequency, and transmits the data to the data analysis module in real time.

[0029] Data analysis phase: The DFCNN+Transformer speech recognition unit receives microphone data, recognizes the command "identify orange and track", and outputs the recognition result; The Tiny YOLO object detection unit receives images from the camera, detects an "orange" object, and outputs the coordinates (x=320, y=240) and category. The temporal behavior analysis unit uses an 8-second sliding window to record the student's operation process: the operation time from the issuance of the voice command to the start of the car (e.g., 2.5 seconds). If the car does not turn correctly (the inertial sensor detects a directional deviation), the error is recorded and the car is retried once, with a retry interval of 1.2 seconds. The Tengine framework optimizes the inference process, with a speech recognition latency of 85 milliseconds and an object detection latency of 92 milliseconds, meeting real-time requirements.

[0030] Teaching guidance stage: The timing analysis results show that "the student retried once due to an error, and the operation time was slightly longer than the class average (2.0 seconds)," triggering the "Difficulty Function Auxiliary Unit" in the teaching guidance module; Example of "Car Target Tracking" technique: The Scratch interface pushes a graphic tutorial on "How to adjust the car's steering angle using target coordinates", while the knowledge graph unit links the knowledge points of "coordinate calculation" and "car motion" and pushes an extended task of "recording changes in target coordinates using variables".

[0031] Feedback Implementation Phase: The Scratch visual hints unit displays a tutorial pop-up window, indicating "Turn right when the target x-coordinate is >320, turn left when it is <320"; The intelligent vehicle's motion feedback unit controls the vehicle to execute a demonstration trajectory: the steering is slowly adjusted according to the "orange" coordinates, and the lights remain constantly green. The voice synthesis prompt unit simultaneously broadcasts, "Please refer to the tutorial, control the car's direction using the target coordinates, and try recording the coordinate changes." The multimodal synchronization unit uses timestamp alignment to ensure synchronous output of the three types of feedback with a delay difference of ≤8 milliseconds.

[0032] Data storage and teacher intervention: The local SD card caches the high-frequency data of this operation (100Hz data from the inertial sensor, operation trajectory), while the cloud database stores the student's operation log (1 error, 2.5 seconds operation time) and tutorial. Teachers can view the student's behavior trajectory report through the interactive interface and find that the student has "weak coordinate calculation logic". Teachers can manually adjust their teaching guidance strategies and push "basic coordinate calculation practice" tasks.

[0033] The comprehensive collection of "data + behavioral process data" provides a complete data foundation for in-depth analysis and solves the problem of one-sided data in existing systems; Analysis depth and real-time performance: Multi-source data association is achieved through time-series behavior analysis units, combined with Tengine inference acceleration, which can both identify students' cognitive obstacles and ensure analysis latency ≤100 milliseconds, meeting the needs of real-time interactive teaching. Personalized and collaborative feedback: The teaching guidance module generates strategies based on individual analysis results, while the feedback execution module achieves "visualization + action + voice" collaboration through multimodal synchronization, improving the accuracy and immersion of feedback and solving the problem of single feedback in existing systems; Scalability and ease of use: The peripheral management unit supports plug-and-play Scratch environment, the teacher interaction interface facilitates teaching intervention, and the system can be flexibly expanded to other AI teaching scenarios such as "robot programming" and "image recognition creation" to adapt to the needs of different educational stages.

[0034] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0035] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. An interactive AI teaching feedback system based on behavioral data analysis, characterized in that, include: The hardware acquisition module includes a control unit based on a Raspberry Pi, and connected to it a camera, microphone, and an inertial sensor integrated into the smart car for collecting student operation data. The data analysis module integrates a DFCNN+Transformer speech recognition unit for processing voice commands and a Tiny YOLO-based target detection unit for processing visual operation data. It also includes a temporal behavior analysis unit for performing correlation analysis on operation result data from the speech recognition unit and the target detection unit, as well as behavior process data consisting of operation duration and error retry frequency. The teaching guidance module is used to dynamically generate personalized teaching guidance strategies for Scratch programming or smart car operation practice based on the correlation analysis results of the temporal behavior analysis unit. The feedback execution module is used to execute the teaching guidance strategy and to perform multimodal interaction by integrating the following methods: providing visual prompts based on analysis results in the Scratch graphical programming environment, driving the smart car to perform specific actions that represent the correctness of the operation or provide demonstrations, and generating speech synthesis prompts that coordinate with the visual prompts and the smart car's actions.

2. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The temporal behavior analysis unit uses a sliding window mechanism with a window duration of 5-10 seconds to perform real-time temporal correlation analysis on the module switching sequence of students in the Scratch programming environment and the control instruction sequence in the operation of the smart car, in order to identify operating habits and cognitive impairment patterns.

3. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The teaching guidance module includes: The error operation step-by-step guidance unit is used to generate a step-by-step operation guidance animation triggered in the Scratch environment when the continuous operation errors of students are identified based on time-series correlation analysis and exceed a preset threshold. The Difficulty Function Assistance Unit is used to generate and push usage tips for a certain function when it is detected that the time a student spends in a certain programming module or car control task exceeds a threshold. The personalized task push unit is used to identify students' knowledge gaps based on the combined results of object detection and speech recognition, and generate targeted programming tasks or car control challenges.

4. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, In the feedback execution module, the specific actions of the smart car include preset light flashing patterns, turning actions, or travel trajectories, which are used to provide immediate and concrete physical feedback on the student's operation results.

5. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The data analysis module uses the Tengine inference framework to accelerate the forward inference of the DFCNN+Transformer speech recognition unit and the Tiny YOLO-based object detection unit, so that the end-to-end latency of speech recognition and object detection is less than 100 milliseconds, in order to meet the real-time requirements of teaching interaction.

6. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The system adopts a hybrid storage architecture, using a local SD card to cache high-frequency behavioral data collected in real time, and using a cloud-based distributed database to store structured user information, course content, and unstructured operation logs; wherein, the cloud-based database provides historical data support for the time-series behavioral analysis unit.

7. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The system also includes a teacher interaction interface, which generates a heatmap of the class’s overall learning behavior and a report on the behavior trajectory of individual students based on the output of the time-series behavior analysis unit. The interface allows teachers to manually trigger, adjust or override strategies automatically generated by the teaching guidance module based on the report.

8. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The teaching guidance module also integrates a knowledge graph unit, which semantically associates Scratch programming knowledge points with intelligent vehicle operation skills, and dynamically generates interdisciplinary composite teaching tasks based on the association analysis results of the temporal behavior analysis unit.

9. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The feedback execution module is equipped with a multimodal synchronization unit. The multimodal synchronization unit uses timestamp alignment technology to ensure that Scratch visual prompts, smart car actions and voice synthesis prompts achieve millisecond-level coordinated output, so as to provide a coherent and immersive interactive feedback experience.

10. The interactive AI teaching feedback system based on behavioral data analysis according to claim 1, characterized in that, The hardware acquisition module also includes a peripheral management unit for unified management of the driver interfaces of the camera, microphone, and smart car, supporting plug-and-play functionality and data synchronization of the peripherals in the Scratch programming environment.

Images