An Augmented Reality-Based Interactive CAD Teaching Method and System

Abstract

This invention discloses an augmented reality-based CAD interactive teaching method and system, belonging to the technical field of teaching systems. Specifically, it includes: real-time acquisition of video operation sequences and synchronized audio explanations from the teacher's end; decomposing video operations into discrete instruction nodes and extracting key parameter knowledge tags from the audio; establishing a spatiotemporal mapping relationship between the two; comparing the deviation between student operations and teacher instruction nodes using the operation trajectory marked on a heatmap as a benchmark; when the deviation exceeds a preset threshold, triggering an interactive circuit breaker mechanism to freeze the student's main interface, loading a matching correction micro-video in an independent sandbox window, and activating the interactive training module; collecting student operation response data through this module; and when the student operation response data consistently meets the standards, releasing the main interface from the circuit breaker state and activating the next operation instruction node. This invention improves the efficiency of CAD interactive teaching through a closed-loop teaching chain of "monitoring-circuit breaker-targeted training-unlocking upon meeting the standards."

Description

Technical Field

[0001] This invention relates to the field of teaching system technology, specifically to an augmented reality-based CAD interactive teaching method and system. Background Technology

[0002] In CAD operation teaching scenarios, the limitations of traditional teaching methods are particularly prominent due to the cumbersome operation steps, rigorous command logic, and high precision requirements for parameter adjustment. Currently, mainstream CAD teaching relies heavily on the teacher transmitting operation videos and synchronous audio explanations to the student's end in real time, requiring students to simultaneously imitate the teacher's operation process to complete exercises. However, in actual teaching, there are significant differences in students' adaptability to the operation pace and their speed of understanding complex commands, making it very easy for them to "fall behind the teacher's operation process."

[0003] On the one hand, teachers' demonstrations often proceed at an average pace, which makes it difficult for students with weak foundations to digest key steps (such as "setting the distance parameter of the offset command" and "the logical judgment of polyline closure") in a short time. They often miss subsequent operations due to a stumbling block at a certain step, resulting in a break in the operation chain. On the other hand, for students who are proficient in the operation, the fixed pace of the demonstration will waste time and reduce learning efficiency.

[0004] On the other hand, in the existing transmission mode, the teacher's operation video and voice explanation are output linearly, and students cannot immediately rewind or pause at the points where they are stuck. At the same time, the precision requirements of CAD operation mean that even if students keep up with the operation rhythm, they may still make operation errors due to unclear parameter adjustment details, and such errors are difficult to identify in real time in the traditional mode.

[0005] When students' operations lag behind the teacher's demonstration, the system neither automatically slows down the transmission pace on the teacher's end to avoid affecting other students, nor can it provide targeted remedial support for lagging students. Ultimately, this leads to students gradually accumulating operational blind spots, affecting the overall learning outcome.

[0006] Therefore, how to address the problems of "poor adaptability of operation rhythm" and "difficulty in real-time correction of lagging operations" in CAD teaching, and how to build a teaching support mechanism that can dynamically match the individual student's rhythm, accurately capture operation deviations, and provide real-time interactive corrections, has become a key challenge to improve the efficiency of CAD teaching. Summary of the Invention

[0007] The purpose of this invention is to provide an augmented reality-based interactive CAD teaching method and system, which solves the following technical problems:

[0008] The key challenge in improving the efficiency of CAD teaching is to build a teaching support mechanism that can dynamically match the individual pace of students, accurately capture operational deviations, and provide real-time interactive corrections.

[0009] The objective of this invention can be achieved through the following technical solutions:

[0010] An augmented reality-based interactive CAD teaching method includes the following steps:

[0011] S1. Real-time acquisition of teacher-side operation data stream, the operation data stream including video operation sequence and synchronous voice explanation information;

[0012] S2. Decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge tags from the voice explanation information; establish a spatiotemporal mapping relationship between operation instruction nodes and knowledge tags, and generate operation trajectories with heat map annotations and floating parameter capsules;

[0013] S3. Deploy an operation matching monitoring engine on the student terminal interface, and use the operation trajectory marked by the heat map as a visual benchmark to compare the deviation between the student's operation action and the operation instruction node generated by the teacher terminal in real time.

[0014] S4. When the deviation exceeds the preset deviation threshold, the interactive circuit breaker mechanism is triggered to freeze the main operation interface of the student terminal. In a sandbox window independent of the main operation interface, a correction micro-video matching the current erroneous operation instruction node is loaded, and the interactive training module is activated simultaneously. The interactive training module reuses the floating parameter capsule generated in S2 as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node.

[0015] S5. The interactive training module collects student operation response data in real time. When the detected student operation response data continuously meets the standard, the main interface circuit breaker state is lifted and the next operation instruction node is activated.

[0016] As a further aspect of the present invention: in step S2, the specific process of generating the operation trajectory with thermal icon annotations and the suspended parameter capsule is as follows:

[0017] The video operation sequence is analyzed frame by frame to detect modeling command trigger events and mouse trajectory key points in the video stream. The continuous operation is decomposed into discretized operation instruction nodes with time sequence numbers. Simultaneously, the voice explanation information is subjected to voiceprint separation and semantic segmentation to extract key parameter knowledge tags containing dimensional tolerances, material properties and geometric constraints.

[0018] Based on the timestamp of the operation instruction node and the voice start time of the knowledge tag, a spatiotemporal mapping relationship between the operation instruction node and the knowledge tag is established. A pre-built student historical operation database is called, and a high-frequency operation area density distribution map is generated by using the Gaussian kernel density estimation algorithm based on the set of past student operation coordinate points stored in the database. This map is then overlaid on the teacher's model view to form a heat map annotating the operation trajectory. At the same time, the key parameter knowledge tags are encapsulated as interactive floating parameter capsules, which are dynamically anchored to the three-dimensional spatial coordinates of the corresponding operation instruction node, thus obtaining the operation trajectory with heat map annotation and the floating parameter capsule.

[0019] As a further aspect of the present invention: in step S3, the specific process for obtaining the deviation is as follows:

[0020] An operation matching and monitoring engine deployed on the student's interface captures the mouse trajectory coordinate sequence and command triggering events during student operations in real time. Using the operation trajectory marked on the heatmap as a visual benchmark, the spatial overlap between the student's current mouse trajectory coordinates and the high-frequency area of ​​the corresponding command node on the heatmap is calculated to obtain the spatial offset. At the same time, the time sequence of the student's triggered modeling command is compared with the time sequence number of the teacher's operation command node to obtain the time delay. The deviation is generated by weighted fusion of the spatial offset and the time delay.

[0021] As a further aspect of the present invention: in step S4, the specific process of loading and correcting the micro-video is as follows:

[0022] Send an API call signal to the student end to disable all modeling command operation permissions. At the same time, extract the unique identifier of the current erroneous operation instruction node through the node matching engine, search for the video segment bound to the identifier in the pre-built micro-video resource library, obtain the memory address and start time code of the video segment, and push the video segment to the video player in the sandbox window in real time for playback through the streaming media transmission protocol.

[0023] As a further aspect of the present invention: the specific process of reconstructing the simulation training scene in S4 is as follows:

[0024] The floating parameter capsules generated by S2 are parsed into structured data using a parameter converter, extracting parameter names, numerical ranges, and physical units. Based on the extraction results, visually adjustable controls, including sliders, numerical input boxes, and unit toggles, are generated in the sandbox window. Simultaneously, the scene reconstruction engine is invoked to extract key geometric features, constraints, and erroneous parameter values ​​from the current erroneous operation command node. A simplified simulation training scene is constructed based on the extracted elements, removing unnecessary topological structures from the original model and retaining geometric entities and constraint relationships directly associated with the erroneous node. Finally, the generated adjustable controls are dynamically bound to the corresponding geometric elements in the simulation scene, establishing a real-time linkage mechanism between parameter adjustment and scene changes.

[0025] As a further aspect of the present invention: S4 further includes automatically activating the next operation instruction node when the deviation is less than or equal to a preset deviation threshold, and writing the current student operation data into the historical operation database for updating the heat map annotation.

[0026] As a further aspect of the present invention: the specific process of ensuring that the response data continuously meets the standard in step S5 is as follows:

[0027] The system captures students' operation sequences on adjustable controls in the sandbox window and their geometric reconstruction actions in the simulation training scenario in real time. It records parameter adjustment values, operation timestamps, and scene state changes at a preset high-frequency sampling period. When the system detects that the parameter adjustment value falls within the target range multiple times in a row and the spatial matching degree between the geometric reconstruction action and the reference template reaches the preset standard, it is determined that the operation response data is continuously up to standard.

[0028] As a further aspect of the present invention: S5 further includes real-time collection of student operation correction data through a sensor array embedded in the sandbox window. The operation correction data includes the distribution of micro-video viewing time, the number of parameter adjustment attempts, and the scene reconstruction response time. The collected data is written into the historical operation database by combining the student ID and the error node ID as key values, and the operation latency rate index of the student is updated. The sliding average of the latency rate of the most recent N operations is extracted and input into a preset piecewise linear function to calculate the playback rate adjustment factor, where N is a preset threshold. The playback rate adjustment factor is applied to the inter-frame interval time corresponding to the next operation instruction node, so that the inter-frame interval time is multiplied by the adjustment factor to obtain the corrected inter-frame interval time.

[0029] An augmented reality-based interactive CAD teaching system, used to implement the aforementioned augmented reality-based interactive CAD teaching method, includes:

[0030] The teaching acquisition module is used to acquire the teacher's operation data stream in real time. The operation data stream includes video operation sequences and synchronous audio explanation information.

[0031] The data analysis module is used to decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge tags from the voice explanation information; establish a spatiotemporal mapping relationship between operation instruction nodes and knowledge tags, and generate operation trajectories with heatmap annotations and floating parameter capsules;

[0032] The teaching monitoring module is used to deploy an operation matching monitoring engine on the student's interface, using the operation trajectory marked by the heat map as a visual benchmark to compare the deviation between the student's operation actions and the operation instruction nodes generated by the teacher in real time.

[0033] The teaching adjustment module is used to trigger an interactive circuit breaker mechanism to freeze the main operation interface of the student terminal when the deviation exceeds a preset deviation threshold. It loads a correction micro-video that matches the current erroneous operation instruction node in a sandbox window independent of the main operation interface and simultaneously activates the interactive training module. The interactive training module reuses the floating parameter capsule generated in the data analysis module as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node.

[0034] The result generation module is used to collect student operation response data in real time through the interactive training module. When the detected student operation response data continuously meets the standard, the main interface circuit breaker state is lifted and the next operation instruction node is activated.

[0035] The beneficial effects of this invention are:

[0036] 1) This invention decomposes the teacher-side operation data stream into discrete operation instruction nodes, extracts key parameter knowledge tags by combining voice explanation, and generates operation trajectories with heatmap annotations and floating parameter capsules. It can be understood that the heatmap annotations present high-frequency operation areas based on historical operation data, which can intuitively guide students to focus on core operation steps; the floating parameter capsules dynamically associate operation nodes with parameter information, helping students understand the correspondence between operation steps and parameters in real time. Through the combination of structured and visual methods, the understanding threshold of students' complex operation logic is reduced, making the cumbersome operation process clear and easy to understand, and helping students quickly establish an operation cognitive framework.

[0037] 2) This invention achieves real-time and precise intervention in student operations through an operation matching monitoring engine and an interactive circuit breaker mechanism. By comparing the deviation between student operations and teacher instruction nodes in real time, when the deviation is too large, a targeted correction micro-video is loaded through a sandbox window and a simulated training scenario is constructed, allowing students to focus on correcting errors in an independent environment. At the same time, the interactive training module reuses the floating parameter capsule as an adjustable control, enabling students to strengthen their mastery of key parameters through actual operation, avoiding the accumulation and spread of incorrect operations, significantly improving the pertinence and effectiveness of operation practice, and helping students correct deviations in a timely manner.

[0038] 3) This invention collects student operation response data through an interactive training module and dynamically adjusts the teaching progress and pace to achieve personalized teaching adaptation. This invention only activates the next step after the student's operation has been continuously up to standard, ensuring that students with weak foundations can solidly master the current content. At the same time, it adjusts the interval between subsequent operation frames based on student correction data to adapt to the learning pace of different students, avoiding wasting time for proficient students due to waiting. This makes the teaching process more in line with individual differences, improves the overall teaching efficiency, and enables students of different levels to receive appropriate learning support. Attached Figure Description

[0039] The invention will now be further described with reference to the accompanying drawings.

[0040] Figure 1 This is a schematic diagram of a CAD interactive teaching method based on augmented reality according to the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Please see Figure 1 As shown, this invention is a CAD interactive teaching method based on augmented reality, comprising the following steps:

[0043] S1. Real-time acquisition of teacher-side operation data stream, the operation data stream including video operation sequence and synchronous voice explanation information;

[0044] S2. Decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge tags from the voice explanation information; establish a spatiotemporal mapping relationship between operation instruction nodes and knowledge tags, and generate operation trajectories with heat map annotations and floating parameter capsules;

[0045] S3. Deploy an operation matching monitoring engine on the student terminal interface, and use the operation trajectory marked by the heat map as a visual benchmark to compare the deviation between the student's operation action and the operation instruction node generated by the teacher terminal in real time.

[0046] S4. When the deviation exceeds the preset deviation threshold, the interactive circuit breaker mechanism is triggered to freeze the main operation interface of the student terminal. In a sandbox window independent of the main operation interface, a correction micro-video matching the current erroneous operation instruction node is loaded, and the interactive training module is activated simultaneously. The interactive training module reuses the floating parameter capsule generated in S2 as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node.

[0047] S5. The interactive training module collects student operation response data in real time. When the detected student operation response data continuously meets the standard, the main interface circuit breaker state is lifted and the next operation instruction node is activated.

[0048] In a preferred embodiment of the present invention, the specific process of generating the operation trajectory with heat map annotations and the suspended parameter capsule in step S2 is as follows:

[0049] The video operation sequence is analyzed frame by frame to detect modeling command trigger events and mouse trajectory key points in the video stream. The continuous operation is decomposed into discretized operation instruction nodes with time sequence numbers. Simultaneously, the voice explanation information is subjected to voiceprint separation and semantic segmentation to extract key parameter knowledge tags containing dimensional tolerances, material properties and geometric constraints.

[0050] Based on the timestamp of the operation instruction node and the voice start time of the knowledge tag, a spatiotemporal mapping relationship between the operation instruction node and the knowledge tag is established. A pre-built student historical operation database is called, and a high-frequency operation area density distribution map is generated by using the Gaussian kernel density estimation algorithm based on the set of past student operation coordinate points stored in the database. This map is then overlaid on the teacher's model view to form a heat map annotating the operation trajectory. At the same time, the key parameter knowledge tags are encapsulated as interactive floating parameter capsules, which are dynamically anchored to the three-dimensional spatial coordinates of the corresponding operation instruction node, thus obtaining the operation trajectory with heat map annotation and the floating parameter capsule.

[0051] When analyzing video operation sequences frame by frame, such as a teacher using CAD to draw a hole in a mechanical part, the system will identify key events in the video frame by frame, such as the moment the "drill command" is clicked and the coordinates of the hole selected by the mouse. The continuous drilling process will be broken down into discrete operation instruction nodes with serial numbers, such as "activate the drilling tool", "specify the hole center", and "set the hole diameter". When processing audio explanations, if the teacher explains "this hole is 10 mm in diameter, made of cast iron, and perpendicular to the end face", the system will separate the teacher's voice and segment the semantics, extracting key parameter knowledge tags containing dimensions, materials, and geometric constraints. A correlation is established between the timestamp of the operation node and the start time of the voice recording. For example, the "set aperture" node occurs at the 8th second, while the corresponding parameter explanation starts at the 7th second, thus binding the two together. The click coordinates of past students punching holes are called from the historical database, and an algorithm generates a distribution map of the hole positions that most students click on. This map is then overlaid on the teacher's view to form a heat map. For example, when most students are likely to click on a certain area when specifying the center of the hole, that area will be marked in dark. At the same time, parameters such as "diameter 10" are encapsulated into clickable floating capsules and dynamically fixed at the three-dimensional coordinates of the hole position, allowing students to intuitively see the correspondence between the operation and the parameter.

[0052] Breaking down complex, continuous operations into step-by-step nodes reduces the difficulty for students to understand and makes the operational logic clearer. Extracting parameter labels and associating them with operation nodes ensures that students accurately obtain key information at each step, avoiding a disconnect between parameters and operations. Heatmaps generated based on historical data guide students to focus on high-frequency operation areas, reducing invalid attempts. Floating parameter capsules allow for interactive parameters, making it convenient for students to view and adjust them at any time. These practices lay the foundation for subsequent real-time monitoring of student operational deviations and precise error correction, helping to achieve a precise correspondence between operation steps and explanations. Ultimately, this helps the system dynamically adapt to the students' pace, improving the effectiveness of CAD teaching.

[0053] In another preferred embodiment of the present invention, the specific process of obtaining the deviation in step S3 is as follows:

[0054] An operation matching and monitoring engine deployed on the student's interface captures the mouse trajectory coordinate sequence and command triggering events during student operations in real time. Using the operation trajectory marked on the heatmap as a visual reference, the spatial overlap between the student's current mouse trajectory coordinates and the high-frequency area of ​​the corresponding command node on the heatmap is calculated to obtain the spatial offset. At the same time, the time sequence of the student's triggered modeling command is compared with the time sequence number of the teacher's operation command node to obtain the time delay. The deviation is generated by weighted fusion of the spatial offset and the time delay.

[0055] The operation matching and monitoring engine running on the student's interface continuously monitors the student's actions. For example, when a student drags the mouse to draw a shape, the engine records the coordinates of every position the mouse moves through, as well as the student's actions such as clicking commands like "draw circle" and "trim." Using the previously generated heatmap as a reference, for example, if the area where the mouse frequently moves when the teacher draws a circle is in the middle right of the screen (which is darker on the heatmap), the engine compares the student's mouse path with this dark area, calculating the degree of overlap. For instance, if two-thirds of the student's mouse path is within the dark area, the spatial overlap is considered high. Simultaneously, the teacher's operations are divided into sequentially numbered steps, such as... The first step is "selecting the drawing command," and the second step is "specifying the center." Students also have their own order of steps. For example, a student might specify the center before selecting the drawing command, or the student might still be in the first step when the teacher is performing the second step. These will all be compared. Spatial offset is the degree to which the student's mouse trajectory deviates from the high-frequency area. The greater the deviation, the larger the value. Time delay is the time difference between the student's operation steps and the teacher's steps. The greater the difference, the larger the value. Based on the importance of these two aspects, for example, if the accuracy of the spatial position is more important, then spatial offset will be considered more. Combining the two, a comprehensive deviation will be obtained. For example, if the student's trajectory deviates significantly and the steps are also delayed, the deviation will be larger.

[0056] By comprehensively assessing the differences between student operations and standard operations from both the location and timing of the operation, the system focuses on whether students have clicked the correct points and kept up with the pace of the steps, avoiding the one-sidedness of a single-dimensional judgment. The deviation obtained in this way can accurately reflect the actual situation of the student's operation, allowing the system to know whether the student is operating in the wrong position, too slow, or both. This provides a reliable basis for determining whether to trigger the error correction mechanism, helping the system to intervene and provide guidance at the appropriate time, making the teaching more in line with the student's actual operating state, and helping to achieve precise teaching assistance.

[0057] The specific formula for calculating the deviation is as follows:

[0058] ;

[0059] Where S is the spatial offset value, T is the time delay value, and a and b are preset weight coefficients, with a being less than b.

[0060] In another preferred embodiment of the present invention, the specific process of loading and correcting the micro-video in step S4 is as follows:

[0061] Send an API call signal to the student end to disable all modeling command operation permissions. At the same time, extract the unique identifier of the current erroneous operation instruction node through the node matching engine, search for the video segment bound to the identifier in the pre-built micro-video resource library, obtain the memory address and start time code of the video segment, and push the video segment to the video player in the sandbox window in real time for playback through the streaming media transmission protocol.

[0062] When a student makes a mistake while performing an operation such as drawing a chamfer on a part, the system on the student's interface first issues an instruction to temporarily disable modeling commands such as "chamfer" and "trim," essentially locking these function buttons to prevent the erroneous operation from continuing. Simultaneously, the system identifies the student's current erroneous step as "setting the chamfer distance" and locates a unique marker for this step. This marker acts like a unique number for each step, corresponding to content in a pre-prepared library of instructional videos. Using this marker, the system can find the video specifically explaining how to correctly set the chamfer distance. Next, the system determines the exact path of this video in its storage location and the starting second of the chamfer distance setting section. Then, using a fast video transmission method, this video is sent to a separate small window on the student's interface for playback. This small window does not affect the main interface content. When the video starts playing, it directly finds the key screen explaining this step based on the previously determined start time, skipping other irrelevant content in the video, such as only showing the chamfer distance setting section without watching the entire part drawing process.

[0063] When students make mistakes, promptly restricting their use of related commands prevents the errors from escalating and allows them to focus on correcting the current error. Using specific tags to locate corresponding instructional videos ensures students see explanations perfectly aligned with the current error, avoiding irrelevant distractions. Playing videos starting some time before the error occurred helps students better connect the preceding and following actions, understand the context of the error, and more easily identify their mistakes. Playing videos in a separate small window allows students to clearly see the correction methods without disrupting the main interface's progress. These measures aim to provide students with precise and timely guidance after making mistakes, correcting errors through targeted practice, and ultimately helping the system dynamically adjust the teaching process based on student performance, thereby improving the effectiveness and relevance of error correction in CAD teaching.

[0064] In another preferred embodiment of the present invention, the specific process of reconstructing the simulation training scene in step S4 is as follows:

[0065] The floating parameter capsules generated by S2 are parsed into structured data using a parameter converter, extracting parameter names, numerical ranges, and physical units. Based on the extraction results, visually adjustable controls, including sliders, numerical input boxes, and unit toggles, are generated in the sandbox window. Simultaneously, the scene reconstruction engine is invoked to extract key geometric features, constraints, and erroneous parameter values ​​from the current erroneous operation command node. A simplified simulation training scene is constructed based on the extracted elements, removing unnecessary topological structures from the original model and retaining geometric entities and constraint relationships directly associated with the erroneous node. Finally, the generated adjustable controls are dynamically bound to the corresponding geometric elements in the simulation scene, establishing a real-time linkage mechanism between parameter adjustment and scene changes.

[0066] Suppose a student is drawing a cylinder with a hole, and an incorrect hole depth parameter setting triggers a reconstruction of the simulation training scene. First, the system converts the previously generated floating parameter capsule containing information such as "hole depth," "range 10-50mm," and "unit mm" into clear and ordered data using a parameter converter. This is analogous to unpacking a pre-packaged file and extracting the parameter name, adjustable value range, and unit. Next, based on this extracted information, a usable adjustment tool is created in a separate sandbox window, such as a slider that can move between 10-50, a box for direct number input, and a button to switch between mm and cm. Simultaneously, the scene reconstruction engine identifies key features from the student's incorrect "hole depth setting" operation, such as the hole's cylindrical shape, the requirement that the hole be perpendicular to the cylinder's top surface, and the incorrect value of "hole depth 60mm." Then, based on this extracted content, a simplified practice scene is built, removing structures unrelated to the hole depth from the original complex model, leaving only the cylinder, the hole, and the constraint that the hole must be perpendicular to the top surface. Finally, bind the prepared sliders, input boxes, and other tools to the holes in the scene. When students drag the sliders or enter numbers, the depth of the holes in the scene will change in real time, allowing students to intuitively see the effect of parameter adjustments.

[0067] Parsing parameter capsules into structured data and generating adjustable controls allows students to directly manipulate key parameters during practice, avoiding the hassle of complex parameter formats. Extracting error-related geometric features and constraints and simplifying the scene eliminates irrelevant information, allowing students to focus on correcting current errors. Binding controls to scene elements enables real-time interaction, allowing students to intuitively experience the impact of parameter changes on the model and deepen their understanding of parameter meaning. The aim is to provide students with a targeted and easy-to-use error-correction practice environment, helping them quickly master the correct methods for incorrect operations. This plays a crucial role in dynamically matching students' learning pace, accurately correcting operational deviations, and improving the efficiency of CAD teaching, enabling students to effectively correct errors in practice and consolidate their knowledge.

[0068] In another preferred embodiment of the present invention, step S4 further includes automatically activating the next operation instruction node when the deviation is less than or equal to a preset deviation threshold, and writing the current student operation data into the historical operation database for updating the heat map annotation.

[0069] When students draw straight lines, the system continuously compares their actions with the teacher's standard actions. If a student's click on the line command and drag the mouse to define the endpoint's trajectory overlaps significantly with the teacher's high-frequency areas, and the timing of the click and the teacher's corresponding steps is similar, the calculated deviation will be less than or equal to a preset standard. Once this condition is met, the system automatically unlocks the next step. For example, if a student has completed "drawing a straight line," they can directly proceed to "annotating dimensions" without manual intervention. Simultaneously, the system records the student's mouse click position and the order of their actions in a database storing past student actions. This newly added data is aggregated with previously stored information from other students. When generating heatmap annotations, this new data is incorporated to ensure that the annotations of high-frequency areas better reflect the current student's actual actions. For instance, if previously many students frequently drew lines in a certain area, but now more students are operating in a different area, the darker areas of the heatmap will change accordingly.

[0070] Automatically advancing to the next step when students' operations meet the requirements makes the learning process smoother, eliminating the need to wait for additional instructions and preventing pauses due to correct operations that disrupt the learning pace. Storing student operation data in a database and updating the heatmap ensures that the heatmap always reflects the latest operation status. Subsequent students can refer to high-frequency operation areas that are more relevant to the current learning group, reducing unnecessary trial and error. The goal is to allow the teaching process to proceed naturally based on students' actual operation, while continuously optimizing teaching reference information. This greatly helps the solution to dynamically adapt to the learning pace of different students, improve the accuracy and effectiveness of teaching, and make the entire teaching process more in line with students' actual operation status, thereby improving learning efficiency.

[0071] In another preferred embodiment of the present invention, the specific process of ensuring that the response data continuously meets the standard in step S5 is as follows:

[0072] The system captures students' operation sequences on adjustable controls in the sandbox window and their geometric reconstruction actions in the simulation training scenario in real time. It records parameter adjustment values, operation timestamps, and scene state changes at a preset high-frequency sampling period. When the system detects that the parameter adjustment value falls within the target range multiple times in a row and the spatial matching degree between the geometric reconstruction action and the reference template reaches the preset standard, it is determined that the operation response data is continuously up to standard.

[0073] When students practice adjusting the chamfer parameters of a part in the sandbox window, the system continuously monitors their actions. Actions such as dragging sliders to change the chamfer size and redrawing the chamfer in the scene are captured in real time. The system records the adjusted chamfer value, the time of the operation, and the change in the chamfer shape in the scene at set short time intervals, such as every short while. When it is observed that the chamfer values ​​adjusted by the student multiple times are within the correct range, for example, the correct range is 2-5 mm, and the student adjusts it to 3 mm, 4 mm, and 3.5 mm three times in a row, and at the same time, the shape and position of the chamfer redrawn by the student have a high degree of spatial overlap with the correct template provided by the teacher, with no obvious deviation, then the student's operation response data is considered to be consistently up to standard.

[0074] By capturing operations in real time and recording data frequently, a comprehensive understanding of students' practice can be gained, avoiding the omission of key operational information. Requiring parameters to meet standards repeatedly eliminates the possibility of accidental correctness, ensuring students truly understand the parameter setting requirements. Checking the matching degree between geometric reconstruction actions and the reference template ensures that students not only set parameters correctly but also performed actions that met the standards. The purpose is to rigorously verify whether students have truly mastered the methods for correcting errors, rather than merely memorizing a value or step. This plays a crucial role in achieving accurate error correction in the final solution, ensuring students solidly master each operational step, and improving the effectiveness of CAD teaching. It also gives students more confidence and ability to perform subsequent operations after passing the standard assessment.

[0075] In another preferred embodiment of the present invention, step S5 further includes collecting student operation correction data in real time through a sensor array embedded in a sandbox window. The operation correction data includes the distribution of micro-video viewing time, the number of parameter adjustment attempts, and the scene reconstruction response time. The collected data is written into the historical operation database by combining the student ID and the error node ID as a key value, and the operation latency rate index of the student is updated. The sliding average of the latency rate of the most recent N operations is extracted and input into a preset piecewise linear function to calculate the playback rate adjustment factor, where N is a preset threshold. The playback rate adjustment factor is applied to the inter-frame interval time corresponding to the next operation instruction node, so that the inter-frame interval time is multiplied by the adjustment factor to obtain the corrected inter-frame interval time.

[0076] When a student corrects an error in the "Setting the height of the cuboid" step within the sandbox window, the recording tool in the sandbox continuously monitors the student's actions. For example, the distribution of viewing time—attentive for the first 20 seconds, pausing for 5 seconds, and fast-forwarding for the last 10 seconds—is recorded. The number of attempts—from the incorrect 150 to 120 and then to the correct 100—is also recorded. Similarly, the 40 seconds it takes for the student to redraw the cuboid and achieve the correct shape is recorded. This data is then linked to the student's unique identifier and the identifier of the "Setting the height of the cuboid" error node, essentially labeling each piece of data with "whose" and "which error" information. This data is then stored in a database recording past operations, while simultaneously updating the student's operational latency metrics. Next, an average delay value is calculated from the student's recent (e.g., the last 5) operation delays. Then, an adjustment coefficient is obtained based on the preset different delay ranges and corresponding adjustment ranges. For example, when the average delay is large, this coefficient will be larger. This coefficient is then used to adjust the playback interval of the next operation step. For example, if the original playback interval of each frame in the next step is 2 seconds, it becomes 3 seconds after multiplying by this coefficient, thus slowing down the playback rhythm of the next step.

[0077] Collecting data on students correcting errors allows for a detailed understanding of each student's learning characteristics when facing different mistakes. For example, some students need to watch more videos, while others need to try different parameters multiple times. Storing data by student and error node identifiers accurately tracks each student's performance on specific errors, avoiding data confusion. Calculating the adjustment coefficient using the average delay of recent iterations reflects the student's current learning pace, providing greater stability than single-item data. Adjusting the playback interval for the next step allows fast-paced students to learn without waiting, while slower learners can keep up. The goal is to flexibly adjust the teaching pace according to students' actual situations, achieving personalized teaching. This is crucial for the final solution to dynamically match students' learning pace, enabling each student to learn at a suitable pace and improving the overall effectiveness of CAD teaching. It makes teaching more aligned with students' needs, avoiding the negative impact of an unsuitable pace on learning outcomes.

[0078] The present invention also includes an augmented reality-based CAD interactive teaching system for implementing the above-described augmented reality-based CAD interactive teaching method, comprising:

[0079] The teaching acquisition module is used to acquire the teacher's operation data stream in real time. The operation data stream includes video operation sequences and synchronous audio explanation information.

[0080] The data analysis module is used to decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge tags from the voice explanation information; establish a spatiotemporal mapping relationship between operation instruction nodes and knowledge tags, and generate operation trajectories with heatmap annotations and floating parameter capsules;

[0081] The teaching monitoring module is used to deploy an operation matching monitoring engine on the student's interface, using the operation trajectory marked by the heat map as a visual benchmark to compare the deviation between the student's operation actions and the operation instruction nodes generated by the teacher in real time.

[0082] The teaching adjustment module is used to trigger an interactive circuit breaker mechanism to freeze the main operation interface of the student terminal when the deviation exceeds a preset deviation threshold. It loads a correction micro-video that matches the current erroneous operation instruction node in a sandbox window independent of the main operation interface and simultaneously activates the interactive training module. The interactive training module reuses the floating parameter capsule generated in the data analysis module as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node.

[0083] The result generation module is used to collect student operation response data in real time through the interactive training module. When the detected student operation response data continuously meets the standard, the main interface circuit breaker state is lifted and the next operation instruction node is activated.

[0084] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. An augmented reality-based interactive CAD teaching method, characterized in that, Includes the following steps: S1. Real-time acquisition of teacher-side operation data stream, the operation data stream including video operation sequence and synchronous voice explanation information; S2. Decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge tags from the voice explanation information; establish a spatiotemporal mapping relationship between operation instruction nodes and knowledge tags, and generate operation trajectories with heat map annotations and floating parameter capsules; S3. Deploy an operation matching monitoring engine on the student terminal interface, and use the operation trajectory marked by the heat map as a visual benchmark to compare the deviation between the student's operation action and the operation instruction node generated by the teacher terminal in real time. The specific process for obtaining the deviation is as follows: An operation matching and monitoring engine deployed on the student's interface captures the mouse trajectory coordinate sequence and command triggering events during student operations in real time. Using the operation trajectory marked on the heatmap as a visual reference, the spatial overlap between the student's current mouse trajectory coordinates and the high-frequency area of ​​the corresponding command node on the heatmap is calculated to obtain the spatial offset. At the same time, the time sequence of the student's triggered modeling command is compared with the time sequence number of the teacher's operation command node to obtain the time delay. The deviation is generated by weighted fusion of the spatial offset and the time delay. S4. When the deviation exceeds the preset deviation threshold, the interactive circuit breaker mechanism is triggered to freeze the main operation interface of the student terminal. In a sandbox window independent of the main operation interface, a correction micro-video matching the current erroneous operation instruction node is loaded, and the interactive training module is activated simultaneously. The interactive training module reuses the floating parameter capsule generated in S2 as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node. S5. The interactive training module collects student operation response data in real time. When the detected student operation response data continuously meets the standard, the main interface circuit breaker state is released and the next operation instruction node is activated. The system collects student operation correction data in real time through a sensor array embedded in the sandbox window. This data includes the distribution of micro-video viewing time, the number of parameter adjustment attempts, and the scene reconstruction response time. The collected data is written into the historical operation database using a key-value pair of student ID and error node ID, updating the student's operation latency rate index. The system extracts the sliding average of the latency rates of the most recent N operations and inputs it into a preset piecewise linear function to calculate the playback rate adjustment factor, where N is a preset threshold. The playback rate adjustment factor is then applied to the inter-frame interval time corresponding to the next operation instruction node, multiplying the inter-frame interval time by the adjustment factor to obtain the corrected inter-frame interval time.

2. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that, In step S2, the specific process of generating the operation trajectory with thermal icon annotations and the suspended parameter capsule is as follows: The video operation sequence is analyzed frame by frame to detect modeling command trigger events and mouse trajectory key points in the video stream. The continuous operation is decomposed into discretized operation instruction nodes with time sequence numbers. Simultaneously, the voice explanation information is subjected to voiceprint separation and semantic segmentation to extract key parameter knowledge tags containing dimensional tolerances, material properties and geometric constraints. Based on the timestamp of the operation instruction node and the voice start time of the knowledge tag, a spatiotemporal mapping relationship between the operation instruction node and the knowledge tag is established. A pre-built student historical operation database is called, and a high-frequency operation area density distribution map is generated by using the Gaussian kernel density estimation algorithm based on the set of past student operation coordinate points stored in the database. This map is then overlaid on the teacher's model view to form a heat map annotating the operation trajectory. At the same time, the key parameter knowledge tags are encapsulated as interactive floating parameter capsules, which are dynamically anchored to the three-dimensional spatial coordinates of the corresponding operation instruction node, thus obtaining the operation trajectory with heat map annotation and the floating parameter capsule.

3. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that, In step S4, the specific process of loading and correcting the micro-video is as follows: Send an API call signal to the student end to disable all modeling command operation permissions. At the same time, extract the unique identifier of the current erroneous operation instruction node through the node matching engine, search for the video segment bound to the identifier in the pre-built micro-video resource library, obtain the memory address and start time code of the video segment, and push the video segment to the video player in the sandbox window in real time for playback through the streaming media transmission protocol.

4. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that, In S4, the specific process of reconstructing the simulation training scene is as follows: The floating parameter capsule generated by S2 is parsed into structured data by the parameter converter, and the parameter name, numerical range and physical unit are extracted. Based on the extraction results, a visual adjustable control is generated in the sandbox window, including a slider, numerical input box and unit switch. At the same time, the scene reconstruction engine is called to extract key geometric features, constraints and error parameter values ​​from the current erroneous operation command node. A simplified simulation training scenario is constructed based on extracted elements. Unnecessary topological structures of the original model are removed, while geometric entities and constraints directly associated with error nodes are retained. Finally, the generated adjustable controls are dynamically bound to the corresponding geometric elements in the simulation scenario, establishing a real-time linkage mechanism between parameter adjustment and scenario changes.

5. The CAD interactive teaching method based on augmented reality according to claim 2, characterized in that, S4 further includes automatically activating the next operation instruction node when the deviation is less than or equal to a preset deviation threshold, and writing the current student operation data into the historical operation database to update the heat map annotation.

6. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that, In S5, the specific process of ensuring that the response data continuously meets the standard is as follows: The system captures students' operation sequences on adjustable controls in the sandbox window and their geometric reconstruction actions in the simulation training scenario in real time. It records parameter adjustment values, operation timestamps, and scene state changes at a preset high-frequency sampling period. When the system detects that the parameter adjustment value falls within the target range multiple times in a row and the spatial matching degree between the geometric reconstruction action and the reference template reaches the preset standard, it is determined that the operation response data is continuously up to standard.

7. An augmented reality-based interactive CAD teaching system, used to implement the augmented reality-based interactive CAD teaching method according to any one of claims 1-6, characterized in that, include: The teaching acquisition module is used to acquire the teacher's operation data stream in real time. The operation data stream includes video operation sequences and synchronous audio explanation information. The data analysis module is used to decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge tags from the voice explanation information; establish a spatiotemporal mapping relationship between operation instruction nodes and knowledge tags, and generate operation trajectories with heatmap annotations and floating parameter capsules; The teaching monitoring module is used to deploy an operation matching monitoring engine on the student's interface, using the operation trajectory marked by the heat map as a visual benchmark to compare the deviation between the student's operation actions and the operation instruction nodes generated by the teacher in real time. The teaching adjustment module is used to trigger an interactive circuit breaker mechanism to freeze the main operation interface of the student terminal when the deviation exceeds a preset deviation threshold. It loads a correction micro-video that matches the current erroneous operation instruction node in a sandbox window independent of the main operation interface and simultaneously activates the interactive training module. The interactive training module reuses the floating parameter capsule generated in the data analysis module as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node. The result generation module is used to collect student operation response data in real time through the interactive training module. When the detected student operation response data continuously meets the standard, the main interface circuit breaker state is lifted and the next operation instruction node is activated.

Images