CAD interactive teaching method and system based on augmented reality

Abstract

The invention discloses a CAD interactive teaching method and system based on augmented reality, and belongs to the technical field of teaching systems.The method specifically comprises the steps that a video operation sequence and synchronous voice explanation information of a teacher end are obtained in real time, video operation is disassembled into discretized instruction nodes, key parameter knowledge tags are extracted from voice, and the key parameter knowledge tags are stored in a database; establishing a space-time mapping relation between the two; taking an operation track marked by the thermodynamic diagram as a reference, and comparing the deviation degree of student operation and teacher instruction nodes; when the deviation degree exceeds a preset threshold value, triggering an interactive fusing mechanism to freeze a student side main interface, loading a matched correction micro video in an independent sandbox window, and activating an interactive training module; according to the CAD interactive teaching system and method, the module is used for collecting student operation response data, when it is detected that the student operation response data continuously reaches the standard, the fusing state of the main interface is relieved, the next operation instruction node is activated, and the CAD interactive teaching efficiency is improved through a closed-loop teaching link of monitoring, fusing, targeted training and standard unlocking.

Description

Technical Field

[0001] The present invention relates to the technical field of teaching systems, and in particular to a CAD interactive teaching method and system based on augmented reality. Background Art

[0002] In CAD operation teaching scenarios, the limitations of traditional teaching models are particularly prominent due to its cumbersome operation steps, rigorous instruction logic, and high parameter adjustment precision requirements. Currently, mainstream CAD teaching relies on the teacher's real-time transmission of operation videos and synchronized voice explanations to the student's end. Students need to imitate the teacher's operation process to complete the exercises. However, in actual teaching, there are significant differences in individual students' ability to adapt to the operation rhythm and the speed of understanding complex instructions, which makes it very easy for them to "not keep up with the teacher's operation process": On the one hand, teachers' operation demonstrations often proceed at an even pace. For students with weak foundations, it is difficult to digest key steps (such as "distance parameter setting of offset command" and "logical judgment of polyline closure") in a short period of time. They often miss subsequent operations due to a certain link being stuck, resulting in a break in the operation chain. For students who are proficient in operation, fixed-pace demonstrations will cause a waste of time and reduce learning efficiency.

[0003] Furthermore, the existing transmission model uses a linear output of the teacher's video and audio instructions, preventing students from instantly retracing or pausing at any point where they experience a delay. Furthermore, the precision required for CAD operations means that even if students keep up with the pace, they may still make operational errors due to unclear parameter adjustment details. These errors are difficult to identify in real time using traditional methods. When students' operations lag behind the teacher's demonstration, the system can neither automatically slow down the transmission rhythm on the teacher's side to avoid affecting other students, nor provide targeted remedial support for the lagging students. Ultimately, students gradually accumulate blind spots in operations, affecting the overall learning effect. Therefore, how to address the problems of "poor adaptability of operation rhythm" and "difficulty in correcting lagging operations in real time" in CAD teaching, and build a teaching assistance mechanism that can dynamically match students' individual rhythm, accurately capture operation deviations and provide instant interactive corrections, has become a key challenge to improving CAD teaching efficiency. Summary of the Invention

[0004] The purpose of the present invention is to provide a CAD interactive teaching method and system based on augmented reality to solve the following technical problems: How to build a teaching assistance mechanism that can dynamically match students' individual rhythm, accurately capture operational deviations, and provide instant interactive corrections has become a key challenge to improving CAD teaching efficiency.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A CAD interactive teaching method based on augmented reality includes the following steps: S1. Real-time acquisition of the teacher-side operation data stream, which includes a video operation sequence and synchronous voice explanation information; S2. Decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge labels from the voice explanation information; establish a spatiotemporal mapping relationship between the operation instruction nodes and the knowledge labels, and generate operation trajectories and suspended parameter capsules with heat map annotations; S3. Deploy an operation matching monitoring engine on the student-side interface, using the operation trajectory marked on the heat map as a visual reference, and compare the deviation between the student's operation action and the operation instruction node generated by the teacher-side in real time; S4. When the deviation exceeds a preset deviation threshold, the interactive fuse mechanism is triggered to freeze the student-side operation main interface, load a corrected micro-video matching the current erroneous operation instruction node in a sandbox window independent of the operation main interface, and simultaneously activate the interactive training module. The interactive training module reuses the suspended parameter capsule generated in S2 as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node; S5. The interactive training module collects student operation response data in real time. When the detected student operation response data continuously meets the standard, the main interface fuse state is released and the next operation instruction node is activated.

[0006] As a further solution of the present invention: In S2, the specific process of generating the operation trajectory with thermal map annotation and the suspended parameter capsule is as follows: The video operation sequence is parsed frame by frame to detect modeling command trigger events and mouse trajectory key points in the video stream, and the continuous operations are disassembled into discrete operation instruction nodes with time sequence numbers; the voice explanation information is simultaneously subjected to voiceprint separation and semantic segmentation to extract key parameter knowledge labels including dimensional tolerances, material properties and geometric constraints; Based on the timestamp of the operation instruction node and the speech start time of the knowledge label, a spatiotemporal mapping relationship between the operation instruction node and the knowledge label is established; the pre-built student history operation database is called, and according to the previous student operation coordinate point set stored in the database, a Gaussian kernel density estimation algorithm is used to generate a high-frequency operation area density distribution map, which is superimposed on the teacher-side model view to form a heat map to mark the operation trajectory; at the same time, the key parameter knowledge label is encapsulated into an interactive suspended parameter capsule, which is dynamically anchored to the three-dimensional spatial coordinates of the corresponding operation instruction node, thereby obtaining the operation trajectory and suspended parameter capsule with heat map annotation.

[0007] As a further solution of the present invention: in S3, the specific process of obtaining the deviation degree is: The operation matching monitoring engine deployed on the student-side interface captures the mouse trajectory coordinate sequence and command triggering events during student operations in real time. Taking the operation trajectory marked on the heat map as the visual benchmark, the spatial overlap between the student's current mouse trajectory coordinates and the high-frequency area of ​​the corresponding instruction node on the heat map is calculated to obtain the spatial offset; at the same time, the time sequence of the student-triggered modeling command is compared with the timing number of the teacher-side operation instruction node to obtain the time delay; the deviation degree is generated by weighted fusion of the spatial offset and time delay.

[0008] As a further solution of the present invention: In S4, the specific process of loading and correcting the micro-video is as follows: An API call signal is sent to the student end to disable all modeling command operation permissions. At the same time, the unique identifier of the current error operation instruction node is extracted through the node matching engine, and the video clip bound to the identifier is retrieved from the pre-built micro-video resource library; the memory address and starting time code of the video clip are obtained, and the video clip is pushed to the video player in the sandbox window in real time through the streaming transmission protocol for playback.

[0009] As a further solution of the present invention: in S4, the specific process of reconstructing the simulation training scene is: The suspended parameter capsules generated by S2 are parsed into structured data through a parameter converter, and the parameter names, numerical ranges and physical units are extracted. Based on the extraction results, visual adjustable controls are generated in the sandbox window, including sliders, numerical input boxes and unit switches. At the same time, the scene reconstruction engine is called to extract key geometric features, constraints and erroneous parameter values ​​from the current erroneous operation instruction node. A simplified simulation training scene is constructed based on the extracted elements, in which unnecessary topological structures of the original model are removed and geometric entities and constraint relationships directly associated with the erroneous nodes are retained. Finally, the generated adjustable controls are dynamically bound to the corresponding geometric elements in the simulation scene to establish a real-time linkage mechanism between parameter adjustment and scene changes.

[0010] As a further solution of the present invention: S4 also includes automatically activating the next operation instruction node when the deviation degree is less than or equal to a preset deviation threshold, and writing the current student operation data into the historical operation database for updating the thermal map annotation.

[0011] As a further solution of the present invention: In S5, the specific process of ensuring that the response data continuously meets the standards is as follows: The student's operation sequence on the adjustable controls in the sandbox window and the geometric reconstruction actions in the simulation training scene are captured in real time, and the parameter adjustment value, operation timestamp and scene state change are recorded according to the preset high-frequency sampling period; when it is detected that the parameter adjustment value falls into the target range for multiple consecutive times and the spatial matching degree of the geometric reconstruction action and the reference template reaches the preset standard, it is determined that the operation response data continues to meet the standard.

[0012] As a further solution of the present invention: S5 also includes real-time collection of student operation correction data through the sensor array embedded in the sandbox window, and the operation correction data includes the distribution of micro-video viewing time, the number of parameter adjustment attempts and the scene reconstruction response time; the collected data is written into the historical operation database according to the combined key value of the student ID and the error node ID, and the operation delay rate indicator of the student is updated; the sliding average of the most recent N operation delay rates is extracted and input into a preset piecewise linear function to calculate the playback rate adjustment factor, wherein N is a preset threshold, and the playback rate adjustment factor is applied to the inter-frame interval corresponding to the next operation instruction node, so that the inter-frame interval is multiplied by the adjustment factor to obtain the corrected inter-frame interval.

[0013] An augmented reality-based CAD interactive teaching system, used to implement the aforementioned augmented reality-based CAD interactive teaching method, comprises: The teaching acquisition module is used to obtain the teacher's operation data stream in real time, which includes the video operation sequence and synchronous voice explanation information; A data analysis module is used to decompose the video operation sequence into discrete operation instruction nodes and extract key parameter knowledge labels from the voice explanation information; establish a spatiotemporal mapping relationship between the operation instruction nodes and the knowledge labels, and generate operation trajectories and suspended parameter capsules with heat map annotations; The teaching monitoring module is used to deploy an operation matching monitoring engine on the student-side interface, using the operation trajectory marked on the heat map as a visual reference to compare the deviation between the student's operation action and the operation instruction node generated by the teacher-side in real time; A teaching adjustment module is configured to trigger an interactive fuse mechanism to freeze the student-side operation main interface when the deviation exceeds a preset deviation threshold, load a corrected micro-video matching the current erroneous operation instruction node in a sandbox window independent of the operation main interface, and simultaneously activate an interactive training module. The interactive training module reuses the suspended parameter capsule generated in the data analysis module as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node; The result generation module is used to collect student operation response data in real time through the interactive training module. When the detected student operation response data continues to meet the standard, the main interface fuse state is released and the next operation instruction node is activated.

[0014] Beneficial effects of the present invention: 1) The present invention decomposes the teacher-side operation data stream into discrete operation instruction nodes, extracts key parameter knowledge labels in combination with voice explanation, and generates operation trajectories and suspended parameter capsules with thermal map annotations. It can be understood that the thermal map annotations present high-frequency operation areas based on historical operation data, which can intuitively guide students to focus on core operation links; the suspended parameter capsules dynamically associate operation nodes with parameter information, helping students to understand the correspondence between operation steps and parameters in real time. Through the combination of structuring and visualization, it lowers the threshold for students to understand complex operation logic, makes cumbersome operation processes clear and identifiable, and helps students quickly establish an operation cognitive framework.

[0015] 2) This invention achieves real-time and precise intervention in student operations through an operation matching monitoring engine and an interactive fuse mechanism. By comparing the deviation between student operations and teacher instruction nodes in real time, when the deviation is too large, a targeted correction micro-video is loaded through a sandbox window and a simulation training scenario is constructed, allowing students to focus on correcting errors in an independent environment. At the same time, the interactive training module reuses suspended parameter capsules as adjustable controls, allowing students to strengthen their grasp of key parameters through actual operations, avoiding the accumulation and spread of erroneous operations, significantly improving the pertinence and effectiveness of operation exercises, and helping students correct deviations in a timely manner.

[0016] 3) The present invention collects student operation response data through the interactive training module, dynamically adjusts the teaching progress and rhythm, and realizes personalized teaching adaptation. The present invention activates the next link only after the student's operation continues to meet the standard, ensuring that students with weak foundations can firmly grasp the current content; at the same time, the interval between subsequent operation frames is adjusted based on the student's correction data to adapt to the learning rhythm of different students, avoiding skilled students wasting time due to waiting, thereby making the teaching process more in line with individual differences, improving overall teaching efficiency, and enabling students of different levels to obtain appropriate learning support. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be further described below with reference to the accompanying drawings.

[0018] Figure 1 This is a flow chart of a CAD interactive teaching method based on augmented reality according to the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0020] See also Figure 1As shown, the present invention is a CAD interactive teaching method based on augmented reality, comprising the following steps: S1. Real-time acquisition of the teacher-side operation data stream, which includes a video operation sequence and synchronous voice explanation information; S2. Decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge labels from the voice explanation information; establish a spatiotemporal mapping relationship between the operation instruction nodes and the knowledge labels, and generate operation trajectories and suspended parameter capsules with heat map annotations; S3. Deploy an operation matching monitoring engine on the student-side interface, using the operation trajectory marked on the heat map as a visual reference, and compare the deviation between the student's operation action and the operation instruction node generated by the teacher-side in real time; S4. When the deviation exceeds a preset deviation threshold, the interactive fuse mechanism is triggered to freeze the student-side operation main interface, load a corrected micro-video matching the current erroneous operation instruction node in a sandbox window independent of the operation main interface, and simultaneously activate the interactive training module. The interactive training module reuses the suspended parameter capsule generated in S2 as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node; S5. The interactive training module collects student operation response data in real time. When the detected student operation response data continuously meets the standard, the main interface fuse state is released and the next operation instruction node is activated.

[0021] In a preferred embodiment of the present invention, the specific process of generating the operation trajectory and the suspended parameter capsule with the thermal map annotation in S2 is as follows: The video operation sequence is parsed frame by frame to detect modeling command trigger events and mouse trajectory key points in the video stream, and the continuous operations are disassembled into discrete operation instruction nodes with time sequence numbers; the voice explanation information is simultaneously subjected to voiceprint separation and semantic segmentation to extract key parameter knowledge labels including dimensional tolerances, material properties and geometric constraints; Based on the timestamp of the operation instruction node and the speech start time of the knowledge label, a spatiotemporal mapping relationship between the operation instruction node and the knowledge label is established; the pre-built student history operation database is called, and according to the previous student operation coordinate point set stored in the database, a Gaussian kernel density estimation algorithm is used to generate a high-frequency operation area density distribution map, which is superimposed on the teacher-side model view to form a heat map to mark the operation trajectory; at the same time, the key parameter knowledge label is encapsulated into an interactive suspended parameter capsule, which is dynamically anchored to the three-dimensional spatial coordinates of the corresponding operation instruction node, thereby obtaining the operation trajectory and suspended parameter capsule with heat map annotation.

[0022] When parsing a video operation sequence frame by frame, for example, when a teacher uses CAD to draw a drilling operation for a mechanical part, the system will identify key events frame by frame in the video, such as the moment the "drilling command" is clicked and the coordinate point of the hole selected by the mouse, and split the continuous drilling process into discrete operation instruction nodes with serial numbers, such as "activate the drilling tool", "specify the hole center", and "set the aperture". When processing voice explanations synchronously, if the teacher explains "this hole has a diameter of 10, is made of cast iron, and is perpendicular to the end face", the system will separate the teacher's voice and segment the semantics, and extract key parameter knowledge labels including size, material, and geometric constraints. An association is established based on the timestamp of the operation node and the start time of the voice. For example, the "Set Aperture" node occurs at the 8th second, and the corresponding parameter explanation starts at the 7th second, thus binding the two together. The click coordinates of students when drilling holes in the past are called from the historical database, and an algorithm is used to generate a distribution map of the hole areas that most students often click on, which is superimposed on the teacher's view to form a thermal trajectory. For example, most students tend to click on a certain area when specifying the center of the hole, and this area will be marked in dark color. At the same time, parameters such as "diameter 10" are encapsulated into clickable floating capsules, which are dynamically fixed at the three-dimensional coordinates of the hole position, allowing students to intuitively see the correspondence between operations and parameters.

[0023] Breaking down complex, continuous operations into step-by-step nodes can reduce the difficulty for students to understand and make the operational logic clearer. Extracting parameter labels and associating them with operational nodes ensures that students accurately obtain key information at the corresponding steps, avoiding disconnection between parameters and operations. Thermal trajectories generated based on historical data can guide students to focus on high-frequency operation areas and reduce ineffective attempts. Floating parameter capsules make parameters interactive, allowing students to easily view and adjust them at any time. These practices provide a foundation for subsequent real-time monitoring of student operational deviations and precise error correction, helping to achieve a precise correspondence between operational steps and explanations. Ultimately, this helps the system dynamically adapt to the student's pace and improve the effectiveness of CAD teaching.

[0024] In another preferred embodiment of the present invention, in S3, the specific process of obtaining the deviation degree is: The operation matching monitoring engine deployed on the student-side interface captures the mouse trajectory coordinate sequence and command triggering events during student operations in real time. Taking the operation trajectory marked on the heat map as the visual benchmark, the spatial overlap of the student's current mouse trajectory coordinates and the high-frequency area of ​​the corresponding instruction node on the heat map is calculated to obtain the spatial offset; at the same time, the time sequence of the student-triggered modeling command is compared with the timing number of the operation instruction node on the teacher's side to obtain the time delay, and the deviation degree is generated by weighted fusion of the spatial offset and time delay.

[0025] The operation matching monitoring engine running on the student interface will always pay attention to the student's operations. For example, when a student uses the mouse to drag and draw a shape, the engine will record the coordinates of each position the mouse moves through, as well as the student's actions of clicking commands such as "Draw Circle" and "Trim". With the previously generated heat map marking trajectory as a reference, for example, when the teacher draws a circle, the area where the mouse moves frequently is in the middle of the right side of the screen. The color of this area on the heat map is darker. The trajectory of the mouse when the student draws the circle is compared with this dark area to calculate the amount of overlap in the trajectory. For example, if two-thirds of the student's mouse path is in the dark area, the spatial overlap is higher. At the same time, the teacher's operations are divided into sequentially numbered steps, such as The first step is to "select the drawing command" and the second step is to "specify the center of the circle". Students' operations also have their own sequence of steps. For example, students specify the center of the circle before selecting the drawing command, or students are still in the first step when the teacher is in the second step. These will be compared; the spatial offset is to see the degree to which the student's mouse trajectory deviates from the high-frequency area. The greater the deviation, the larger the value. The time delay is to see the time difference between the student's operation steps and the teacher's steps. The greater the difference, the larger the value. Then, based on the importance of these two aspects, for example, if the accuracy of spatial position is more important, the spatial offset should be considered more. The two are combined to obtain a comprehensive deviation degree. For example, if the student's trajectory deviates a lot and the steps are also delayed, the deviation degree will be larger.

[0026] From the two aspects of the operation position and time, the system comprehensively judges the difference between the student's operation and the standard operation, paying attention to whether the student has clicked the right place and whether the student has kept up with the rhythm of the steps, avoiding the one-sidedness of single-dimensional judgment; the deviation degree obtained in this way can accurately reflect the actual situation of the student's operation, letting the system know whether the student's operation position is wrong, the rhythm is slow, or both, providing a reliable judgment basis for whether to trigger the error correction mechanism later, helping the system to intervene and guide at the right time, making the teaching more in line with the students' actual operation status, and helping to achieve accurate teaching assistance.

[0027] The specific calculation formula of the deviation degree is: ; Wherein, S is the spatial offset value, T is the time delay value, a and b are preset weight coefficients, and a is smaller than b.

[0028] In another preferred embodiment of the present invention, in S4, the specific process of loading and correcting the micro-video is as follows: An API call signal is sent to the student end to disable all modeling command operation permissions. At the same time, the unique identifier of the current error operation instruction node is extracted through the node matching engine, and the video clip bound to the identifier is retrieved from the pre-built micro-video resource library; the memory address and starting time code of the video clip are obtained, and the video clip is pushed to the video player in the sandbox window in real time through the streaming transmission protocol for playback.

[0029] When a student makes an error while performing an operation such as chamfering a part, the system on the student interface first issues a command, temporarily preventing the student from using modeling commands such as "Chamfer" and "Trim," effectively locking these buttons to prevent further incorrect operations. Simultaneously, the system identifies the step where the student made the error, "Setting the Chamfer Distance," and identifies a unique tag for this step. This tag, like a unique number for each step, can be mapped to content in a pre-prepared instructional video library. Using this tag, the system can locate the video in the library that specifically explains how to correctly set the chamfer distance. The system then determines the specific path of this video in its storage location and the second at which the instruction on setting the chamfer distance begins. Using a fast video transfer method, the video is then played in a separate window on the student interface, without interfering with the main interface. When the video begins, it will directly select the key scene explaining the step based on the previously determined start time, skipping irrelevant content. For example, the system can skip watching the entire part drawing process to focus solely on the chamfer distance setting.

[0030] When students make an error, timely restricting them from continuing to use related commands can prevent the error from expanding further and allow students to focus on correcting the current error; finding the corresponding teaching video through exclusive tags can ensure that students see the explanation content that fully matches the current error without being disturbed by irrelevant information; playing the video from a period before the error occurs can help students better connect previous and subsequent operations, understand the context in which the error occurred, and more easily understand where they went wrong; playing the video in a separate small window can allow students to clearly see the correction method without affecting the operation progress of the main interface. The purpose of these practices is to allow students to receive accurate and timely guidance after making an error, correct the error through targeted exercises, and then help the system achieve the ultimate goal of dynamically adjusting the teaching process according to the student's operation situation, thereby improving the effectiveness and pertinence of the error correction link in CAD teaching.

[0031] In another preferred embodiment of the present invention, in S4, the specific process of reconstructing the simulation training scene is: The suspended parameter capsules generated by S2 are parsed into structured data through a parameter converter, and the parameter names, numerical ranges and physical units are extracted. Based on the extraction results, visual adjustable controls are generated in the sandbox window, including sliders, numerical input boxes and unit switches. At the same time, the scene reconstruction engine is called to extract key geometric features, constraints and erroneous parameter values ​​from the current erroneous operation instruction node. A simplified simulation training scene is constructed based on the extracted elements, in which unnecessary topological structures of the original model are removed and geometric entities and constraint relationships directly associated with the erroneous nodes are retained. Finally, the generated adjustable controls are dynamically bound to the corresponding geometric elements in the simulation scene to establish a real-time linkage mechanism between parameter adjustment and scene changes.

[0032] Suppose a student is drawing a cylinder with a hole and incorrectly sets the hole depth parameter, triggering a reconstruction of the simulation training scene. First, the system converts the previously generated floating parameter capsule containing information such as "hole depth," "range 10-50mm," and "units in mm" into clear and organized data using a parameter converter. This is like unpacking a packaged file, extracting the parameter name, adjustable value range, and unit. Next, based on this extracted information, a separate sandbox window creates directly usable adjustment tools, such as a slider that slides between 10 and 50, a box for direct numerical input, and a button to switch between mm and cm. Simultaneously, the scene reconstruction engine identifies key features of the hole's incorrect "hole depth setting" operation, such as the cylindrical shape and the requirement that the hole be perpendicular to the cylinder's top surface, as well as the incorrect value of "hole depth 60mm" set by the student. Based on this extracted information, a simplified practice scene is constructed, removing irrelevant features from the original complex model, leaving only the cylinder and the hole, as well as the constraint that the hole must be perpendicular to the top surface. Finally, bind the prepared sliders, input boxes and other tools to the holes in the scene. When students drag the sliders or enter numbers, the depth of the holes in the scene will change in real time, allowing students to intuitively see the effects of parameter adjustments.

[0033] Parsing parameter capsules into structured data and generating adjustable controls allows students to directly manipulate key parameters during practice, avoiding the hassle of complex parameter formats. Extracting error-related geometric features and constraints and simplifying the scene eliminates irrelevant information interference, allowing students to focus on correcting current errors. Binding controls to scene elements for real-time linkage allows students to intuitively experience the impact of parameter changes on the model and deepen their understanding of the meaning of the parameters. The purpose is to provide students with a highly targeted and easy-to-operate error correction practice environment, helping them quickly master the correct methods for incorrect operations. This plays an important role in the program's ultimate goal of dynamically matching students' learning rhythm, accurately correcting operational deviations, and improving CAD teaching efficiency. It allows students to effectively correct errors in practice and consolidate their knowledge.

[0034] In another preferred embodiment of the present invention, said S4 also includes automatically activating the next operation instruction node when the deviation degree is less than or equal to a preset deviation threshold, and writing the current student operation data into the historical operation database for updating the thermal map annotation.

[0035] When a student draws a line, the system continuously compares their actions with the teacher's standard. If the student's click on the line command and drags the mouse to determine the endpoints overlaps significantly with the teacher's high-frequency trajectory, and the timing of the clicks is similar to the teacher's corresponding steps, the calculated deviation will be less than or equal to the preset standard. Once this condition is met, the system automatically unlocks the next step. For example, if a student has completed "Draw a Line," they can proceed directly to "Dimension" without having to manually open the step. At the same time, the system records the student's mouse click location and the order of their actions in a database that stores past student actions. This newly added data is combined with previously stored information about other students' actions. When generating heatmap annotations, this new data is factored in, ensuring that the annotations for high-frequency areas more closely reflect the students' actual actions. For example, if most students previously drew a line with endpoints in one area, but now more students are working in another area, the darker areas of the heatmap will shift accordingly.

[0036] When the student's operation meets the requirements, it will automatically proceed to the next step, making the learning process smoother. There is no need to wait for additional instructions, and the learning rhythm will not be affected by stagnation due to correct operation. Storing the student's operation data in the database and updating the heat map can make the heat map always reflect the latest operation status. Subsequent students can refer to the high-frequency operation area that is more suitable for the current learning group to reduce unnecessary trial and error. The purpose is to allow the teaching process to proceed naturally according to the actual operation of the students, while continuously optimizing the teaching reference information. This is of great help for the program to ultimately achieve dynamic adaptation to the learning rhythm of different students and improve the accuracy and effectiveness of teaching. It can make the entire teaching process more in line with the students' actual operation status and improve learning efficiency.

[0037] In another preferred embodiment of the present invention, the specific process of the response data continuously meeting the standards in S5 is as follows: The student's operation sequence on the adjustable controls in the sandbox window and the geometric reconstruction actions in the simulation training scene are captured in real time, and the parameter adjustment value, operation timestamp and scene state change are recorded according to the preset high-frequency sampling period; when it is detected that the parameter adjustment value falls into the target range for multiple consecutive times and the spatial matching degree of the geometric reconstruction action and the reference template reaches the preset standard, it is determined that the operation response data continues to meet the standard.

[0038] When students practice adjusting the chamfer parameters of a part in the sandbox window, the system will constantly monitor their operations. Actions such as dragging the slider to change the chamfer size and redrawing the chamfer in the scene will be captured in real time. The system will record the chamfer value adjusted by the student, the time the operation occurred, and the changes in the chamfer shape in the scene at set short time intervals, such as every few minutes. If it is observed that the chamfer value adjusted by the student multiple times is within the correct range, for example, the correct range is 2-5 mm, and the student adjusts it to 3 mm, 4 mm, and 3.5 mm three times in a row, and the chamfer shape and position redrawn by the student have a high degree of spatial overlap with the correct template provided by the teacher, with no significant deviation, then the student's operational response data will be considered to be consistently meeting the standards.

[0039] By capturing operations in real time and recording data at high frequency, we can fully understand the students' practice status and avoid missing key operation information; requiring parameters to meet the standards for multiple times in a row can eliminate the situation where students accidentally operate correctly and ensure that students truly understand the requirements of parameter settings; checking the matching degree of geometric reconstruction actions with reference templates can ensure that students not only set parameters correctly, but also meet the standards for actual operation actions. The purpose is to strictly verify whether students have truly mastered the method of correcting errors, rather than temporarily memorizing a certain value or step. This plays an important role in the ultimate realization of accurate error correction, ensuring that students have a solid grasp of each operation link, and improving the effectiveness of CAD teaching. It can enable students to have more confidence and ability to perform subsequent operations after passing the standard test.

[0040] In another preferred embodiment of the present invention, the S5 also includes real-time collection of student operation correction data through a sensor array embedded in the sandbox window, wherein the operation correction data includes the distribution of micro-video viewing time, the number of parameter adjustment attempts and the scene reconstruction response time; the collected data is written into the historical operation database according to the combined key value of the student ID and the error node ID, and the operation delay rate index of the student is updated; the sliding average of the most recent N operation delay rates is extracted and input into a preset piecewise linear function to calculate the playback rate adjustment factor, wherein N is a preset threshold, and the playback rate adjustment factor is applied to the inter-frame interval corresponding to the next operation instruction node, so that the inter-frame interval is multiplied by the adjustment factor to obtain the corrected inter-frame interval.

[0041] When students correct the incorrect operation of "setting the height of the rectangle" in the sandbox window, the recording tool in the sandbox window will always pay attention to the students' operations. For example, when students watch the micro-video explaining the error, they watch carefully for the first 20 seconds, pause for 5 seconds in the middle, and fast-forward for the last 10 seconds. The distribution of these viewing times will be recorded; when students adjust the height parameter, they try 3 times from the incorrect 150 to 120 and then to the correct 100. The number of attempts will also be recorded; when students redraw the rectangle, it takes 40 seconds from the beginning of the operation to the completion of the required shape. This response time will also be recorded. Afterwards, these data will be bound to the unique identifier of the student and the identifier of the error node "setting the height of the rectangle", just like labeling each piece of data with "whose" and "which error", and then stored in the database that records past operations, while updating the student's operational delay indicator. Next, an average delay value is calculated from the student's recent operation delays (for example, the last 5 times), and then an adjustment coefficient is obtained based on the preset different delay ranges corresponding to different adjustment amplitudes. For example, when the average delay is larger, this coefficient will be larger. This coefficient is then used for the playback interval of the next operation step. For example, the original playback interval of each frame in the next step is 2 seconds, which becomes 3 seconds after multiplying by this coefficient, making the playback rhythm of the next step slower.

[0042] By collecting various data when students correct their errors, we can gain a detailed understanding of each student's learning characteristics when faced with different errors. For example, some students need to watch more videos, while others need to try parameters multiple times. Storing data by student and error node identifiers can accurately track each student's performance on specific errors and avoid data confusion. Using the average delay of the most recent times to calculate the adjustment coefficient can reflect the student's current learning pace and is more stable than single-time data. Adjusting the playback interval of the next step can prevent students with a fast pace from waiting and allow students with a slow pace to keep up. The purpose is to flexibly adjust the teaching rhythm according to the students' actual situation and achieve personalized teaching. This is of great help in the program's ultimate realization of dynamically matching students' learning rhythms, allowing each student to learn at a pace that suits them, and improving the overall effectiveness of CAD teaching. It can make teaching more in line with students' needs and avoid the impact of learning effects due to an unsuitable rhythm.

[0043] The present invention also includes an augmented reality-based CAD interactive teaching system for implementing the aforementioned augmented reality-based CAD interactive teaching method, comprising: The teaching acquisition module is used to obtain the teacher's operation data stream in real time, which includes the video operation sequence and synchronous voice explanation information; A data analysis module is used to decompose the video operation sequence into discrete operation instruction nodes and extract key parameter knowledge labels from the voice explanation information; establish a spatiotemporal mapping relationship between the operation instruction nodes and the knowledge labels, and generate operation trajectories and suspended parameter capsules with heat map annotations; The teaching monitoring module is used to deploy an operation matching monitoring engine on the student-side interface, using the operation trajectory marked on the heat map as a visual reference to compare the deviation between the student's operation action and the operation instruction node generated by the teacher-side in real time; A teaching adjustment module is configured to trigger an interactive fuse mechanism to freeze the student-side operation main interface when the deviation exceeds a preset deviation threshold, load a corrected micro-video matching the current erroneous operation instruction node in a sandbox window independent of the operation main interface, and simultaneously activate an interactive training module. The interactive training module reuses the suspended parameter capsule generated in the data analysis module as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node; The result generation module is used to collect student operation response data in real time through the interactive training module. When the detected student operation response data continues to meet the standard, the main interface fuse state is released and the next operation instruction node is activated.

[0044] The above is a detailed description of an embodiment of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A CAD interactive teaching method based on augmented reality, characterized in that: The following steps are involved: S1. Real-time acquisition of the teacher-side operation data stream, which includes a video operation sequence and synchronous voice explanation information; S2. Decompose the video operation sequence into discrete operation instruction nodes, and extract key parameter knowledge labels from the voice explanation information; establish a spatiotemporal mapping relationship between the operation instruction nodes and the knowledge labels, and generate operation trajectories and suspended parameter capsules with heat map annotations; S3. Deploy an operation matching monitoring engine on the student-side interface, using the operation trajectory marked on the heat map as a visual reference, and compare the deviation between the student's operation action and the operation instruction node generated by the teacher-side in real time; S4. When the deviation exceeds a preset deviation threshold, the interactive fuse mechanism is triggered to freeze the student-side operation main interface, load a corrected micro-video matching the current erroneous operation instruction node in a sandbox window independent of the operation main interface, and simultaneously activate the interactive training module. The interactive training module reuses the suspended parameter capsule generated in S2 as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node; S5. The interactive training module collects student operation response data in real time. When the detected student operation response data continuously meets the standard, the main interface fuse state is released and the next operation instruction node is activated.

2. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that: In S2, the specific process of generating the operation trajectory and the suspended parameter capsule with the thermal map annotation is as follows: The video operation sequence is parsed frame by frame to detect modeling command trigger events and mouse trajectory key points in the video stream, and the continuous operations are disassembled into discrete operation instruction nodes with time sequence numbers; the voice explanation information is simultaneously subjected to voiceprint separation and semantic segmentation to extract key parameter knowledge labels including dimensional tolerances, material properties and geometric constraints; Based on the timestamp of the operation instruction node and the speech start time of the knowledge label, a spatiotemporal mapping relationship between the operation instruction node and the knowledge label is established; the pre-built student history operation database is called, and according to the previous student operation coordinate point set stored in the database, a Gaussian kernel density estimation algorithm is used to generate a high-frequency operation area density distribution map, which is superimposed on the teacher-side model view to form a heat map to mark the operation trajectory; at the same time, the key parameter knowledge label is encapsulated into an interactive suspended parameter capsule, which is dynamically anchored to the three-dimensional spatial coordinates of the corresponding operation instruction node, thereby obtaining the operation trajectory and suspended parameter capsule with heat map annotation.

3. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that: In S3, the specific process of obtaining the deviation degree is as follows: The operation matching monitoring engine deployed on the student-side interface captures the mouse trajectory coordinate sequence and command triggering events during student operations in real time. Taking the operation trajectory marked on the heat map as the visual benchmark, the spatial overlap of the student's current mouse trajectory coordinates and the high-frequency area of ​​the corresponding instruction node on the heat map is calculated to obtain the spatial offset; at the same time, the time sequence of the student-triggered modeling command is compared with the timing number of the operation instruction node on the teacher's side to obtain the time delay, and the deviation degree is generated by weighted fusion of the spatial offset and time delay.

4. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that: In S4, the specific process of loading and correcting the micro-video is as follows: An API call signal is sent to the student end to disable all modeling command operation permissions. At the same time, the unique identifier of the current error operation instruction node is extracted through the node matching engine, and the video clip bound to the identifier is retrieved from the pre-built micro-video resource library; the memory address and starting time code of the video clip are obtained, and the video clip is pushed to the video player in the sandbox window in real time through the streaming transmission protocol for playback.

5. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that: In S4, the specific process of reconstructing the simulation training scene is: The suspended parameter capsules generated by S2 are parsed into structured data through a parameter converter, and the parameter names, value ranges, and physical units are extracted. Based on the extracted results, visual adjustable controls are generated in the sandbox window, including sliders, value input boxes, and unit switches. At the same time, the scene reconstruction engine is called to extract key geometric features, constraints, and incorrect parameter values ​​from the current incorrect operation instruction node. A simplified simulation training scenario is constructed based on the extracted elements, in which unnecessary topological structures of the original model are removed, and the geometric entities and constraint relationships directly associated with the erroneous nodes are retained; finally, the generated adjustable controls are dynamically bound to the corresponding geometric elements in the simulation scenario, establishing a real-time linkage mechanism between parameter adjustment and scenario changes.

6. The CAD interactive teaching method based on augmented reality according to claim 2, characterized in that: Said S4 also includes automatically activating the next operation instruction node when the deviation degree is less than or equal to a preset deviation threshold, and writing the current student operation data into the historical operation database for updating the heat map annotation.

7. The CAD interactive teaching method based on augmented reality according to claim 1, characterized in that: In S5, the specific process of the response data continuously meeting the standards is as follows: The student's operation sequence on the adjustable controls in the sandbox window and the geometric reconstruction actions in the simulation training scene are captured in real time, and the parameter adjustment value, operation timestamp and scene state change are recorded according to the preset high-frequency sampling period; when it is detected that the parameter adjustment value falls into the target range for multiple consecutive times and the spatial matching degree of the geometric reconstruction action and the reference template reaches the preset standard, it is determined that the operation response data continues to meet the standard.

8. The CAD interactive teaching method based on augmented reality according to claim 7, characterized in that: The S5 also includes collecting student operation correction data in real time through the sensor array embedded in the sandbox window, and the operation correction data includes the distribution of micro-video viewing time, the number of parameter adjustment attempts and the scene reconstruction response time; writing the collected data into the historical operation database according to the combined key value of the student ID and the error node ID, and updating the student's operation delay rate indicator; extracting the sliding average of the last N operation delay rates and inputting a preset piecewise linear function to calculate the playback rate adjustment factor, wherein N is a preset threshold, and applying the playback rate adjustment factor to the inter-frame interval corresponding to the next operation instruction node, so that the inter-frame interval is multiplied by the adjustment factor to obtain the corrected inter-frame interval.

9. A CAD interactive teaching system based on augmented reality, used to implement the CAD interactive teaching method based on augmented reality according to any one of claims 1 to 8, characterized in that: include: The teaching acquisition module is used to obtain the teacher's operation data stream in real time, which includes the video operation sequence and synchronous voice explanation information; A data analysis module is used to decompose the video operation sequence into discrete operation instruction nodes and extract key parameter knowledge labels from the voice explanation information; establish a spatiotemporal mapping relationship between the operation instruction nodes and the knowledge labels, and generate operation trajectories and suspended parameter capsules with heat map annotations; The teaching monitoring module is used to deploy an operation matching monitoring engine on the student-side interface, using the operation trajectory marked on the heat map as a visual reference to compare the deviation between the student's operation action and the operation instruction node generated by the teacher-side in real time; A teaching adjustment module is configured to trigger an interactive fuse mechanism to freeze the student-side operation main interface when the deviation exceeds a preset deviation threshold, load a corrected micro-video matching the current erroneous operation instruction node in a sandbox window independent of the operation main interface, and simultaneously activate an interactive training module. The interactive training module reuses the suspended parameter capsule generated in the data analysis module as an adjustable control and reconstructs the simulation training scene based on the erroneous operation instruction node; The result generation module is used to collect student operation response data in real time through the interactive training module. When the detected student operation response data continues to meet the standard, the main interface fuse state is released and the next operation instruction node is activated.

Images