Hybrid training device for dental practice training, and operation method thereof

Abstract

A hybrid training device for dental procedure training according to various embodiments of the present disclosure includes a memory for storing at least one process for executing a dental procedure training application, and a processor for executing a dental procedure training application according to the process, wherein the processor includes an Augmented Reality (AR) procedure training module, and the Augmented Reality procedure training module can display a first virtual tooth model and a first virtual handpiece in a first virtual space, track the position of a first actual handpiece to generate a first cutting vector, calculate a first cutting volume based on the first cutting vector, and update a first virtual tooth model cut based on the first cutting volume.

Description

Technology Field

[0001] The present invention relates to a hybrid educational technology for dental procedure training. The present invention relates to a hybrid educational platform that is expanded into a virtual-real integrated procedure environment based on a conventional procedural root canal treatment simulation system (e.g., a root canal training application). The root canal treatment training application includes a configuration that generates a tooth model based on real-world images and dental model data, sets anatomical and simulation landmarks, and provides a step-by-step training UI / UX. By extending this analytical digital simulation structure into an augmented reality and real-world coaching environment, the present invention realizes data continuity between digital training, AR training, and real-world training. Background Technology

[0002] Recently, training systems based on digital simulation are being actively researched in the field of dental procedural education. Conventional dental procedural education has been conducted primarily through practice using actual tooth models; however, the learning effect was limited due to spatial constraints, material costs, and the inefficiency of training repetition. Accordingly, digital procedural training systems have been developed that utilize computer graphics technology to display three-dimensional (3D) virtual tooth models on a screen and simulate the cutting process through touch or mouse operations. However, this digital method lacks physical responsiveness and spatial awareness, making it difficult to provide training effects similar to actual clinical practice, and there were limitations in providing real-time feedback on precise movements such as the learner's procedural angle or cutting depth.

[0003] Meanwhile, although advancements in Augmented Reality (AR) technology have enabled the overlaying of virtual tooth models or surgical tools onto real-world spaces, conventional AR-based training systems have remained limited to merely visualizing virtual objects, failing to properly perform precise position tracking of actual tools or digital analysis of cutting actions. Furthermore, because the physical training phase using physical models and the digital simulation phase are separated, it has been impossible for learners to continuously study the same surgical case in both digital and physical environments, or to receive improvement feedback by comparing and analyzing actual surgical results with virtual models. Therefore, there is a need for a hybrid educational device that organically integrates digital, augmented reality, and physical training, and can evaluate and coach surgical results by quantifying actual surgical actions. The problem to be solved

[0004] The technical problem to be solved through some embodiments of the present disclosure is to quantitatively analyze the cutting direction and depth of the operator by converting the movement of the actual handpiece into a cutting vector and a cutting volume and applying them to a virtual tooth model.

[0005] The technical problem to be solved through some embodiments of the present disclosure is to precisely align a real tooth model and a virtual tooth model to provide real-time visual coaching feedback through augmented reality even during real-world training.

[0006] The technical problem to be solved through some embodiments of the present disclosure is to objectively quantify the accuracy of the procedure by automatically comparing the correct answer model and the procedure result model to evaluate under- and over-cutting.

[0007] The technical problem to be solved through some embodiments of the present disclosure is to improve data continuity between learning stages and learning efficiency by integrating digital training, augmented reality training, and physical training.

[0008] The technical problems of the present disclosure are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0009] A hybrid training device for dental procedure training according to some embodiments of the present disclosure for solving the above technical problem comprises: a memory storing at least one process for executing a dental procedure training application; and a processor executing the dental procedure training application according to the process, wherein the processor comprises an Augmented Reality (AR) procedure training module, wherein the AR procedure training module can display a first virtual tooth model and a first virtual handpiece in a first virtual space, track the position of a first actual handpiece to generate a first cutting vector, calculate a first cutting volume based on the first cutting vector, and update the first virtual tooth model cut based on the first cutting volume.

[0010] In some embodiments, the processor further includes a physical procedure training module that provides digital coaching, wherein the physical procedure training module recognizes a physical tooth model with a second AR camera and matches it with a second virtual tooth model, displays the second virtual tooth model and a second virtual handpiece in a second virtual space, tracks the position of the second actual handpiece to generate a second cutting vector, calculates a second cutting volume based on the second cutting vector, updates the second virtual tooth model cut based on the second cutting volume, and provides visual feedback on the cutting process in the second virtual space.

[0011] In some embodiments, the first cutting vector is defined to include a surface contact point (OP-start-point) at the start of drilling and a maximum point (OP-end-point) at the end of drilling, and the first cutting volume is calculated based on the first cutting vector and a user-set radius (OP-radius), and the second cutting vector is defined to include a surface contact point at the start of drilling and a maximum point at the end of drilling, and the second cutting volume can be calculated based on the second cutting vector and a user-set radius.

[0012] In some embodiments, the augmented reality procedure training module analyzes an image of the real space acquired from the first AR camera, tracks the position and orientation of the first real handpiece with a 5-sided tracking marker attached, and aligns and corrects the position and orientation of the first virtual handpiece in real time based on the position and orientation of the first real handpiece, and the real procedure training module analyzes an image of the real space acquired from the second AR camera, tracks the position and orientation of the second real handpiece with a 5-sided tracking marker attached, and aligns and corrects the position and orientation of the second virtual handpiece in real time based on the position and orientation of the second real handpiece.

[0013] In some embodiments, the augmented reality procedure training module compares a first correct answer model with a first procedure result model in which the first virtual tooth model is cut to calculate a first undercut area, a first overcut area, and a first normal cut area, calculates a first procedure evaluation score based on the first undercut area, the first overcut area, and the first normal cut area, and visually provides the first procedure evaluation score to the learner, and the real procedure training module compares a second correct answer model with a second procedure result model in which the second virtual tooth model is cut to calculate a second undercut area, a second overcut area, and a second normal cut area, calculates a second procedure evaluation score based on the second undercut area, the second overcut area, and the second normal cut area, and visually provides the second procedure evaluation score to the learner.

[0014] In some embodiments, the memory stores in advance coordinate data of a Dentiform Origin marker and coordinate data of a plurality of tooth socket points for recognizing the actual tooth model, and the actual procedure training module can precisely align the second virtual tooth model to the actual tooth model using the coordinate data of the Dentiform Origin marker and the coordinate data of the tooth socket points when the actual Dentiform is recognized.

[0015] In some embodiments, the augmented reality surgical training module tracks the movement of the first physical handpiece in real time to calculate a first cutting angle, a first cutting depth, and a first cutting speed, and displays real-time feedback including at least one of a vector, color highlighting, or warning display on the first virtual tooth model in the first virtual space based on the first cutting angle, the first cutting depth, and the first cutting speed, and the physical surgical training module tracks the movement of the second physical handpiece in real time to calculate a second cutting angle, a second cutting depth, and a second cutting speed, and displays real-time feedback including at least one of a vector, color highlighting, or warning display on the second virtual tooth model in the second virtual space based on the second cutting angle, the second cutting depth, and the second cutting speed.

[0016] In some embodiments, the augmented reality procedure training module may calculate a first procedure time deduction based on a first procedure time, calculate a first procedure action deduction based on the number of first procedure actions, and calculate a first final evaluation score based on the first procedure evaluation score, the first procedure time deduction, and the first procedure action deduction, and the real procedure training module may calculate a second procedure time deduction based on a second procedure time, calculate a second procedure action deduction based on the number of second procedure actions, and calculate a second final evaluation score based on the second procedure evaluation score, the second procedure time deduction, and the second procedure action deduction.

[0017] In some embodiments, the augmented reality procedure training module and the real procedure training module can use a machine learning algorithm to compare and analyze the uniformity of procedure results, angle error, and cutting depth distribution with the learner's past history, and provide improvement guidelines for future training stages.

[0018] A method of operation performed by an apparatus according to some embodiments of the present disclosure may include: displaying a first virtual tooth model and a first virtual handpiece in a first virtual space by an augmented reality surgical training module of the apparatus; generating a first cutting vector by tracking the position of a first actual handpiece by the augmented reality surgical training module; calculating a first cutting volume based on the first cutting vector by the augmented reality surgical training module; and updating the first virtual tooth model cut based on the first cutting volume by the augmented reality surgical training module. Effects of the invention

[0019] A hybrid training device for dental procedure training and a method of operation thereof according to some embodiments of the present disclosure can quantitatively analyze the cutting direction and depth of a practitioner by converting the movement of an actual handpiece into a cutting vector and a cutting volume and applying them to a virtual tooth model.

[0020] A hybrid training device for dental procedure training and a method of operation thereof according to some embodiments of the present disclosure can provide real-time visual coaching feedback through augmented reality even during physical training by precisely aligning a real tooth model and a virtual tooth model.

[0021] A mixed-type training device for dental procedure training and a method of operation thereof according to some embodiments of the present disclosure can objectively quantify the accuracy of a procedure by automatically comparing a correct answer model and a procedure result model to evaluate under- or over-cutting.

[0022] A hybrid training device for dental procedure training and a method of operation thereof according to some embodiments of the present disclosure can improve data continuity between learning stages and learning efficiency by integrating digital training, augmented reality training, and physical training.

[0023] The effects according to some embodiments of the present disclosure are not limited to those exemplified above, and a wider variety of effects are included in the present disclosure. Brief explanation of the drawing

[0024] FIG. 1 is an overall flowchart showing an operation scenario of a mixed-type training device for dental procedure training according to one embodiment of the present invention. FIG. 2 is an exemplary block diagram of a mixed-type training device for dental procedure training according to one embodiment of the present invention. FIG. 3 is a flowchart showing the operation flow of a mixed-type training device for dental procedure training according to one embodiment of the present invention. FIG. 4 is a diagram illustrating the process of making a tooth model for a lesson for mixed education according to one embodiment of the present invention. FIG. 5 is a diagram illustrating a cutting vector (OP vector) according to one embodiment of the present invention. FIG. 6 is a diagram illustrating the operation of calculating and quantitatively evaluating the difference between a correct answer model and a result model according to one embodiment of the present invention. FIG. 7 is a drawing for explaining a cap-shaped handpiece tracking device with a 5-sided tracking mark applied according to one embodiment of the present invention. FIG. 8 is a drawing illustrating a virtual space according to one embodiment of the present invention. FIG. 9 is a drawing illustrating a virtual space according to one embodiment of the present invention. FIGS. 10 to 14 are drawings for explaining the production of a training 3D printing model according to an embodiment of the present invention. FIGS. 15 and 16 are drawings illustrating a dentiform for mapping a real tooth model to a virtual tooth model according to one embodiment of the present invention. Specific details for implementing the invention

[0025] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. The advantages and features of the present disclosure and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the attached drawings. However, the technical concept of the present disclosure is not limited to the following embodiments but can be implemented in various different forms. The following embodiments are provided merely to complete the technical concept of the present disclosure and to fully inform those skilled in the art of the scope of the present disclosure, and the technical concept of the present disclosure is defined only by the scope of the claims. That is, the embodiments of the present disclosure are exemplified for the purpose of explaining the technical concept of the present disclosure. The scope of rights according to the present disclosure is not limited to the embodiments presented below or the specific description thereof.

[0026] It should be noted that when assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the present disclosure, if it is determined that a detailed description of related known components or functions could obscure the essence of the present disclosure, such detailed description is omitted.

[0027] Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which this disclosure pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise. The terms used in this specification are for describing embodiments and are not intended to limit this disclosure. In this disclosure, the singular form may include the plural form unless specifically stated otherwise in the text, and this applies likewise to the singular expressions described in the claims.

[0028] Additionally, terms such as first, second, A, B, (a), (b), etc., may be used to describe the components of the present disclosure. These terms are intended only to distinguish the components from other components and do not limit the nature, order, sequence, importance, etc. of the components. Where it is stated that a component is "connected," "coupled," or "joined" to another component, it should be understood that the component may be directly connected or joined to the other component, but that another component may also be "connected," "coupled," or "joined" between each component.

[0029] As used in the specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements. That is, expressions such as "comprising," "comprising," "having," etc., as used in this disclosure should be understood as open-ended terms implying the possibility of including other embodiments, unless otherwise stated in the phrase or sentence containing such expressions.

[0030] The expression “based on” as used in this disclosure is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and this expression does not exclude additional factors affecting an act or action of a decision or judgment.

[0031] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the accompanying drawings, identical or corresponding components are given the same reference numerals. Furthermore, in the description of the embodiments below, the description of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0032] FIG. 1 is an overall flowchart illustrating an operation scenario of a hybrid training device for dental procedure training according to one embodiment of the present invention. The overall flow illustrating an operation scenario of a hybrid training device for dental procedure training according to one embodiment may include the steps of running an app for a portable device (101), selecting a lesson (102), a lecture lesson (103), a practice lesson (104), selecting a practice method (105), performing digital procedure training (106), performing augmented reality procedure training (107), performing digital coaching physical procedure training (108), analyzing training results and reporting (109), and saving learning records (110).

[0033] First, in one embodiment, at the step of running an app for a portable device (101), the user can run a dental procedure education application through a hybrid education device. Here, the hybrid education device on which the dental procedure education application can be run is a portable device and may include not only general portable devices such as smartphones or tablets, but also a wide range of mobile devices capable of providing augmented reality functions, such as augmented reality (AR) glasses and mixed reality (MR) equipment (e.g., equipment for implementing an AR environment using the pass-through function of MetaQuest3). That is, the hybrid education device can be implemented as various types of mobile devices combined with displays and sensors, and can be configured to run the same learning scenario depending on the type of hardware supporting an AR or MR environment. On the initial screen, the user can specify the target tooth for learning, the topic of the procedure (e.g., caries removal, crown prep, endodontic access, etc.), the difficulty level, and the recommended time, and the correct answer model and virtual tooth model corresponding to the selected lesson can be loaded from an internal database. In this case, the hybrid education device can ensure data continuity between steps by maintaining the same lesson identifier and timestamp as a common key in subsequent steps.

[0034] In one embodiment, when a user selects a lecture lesson (103), the hybrid education device may provide theoretical content such as three-dimensional anatomical visualization, tool usage, and cutting principles to facilitate conceptual learning. The lecture lesson may include videos and interactive animations, and may check readiness for practice through key checkpoints (quizzes, mini-tasks). When the lecture lesson is completed, the user may return to the lesson selection (102) on the hybrid education device and switch to a practice lesson (104).

[0035] In one embodiment, the hybrid training device may present the procedure goals, tolerances, and evaluation items of the same lesson in advance during the practice lesson (104). Subsequently, in the step of selecting the practice method (105), the user may select one of three methods through the hybrid training device. (i) If the user selects to perform digital procedure training (106), the user may perform a cutting simulation by manipulating a virtual tooth model on the screen of the hybrid training device. The hybrid training device may update the virtual tooth model in real time by generating a cutting vector based on the user's input trajectory and calculating the cutting volume by applying a user-set radius. At this time, the difference from the correct model may be calculated to visualize the normal cutting area (E1), undercutting area (E2), and overcutting area (E3) as a color or transparency overlay. (ii) When the user selects to perform augmented reality procedure training (107), the hybrid training device places a virtual tooth model and a virtual handpiece on the camera image and tracks the position and orientation of the actual handpiece with a tracking marker attached, thereby detecting the drilling start point (OP-start-point) and end point (OP-end-point) to calculate the cutting vector and cutting volume. In this process, the hybrid training device can induce correction close to the actual operation by providing key feedback such as the entry direction, angle, and depth as an AR overlay (vector, gauge, warning display). (iii) When the user selects to perform digital coaching physical procedure training (108), the hybrid training device recognizes the physical tooth model and precisely aligns the virtual tooth model based on the Dentiform origin marker and tooth socket point coordinates, and then provides visual coaching by simultaneously digitally reproducing the actual cutting action.

[0036] In one embodiment, the results of each practice method can be integrated and processed in the training result analysis and report (109) stage. The hybrid training device can calculate E1, E2, and E3 through the intersection and difference operations of the correct answer model and the procedure result model, and calculate a weighted final evaluation score by reflecting metadata such as procedure time (TO) and the number of procedure actions (e.g., number of OP vectors). The hybrid training device can generate a report consisting of a snapshot, a 3D heatmap, a trajectory graph, and a score table for each item based on the analysis results, and can submit it to a teacher's dashboard or display it immediately on a learner's terminal.

[0037] Finally, in one embodiment, at the learning record storage (110) step, the hybrid education device may accumulate a lesson identifier, a method of execution (digital / augmented reality / physical), session time, final evaluation score, snapshot / video evidence, and improvement recommendations in the history storage of the user account. The stored records can be utilized in subsequent sessions for automatic difficulty adjustment, guidance on correcting weak areas, adaptive updates of weights (w2~w6), recommendations for retrying the same case, etc., and through a loop to lesson selection (102), the learner can continue cyclical training by selecting a lecture lesson or a practice lesson again. Through this flow, the present invention implements data linkage between digital procedure training (106), augmented reality procedure training (107), and digital coaching physical procedure training (108) as a consistent user journey, thereby enabling the learner to perform the entire process from concept learning to physical implementation and quantitative evaluation on a single platform.

[0038] FIG. 2 is an exemplary block diagram of a hybrid training device (200) for dental procedure training according to one embodiment of the present invention. In one embodiment, the hybrid training device (200) may include a processor (210), a communication interface (220), a memory (230), and a BUS (240).

[0039] The hybrid educational device (200) of the present invention can extend and implement the tooth modeling module, landmark setting module, and training information provision module defined in the procedural root canal treatment simulation device of Registered Patent No. 10-2757294. For example, the tooth modeling module can generate a CT / CBCT-based 3D surface model based on image data of a real dental model, and separate the root canal, enamel, and dentin regions to convert them into a digital tooth model for lessons. The landmark setting module can automatically recognize anatomical reference points such as CEJ, pulp horn point, and canal orifice point, and be used to set the starting point (OP-start-point) and ending point (OP-end-point) of the cutting vector (OP vector). In addition, the training information provision module is composed of an interface including multiple viewpoints (Frontal, Buccal, Scope, Overview, etc.) and can provide real-time procedure guidance to the learner. In other words, the existing procedural training steps (digital modeling-UX simulation-evaluation) are maintained with the same data structure (landmark coordinates, cutting vector, cutting volume), and by linking this to real dental model training and augmented reality visual feedback, users can experience the same evaluation and coaching system as digital simulation even in precision training using a real handpiece.

[0040] First, the processor (210) may be a component that performs the core computation module of the present invention, serving as a central control unit for executing a dental procedure training application. The processor (210) can drive an augmented reality procedure training module and a real procedure training module, respectively, to control a series of processes such as calculating a cutting vector based on tracking data of a real handpiece, calculating a cutting volume, updating a virtual tooth model, and generating visual feedback. Additionally, the processor (210) can execute an algorithm that compares a correct answer model with a procedure result model to distinguish between undercut, overcut, and normal cutting areas, and calculates a procedure evaluation score. Such computations by the processor (210) can be used to quantify the learner's cutting behavior data and provide real-time feedback and a final report.

[0041] In one embodiment, the communication interface (220) can perform data transmission and reception between the hybrid education device (200) and an external device. Specifically, the communication interface (220) can receive real-time video data acquired from an augmented reality (AR) camera and receive position and posture information of an actual handpiece with a tracking marker attached, and transmit it to the processor (210). Additionally, the communication interface (220) can be connected to a learner's terminal or a cloud server to upload a treatment result model and a learning report, or to download lesson content and correct answer model data from a remote lecture management system. The communication method may include at least one of wired (USB, LAN) or wireless (Bluetooth, Wi-Fi, etc.).

[0042] In one embodiment, the memory (230) may be a storage device that stores instructions and data for executing a dental procedure training application. The memory (230) includes a computer program executable on the processor (210), and the program may include detailed functions such as an augmented reality procedure training module, a real procedure training module, and an evaluation and report module. Additionally, the memory (230) may store a virtual tooth model, a correct answer model, Dentiform origin marker coordinates, tooth socket point coordinates, the learner's procedure history, evaluation score, training report data, etc.

[0043] In one embodiment, the BUS (240) may provide a data transmission path between the processor (210), the communication interface (220), and the memory (230). The BUS (240) is responsible for the exchange of internal data signals and control signals and may be implemented in a parallel or serial communication structure that supports high-speed data transmission. Through this, real-time procedure data received through the communication interface (220) can be transmitted to the processor (210), and the calculation results can be immediately written to the memory (230) or transmitted to an external server.

[0044] In this way, the hybrid training device (200) illustrated in FIG. 2 can function as an integrated platform that supports both digital simulation and real cutting training by organically linking hardware and software. The processor (210) can control a virtual tooth model in an augmented reality environment by executing a computer program stored in memory (230), exchange data in real time with an external camera and sensor device through a communication interface (220), and efficiently manage the computational flow of the entire system through a BUS (240).

[0045] In one embodiment, the processor (210) can update the virtual tooth model in real time by scanning the pose history to detect the first contact point (OP-start point) and the deepest point (OP-end point) of the drilling, and by creating and applying a cylindrical cutting volume by combining it with the user-set OP-radius stored in the memory (230). Additionally, the processor (210) can render the comparison result with the correct answer model residing in the memory (230) as a visual overlay (vector, color highlighting, warning display), and save and transmit the final evaluation score and snapshot in the form of a report.

[0046] Furthermore, the hybrid educational device (200) may optionally provide a supervised mode for instructors and an autonomous learning mode for learners, and may support multi-device synchronization (e.g., real-time feedback monitoring on a teacher's tablet, simultaneous management of multiple learner sessions) through a communication interface (220). If necessary, the coordinate data of the dental model origin marker and the coordinate data of the tooth socket points stored in the memory (230) may be updated through an orthodontic procedure, and hash / signature information may be stored together for version control and integrity verification.

[0047] According to various embodiments, at least one of the components included in the hybrid education device (200) may be omitted, or other components may be added to the hybrid education device (200). Additionally or generally, some components may be implemented as integrated units or as a single or multiple entities. At least some of the components within the hybrid education device (200) may be connected to each other via a bus (240), GPIO (general purpose input / output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface), etc., to exchange data and / or signals. For example, the bus (240) provides communication functions between components of the hybrid education device (200). The bus (240) may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0048] According to various embodiments, the processor (210) of the mixed-type educational device (200) may be configured to perform operations or data processing regarding the control and / or communication of each component (e.g., memory (230)) of the mixed-type educational device (200). The processor (210) may be operatively connected to the components of the mixed-type educational device (200), for example. The processor (210) may load commands or data received from other components of the mixed-type educational device (200) into the memory (230), process commands or data stored in the memory (230), and store result data. The mixed-type educational device (200) may be equipped with one or more processors (210), and the processor (210) disclosed herein may mean a set of one or more processors (210). The processor (210) may control the overall operation of each component of the mixed-type educational device (200). The processor (210) may be configured to include a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), or any form of processor well known in the art of the present disclosure. Additionally, the processor (210) may perform operations for at least one application or program for executing the method according to the embodiments of the present disclosure.

[0049] According to various embodiments, the memory (230) of the mixed-type educational device (200) may store various data, commands and / or information. The memory (230) may load one or more computer programs to execute the method according to the embodiments of the present disclosure. For example, the memory (230) may be implemented as volatile memory such as RAM, or may include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, etc., a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure belongs, but the technical scope of the present disclosure is not limited thereto.

[0050] A computer program may include one or more instructions that cause a processor (210) to perform a method / operation according to various embodiments of the present disclosure when loaded into memory (230). That is, the processor (210) may perform a method / operation according to various embodiments of the present disclosure by executing the one or more instructions. In such cases, a method according to some embodiments of the present disclosure may be implemented through a hybrid educational device (200). Additionally, memory (230) may store various instructions for implementing the operations described in the present disclosure.

[0051] According to various embodiments, the communication interface (220) of the hybrid education device (200) may establish a wired or wireless communication channel with an external camera, a sensor device, a first actual handpiece, a second actual handpiece, an external user interface, etc., and transmit and receive various data. According to one embodiment, the communication interface (220) may include at least one port for being connected via a wired cable to the external camera, a sensor device, a first actual handpiece, a second actual handpiece, an external user interface, etc., in order to communicate via a wired connection with the external camera, a sensor device, a first actual handpiece, a second actual handpiece, an external user interface, etc. In the above case, the communication interface (220) may perform communication with the external camera, a sensor device, a first actual handpiece, a second actual handpiece, an external user interface, etc., which are wired through the at least one port. According to one embodiment, the communication interface (220) may include a cellular communication module and be configured to be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or Wimax). According to various embodiments, the communication interface (220) may include a short-range communication module to transmit and receive data with an external device using short-range communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB), but is not limited thereto. According to one embodiment, the communication interface (220) may include a contactless communication module for contactless communication. Contactless communication may include at least one contactless proximity communication technology, such as, for example, near field communication (NFC) communication, radio frequency identification (RFID) communication, or magnetic secure transmission (MST) communication.

[0052] A hybrid educational device (200) according to various embodiments will not hinder the implementation of the various embodiments disclosed in this document even if some of the components shown in FIG. 2 are omitted or substituted. For example, the hybrid educational device (200) may additionally include an input / output interface, a display, etc., to receive various information, commands, etc. from an administrator or user, or to output various information, data, etc. to an administrator or user through the input / output interface, the display, etc. For example, the display may display various screens based on the control of the processor (210). For example, the display may be implemented in the form of a touch sensor panel (TSP) capable of recognizing contact or proximity of various external objects (e.g., fingers). The touch sensor panel may have various structures and types, and the present disclosure may be applicable regardless of the structure and type of the touch sensor panel. For example, the display may include a capacitive sensor to recognize contact or proximity of various external objects. The capacitive sensor may be composed of a plurality of capacitors, and the capacitive sensor may apply an electrical signal to the capacitors. A capacitor can charge and discharge electric charge in response to the application of an electric signal. When an electric signal is applied to a capacitor, electric charge can be charged in the capacitor depending on the magnitude of the voltage of the electric signal. For example, a display can receive touch input based on signals collected from a capacitance sensor. For example, the display can receive a user's tap touch, double tap touch, sliding touch, drag and drop touch, and long touch. In addition, the input / output interface may include various input / output devices such as a keyboard, mouse, and speaker, in addition to the touch screen.

[0053] In the present disclosure, the operation described as being performed by the mixed-type educational device (200) may be understood, unless otherwise explained, as an operation performed by the processor (210) of the mixed-type educational device (200) executing an instruction stored in memory (220).

[0054] FIG. 3 is a flowchart showing the operation method (300) of a mixed-type educational device for dental procedure training according to one embodiment of the present invention.

[0055] The operation method (300) of the hybrid training device (200) may include the step of executing an augmented reality surgical training module, the step of displaying a first virtual tooth model and a first virtual handpiece in a first virtual space (S310), the step of generating a first cutting vector by tracking the position of a first actual handpiece (S320), the step of calculating a first cutting volume based on the first cutting vector by the augmented reality surgical training module (S330), and the step of updating a first virtual tooth model cut based on the first cutting volume (S340). Each step is described in detail below.

[0056] In one embodiment, the mixed-type training device (200) can display a first virtual tooth model and a first virtual handpiece in a first virtual space by means of an augmented reality procedure training module (S310). The mixed-type training device (200) can initialize the augmented reality procedure training module, establish the first virtual space in a coordinate system generated through camera pose estimation, and then place a first virtual tooth model (mesh / volume data including crown, root, and root canal shapes) corresponding to a lesson identifier at an appropriate scale. The mixed-type training device (200) can initially place a basic pose of the first virtual handpiece based on the user's dominant hand, viewing direction, and working distance parameters, and display visual aid elements such as tool length, bur diameter, and angle gauge together as a HUD or overlay. In one embodiment, for a smooth mixed-reality experience, the mixed-type training device (200) can configure a rendering pipeline to match the brightness, shadows, and subject occlusion of virtual objects with the actual scene by applying environment mapping (plane detection and lighting estimation) and an occlusion mask.

[0057] In one embodiment, the hybrid training device (200) can generate a first cutting vector by tracking the position of the first actual handpiece by an augmented reality surgical training module (S320). The hybrid training device (200) can collect poses (position, posture, confidence) per frame from a multi-faceted tracking marker or IMU sensor attached to the first actual handpiece and refine them into a temporally continuous pose sequence through state estimation techniques such as outlier removal and Kalman filtering. The hybrid training device (200) can define the first cutting vector by responding to a user's drilling start input (at least one of a screen button, voice command, or IoT button signal) and performing a collision check between the pose sequence and the surface of the first virtual tooth model, thereby detecting the moment of first contact as the OP-start point and the point where the depth value based on the tooth coordinate system is maximum in the drilling section as the OP-end point. The hybrid training device (200) can acquire a drill radius specified by a lesson setting or user setting as an OP-radius, and record metadata (including timestamp, pose reliability, and frame index) combining the first cutting vector and the OP-radius in a buffer to provide for volume calculation in a subsequent step. Optionally, when multiple cutting vectors are generated in a continuous operation, the hybrid training device (200) can automatically split or merge the vectors by applying a cutting action segmentation criterion (stop time threshold, speed threshold, direction change threshold).

[0058] In one embodiment, the hybrid training device (200) can calculate a first cutting volume based on a first cutting vector by an augmented reality procedure training module (S330). The hybrid training device (200) can calculate the first cutting volume by constructing a cylindrical or frustoconical cutting solid (applying a correction factor according to the burr shape) based on the length and direction of the first cutting vector and the OP-radius, and by performing a CSG (Boolean difference) operation in the case of a mesh-based model or a 3D mask erasure operation in the case of a voxel-based model. The hybrid training device (200) may also apply a post-processing shader that adds fine noise or a scallop pattern to the cutting volume boundary according to the vector direction, tool rotation speed, and contact time to simulate the physical roughness of the cutting surface. A hybrid education device (200) can prevent frame drops by decomposing long cutting vectors into subsegments and applying volumes incrementally, or by using a GPU compute kernel to perform parallel voxel erasure, in order to ensure AR real-time performance.

[0059] In one embodiment, the hybrid training device (200) can update a first virtual tooth model cut based on a first cutting volume by an augmented reality procedure training module (S340). The hybrid training device (200) can apply the calculated first cutting volume to the first virtual tooth model to update the geometry and immediately reflect the rendering result in the first virtual space. For example, the hybrid training device (200) can provide immediate visual feedback by comparing the updated model with the correct answer model, overlaying the normal cutting area, undercut area, and overcut area as a real-time color heatmap, and displaying the current cutting angle, entry direction, depth, and speed as a vector gauge or numeric HUD. In one embodiment, at the end of the session, the hybrid training device (200) can transmit metadata (procedure time, number of vectors, average angle error, etc.) of all generated first cutting vectors and corresponding first cutting volumes to an evaluation module to accumulate basic data for calculating the final score and generating a report. Optionally, the hybrid training device (200) can enable coaching sessions by streaming the updated model and feedback in real time to a remote instructor console, or maintain synchronization of common lesson identifiers and timestamps for data alignment with the physical training phase of the same lesson.

[0060] In one embodiment, the hybrid training device (200) can prevent over- or under-applying of volume due to misrecognition by automatically excluding or interpolating frames where the pose reliability is below a threshold value when generating the first cutting vector. In one embodiment, the hybrid training device (200) can prevent over-penetration by dynamically adjusting the OP-radius according to the lesson difficulty or by visualizing the upper / lower cutting depth limit as a guide plane. In one embodiment, if the cutting result does not satisfy a specific critical quality standard, the hybrid training device (200) can identify a cause vector (excessive angle variation, entry direction error, etc.) through backtracking analysis and suggest an improvement scenario (resetting the entry direction, changing the bur diameter, etc.).

[0061] As described above, the flowchart illustrated in FIG. 3 can demonstrate that the augmented reality surgical training module can provide the learner with clinically accurate feedback and a quantitative evaluation-based training experience by quantifying the operation of the first actual handpiece in the first virtual space as a first cutting vector, calculating the first cutting volume accordingly, and updating the first virtual tooth model in real time. Furthermore, while the series of steps illustrated in FIG. 3 describes exemplary procedures performed in the augmented reality surgical training module, the present invention is not limited thereto, and substantially identical or similar steps may be performed in the actual surgical training module.

[0062] For example, in the physical procedure training module, a corresponding cutting vector is generated by tracking the position and orientation of the physical handpiece recognized by the AR camera based on the real space where the physical tooth model is located, instead of the first virtual space, and the cutting result can be visualized by calculating the cutting volume using the same computational method. In this case, the cutting result in the real space is mapped in real time to the virtual tooth model within the virtual environment, allowing the same algorithmic computation as the processing step of the augmented reality procedure training module to be performed.

[0063] Accordingly, terms such as "first," "second," etc. used in this specification are merely identification symbols to clarify functional distinctions between different modules (i.e., augmented reality procedure training module and physical procedure training module) and do not necessarily mean physically different components. For example, the first physical handpiece and the second physical handpiece may be the same device, and the first virtual tooth model and the second virtual tooth model may also refer to the same data structure or the same 3D rendering object.

[0064] That is, the notations "first" and "second" are intended to distinguish cases where the same hardware or software components are used sequentially or in parallel in multiple modules, and the technical scope of the present invention is not limited to the performing entity of each step or the module classification. Consequently, the augmented reality procedure training module and the physical procedure training module can share a common data format, tracking algorithm, cutting vector calculation logic, and cutting volume calculation engine, and through this integrated structure, the training results between the two modules can be linked and compared and analyzed. Accordingly, the description of the "first OO" configuration, such as the first virtual space, the first actual handpiece, the first virtual handpiece, and the first virtual tooth model in this disclosure, may be applied to the "second OO" configuration in the same way or with minor modifications appropriately.

[0065] FIG. 4 is a diagram illustrating the process of creating a tooth model for mixed education according to an embodiment of the present invention. FIG. 4 can explain the data flow and outputs between actual teeth, CT scan 3D data, clinical information records excluding patient personal identification information, a digital tooth model for lessons (an example of intraoral rendering), and a physical tooth model for lessons (an example of 3D printing). Additionally, a mixed education device (200) may be involved in some or all of the tooth model creation process. The following describes in detail the linkage procedures between each element.

[0066] First, in one embodiment, actual teeth refer to teeth of dental patients or standard reference models; for example, specimens with clinical characteristics, such as molars with carious lesions, incisors for restoration, or teeth for root canal treatment, may be used. Actual teeth may be subjected to radiographic acquisition, computed tomography, and oral scanner imaging; for example, radiographs may be accumulated as reference material to supplementarily verify the depth of the lesion and the morphology of the root canal. These radiographs may be utilized as a basis for establishing anatomical constraints of the 'correct answer model' when designing lessons in subsequent stages.

[0067] Next, CT scan 3D data refers to raw volume data acquired for actual teeth, and in one embodiment, can be acquired at a resolution of 20-50 micrometers using a microcomputer tomography device. For example, the acquired raw volume may be subjected to a metal artifact correction filter to reduce metal prosthesis-induced artifacts and an anisotropic diffusion filter to suppress noise. Subsequently, a combination of threshold-based segmentation and graph-cut segmentation may be applied to segment the enamel, dentin, pulp cavity, and root canal layer by layer, and mesh holes in thin structures at the root apex may be filled by manual correction. The preprocessed CT scan 3D data can be converted into a surface mesh (e.g., STL, OBJ) through contour extraction, and can be refined to a quality suitable for both educational rendering and 3D printing by performing, for example, mesh simplification, removal of small triangles, normal direction alignment, and scale normalization. For example, a hybrid educational device (200) may be connected to a CT scanner to receive the above information.

[0068] In one embodiment, the hybrid training device (200), in the clinical information recording step excluding the patient's personal identification information, may remove all identifiable information (name, date of birth, medical record number, face image, location information, etc.) for data governance and privacy protection, and then structure and store only training-related attributes, such as, for example, tooth location, caries category, presence or absence of existing restorations, root canal morphology classification, recommended cutting boundary, and allowable cutting depth. In one embodiment, this structured information may be used as lesson metadata and may be used as a label attached to both the digital tooth model for the lesson and the physical tooth model for the lesson, and may be encoded, for example, as a QR code or serial number so that the device can automatically call the corresponding lesson.

[0069] Subsequently, CT scan 3D data can be derived into two types of outputs. First, in one embodiment, a digital tooth model for lessons can be generated through mesh retopology and material mapping to enable real-time rendering in a 3D graphics engine. For example, the 3D graphics engine may be included in a hybrid training device (200). For example, the tooth's outer shape may be configured to express enamel reflective highlights, and the interior may be configured to allow identification of the root canal and pulp cavity through transparency shaders and segment visualization. This digital tooth model for lessons can be placed in a first virtual space on a portable device and an augmented reality camera and used as a common base model for real-world procedural training that provides digital procedural training, augmented reality procedural training, and digital coaching. For example, the correct answer model may be derived based on recommended cutting boundaries and allowable angles included in clinical information records excluding the patient's personal identification information, and may provide normal cutting areas, undercutting areas, and overcutting areas as color heatmaps during the learning phase.

[0070] Second, in one embodiment, a real tooth model for lessons can be produced by converting CT scan 3D data into a solid form suitable for 3D printing. For example, to preserve the empty space of the pulp cavity and root canal, the interior can be separated into a separate part, and then a lower root socket interface (cylindrical or polygonal socket) for inserting a dentiform can be additionally designed. In addition, to verify the bur diameter and cutting radius, test surface indicators (guide rings, depth markings) can be added in the form of fine protrusions, and the material can be selected from photocurable resin, nylon, or composite resin to control mechanical strength and cutting sensation. For example, to simulate the tactile sensation of the inner wall of the root canal, a silicone-based filler can be filled and cured so that the difference in resistance can be felt when contacting the actual handpiece. For example, a hybrid educational device (200) can communicate with a 3D printer and, through the 3D printer, participate in the production of a practical tooth model for lessons as described above and process the necessary data.

[0071] In addition, in one embodiment, a radiograph obtained from an actual tooth can be used as reference data in the quality verification stage of a digital tooth model for lessons and a physical tooth model for lessons. For example, by comparing the root canal curvature angle of the radiograph with the root canal curvature calculated from CT scan 3D data, it can be checked whether the error is below a threshold value, and if the error is large, mesh correction or resetting of printing parameters (scale, shrinkage correction) can be performed. Furthermore, the clinical information record, excluding the patient's personal identification information, is designed to include only the dataset version, production date, and origin tooth identification category for cross-verification, thereby ensuring data integrity during lesson redistribution and follow-up.

[0072] Finally, in one embodiment, the digital tooth model for lessons and the physical tooth model for lessons can be aligned to share the same coordinate system definition and reference points (e.g., cusp apex reference point, marginal ridge reference line, root apex reference point). For example, the socket reference plane of the physical tooth model for lessons can be aligned with the reference plane of the digital tooth model for lessons, and the error can be minimized during augmented reality spatial alignment using the Dentiform Origin and tooth socket points. This alignment consistency can serve as a basis for the training device to mutually convert, compare, and reproduce cutting vectors, cutting volumes, and evaluation results between digital procedure training, augmented reality procedure training, and physical procedure training that provides digital coaching.

[0073] FIG. 5 is a diagram illustrating a cutting vector (OP vector) according to an embodiment of the present invention. In the screen (501) shown in FIG. 5, a virtual tooth model (white silhouette), a cutting area (red shading), and a cutting vector (OP vector) (yellow arrow) indicating the direction and section of the cutting may be shown. A yellow circle may indicate a surface contact point (OP-start-point), and the end point of the arrow may indicate a maximum point (OP-end-point).

[0074] In one embodiment, the surface contact point (OP-start-point) may be defined as the surface coordinates p_s at time t_s, when the first valid collision with the surface of the virtual tooth model is detected while the augmented reality procedure training module or the digital procedure training module tracks the first actual handpiece or user input cursor. In this case, the valid collision may be limited to, for example, cases where (i) a drilling start signal (button / voice / IoT input) by the user is active, (ii) the reliability index of the estimated pose is above a threshold value, and (iii) the angle θ=cos^-1(n·a) between the surface normal vector n(p) of the virtual tooth model and the tool axis vector a(t) is within an allowable range. The deepest point (OP-end-point) may be defined as the coordinates p_e at time t_e, when the depth value based on the tooth coordinate system becomes maximum in the continuous pose sequence until the drilling end signal is input. To remove noise, p_s and p_e can be calculated from a trajectory smoothed by a Kalman filter or a low-pass filter.

[0075] The cutting vector (OP vector) v=p_e-p_s defined in this way can be expressed in terms of length L=∥v∥ and direction v^=v / ∥v∥, and can be used as a basic generation factor for the cutting volume along with a user-defined radius (OP-radius) r. In one embodiment, the cutting volume is a cylindrical solid V_cyl={x|dist(x,p_s p_e) It can be approximated as {r}, and if the burr shape is frustum, it can be extended into a frustum solid with a linear taper by introducing a longitudinal radius correction factor α. For computational convenience, the hybrid training device (200) can divide v into subsegments of a certain length and apply cumulative CSG (Boolean difference) calculation or voxel masking, and the red shading in FIG. 5 can conceptually represent the result of applying such a cutting volume to a virtual tooth model.

[0076] In one embodiment, multiple cutting vectors (OP vectors) may be generated during a single session, and each vector is<p_s, p_e, r, t_s, t_e, θ, conf> It can be recorded in chronological order, including metadata such as. The number of recorded cutting vectors (OP vectors) can be aggregated as the number of procedure actions and used as an evaluation item, and the directional dispersion, angle deviation, and depth exceedance of individual vectors can be provided as real-time feedback via a warning overlay (vector gauge, color highlighting). Additionally, if a pattern is detected where θ exceeds the critical angle and enters the enamel excessively obliquely, the hybrid training device (200) can lower the validity of the cutting vector (OP vector) or immediately display a correction guide.

[0077] Additionally, in the representation of FIG. 5, the red square slope around the surface contact point (OP-start-point) can be interpreted as a guide patch that estimates the cutting entry plane, which visualizes the local tangent plane based on the surface normal and the cutting direction, thereby helping the learner intuitively adjust the entry angle. Optionally, the hybrid training device (200) can decompose the projection of the cutting vector (OP vector) with respect to anatomical reference lines such as cusps, pits, and grooves, calculate the correlation with the anatomical structure preservation score, and display it in parallel as a heatmap.

[0078] Furthermore, after the cutting vector (OP vector) is calculated, the mixed-type training device (200) can calculate the normal cutting area, undercutting area, and overcutting area through cross-difference operations with the correct answer model, which can be directly linked to the calculation of the procedure evaluation score and final evaluation score in the subsequent stage.

[0079] FIG. 6 is a diagram illustrating the operation of calculating and quantitatively evaluating the difference between a correct answer model and a result model according to an embodiment of the present invention. Each screen of FIG. 6 can conceptually represent a procedure for comparing and analyzing a correct answer model and a result model by overlaying them after a simulation or real-world-based cutting is performed on the same lesson.

[0080] First, the screen (601) displays a virtual tooth model in an initial state that serves as a reference for comparison. In the screen (601), a reference surface and internal structure are defined prior to the cutting process, and subsequent operations can be performed in a state that corresponds to this reference coordinate system.

[0081] The screen (602) represents the correct answer model, and area A, which is marked in blue, may represent the ideal cutting target area. In one embodiment, the correct answer model may be predefined as three-dimensional volume data reflecting clinical guidelines, root canal access paths, allowable cutting depths, and angle ranges, and may also load layer-specific tolerances (depth error, angle error, boundary flatness) from lesson metadata, for example. Area A of the correct answer model may be represented as a set of polyhedra in a mesh-based tooth model and a set of voxels in a voxel-based tooth model.

[0082] The screen (603) displays the result of the cutting performed by the learner, and the area B marked in red may represent the cutting volume of the result model. The area B of the result model may be composed of the cumulative value of a Boolean difference or voxel masking operation applied stepwise with the cutting vector and user-set radius as inputs, and, for example, tool shape correction, surface scallop correction, and morphological operations (open / closed) for noise suppression may be applied in advance.

[0083] The screen (604) shows the result of decomposing three sub-regions by overlapping and comparing region A of the correct answer model and region B of the result model. Specifically, the purple region E1 inside the yellow border is the intersection, which is the "normal truncation region (= A It may mean "B". The blue area E2 may mean the difference "under-truncation area (= A - E1)", and the red area E3 may mean the difference "over-truncation area (= B - E1)". In one embodiment, the mixed-type educational device (200) can calculate the volumes (or number of voxels) of areas E1, E2, and E3 as |E1|, |E2|, and |E3|, respectively, and calculate the item-specific index by normalizing them to the volume |A| of the correct answer model.

[0084] Below, the quantitative evaluation procedure based on Fig. 6 will be explained in more detail.

[0085] (1) Alignment and matching step: Rigid transformation (rotation and translation) and, if necessary, fine nonlinear transformation can be performed so that the coordinate system of the result model matches the coordinate system of the correct model. In the case of augmented reality-based real-world training, initial matching can be performed using the Dentiform Origin and tooth socket points, and then fine correction can be performed by boundary feature point matching.

[0086] (2) Binarization and tolerance application step: Deviations of 0.1 to 0.2 millimeters caused by fine vibrations at the boundary or equipment errors can be ignored by applying a critical tolerance τ, and a threshold-based expansion operation can be applied so that the area within the tolerance is incorporated into E1.

[0087] (3) Partitioning step: Regions E1, E2, and E3 can be derived through set operations, and their respective volumes can be calculated. In the case of a mesh-based system, computational load can be reduced by using a fast collision checker and a spatial partitioning structure (e.g., BVH), and in the case of a voxel-based system, voxel counting can be performed using GPU parallel processing.

[0088] (4) Indicator calculation step: The basic evaluation formula of the present invention can be defined as follows.

[0089] Procedure evaluation score = {|E1|(w_2*|E2|+w_3*|E3|)} / |A|, where w_2 is the undercut penalty weight and w_3 is the overcut penalty weight, and w_2 and w_3 can be set asymmetrically depending on lesson characteristics (whether nerve access is available, necessity of preserving adjacent faces, etc.). Additionally, the final evaluation score can be calculated by combining deduction items for operation indicators such as the number of multiple cutting vectors and cutting time.

[0090] The hybrid training device (200) can calculate the penalty for a procedure based on the number of cutting vectors (OP vectors) OC. For example, given the appropriate number of cutting vectors O_min defined by the lesson designer and the allowable maximum number of cutting vectors O_max, the penalty for a procedure is

[0091] "Procedure Procedure Deduction=MIN(O_max-O_min, MAX(OC-O_min, 0)) / (O_max-O_min)"

[0092] It can be calculated as such. This can serve as an indicator to suppress unnecessary cutting attempts (segmented approach, frequent lift-and-place motions) and to induce planned continuous drilling behavior. When calculating OC, the hybrid training device (200) can be configured to logically combine explicit drilling start / end signals (UI, voice, handpiece tracking cap button, etc.) and actual surface contact events to minimize the occurrence of false cutting vectors.

[0093] The mixed-type training device (200) can calculate the treatment time deduction by normalizing the actual time required TO based on the minimum time T_min and maximum time T_max. For example,

[0094] Procedure time deduction=MIN(T_max-T_min, MAX(TO-T_min, 0)) / (T_max-T_min)

[0095] It can be defined as such. Through this, a penalty for an excessively long procedure can be imposed, but T_min can be set as a lower limit to prevent a decrease in safety due to an excessively short procedure. The hybrid training device (200) can separately measure idle time during cutting and actual cutting contact time, and support a scenario-specific policy to adopt only the total time or effective contact time as TO depending on the nature of the lesson.

[0096] The mixed-type educational device (200) can combine the three indicators above to calculate the final evaluation score as follows:

[0097] Final evaluation score = Procedure evaluation score * w_4 - Procedure action deduction * w_5 - Procedure time deduction * w_6, where w_4, w_5, and w_6 can be set as upper weights that can be adjusted according to the lesson objectives (e.g., priority of formation accuracy, priority of tissue preservation, priority of time efficiency, etc.). In one embodiment, the hybrid training device (200) can prioritize accuracy learning by setting w_4 high and w_5 and w_6 moderately for beginner lessons, and conversely, can be configured to simultaneously evaluate proficiency (behavioral economy and time management) by increasing w_5 and w_6 for advanced lessons. The hybrid training device (200) can ensure UI consistency by scaling and clamping the calculated final evaluation score to a range of 0 to 100.

[0098] In one embodiment, the hybrid training device (200) loads initial values ​​of weights w_2, w_3, w_4, w_5, and w_6 from lesson metadata and can subsequently adjust them by statistical analysis of session-accumulated data or by a machine learning-based tuner (e.g., Bayesian optimization or a cross-validation-based regression model). For example, if a specific error pattern (continuous overcutting or undercutting in a specific direction) appears significantly in a user who repeatedly learns the same lesson, the hybrid training device (200) can increase the correction induction power by fine-tuning w_2 or w_3 with personalized weights. Additionally, the lesson designer can pre-define O_min, O_max, T_min, and T_max according to case difficulty and clinical safety limits, and the hybrid training device (200) can stepwise reset the corresponding thresholds according to the user's proficiency level.

[0099] For transparency in the calculation process, the hybrid training device (200) can record intermediate results for each item (e.g., |E1|, |E2|, |E3|, procedure evaluation score, procedure action deduction, procedure time deduction) in a session report and provide them visually to the user in combination with a heatmap overlay (normal / underperforming / overperforming). In one embodiment, the hybrid training device (200) can display a contribution decomposition (Shapley value-like descriptive indicator) for the final evaluation score to help the user intuitively understand which items lost points. In addition, the hybrid training device (200) can estimate the error bias between the virtual and the real by comparing the scores of the AR session and the real session in parallel in the same coordinate system (based on the Dentiform Origin and tooth-specific socket points) and automatically generate a personalized guide for the next session (e.g., angle correction suggestion, approach path redesign).

[0100] (5) Auxiliary indicator step (optional): To improve the reliability of the accuracy indicator, auxiliary indicators such as Jaccard coefficient |A∩B| / |A∪B|, Dice coefficient 2|A∩B| / (|A|+|B|), maximum depth error, average surface distance, and direction angle deviation may be calculated and included in the report.

[0101] In terms of visualization, the hybrid educational device (200) can provide intuitive feedback to the user by overlaying areas E1, E2, and E3 with different colors, such as on the screen (604). For example, a vector guide indicating the direction requiring additional cutting can be displayed in area E2, and a message indicating the need for correction in the prosthetic design can be displayed in area E3 along with an overcutting warning icon. Additionally, a 2D map projected in the upper, lower, buccal, and lingual directions and a cross-sectional view can be provided simultaneously to check spatial deviations from various angles.

[0102] In particular, in one embodiment of the present invention, Area A of the correct answer model is loaded identically at the start of the lesson and can be used consistently among digital procedure training, augmented reality procedure training, and physical procedure training providing digital coaching. Accordingly, Area B of the result model generated in each method is decomposed into Areas E1, E2, and E3 through the same evaluation pipeline and scored using the same weighting system, allowing for quantitative comparison between methods (e.g., improvement in physical performance compared to digital).

[0103] Furthermore, the mixed-type educational device (200) can identify recurring error patterns (e.g., excessive angle, insufficient adjacent part of a specific cusp) by comparing the spatial distribution of areas E2 and E3 with the learner's past history, and can automatically generate recommended practice modules, entry angle correction guides, and bur diameter adjustment suggestions for the next session.

[0104] FIG. 7 is a drawing for explaining a cap-shaped handpiece tracking device (701) with a 5-sided tracking mark applied according to an embodiment of the present invention. FIG. 7 can explain the interrelationships between the handpiece tracking device (701), an augmented reality camera, a 5-sided tracking marker, a cap coupling part, a drill tip reference point (TCP, Tool Center Point), and a QR mark. The configuration and operation are described in detail sequentially below.

[0105] First, the handpiece tracking device (701) may be composed of a cap coupling part detachably coupled to the outer circumference of the head part of the handpiece, and five-sided tracking markers arranged on the front, rear, left, right, and top (total of five sides) of the cap coupling part. In one embodiment, the cap coupling part may be molded from medical polycarbonate or a high-rigidity resin and designed to withstand sterilization and repeated washing, and may be implemented, for example, with a slit-type clamp structure or a screw-type ring structure to allow tolerance correction according to the outer diameter of the handpiece head. The five-sided tracking markers may be printed to have different patterns so that an augmented reality camera can identify and track them from any direction, and for example, at least one of a square marker of the ArUco / AprilTag series, a marker with special corner features emphasized, or an isosceles trapezoidal marker for distortion reduction may be applied.

[0106] An augmented reality camera (not shown) can detect the contour and corner features of five-sided tracking markers from real-time video and calculate the pose (position / orientation) of each marker face through a Perspective-n-Point (PnP) solution based on a projection model. In one embodiment, when multiple faces are simultaneously present in the field of view, the hybrid training device (200) can perform multi-sided fusion pose estimation using visibility reliability, observation angle, area within the image, distortion index, etc. as weights to integrate them into a single pose. For example, if three faces—front, left, and top—are detected simultaneously, a robust handpiece estimation pose can be derived through Kalman filter-based fusion or RANSAC-based disturbance removal procedures that consider the covariance of each pose. Meanwhile, if a specific face is temporarily obscured, tracking continuity can be maintained by interpolating with a predicted pose that combines the previous state and IMU (optional may be included) inertial information.

[0107] Tool Center Point (TCP) correction is a core procedure of the present invention. By pre-learning and recording fixed transformations (rotation and translation) from the reference coordinate system of a 5-sided tracking marker (e.g., defined based on the top face center and edge reference) to the Tool Center Point, the 3D position and orientation of the drill tip can be immediately calculated at runtime using only the estimated pose of the handpiece. In one embodiment, the correction may be performed by (i) a method of sampling the position while continuously contacting the drill tip with a spherical contact jig, or (ii) a multi-pose measurement method with an optical reference frame. For example, it may include a procedure for obtaining fixed transformations as least-squares solutions from a 20-30 pose distribution and requesting re-correction if the residual exceeds a threshold value.

[0109] In one embodiment, the 5-sided tracking marker may include a QR mark. In one embodiment, the QR mark may additionally serve as a configuration for encoding metadata such as the serial number of the handpiece tracking device, drill diameter, drill tip reference point calibration file identifier, and lesson ID. For example, the QR mark may allow the augmented reality camera to automatically load device settings upon initial recognition.

[0110] In terms of real-time rendering, the hybrid training device (200) can align a virtual handpiece within a first virtual space or a second virtual space based on the estimated handpiece pose and the drill tip reference point, and display a drill axis direction vector, an entry angle gauge, and a depth gauge as overlays. For example, if the angle error between the drill axis and the tooth surface normal vector exceeds a critical angle, it can be switched to a color warning (red), and if the entry angle is within the acceptable range, it can be displayed as a color stable (green). Additionally, tracking quality indicators (number of corners, reprojection error, number of visible faces) are displayed on the HUD so that the learner and the instructor can immediately determine the tracking reliability.

[0111] In terms of durability and infection control, the cap attachment can be implemented with a structure combining a sterile polymer body and a disposable thin outer shell. For example, the outer shell may be provided in the form of a heat-shrink polymer or a sterile pouch, allowing for replacement after each session to reduce the risk of cross-contamination. The marker pattern may be formed using laser etching or UV-curing ink to prevent printing smudging caused by alcohol or hypochlorous acid cleaning. Additionally, a matte coating to reduce surface diffuse reflection and a fine texture to suppress specular highlights under dental lighting may be applied.

[0112] For accuracy and latency removal, the hybrid training device (200) can perform frame filtering based on reprojection error and time synchronization (timestamp correction) to correct the latency of the image processing pipeline. For example, by using an extrapolated pose by modeling the difference between the augmented reality camera frame timestamp and the time of application of the handpiece estimated pose, visual and motion alignment errors can be minimized in a 60fps display-based environment. Optionally, a small IMU can be added to the handpiece tracking device to improve the stability of pose prediction even in rapid acceleration and occlusion situations.

[0113] Finally, the handpiece tracking device (701) can be linked to the same cutting vector calculation logic and cutting volume calculation engine by sharing a common data format (pose, reliability, reprojection error, visible surface index) with the augmented reality procedure training module and the real procedure training module that provides digital coaching. For example, the cutting vector can be defined by detecting the surface contact point (OP-start-point) and the deepest point (OP-end-point) in the drill tip reference point trajectory provided by the hybrid training device (200), and a real-time feedback loop can be formed that is immediately reflected in the virtual tooth model.

[0114] FIG. 8 is a drawing illustrating a virtual space according to one embodiment of the present invention.

[0115] FIG. 8 illustrates an example in which a hybrid training device (200) according to an embodiment of the present invention creates a first virtual space to superimpose a first virtual tooth model and a first virtual handpiece on an augmented reality screen (801) and provides real-time guidance and evaluation assistance information. The hybrid training device (200) can precisely align and display a rendered 3D object against a real scene (a real dental model, a work surface, a user's hand, etc.) acquired by a first AR camera through an augmented reality procedure training module.

[0116] In one embodiment, the hybrid training device (200) may place a first virtual tooth model corresponding to a translucent tooth in the center of the screen and provide a transparency mode to intuitively check the internal anatomical structure. The transparent toggle in the right UI list can be configured to change the surface shader of the first virtual tooth model to make the enamel translucent and to visualize the root canal and pulp cavity. The gum and orifice toggles can be configured to configure a field of view suitable for the training goal by turning the visibility of the gum mesh and the root canal opening marker layers on and off, respectively.

[0117] In one embodiment, the hybrid training device (200) may align the first virtual handpiece at the top with a pose transmitted from a cap-shaped handpiece tracking device with a 5-sided tracking marker, and display cutting vector candidates with green directional arrows based on the drill tip reference point position. The arrows may be configured to change color or thickness in proportion to the angular error between the current tool axis and the first virtual tooth model surface normal, and may switch to a warning animation if they exceed the acceptable range. The hybrid training device (200) may support entry depth control by displaying the projected distance (e.g., "2.7 cm") from the drill tip to the current target contact point numerically along with a concentric circle gauge.

[0118] In one embodiment, the hybrid education device (200) can guide the current viewing direction by corresponding it to the subject dental coordinate system using a circular viewpoint indicator UI (e.g., Buccal View notation) at the bottom of the screen. When the user performs a viewpoint rotation, the hybrid education device (200) can synchronizely update the label and orientation marker of the first virtual tooth model while maintaining a reference axis based on the Dentiform origin and the tooth socket point. The Reposition control at the bottom right is an AR reference plane reset function, which allows the hybrid education device (200) to reset the origin and scale of the first virtual space by recognizing the reference marker (e.g., QR code or origin marker).

[0119] In one embodiment, the hybrid training device (200) can clearly distinguish the drilling effective section through the Handpiece ON status indicator on the upper left. When the user starts drilling in the effective section, the hybrid training device (200) can automatically detect the surface contact point (OP-start-point) and calculate the deepest point (OP-end-point) at the end of drilling to record the cutting vector (OP vector). Based on the recorded cutting vector (OP vector) and the user-set radius (OP-radius), the cutting volume is calculated immediately, and the result is reflected in the first virtual tooth model to visualize the cutting progress status in real time.

[0120] In one embodiment, the hybrid training device (200) performs a cross-difference operation with the correct answer model in the background on the cutting history accumulated during the session, and can progressively display normal cutting areas, undercutting areas, and overcutting areas as a color heatmap. If necessary, the hybrid training device (200) can be configured so that the learner selectively reviews only specific error sections by individually toggling the heatmap overlay and direction vector guide through a simple menu on the right side of the screen. Additionally, the hybrid training device (200) can operate such that the Back button and pause icon at the bottom left trigger a session stage transition and a data snapshot save.

[0121] In one embodiment, when the hybrid education device (200) recognizes a QR mark within the scene, it can automatically load the metadata of the corresponding lesson (correct answer model ID, allowable tolerance, recommended entry path, weight settings) to configure the first virtual space for each lesson. In addition, the hybrid education device (200) can evaluate tracking reliability and reprojection error for each frame and, if a certain threshold is exceeded, recommend repositioning to the user or switch to a temporary inertial prediction mode to maintain visual continuity.

[0122] FIG. 9 is a drawing illustrating a virtual space according to an embodiment of the present invention. FIG. 9 describes a scene in which a hybrid training device (200) interacts with and displays and controls a handpiece tracking cap, an actual handpiece, simulation virtual guide information, and a virtual model in a virtual space (901) according to an embodiment of the present invention. First, the virtual space (901) illustrated in FIG. 9 may be a screen connected to the virtual space (801) of FIG. 8 described earlier. Therefore, the description of the virtual space (801) of FIG. 8 may be applied equally to the virtual space (901) of FIG. 9. The hybrid training device (200) first recognizes the multi-faceted tracking marker of the handpiece tracking cap to estimate the pose (position / position) of the actual handpiece and calculates a drill tip reference point based on that pose. Subsequently, a virtual model aligned by the Dentiform Origin and tooth socket point (e.g., the first virtual tooth model of the corresponding lesson) is superimposed onto a real-world surface, and simulation virtual guide information is provided as an overlay.

[0123] The hybrid training device (200) activates, in stages, the drill axis direction vector, entry angle / depth gauge, allowable angle cone, cutting prohibition zone boundary, root canal opening marker, etc., as simulation virtual guide information. For example, when the user aligns the actual handpiece with the root canal approach path, the vector stabilizes in green, and when it deviates from the allowable angle, an audible or vibration notification is generated along with a warning animation. Additionally, the hybrid training device (200) periodically evaluates the tracking reliability calculated from the number of visible surfaces of the handpiece tracking cap and the reprojection error, and when the reliability is below a threshold, it automatically relaxes the sensitivity of the guide or guides a realignment operation.

[0124] The hybrid training device (200) records the surface contact point (OP-start-point) at the point where the drill tip of the actual handpiece penetrates the surface of the virtual model, and extracts the deepest point (OP-end-point) at the end of the drilling to generate a cutting vector. The generated cutting vector is combined with a user-set radius and immediately converted into a cutting volume, and the result of the conversion is reflected in a real-time cutting simulation on the virtual model. At this time, the hybrid training device (200) manages the accumulated cutting history within the same session by layer, allowing the user to refer to and review past paths or retrain by returning only to specific areas.

[0125] The hybrid training device (200) generates automatic coaching sentences using the mutual positional relationship between the virtual model and the simulated virtual guide information. For example, if a risk of overcutting is detected along the inner slope of the cusp of the drill tip, phrases such as "decrease 5° to the buccal side" or "maintain depth 0.6 mm" are provided on the edge of the screen or as a head-up label. In addition, user interaction logs (guide toggle, viewpoint switch, Back / Reposition selection, etc.) and cutting vector / cutting volume data are stored together as timestamps so that the training result analysis and report module can subsequently generate an error heatmap and time-motion chart against the correct model.

[0126] In addition, the hybrid educational device (200) automatically reloads the virtual model and guide parameters (allowable tolerance, recommended entry path, weight) by matching the QR-based auxiliary identifier and the lesson metadata even when the handpiece tracking cap is replaced or the lesson is changed. In this way, in the virtual space illustrated in FIG. 9, the physical operation of the actual handpiece and the digital feedback of the virtual model are closely combined through the simulation virtual guide information, so that the learner can immediately receive coaching regarding direction, depth, and speed and repeatedly correct within the same screen.

[0127] FIGS. 10 to 14 are drawings for explaining the production of a training 3D printing model according to an embodiment of the present invention.

[0128] FIG. 10 illustrates the state in which a hybrid educational device (200) loads and aligns / organizes computed tomography-based 3D data to produce a tooth model for lessons. The hybrid educational device (200) can, for example, mesh computed tomography raw voxel data obtained from actual patient teeth by iso-surface extraction, and perform Laplacian smoothing and region segmentation to remove surface noise and separate oral soft tissues. The hybrid educational device (200) can integrate the crown, cervical, and root into a single digital tooth model, and then automatically estimate the centerline of the root canal to set an anatomy-based reference axis. Additionally, to correct for real-world output errors, the entire scale can be derived at training magnifications of 1x, 2x, and 3x to manage according to the difficulty level of the lesson.

[0129] FIG. 11 illustrates a hybrid training device (200) designing a socket-type root base for inserting a dentiform and attaching it to the bottom of a digital tooth model. The hybrid training device (200) can automatically generate the shape of the root base by retrieving the coordinate system of the dentiform origin and the tooth socket point to align with the corresponding coordinate system. For example, the outer circumference of the root base may include a key slot defining the insertion direction, and the height direction may include a step surface serving as a reference for the insertion depth. The hybrid training device (200) may apply an external tolerance (e.g., ±0.05 millimeters) by considering the elastic modulus of the dentiform material and the shrinkage rate of the output material, and may add fine ribs to prevent loosening during repeated insertion. The shallow groove markings on the outer surface can be used as socket alignment markers to provide the same reference line even during augmented reality alignment.

[0130] FIG. 12 illustrates an example in which a hybrid educational device (200) performs a segmented design to reproduce the internal shape of the pulp chamber and the root canal. The hybrid educational device (200) can divide the crown portion into an upper part and the root base into a lower part, and design an inclined snap structure and a matching pin at the joining position. For example, an internal channel following the actual curvature of the root canal can be formed on the inner side of the upper part, and an opening reinforcement bead can be placed on the upper side of the lower part to prevent damage even during repeated drilling. The hybrid educational device (200) can design the segmentation surface as an oblique wavy surface to suppress positional errors during assembly and minimize channel damage by ensuring that force is applied only in a specific direction during disassembly. Furthermore, to increase consistency with augmented reality simulations, the coordinate origin of each part is recorded as a relative coordinate to the Dentiform Origin, thereby facilitating positional alignment during virtual-to-real object conversion.

[0131] FIG. 13 illustrates the result of a hybrid educational device (200) producing a three-dimensionally printed upper part and lower part using design data. The hybrid educational device (200) can print with a layer thickness of 50 micrometers to 100 micrometers using, for example, a photopolymerization resin type printer, and remove surface residue by performing isothermal curing and ultrasonic cleaning after removing the support. The joining surface of the upper part and lower part is provided with an intaglio off-row that compensates for curing shrinkage to enable dry-fit assembly, and if necessary, a replaceable internal channel insert can be inserted to adjust tactile feedback of the root canal (e.g., filling with viscous material). Additionally, the hybrid educational device (200) can control optical diffuse reflection by using a translucent material mixed with pigment so that the internal structure appears naturally in an augmented reality transparent mode.

[0132] FIG. 14 illustrates an example in which a hybrid training device (200) inserts and places a completed training 3D printed model into the corresponding tooth socket point of a real dental model. The hybrid training device (200) recognizes the origin marker on the dental model to align the virtual tooth model to the same position on the augmented reality screen, and in that state, can project the cutting guide, direction vector, and depth gauge provided by the augmented reality procedure training module and the real procedure training module onto the real model. For example, the educator can sequentially adjust the difficulty level by replacing and inserting training 3D printed models at magnifications of 1x, 2x, and 3x for the same case, and the hybrid training device (200) can link and store the cutting vector, cutting volume, and procedure evaluation score of each session in a learning record repository. Furthermore, the hybrid training device (200) can simulate interference with surrounding teeth by considering the tooth alignment and occlusal surface curvature of the dental model, and, if necessary, automatically correct the key slot angle of the root base to adjust it to secure an approach path identical to the clinical field of view.

[0133] FIGS. 15 and 16 illustrate a dentiform for mapping a real tooth model to a virtual tooth model according to an embodiment of the present invention. FIGS. 15 and 16 can explain the structure of the dentiform, the definition of the coordinate system, and the mapping procedure.

[0134] First, referring to FIG. 15, a hybrid educational device (200) can recognize a dentiform (1501) having a plurality of sockets formed for inserting a replacement real tooth. The hybrid educational device (200) can calculate the position and orientation of the dentiform (1501) in an augmented reality coordinate system by using a dentiform origin marker (1502) attached to the central surface area of ​​the dentiform (1501) as a tracking reference point. The dentiform (1501) may have a plurality of tooth-specific socket pivots (1503) formed therein, and each tooth-specific socket pivot (1503) may be manufactured with coordinates corresponding to a tooth numbering system to reflect the alignment of the upper or lower jaw teeth. The hybrid training device (200) can use the identifier of the Dentiform origin marker (1502) (e.g., payload of a two-dimensional code) as a key to query the Dentiform-specific correction parameters, the coordinate table of the tooth-specific socket pivot (1503), the tooth number mapping table, etc., that are stored in memory in advance. Accordingly, the hybrid training device (200) can load the reference position of the tooth-specific socket pivot (1503) corresponding to the tooth number selected in a specific training case, and position the initial pose of the virtual tooth model to align with the corresponding tooth-specific socket pivot (1503) on the Dentiform (1501).

[0135] Next, referring to FIG. 16, the hybrid education device (200) can define an orthogonal coordinate system (1601) with the Dentiform origin marker (1602) as the origin. For example, the hybrid education device (200) can set the center of the Dentiform origin marker (1602) to (0,0,0) and determine the X-axis, Y-axis, and Z-axis respectively to correspond to the front-back, left-right, and up-down directions of the Dentiform. Additionally, the hybrid education device (200) can pre-calculate and store relative position and rotation (pose) information for each tooth socket pivot (1603) as a transformation matrix based on the Dentiform origin marker (1602). In one embodiment, the hybrid training device (200) can measure the center point, axial direction, tilt, etc. of each tooth's socket pivot (1603) using a scanning device or a measuring jig during manufacturing, and normalize this into parameters (Tx, Ty, Tz, Rx, Ry, Rz) and save it as a profile. Subsequently, during operation, when the hybrid training device (200) detects the dentiform origin marker (1602) with an augmented reality camera, the pose of the corresponding virtual tooth model can be immediately calculated by synthesizing the pre-offset of each tooth's socket pivot (1603) with the pose of the dentiform origin marker (1602) calculated in real time.

[0136] The hybrid training device (200) can correct projection errors by applying a camera correction matrix and distortion coefficients during the pose synthesis step. Additionally, in one embodiment, the hybrid training device (200) can perform auxiliary pose matching by using the boundary contour of the dentiform (1501) or the circular rim feature point of the tooth-specific socket pocket (1603) as additional reference features when the observation reliability of the dentiform origin marker (1602) falls below a predetermined threshold (e.g., part of the marker is obscured). Furthermore, the hybrid training device (200) can suppress inter-frame shaking and ensure visual stability of the virtual tooth model by applying a Kalman filter, a particle filter, or non-linear least squares-based pose smoothing.

[0137] The hybrid training device (200) may also provide a correction step to increase the precision of the mapping results. For example, after a user inserts a real tooth model into a specific tooth-specific socket pocket (1603), if the user operates a fine-tuning slider or snap alignment button so that the outline of the virtual tooth model on the screen matches the outer edge of the real tooth model, the hybrid training device (200) can estimate the error vector (position and rotation) at that moment and accumulate it in the correction parameters for each dental model (on-device re-correction). This cumulative correction can increase reproducibility when using the same dental model (1501) and reduce systematic deviations caused by individual hand tremors, camera angle differences, etc., in a multi-user environment.

[0138] When the mapping is complete, the hybrid training device (200) defines the coordinate system of the selected tooth-specific socket pocket (1603) as the "tooth local coordinate system" and can perform calculations of the cutting vector and cutting volume in the corresponding coordinate system. For example, the hybrid training device (200) can project the tool axis direction of the handpiece tracking device onto the tooth local coordinate system to decompose the entry angle, inclination angle, and rotation angle, and provide a visual guide in real time. In addition, the hybrid training device (200) can convert the pose of the "correct answer model" for the same tooth position to the same coordinate system and calculate the undercut area, overcut area, and normal cutting area on a frame-by-frame basis.

[0139] In one embodiment, the hybrid education device (200) assigns an arbitrary identification number to the dentiform origin marker (1602) and stores the coordinate profile for each dentiform in a cloud server or local database, thereby maintaining consistent mapping quality even between different dentiforms (1501). Additionally, when micro-dimensional changes due to thermal expansion or material hardening of the dentiform (1501) are detected, the hybrid education device (200) can estimate and correct global scale factors and anisotropic deformation by performing scale correction using a multi-point standard (combination of multiple tooth-specific socket pockets (1603)).

[0140] Various embodiments of the present disclosure and effects according to the embodiments have been described so far with reference to FIGS. 1 to 16.

[0141] The effects according to the technical concept of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description in the specification.

[0142] The technical concept of the present disclosure, described so far with reference to FIGS. 1 through 16, may be implemented as computer-readable code on a computer-readable medium. The computer-readable recording medium may be, for example, a removable recording medium (CD, DVD, Blu-ray disc, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). The computer program recorded on the computer-readable recording medium may be transmitted to another computing device via a network such as the Internet and installed on the other computing device, thereby being used on the other computing device.

[0143] In the foregoing, although all components constituting the embodiments of the present disclosure have been described as being combined or operating together, the technical concept of the present disclosure is not necessarily limited to such embodiments. That is, within the scope of the purpose of the present disclosure, all components may be selectively combined in one or more ways to operate.

[0144] Although operations are depicted in a specific order in the drawings, it should not be understood that the operations must be executed in the specific order depicted or in a sequential order, or that all depicted operations must be executed to obtain the desired result. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various configurations in the embodiments described above should not be understood as a necessary separation, and it should be understood that the described program components and systems can generally be integrated together into a single software product or packaged into multiple software products.

[0145] Although embodiments of the present disclosure have been described above with reference to the attached drawings, those skilled in the art will understand that the present disclosure may be practiced in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of protection of the present disclosure shall be interpreted by the claims below, and all technical concepts within the equivalent scope shall be interpreted as being included within the scope of rights of the technical concepts defined by the present disclosure.

Claims

Claim 1 A mixed-type educational device for dental procedure training, comprising: a memory for storing at least one process for executing a dental procedure training application; The apparatus includes a processor that executes the dental procedure training application according to the above process, wherein the processor includes an Augmented Reality (AR) procedure training module and a physical procedure training module that provides digital coaching, wherein the Augmented Reality procedure training module displays a first virtual tooth model and a first virtual handpiece in a first virtual space, tracks the position of a first actual handpiece to generate a first cutting vector, calculates a first cutting volume based on the first cutting vector, and updates the first virtual tooth model cut based on the first cutting volume, wherein the physical procedure training module recognizes the physical tooth model with a second AR camera and matches it with a second virtual tooth model, displays the second virtual tooth model and a second virtual handpiece in a second virtual space, tracks the position of a second actual handpiece to generate a second cutting vector, calculates a second cutting volume based on the second cutting vector, updates the second virtual tooth model cut based on the second cutting volume, and provides visual feedback on the cutting process in the second virtual space, wherein the first cutting vector is drilling A mixed-type training device for dental procedure training, characterized in that the first cutting volume is defined including the surface contact point at the start time (OP-start-point) and the deepest point at the end time of drilling (OP-end-point), the first cutting volume is calculated based on the first cutting vector and the user-set radius (OP-radius), the second cutting vector is defined including the surface contact point at the start time of drilling and the deepest point at the end time of drilling, and the second cutting volume is calculated based on the second cutting vector and the user-set radius. Claim 2 delete Claim 3 delete Claim 4 A hybrid training device for dental procedure training according to claim 1, wherein the augmented reality procedure training module analyzes an image of the real space acquired from the first AR camera, tracks the position and posture of the first actual handpiece attached with a 5-sided tracking marker, and aligns and corrects the position and posture of the first virtual handpiece in real time based on the position and posture of the first actual handpiece, and the real procedure training module analyzes an image of the real space acquired from the second AR camera, tracks the position and posture of the second actual handpiece attached with a 5-sided tracking marker, and aligns and corrects the position and posture of the second virtual handpiece in real time based on the position and posture of the second actual handpiece. Claim 5 A mixed-type educational device for dental procedure training according to claim 1, wherein the augmented reality procedure training module compares a first correct answer model with a first procedure result model in which the first virtual tooth model is cut to calculate a first undercut area, a first overcut area, and a first normal cut area, calculates a first procedure evaluation score based on the first undercut area, the first overcut area, and the first normal cut area, and visually provides the first procedure evaluation score to a learner, and the real procedure training module compares a second correct answer model with a second procedure result model in which the second virtual tooth model is cut to calculate a second undercut area, a second overcut area, and a second normal cut area, calculates a second procedure evaluation score based on the second undercut area, the second overcut area, and the second normal cut area, and visually provides the second procedure evaluation score to a learner. Claim 6 A hybrid training device for dental procedure training according to claim 1, wherein the memory stores in advance coordinate data of a Dentiform Origin marker for recognizing the actual tooth model and coordinate data of a plurality of tooth socket points, and the actual procedure training module, when the actual Dentiform is recognized, uses the coordinate data of the Dentiform Origin marker and the coordinate data of the tooth socket points to precisely align the second virtual tooth model to the actual tooth model. Claim 7 A mixed-type training device for dental procedure training according to claim 1, wherein the augmented reality procedure training module tracks the movement of the first physical handpiece in real time to calculate a first cutting angle, a first cutting depth, and a first cutting speed, and displays real-time feedback including at least one of a vector, color highlighting, or warning display on the first virtual tooth model in the first virtual space based on the first cutting angle, the first cutting depth, and the first cutting speed, and the physical procedure training module tracks the movement of the second physical handpiece in real time to calculate a second cutting angle, a second cutting depth, and a second cutting speed, and displays real-time feedback including at least one of a vector, color highlighting, or warning display on the second virtual tooth model in the second virtual space based on the second cutting angle, the second cutting depth, and the second cutting speed. Claim 8 A mixed-type educational device for dental procedure training according to claim 5, wherein the augmented reality procedure training module calculates a first procedure time deduction based on a first procedure time, calculates a first procedure action deduction based on the number of first procedure actions, calculates a first final evaluation score based on the first procedure evaluation score, the first procedure time deduction, and the first procedure action deduction, and the real procedure training module calculates a second procedure time deduction based on a second procedure time, calculates a second procedure action deduction based on the number of second procedure actions, and calculates a second final evaluation score based on the second procedure evaluation score, the second procedure time deduction, and the second procedure action deduction. Claim 9 A mixed-type educational device for dental procedure training according to claim 1, wherein the augmented reality procedure training module and the real procedure training module use a machine learning algorithm to compare and analyze the uniformity of procedure results, angle error, and cutting depth distribution with the learner's past history, and present improvement guidelines for future training stages. Claim 10 A method of operation performed by a device comprises: a step of displaying a first virtual tooth model and a first virtual handpiece in a first virtual space by an augmented reality surgical training module of the device; a step of generating a first cutting vector by tracking the position of a first actual handpiece by the augmented reality surgical training module; a step of calculating a first cutting volume based on the first cutting vector by the augmented reality surgical training module; and a step of updating the first virtual tooth model cut based on the first cutting volume by the augmented reality surgical training module; wherein, by a real surgical training module providing digital coaching of the device, the real tooth model is recognized by a second AR camera and matched with a second virtual tooth model; a step of displaying the second virtual tooth model and a second virtual handpiece in a second virtual space; a step of generating a second cutting vector by tracking the position of a second actual handpiece; a step of calculating a second cutting volume based on the second cutting vector; and a step of updating the second virtual tooth model cut based on the second cutting volume. A method of operation of a device comprising the step of providing visual feedback on a cutting process in the second virtual space; wherein the first cutting vector is defined including a surface contact point (OP-start-point) at the start of drilling and a deep point (OP-end-point) at the end of drilling, the first cutting volume is calculated based on the first cutting vector and a user-set radius (OP-radius), the second cutting vector is defined including a surface contact point at the start of drilling and a deep point at the end of drilling, and the second cutting volume is calculated based on the second cutting vector and a user-set radius.

Images