Systems and methods for simulating medical and dental procedures

Abstract

Disclosed are systems, devices, methods, and other implementations, including a system for simulating a medical or dental procedure that includes a simulation structure with one or more simulation parts emulating anatomical parts, a plurality of sensors distributed in a procedure area in the simulation structure at which a simulation of a medical or dental procedure is being performed, with at least some of the plurality of sensors configured to track mechanical displacements in the procedure area caused through application of an operation tool handled by a user, and a computing device. The computing device is configured to determine, based on the tracked mechanical displacements, a dynamic physical state of the simulation parts and the operation tool, resulting from performance of the procedure, and determine, based at least in part on the determined dynamic physical state, procedure performance feedback data representative of the procedure's performance by the user.

Description

Attorney Reference No.: 7551-0010W001SYSTEMS AND METHODS FOR SIMULATING MEDICAL AND DENTAL PROCEDURESCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of, and priority to, U.S. Provisional Application No. 63 / 753,744, filed February 4, 2025, entitled “Systems and Methods for Simulating Medical and Dental Procedures,” the content of which is incorporated herein by reference in its entirety.TECHNICAL FIELD

[0001] The present disclosure relates generally to an improved medical and / or dental simulation platform used for training medical / dental professionals, and more particularly to a simulation platform using a manikin with realistic modeling of body parts, and a distributed set of sensors (e.g., vibration sensors, inertial sensors, optical sensors, etc.) to detect and collect haptic data, motion data, etc., generated during performance of dental and / or medical procedures on the manikin, and to determine forces and other measurable quantities causing the detected (tracked) data so as to update models of the dental or medical procedures being performed on the affected body parts.BACKGROUND

[0002] When performing dental or medical procedures, precision and accuracy are paramount, especially when using tools to perform such procedures, e.g., operations on a patient’s teeth. Dentists, for example, often rely on their tactile feedback and visual cues to guide their actions. However, these traditional training and feedback methods for such dental and medical procedures can be limiting, particularly in complex procedures or when teaching and training new dentists.Attorney Reference No.: 7551-0010W001SUMMARY

[0003] Current dental simulators and visualization tools lack the ability to provide real-time feedback on tool-to-tooth interaction. They often fail to accurately track which tooth is being operated on, especially when compensating for patient head movements. To address this, advanced visualization and feedback systems are needed to assist dental and / or medical professionals and trainees in real-time, allowing them to see how their tools interact with the teeth, improving both accuracy and outcomes.

[0004] The present disclosure describes cutting-edge dental / medical manikins (optionally with loT-based devices) with integrated 3D tracking sensors designed to track dental and / or medical procedures in real time. The manikin is attachable to a variety of dental chairs and features such as replaceable teeth and soft tissue elements. It provides real-time (and in some embodiments, Al-powered) feedback on procedures, including handpiece angulation and tooth-level tracking, enhancing training and real-life outcomes. The manikin’s design is customizable and visually appealing, aimed at engaging users on social media and improving overall training experiences with modem technology integration. In contrast, existing dental / medical procedure training platforms lack real-time, detailed feedback during dental training and are generally not visually appealing. The manikins of the proposed platform integrate 3D tracking technology and other features to track handpiece movements and tooth- specific details, offering feedback that reduces the reliance on live educators. The flexible implementations accommodates different procedures, while its modem, appealing aesthetic encourages social media sharing, and wide market adoption. Replaceable parts and customizable options further enhance its usability and engagement. The proposed platform is configured to measure, among other things, “time on tooth” and detect which tooth is actively being worked on, and provide visualization and performance metrics on, among other things, tool-on-tooth interactions.

[0005] In some embodiments, the proposed platform utilizes a physics engine (implemented on a computing device) to compute which tooth is being operated on, and various performance parameters (such as time spent working on a tooth). In some example, two (2) inertial measurement units (IMUs) on the tool head give an accurate tool vector in a 3D space. In some embodiments, the teeth and gum set may include a homing button for tool head. The teeth and gum set may also include a pair of IMUs on top and bottom halves of the mouth area to compensate for head movements. In some examples, the proposed platformAttorney Reference No.: 7551-0010W001may include several strategically placed accelerometers to sense the vibration and adjust for position of the head in 3D space. A collision detection system is configured to compute, based on collected sensor data, which tooth is being worked on. The tool’s IMUs, along with teeth and gum sensors can be connected to a custom computing system via a wired or wireless connection. Tools and tool handles can be catalogued for accurate modeling

[0006] The improved performance of the proposed simulation platform is achieved through a combination of one or more of, for example, 3D tracking technology, wireless or wired connectivity (optionally to establish loT functionality), and / or real-time feedback (optionally facilitated by Al). The manikin also offers a highly customizable design, with swappable soft tissue elements and teeth, and an appealing aesthetic that makes it visually striking.Advantages of the proposed platform include: a) real-time, 3D-tracked feedback, providing far more detailed and precise feedback than existing models, b) customizability, providing an adaptable design for different procedures, chair types, and visual preferences, c) aesthetic appeal achieved by a sleek design that encourages users to share on social media, increasing visibility and engagement, and loT integration that allows connectivity to user interface programs, adding a layer of digital innovation for better training experiences.

[0007] Accordingly, in one aspect, a system for simulating medical and / or dental procedures is provided that includes a simulation structure with one or more simulation parts (e.g., drillable replaceable teeth) emulating anatomical parts (of a human or animal), a plurality of sensors distributed in a procedure area (e.g., in the mouth area in the case of dental simulation structure) in the simulation structure at which a simulation of a medical or dental procedure is being performed, with at least some of the plurality of sensors configured to detect / track mechanical displacements in the procedure area caused through application of an operation tool handled by a user (e.g., a trainee) to at least one simulation part from the one or more simulation parts in the procedure area, and a computing device including one or more processor-based devices coupled to one or more memory storage devices. The computing device is configured to determine, based on the tracked mechanical displacements, a dynamic physical state of at least some of the one or more simulation parts and the operation tool, resulting from performance of the medical or dental procedure in the procedure area, and to determine, based at least in part on the determined dynamic physical state, procedure performance feedback data representing of the procedure’s performance by the user.Attorney Reference No.: 7551-0010W001

[0008] Embodiments of the system may include at least some of the features described in the present disclosure, including one or more of the following features.

[0009] The computing device configured to determine the physical state can be configured to perform one or more of, for example, i) track linear and angular movements of the at least some of the one or more simulation parts, the operation tool, a body part of the user, and / or the simulation structure, and / or ii) determine forces applied on the at least some of the one or more simulation parts and / or the operation tool resulting from interaction of the operation tool with the at least one simulation part.

[0010] The tracked mechanical displacements may include one or more of, for example, detected vibrations, and / or detected spatial movements of the at least some of the one or more simulation parts and / or the operation tool.

[0011] The simulation structure may include a manikin with a physical model of a head part having a mouth, with the one or more simulation parts including simulation teeth constituting a training simulation denture set, and with the operation tool including a dental tool handled by the user to perform a simulated dental procedure on at least one tooth from the simulation teeth.

[0012] For at least one tooth from the simulation teeth, the physical model of the head part can further include respective one or more infrared (IR) LEDs and respective one or more IR sensors embedded underneath the at least one tooth. The computing device may further be configured to controllably cause emissions of the one or more IR LEDs into a structure of the at least one tooth during performance of the dental procedure, and determine changes to the structure of the at least one tooth, caused by the dental procedure, based on changes of characteristics of reflected IR signals from surfaces of the internal structure of the at least one tooth, resulting from the changes to the structure of the at least one tooth.

[0013] At least one tooth from the simulation teeth may be a customized multi-layered tooth, shaped and dimensioned according to grid-based specifications specifying coordinates of features of the at least one tooth, with layers of the multi-layered at least one tooth being composed from respective materials having different material characteristics representing different anatomical layers of an actual tooth.

[0014] The head part can include a controllable, lockable jaw, with the controllable jaw including moveable parts that move to place the jaw in one of multiple structuralAttorney Reference No.: 7551-0010W001configurations upon application of force at one or more pressure points actuating movement of one or more of the moveable parts.

[0015] The simulation structure may include a manikin with a physical model of one or more body parts. The one or more simulation parts may include simulation organs and blood vessels associated with the respective one or more body parts, and the operation tool may include a surgical tool handled by the user to perform a simulated medical procedure on at least one of the simulation organs and / or blood vessels.

[0016] The plurality of sensors can include one or more computer numerical control (CNC) probes, with each of the CNC probes placed proximate to at least one of the one or more simulation parts in the simulation structure.

[0017] The each of CNC probes may be placed directly underneath a simulation tooth structure.

[0018] At least some of the one or more CNC probes can be configured to detect linear and angular vibrations.

[0019] The plurality of sensors can include one or more simulation part selection items, actuatable by the user, that are respectively associated with individual ones from the one or more simulation parts, with the one or more simulation part selection items being selectable by the user to indicate an intended simulation part on which the user is to perform the simulated medical or dental procedure, and one or more vibration sensors to detect vibrations caused by performance of the simulated medical or dental procedure.

[0020] Each of the one or more simulation part selection items may include one or more of, for example, a physical actuatable switch and / or a graphical selection item.

[0021] The plurality of sensors can include a first set of inertial measurement units (IMUs) mounted on or within the operation tool manipulated by the user, and a second set of IMU’s disposed proximate to at least some of the one or more simulation parts in the simulation structure.

[0022] The IMU’s of the first set can be aligned with a tip of the operation tool, and configured to measure displacement of the tool, with resultant measured displacement data being used to derive, by the computing device, a vector movement representative of the displacement of the tool.Attorney Reference No.: 7551-0010W001

[0023] The IMU’s of the second set may be configured to measure movement and orientation data representative of movement and orientation of at least some of the one or more simulation parts in the simulation structure, with the movement and orientation data being configured to be communicated to the computing device to adjust a 3D model representative of a state of the simulation structure.

[0024] The computing device configured to determine the dynamic physical state of at least some of the one or more simulation parts and / or the operation tool, may be configured to implement a physics engine modeling physical properties of the one or more simulation parts in the simulation structure and of the operation tool, with the physics engine being configured to analyze interactions between the operation tool and at least one of the one or more simulation parts in the procedure area, based on the modeled physical properties and measurement data detected by the first and second sets of IMU’s, to represent the effect of the interactions on a 3D model of the one or more simulation parts and of the operation tool.

[0025] The physics engine can be configured to continually adjust the 3D model of the one or more simulation parts and the operation tool in response to receipt of updated data detected by the first and second sets of IMU’s.

[0026] The physics engine can be further configured to model tool-to-tooth interactions caused by application of force by the user to the at least one simulation part via the operation tool, and to derive, based on data collected by the first and second sets of IMU’s and analyzed using the tool-to-tooth model, updates to a state of the 3D model representation of the one or more simulation parts.

[0027] The plurality of sensors may include multiple optical sensors configured to capture positioning information associated with one or more trackable objects disposed on at least the operation tool. The computing device may further be configured to track, based on the captured positioning information, spatial location and motion of the operation tool.

[0028] The one or more trackable object may include one or more infrared (IR) LEDs each illuminating cycles of IR emissions at a respective pre-determined cycle frequency, with the multiple optical sensors including multiple macro-lens infrared (IR) cameras configured to each capture at least some of the IR emissions from the one or more IR LEDs. The computing device may be configured to derive the spatial location and motion of theAttorney Reference No.: 7551-0010W001operation tool according to relative positioning and strength of captured IR signals from the IR emissions illuminated by the one or more IR LEDs.

[0029] The one or more trackable object can include one or more visible objects, with the multiple optical sensors including multiple visible range cameras. The computing device can be configured to identify the one or more visible objects appearing in image data captured by each of the multiple visible-range cameras, and derive the spatial location and motion of the operation tool and / or a body part of the user according to relative locations of the one or more visible objects appearing in the captured image data from the multiple visible range cameras.

[0030] The plurality of sensors may include multiple vibration sensors, each embedded at respective locations in the simulation structure and configured to detect vibration caused through contact of the operation tool with one of the one or more simulation parts of the simulation structure. The computing device may further be configured to derive contact location of the operation tool based on amplitude, timing, frequency, and / or peak offset characteristics of vibrations signals measured by at least some of the multiple vibration sensors.

[0031] The multiple vibration sensors can include comprise multiple piezoelectric sensors embedded in a palette area of a dental simulation structure. The computing device configured to derive contact location of the operation tool can be configured to derive the contact location based on one or more of, for example, a multilateration or triangulation procedure applied to the vibration signals measured by the multiple piezoelectric sensors, and / or a machine learning (ML) vibration-to-location model that generates an output label representative of the contact locations according to the measured vibration signals inputted to an ML system implementing the ML vibration-to-location model.

[0032] The plurality of sensors may include multiple pressure sensors, each embedded at respective locations in the simulation structure and configured to detect forces applied to the simulation structure through contact between the operation tool and the one or more of the simulation parts.

[0033] The multiple pressure sensors may include multiple load cells embedded in a jaw and / or skull parts of a head part of a dental simulator, each of the multiple load cells configured to gauge a level of force measured at the respective one of the each of the multiple load cells, and with the computing device being further configured to process the force levelsAttorney Reference No.: 7551-0010W001measured by the multiple load cells, and to generate feedback data, provided to a userinterface, representative of performance of the dental procedure and an associated comfort level that a live patient would feel for similar measured force levels.

[0034] The computing device may further be configured to perform one or more of, for example, track timing information associated with the operation procedure, derive, based on the determined dynamic physical state, updates for a 3D model of the simulation structure resulting from the application of forces corresponding to interactions between the operation tool and the at least some of the one or more simulation parts, and / or render a graphical 3D representation of the 3D model on a display device.

[0035] The computing device configured to determine procedure performance feedback data can be configured to evaluate, using a machine learning system implementing a performance evaluation model, based on tracking data of motion of the operation tool and limbs of the user, performance of the user, with the performance evaluation model being optimized based on training data comprising training samples with training motion tracking data for a plurality of previously evaluated practitioners, and respective output labels representing the evaluation of the performance of the plurality of previously evaluated practitioners upon performing procedures resulting in the respective training motion tracking data.

[0036] In another aspect, a method for simulating a medical or dental procedure is disclosed. The method includes tracking mechanical displacements with a plurality of sensors distributed in a procedure area of a simulation structure comprising one or more simulation parts emulating anatomical parts, with the mechanical displacements caused through application of an operation tool handled by a user to at least one simulation part from the one or more simulation parts in the procedure area in which the user is performing the medical or dental procedure. The method further includes determining, at a computing device, based on the tracked mechanical displacements, a dynamic physical state of at least some of the one or more simulation parts and the operation tool, resulting from performance of the medical or dental procedure in the procedure area, and determining at the computing device, based at least in part on the determined dynamic physical state, procedure performance feedback data representative of the procedure’s performance by the user.Attorney Reference No.: 7551-0010W001

[0037] Embodiments of the method may include at least some of the features described in the present disclosure, including any of the features described in relation to the above system, as well as one or more of the following features.

[0038] Determining the physical state can include one or more of, for example, i) tracking linear and angular movements of the at least some of the one or more simulation parts, the operation tool, a body part of the user, and / or the simulation structure, and / or ii) determining forces applied on the at least some of the one or more simulation parts and / or the operation tool resulting from interaction of the operation tool with the at least one simulation part.

[0039] For at least one tooth from the simulation teeth, the physical model of the head part further can include respective one or more infrared (IR) LEDs and respective one or more IR sensors embedded underneath the at least one tooth. The method can further include controllably causing emissions of the one or more IR LEDs into a structure of the at least one tooth during performance of the dental procedure, and determining changes to the structure of the at least one tooth, caused by the dental procedure, based on changes of characteristics of reflected IR signals from surfaces of the internal structure of the at least one tooth, resulting from the changes to the structure of the at least one tooth.

[0040] The plurality of sensors may include a first set of inertial measurement units (IMUs) mounted on or within the operation tool manipulated by the user, and a second set of IMU’s disposed proximate to at least some of the one or more simulation parts in the simulation structure. The method may further include measuring, by the IMU’s of the second set, movement and orientation data representative of movement and orientation of at least some of the one or more simulation parts in the simulation structure, with the movement and orientation data configured to be communicated to the computing device to adjust a 3D model representative of a state of the simulation structure.

[0041] Determining the dynamic physical state can include implementing a physics engine modeling physical properties of the one or more simulation parts in the simulation structure and of the operation tool, with the physics engine being configured to analyze interactions between the operation tool and at least one of the one or more simulation parts in the procedure area, based on the modeled physical properties and measurement data detected by the first and second sets of IMU’s, to represent the effect of the interactions on a 3D model of the one or more simulation parts and of the operation tool.Attorney Reference No.: 7551-0010W001

[0042] The plurality of sensors may include multiple vibration sensors, each embedded at respective locations in the simulation structure and configured to detect vibration caused through contact of the operation tool with one of the one or more simulation parts of the simulation structure, and the method may further include deriving contact location of the operation tool based on amplitude, timing, frequency, and / or peak offset characteristics of vibrations signals measured by at least some of the multiple vibration sensors.

[0043] Deriving contact location of the operation tool can include deriving the contact location based on one or more of, for example, a multilateration or triangulation procedure applied to the vibration signals measured by the multiple piezoelectric sensors, and / or a machine learning (ML) vibration-to-location model that generates an output label representative of the contact locations according to the measured vibration signals inputted to an ML system implementing the ML vibration-to-location model.

[0044] Determining the procedure performance feedback data may include evaluating, using a machine learning system implementing a performance evaluation model, based on tracking data of motion of the operation tool and limbs of the user, performance of the user. The performance evaluation model may be optimized based on training data comprising training samples with training motion tracking data for a plurality of previously evaluated practitioners, and respective output labels representing the evaluation of the performance of the plurality of previously evaluated practitioners upon performing procedures resulting in the respective training motion tracking data.

[0045] The plurality of sensors can include multiple optical sensors configured to capture positioning information associated with one or more trackable objects disposed on at least the operation tool, and the method can further include tracking, based on the captured positioning information, spatial location and motion of the operation tool.

[0046] The one or more trackable object may include one or more infrared (IR) LEDs each illuminating cycles of IR emissions at a respective pre-determined cycle frequency, with the multiple optical sensors including multiple macro-lens infrared (IR) cameras configured to each capture at least some of the IR emissions from the one or more IR LEDs. The method may further include deriving the spatial location and motion of the operation tool according to relative positioning and strength of captured IR signals from the IR emissions illuminated by the one or more IR LEDs.Attorney Reference No.: 7551-0010W001

[0047] In yet another aspect, a method for simulation platform development is disclosed that includes receiving, via a grid-like graphic user-interface, specifications regarding dimensions and contours of one or more customized replaceable simulation teeth, and generating, based on the specification, manufacturing instructions to produce the one or more customized replaceable simulation teeth. The one or more simulation teeth are attachable to a dental simulator, for training dental practitioners, with the dental simulator including sensors that track motion of a dental trainee and / or a dental tool operating on at least one of the one or more customized replaceable simulation teeth. Similar methods can be used for developing simulation platforms for other medical or non-medical fields / domains.

[0048] Other features and advantages of the invention are apparent from the following description, and from the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0049] These and other aspects will now be described in detail with reference to the following drawings.

[0050] FIG. 1 is a diagram of an example system for simulating medical or dental procedures.

[0051] FIG. 2 is a diagram showing part of another example system for simulating medical or dental procedures.

[0052] FIG. 3 is a diagram of a further example simulation system.

[0053] FIG. 4 is a diagram of another example simulation system used for dental procedures simulation and training.

[0054] FIG. 5A is a schematic diagram of an example platform architecture for simulation system.

[0055] FIG. 5B includes images of an example tool and denture set that may be similar to the schematically illustrated tool and denture set of FIG. 5A.

[0056] FIG. 6 is a flow diagram showing various data processing operations that are implemented on an example system with a platform architecture similar to that of FIG. 5A.Attorney Reference No.: 7551-0010W001

[0057] FIG. 7 is a flow diagram of a workflow for an example monitoring and tracking approach used with dental and medical simulation systems.

[0058] FIG. 8 includes diagrams of an example simulation element, configured to be tracked and monitored to assess structural changes thereto, in original and altered forms.

[0059] FIG. 9A-F includes images of an example foldable manakin in various folded or disassembled positions.

[0060] FIGS. 10A-Q include images of various head designs and manakin assemblies that can be used with dental simulation systems described herein.

[0061] FIG. 11 is a photo of a practitioner practicing on a manakin of an example simulation system.

[0062] FIG. 12 is a flowchart of an example procedure for simulating a medical or dental procedure.DETAILED DESCRIPTION

[0063] Disclosed is a proposed simulation and modeling framework to improve the training experience for users of platforms (medical and / or dental trainees) implementing the proposed framework, achieve improved visualization through accurate modeling and a rendering of procedures performed (including to facilitate training and performance of practitioners with actual patients), and provide performance data regarding the procedures performed (e.g., various metrics / analytics characterizing the manner in which the procedures were performed). In some examples, the proposed platform includes a dummy denture and tool heads in 3D space, equipped with a collision detection mechanism to identify and track which tooth is being operated on in real-time. In such examples, a 3D modeling system for dentures and dental tools can be implemented. Such a modeling can include, for example, a real-time collision detection implementation (procedure / algorithm) that is configured to identify the tooth being worked on, can record and analyze the time spent on each tooth to provide feedback on the procedure, can compensate, e.g., using multiple IMUs and accelerometers, for head movements, etc.

[0064] With reference to FIG. 1, a diagram of an example system 100 for simulating medical or dental procedures is shown. The example system 100 is implemented based on anAttorney Reference No.: 7551-0010W001approach that uses multiple computer numerical control (CNC) probes to measure mechanical displacement data resulting from performance of the medical or dental procedure in the simulation structure. As depicted in FIG. 1, the system 100 includes a manikin 110 with, in embodiments in which the simulation system is configured to provide a dental training platform, a denture assembly (also referred to as a denture set) 120 that is fitted within a mouth area of the manakin 110. The denture assembly 120 includes one or more teeth / dentures, such as the teeth 122 and 124. At least some of the teeth included in the denture assembly 120 may be replaceable teeth that can be secured to holding structures. In some embodiments, such holding structures may include sensors, such as the sensor 130, which is mechanically attachable to the replaceable tooth 124, thus acting both to mechanically secure the denture 124 to the denture assembly (optionally in a manner mimicking the manner in which teeth are anchored into sockets defined in the jawbone), and to sense and measure forces applied to the respective tooth (e.g., based on mechanical vibration measured), extent of loss of dental material (e.g., through optical measurements, as will be discussed in greater detail below), motion of a dental tool controlled by a trainee, etc.

[0065] In some embodiments, each of the holding structures to which a respective tooth is attached may be implemented using a sensor device mechanically structured to attach to the corresponding replaceable structure. For example, as illustrated in FIG. 1, a sensor head 132 of the sensor 130 may be structured to have a conical shape that tapers in a direction extending away from the socket in which the sensor is placed, and configured to be matingly received within a carved volume defined by a base 125 for the tooth 124 (a view of the denture 124 and the sensor 130 is also provided inset 160). Although not shown in FIG. 1, in some embodiments, the sensor head 132 and the carved volume in the base 125 may have complementary threads to provide further mechanical stability to the resulting attachment of the tooth’s base and the sensor head. Other fastening / securing mechanism to secure the teeth to the denture assembly 120 may also be used. In some embodiments, to reduce costs and implementation complexity, only some of the holding structures for the various replaceable dentures may include sensory elements and circuitry. In such embodiments, the lower number of sensors are configured to measure and sense mechanical forces, motions of the dental tools (and / or other structures) in the mouth area, structural changes within the mouth areas (e.g., removal of tooth materials as a result of the operation of dental tools), etc., for multiple (and in some case all) teeth and other structures within the mouth. In such situations, the measurements made by the sensors can be processed and analyzed toAttorney Reference No.: 7551-0010W001determine (approximate) individual measurable quantities (e.g., forces applied to individual teeth) based on correlation between measurements made by one sensor, and measurements made by at least one sensor, for a particular event. For instance, if a first sensor measures a certain level of force (based on vibrations) applied to a nearby tooth, and a second sensor, located elsewhere in the mouth, obtains a second measurement for the same event, the particular force applied to the denture in question can be estimated based on the two measurements (and possibly additional measurements), and known information for the sensors (e.g., their locations within the mouth). It is noted that in some embodiments, the sensors may be formed, or be located in, different structures from the holding structures configured to secure teeth (replaceable or otherwise) to the denture assembly 120. For example, sensors may be placed at strategic locations within a mouth area 112 of the manakin 110 to make local measurements (vibration measurements, optical or electromagnetic measurements, etc.) of detected occurrences, based on which individualized measurable quantities (such as force or motion) can be derived.

[0066] As further shown in FIG. 1, in various embodiments, the denture assembly 120 may be mechanically and / or electrically coupled to a control / communication board 140 that includes circuitry to receive sensor signals from the sensors, process the signals (e.g., determine measurement values based on the signals electrically communicated by the sensors) via a controller (e.g., processor-based controller disposed on the board), and communicate through wired and / or wireless connections the signals or the measurements (whether processed by processor disposed locally on the board 140, or otherwise) to a remote computing system 150 for further processing, analysis, and generation of output for use by a user (e.g., on an interface that may provide a 3D rendering of the simulation for the training session), as will be discussed in greater detail below. The communication of sensors’ signals and measurements to the board 140 may be implemented through a direct wired connection from the sensors, or via an intermediate interfacing device (not shown) that mechanically connects to, for example, a set of pins 142. This type of connection can also provide further structural stability to secure the denture assembly 120 to the manakin 110. Alternatively, in various embodiments, signals measured and / or processed by the various sensors may be transmitted wirelessly to a processing device (whether situated on or in the manakin 110, or located remotely from the manakin 110). It is noted that at least some of the sensors may be connected to an Internet-of-Things (loT) network, and may be individually controlled (e.g., can be individually configured to perform tasks as needed or desired). Such loT sensors canAttorney Reference No.: 7551-0010W001establish wireless communication links (e.g., based on WiFi, Bluetooth Low Energy, and other communication technologies) with other sensors and / or a central remote computing node.

[0067] The one or more sensors positioned proximate to the various teeth may include sensors of various types, and may include computer numerical control (CNC) sensor devices, piezoelectric sensors, optical sensors, sensors to receive and detect electromagnetic signals, thermal sensors, etc. In the example of FIG. 1, the sensor 130 may be a CNC sensor, that may include one or more sensory elements that are configured to sense and / or measure various dynamic events occurring in the mouth area of the manakin 110 and / or with respect to the one or more of the teeth. For example, the CNC sensor may include inertial sensors (e.g., gyroscopes, accelerometers, magnetometers, etc., which may be implemented, for example, using MEMS technology), optical sensors, infrared sensors, etc., positioned within the sensor 130 (and more particularly, within the sensor head 132 or on the exterior surfaces of the sensor 130). In some embodiments, the sensory implementation may include integrating multiple CNC sensors / probes (32 probes; other numbers of CNC probes may be used instead) beneath each tooth of the dental training manikin to accurately measure which tooth is being worked on, measure forces applied to the teeth (e.g., by dental tools operated by the user), and various other measurable quantities. For example, the CNC probes may be configured to detect both linear and angular vibrations generated during the procedure (e.g., based on inertial sensors included with the probes) to provide precise data on the tooth being targeted. This approach can eliminate the need for handpiece tracking while ensuring robust detection of the interaction between the tool and each tooth / denture. The collected vibration data can be processed, allowing for detailed performance analytics, such as time spent on each tooth, forces applied on the teeth, identification of teeth previously worked on, and preparation quality. This durable, sensor-based system provides a reliable and relatively maintenance-free solution, offering dental trainees and instructors an advanced tool for performance assessment and real-time feedback.

[0068] As noted, the sensors (whether CNC probes or other types of sensors) can be embedded underneath the teeth. By embedding sensors underneath the teeth and / or into the palette of the mouth, the sensor data can be used to determine when the dentist is in contact with the teeth. Through signal processing and / or machine learning, a controller can perform a triangulation / multilateration procedures (e.g., based on characteristics of sensor signals,Attorney Reference No.: 7551-0010W001such as their amplitude, received from different embedded sensors) to determine which tooth the user is working on. The sensor data also allows for tracking tooth contact time as a portion of overall task time.

[0069] Thus, various embodiments of the proposed platform include a system for simulating a medical or dental procedure that includes a simulation structure (e.g., a manikin) with one or more simulation parts / objects (which can include replaceable parts) emulating anatomical parts, and a plurality of sensors distributed in a procedure area in the simulation structure at which a simulation of a medical or dental procedure is being performed, with at least some of the plurality of sensors configured to track mechanical displacements (e.g., vibrations, linear and / or angular displacements of various objects) in the procedure area, caused through application of an operation tool handled by a user (e.g., a trainee using a dental drill in the case of a dental procedure, a cutting or fusing instrument in the case of a medical surgical procedure) to at least one simulation part (e.g., a simulation tooth, a simulation body organ, etc.) from the one or more simulation parts in the procedure area. The system further includes a computing device comprising one or more processor-based devices coupled to one or more memory storage devices, with the computing device configured to determine, based on the tracked mechanical displacements, dynamic physical state of at least some of the one or more simulation parts (e.g., simulation teeth) and the operation tool, resulting from performance of the medical or dental procedure in the procedure area, and to determine, based at least in part on the determined dynamic physical state, procedure performance feedback data representing of the procedure’s performance by the trainee. In some examples, the computing device configured to determine the physical state is configured to perform one or more of, for example, i) track linear and angular movements of the at least some of the one or more simulation parts, the operation tool, a body part of the user (e.g., the user hand, as will be discussed in greater detail below), and / or the simulation structure, and / or ii) determine forces applied on the at least some of the one or more simulation parts and / or the operation tool resulting from interaction of the operation tool with the at least one simulation part.

[0070] In some examples, the plurality of sensors can include one or more computer numerical control (CNC) probes, with each of the CNC probes placed proximate to at least one of the one or more simulation parts in the simulation structure. In such examples, the each of CNC probes may be placed directly underneath a simulation tooth structure. At leastAttorney Reference No.: 7551-0010W001some of the one or more CNC probes can be configured to detect linear and angular vibrations.

[0071] In various examples, the tracked mechanical displacements may include one or more of, for example, detected vibrations, and / or detected spatial movements of the at least some of the one or more simulation parts and the operation tool. Examples of sensors that detect / track mechanical displacements include inertial sensors, pressure sensors, proximity sensors to track where something is in the mouth and how close it is to the tooth (in the case of a dental simulation structure), optical sensors (including IR sensors, and image capture sensors such as cameras, whose captured data can be analyzed using analytic processes or machine learning processes to compute displacements of various objects during particular time intervals), etc. Further details about some of the various sensors used to implement the simulation system described herein are provided below. In some embodiments, the simulation structure may include a manikin with a physical model of a head part having a mouth, with the one or more simulation parts including simulation teeth constituting a training simulation denture set, and with the operation tool including a dental tool that is handled / actuated by a trainee to perform a simulated dental procedure on at least one tooth from the simulation teeth. In additional examples, the simulation structure may include a manikin with a physical model of one or more body parts, and with the one or more simulation parts including simulation organs and blood vessels associated with the respective one or more body parts, and with the operation tool including a surgical tool actuated by a trainee to perform a simulated medical procedure on at least one of the simulation organs and / or blood vessels.

[0072] FIG. 2 is a diagram showing part of another example system 200 for simulating medical or dental procedures, with the system 200 including piezoelectric sensors (at least some of those sensors may be embedded in a denture assembly). The diagram of FIG. 2 depicts three sensors 210, 212, and 214 that each includes piezoelectric materials (e.g., piezoelectric crystal materials, piezoelectric ceramics, such as PZT ceramics, etc.) that are each configured to produce electric signals (signals 220, 222, and 224) based on deformations of the piezoelectric materials in response to mechanical vibrations, pressure, and various mechanical forces generated in the simulation area (e.g., the mouth area in the example of FIG. 2) from interaction of a dental tool 230 with the simulation teeth. The use of embedded piezoelectric sensors 210, 212, and 214 (a sensor-embedded configuration may also beAttorney Reference No.: 7551-0010W001realized for other types of sensors, including the CNC sensors depicted of FIG. 1) requires no augmentation of the dental tools that are used in the training system 200. Furthermore, by embedding such sensors into the palette (as schematically depicted for the piezoelectric sensor 214) or into the lower or upper jaw of the manakin 210 (or into other non-exposed areas of the manakin), such embedded sensors are unaffected by the dust and liquids involved in dental practice.

[0073] The signals produced by the different sensors can then be used to derive, using multilateration or triangulation procedures, the forces produced and applied by the dental tool 230 on one or more of the teeth. In the example of FIG. 2, a tooth 204 operated on by the tool 230 causes mechanical changes to the different piezoelectric sensors deployed in the simulation area (on or in the denture assembly 202, or within regions of the simulation area outside of the actual dental assembly). The resultant electrical signals allow accurate determination of the specific tooth being contacted by the tool (e.g., through multilateration techniques, signal correlation techniques, etc.). For example, when a tooth is contacted, vibrations in the model are measured by the sensors. These vibrations attenuate across the model and the resultant signals arrive at the sensors at different times. Thus, by computing signal amplitudes, frequencies, and peak offsets, a precise contact location can be determined. In some embodiments, the measured signals (also referred to as features) may be fed into a machine learning classifier, which identifies, based on the input signals provided to the classifier’s input stage, the tooth that has been contacted. In some embodiments, the sensors’ measurements and signals can be used to determine more advanced information and metrics, beyond just determining which tooth had been contacted. For example, analysis of the signals measured by the various sensors deployed in the manakin can be used to determine performance of the trainee in handling the tool 230. For instance, by determining or estimating the forces (direction and strength) applied to a tooth based on signals produced by the various sensors, the skill and dexterity of the trainee can be evaluated, e.g., determine whether the trainee is applying too much force, whether the force is being applied in the correct direction and at a proper location on the tooth, etc.

[0074] As described herein, the piezoelectric sensors 210, 212, and 214 may be supplemented by additional sensors that may include additional piezoelectric sensors, inertial sensors, etc. As also noted, such additional piezoelectric sensors may be integrated into CNC probes that may each include multiple types of sensors, whether to produce measurementsAttorney Reference No.: 7551-0010W001representative of mechanical forces, or produce measurements to track other measurable quantities such as motion of the tool 230, structural properties of replaceable simulation teeth, etc.

[0075] Thus, in various embodiments, the plurality of sensors (of the system of FIG, 2, for example) can include multiple vibration sensors, each embedded (or otherwise positioned) at respective locations in the simulation structure and configured to detect vibration caused through contact of the operation tool with one of the one or more simulation parts of the simulation structure. In such situations, the computing device is further configured to derive contact location of the operation tool based on amplitude, timing, frequency, and / or peak offset characteristics of vibrations signals measured by at least some of the multiple vibration sensors. The multiple vibration sensors may include multiple piezoelectric sensors embedded in a dental simulation structure, with the computing device configured to derive contact location of the operation tool being configured to derive the contact location based on one or more of, for example, a multilateration or triangulation procedure applied to the vibration signals measured by the multiple piezoelectric sensors, and / or a machine learning (ML) vibration-to-location model that generates an output label representative of a contact location according to the measured vibration signals inputted to an ML system implementing the ML vibration-to-location model.

[0076] The proposed dental / medical procedure simulation and training platform described herein can also be configured to obtain data representing performance of a particular procedure based on the manipulation of a medical or dental instrument (such as a dental drill) performed by the user. In such implementations, the manakin may be equipped with sensors configured to track the movement of a tool being handled by the trainee. Such sensors can be included with, for example, the CNC sensors of FIG. 1, or may be dedicated sensors placed in the simulation area (be it a mouth area of a manakin or some other body part for which a medical trainee may be practicing a procedure), embedded into different parts of the manakin, or placed outside of the manakin, e.g., optical sensors, radio signal sensors / receivers, etc., placed around the manakin at locations sufficiently proximate to detect measurable properties associated with the simulation parts (such as teeth or some other anatomical parts).

[0077] Consider, for example, implementations that are based on optical / visual tracking of the tools employed by the trainee (be it dental tools, medical tools, or tools used in someAttorney Reference No.: 7551-0010W001other field), as well as visually tracking the hands of the trainee (e.g., to assess the quality of the techniques practiced by the trainee, to glean information about forces applied to the target area being worked on, etc.). Optical / visual tracking (camera / vision-based systems) can come in two forms: outside-in tracking (where cameras are mounted into the room and track the user from a distance) and inside-out tracking (where cameras are embedded into the object of interest and track the user from within). Outside-in tracking is typically easier to implement, but requires infrastructuring of the training environment, which is not always practical. In the case of implementing inside-out tracking for dental scenarios, in some embodiments, small, macro-lens, cameras (e.g., infrared cameras) are embedded into the eyes, nose, chest / neck, and / or other areas of the simulation structure (the manikin), to track, for example, infrared LEDs mounted on the far end of the dental tools (i.e., the non-contact end of the tool). In some embodiments, trackable LEDs (and / or other types of signalemitting beacons or trackable structures that can be recognized) can be deployed on the surfaces of the trainee’s hands (or the gloves the trainee may be wearing), or on other body parts and clothing items from which motion of the trainee’s body can be determined.

[0078] In situations where it is undesirable to position LEDs (IR or otherwise) or other types of beacons emitting devices on the trainee’s body, the gloves, or other clothing items of the trainees (e.g., due to the distraction that such emitting / illuminating devices may cause), more passive trackable objects may be used. For example, in some embodiments, passive retroreflective markers / dots can be placed on various clothing items worn by the trainees (lab coats, gloves, and so on). For instance, nitrile gloves with embedded patterns of retroreflective markers can be produced (e.g., little, silver-looking dots that reflect light back in the direction it arrived, to its source), such that a trainee’s hand could be accurately tracked, without adding any extra burden to the trainee, based on reflection of light produced by devices external to the trainee. Such markers could just be incorporated into the material of the glove, with no mechanical / comfort implications to its design.

[0079] By tracking, for example, the end of the tool (and optionally its orientation in the spatial framework associated with the tracking device) the other end of the tool (e.g., the tooltip) and / or other objects appearing in the tracked framework can be identified. By positioning multiple cameras, various tracking issues caused by occlusion can be accounted for. For instance, by detecting fully the tooltip with one camera, but only partially detecting the tool tip with one or more other cameras, information about location and / or orientation ofAttorney Reference No.: 7551-0010W001the tooltip, and thus of the entire tool, can be derived (e.g., through triangulation / multilateration of image data from various cameras, through machine learning models, etc.). This information is used for tracking, over a time period, the trajectory of the tool and / or the trainee. This approach is also applicable across broader categories of surgical training, and training with other tools from different technological domains.

[0080] It is noted that by tracking the trainee’s hand movements and the movement of the tool, the collected tracking data (be it for dental, general surgery, or other domain requiring training) can be later processed and analyzed (possibly at some remote server, such as the server 150 shown in FIG. 1, and / or by a dedicated third-party analytics-generating entity) to derive analytics and metrics that can be used to evaluate the quality of the trainee’s performance of the particular procedure the trainee performed (e.g., the smoothness of the trainee’s motion, the percentage of time that the trainee is moving his / her hands, some metric representative of overall efficiency and competency of the trainee’s techniques, etc.).

[0081] In various embodiments, such evaluations, based on tracking data collected for the movement of the trainees and / or the tools, of the competence and efficiency of trainees performing the training procedures can be performed using trained machine learning systems with evaluations models that are optimized according to training data (samples) that includes, for example, tracking data (be it through videos of the training or a rendering of the tracked motion) and ground truth labels corresponding to actual evaluations by viewers. The evaluation systems (deriving evaluation analytics / metrics based on tracked motion data of the instruments / tools and / or of the trainees) can also produce comparison analytics that compare the performance of the trainees to the performance of a standard practitioner, or to a specific practitioner (e.g., the top trainee in a group that the trainee being evaluated is a member of). Thus, in some embodiments, a computing device for determining procedure performance feedback data can be configured to evaluate, using a machine learning system implementing a performance evaluation model, based on tracking data of motion of the operation tool and limbs of the trainee, performance of the trainee. The performance evaluation model can be optimized based on training data comprising training samples with training motion tracking data for a plurality of previously evaluated practitioners, and respective output labels representing the evaluation of the performance of the plurality of previously evaluated practitioners upon performing procedures resulting in the respective training motion tracking data. Note that the evaluation approach of using training dataAttorney Reference No.: 7551-0010W001compiled from performance data for previously evaluated practitioners can be used with different types of data, collected from a variety of different sensor devices and other sources.

[0082] Further details about inside-out tracking, and implementation of a system therefor, are now provided with reference to FIG. 3, showing a diagram of an example simulation platform 300 that includes a manikin 310 (the simulation structure or model) for dental procedures training. The platform 300 includes multiple infrared cameras (sensors), including example macro-lens infrared cameras 310, 312, 314, and 316 disposed or embedded in and / or within key places in the manikin’s head 302 of the simulation platform / model. In the example dental model of FIG. 3, the infrared cameras are embedded into the eyes, nose, and sternum, looking towards the mouth. As further illustrated in FIG. 3, active infrared LEDs (the trackable objects), including an example infrared LED 322 are embedded into, or disposed on, a far end of a handle of a dental tool 320. The trackable objects (LEDs, radio signal producing elements, etc.) can be secured to the tool 320 through, for example a custom-designed universal mount, adhesives, or other fastening mechanisms. One or more IR LED’s, or some other type of trackable object, can also be placed at other strategic positions on or within the tool.

[0083] The example of IR LEDs (LED’s at other optical frequencies can also be used), deployed as trackable objects on the tool, are detected by tracking sensors (such as the macrolens IR cameras 310312, 314, and 316) deployed in or around the simulation structure (the manakin, and more particularly the manakin head 302). As can be seen in FIG. 3, the detection / tracking sensors (e.g., the IR cameras) are configured to have sufficient sensitivity, and be placed at appropriate strategic locations around the simulation area, that these sensors have intersecting detectability areas (represented by shaded triangular regions, such as detectable area 330 extending from the IR camera 316) that in combination cover the entirety of the simulation area (the mouth area). Generally, at least a portion of simulation area will have multiple intersecting coverage areas to allow for the trackable objects (e.g., the LED’s disposed or within the tool 320) to be detectable by multiple ones of the detection / tracking sensors, thus allowing for determination of location and / or movement of the tool 320 (e.g., through multilateration techniques and / or machine learning processes). The IR LED’s can illuminate / flash at some particular frequency, improving their detectability / recognition by the cameras. The LEDs at the end of the handle (or elsewhere on or within the tool), combined with the multiple cameras in the simulator head, ensure the system is lessAttorney Reference No.: 7551-0010W001vulnerable to occlusion issues. The position of the head of the tool can thus be accurately inferred based on the tracking of the handle.

[0084] Accordingly, in various embodiments, the plurality of sensors in the simulation structure (for dental, medical, or non-medical training) can include multiple optical sensors configured to capture positioning information associated with one or more trackable objects disposed on at least the operation tool, with the computing device being further configured to track, based on the captured positioning information, spatial location and motion of the operation tool. In such embodiments, the one or more trackable object can include one or more infrared (IR) LEDs each illuminating cycles of IR emissions at a respective predetermined cycle frequency, with the multiple optical sensors including multiple macro-lens infrared (IR) cameras configured to each capture at least some of the IR emissions from the one or more IR LEDs. The computing device can be configured to derive a spatial location and motion of the operation tool (and / or a body part of the trainee) according to relative positioning and strength of captured IR signals from the IR emissions illuminated by the one or more IR LEDs. In some examples, the one or more trackable object can include one or more visible objects, the multiple optical sensors can include multiple visible range cameras, and the computing device can be configured to identify the one or more visible objects appearing in image data captured by each of the multiple visible-range cameras, and derive the spatial location and motion of the operation tool and / or a body part of the trainee according to relative locations of the one or more visible objects appearing in the captured image data from the multiple visible range cameras.

[0085] Further example implementations of the proposed platform, which may be used in conjunction with, or independently of, the various embodiments described above (with different types of sensors), include an IMU (inertial measurement unit)-based tracking system to track movement of the various simulation parts (e.g., different body portions of the manakin, different tools being used by a trainee, etc.) comprising the simulation platform. In some embodiments, the IMU may include integrated small-scale electro-mechanical circuits that implement accelerometers, gyroscopes, and magnetometers (implemented using MEMS technology) to measure linear and angular acceleration, tilt, vibration, and other forces, based on changes that the sensed motion causes to the mechanical structure (proof mass) of the sensors. For example, motion on the proof masses can cause predictable changes of the electrical properties (e.g., capacitances) of circuits of an IMU comprising the proof massesAttorney Reference No.: 7551-0010W001and electrodes, resulting in electrical signals representative of the motion and forces acting on the IMU’s circuits. In some embodiments, the IMU’s sensors may be implemented using piezoelectric elements (in a manner similar to the sensors described in relation to FIG. 2), that are deformed (have their structural characteristics change) due to motion and forces experienced by the various simulation parts / objects in the simulation system (e.g., due to the interaction between the tool manipulated by the trainee and the simulation parts in the mouth, due to movement of the tool, etc.). The changes to the physical characteristics of the IMU’s sensing elements produce electrical signals that correlate, and are representative of, the forces and motions detected by the sensing elements of the IMU’s. The electrical signals produced by the IMU’s can be used to determine motion and / or forces experienced by the IMU devices, either on a local processor, or at a remote computing device (through wired and / or wireless communication links). IMU’s can be incorporated into one or more CNC probes.

[0086] FIG. 4 is a diagram of an example simulation system 400 used for dental procedures simulation and training (also referred to “RG VectorDent”), showing a tool 410 operating on a simulation tooth 404 disposed in a manakin head 402. The proposed simulation system uses IMU sensors implements an advanced visualization and feedback systems to assist dentist trainees (as well as dentists in the course of performing procedures on actual patients, and requiring further visualization of the procedure) in real-time, allowing them to see how their tools interact with the teeth, improving both accuracy and outcomes. In various embodiments, the simulation system can be used to model a dummy denture and tool head in 3D space, equipped with a collision detection mechanism to identify and track which tooth is being operated on in real-time.

[0087] The example system 400 includes two example IMU devices, with a first IMU device 430 secured to a palette of the manakin head 402, and a second IMU device 432 positioned, for example, at the closed end (i.e., the tip) of the tool’s head, opposite the end comprising the drill bit attachment section. Additional IMU devices, as well as other types of sensor devices, may be deployed at additional locations of the simulation systems (e.g., on exposed surfaces of the manakin’s head, embedded within the head, on or within the tool, etc.). In the present example, the IMU device may include several devices (each corresponding to one of the four quadrants illustrated for IMU device 430). Each of the one or more IMU sensors that may comprise the IMU device 430 may include sensory elementsAttorney Reference No.: 7551-0010W001to detect motion and forces experienced by the sensory elements, to thus provide one or more sets of signals representative of the motion and forces detected / experienced.

[0088] As described herein, the different sets of signals, corresponding to different sets of measurements, provide higher resolution and greater detection sensitivity of the motions and forces that produce those separate measurements, and can therefore produce a more detailed and accurate estimate of the motions and forces experience by the manakin head, by the simulation objects (e.g., the teeth), etc. For example, if there are four different IMU sensors included with the IMU device 430, operation of the tool 410 on the tooth worked on will cause vibrations as will as tilting of manakin head. These motions and forces are detected by the various sensing elements of the IMU device 430, based on which estimates of those motions and forces can be derived by correlating the measurements according to different techniques (e.g., multilateration computation, ML models trained to output an estimate of the force and / or location based on multiple measurements produced by multiple sensors, etc.). For the IMU device 432 (which may also include multiple IMU sensors), the movement of the tool 410, and the vibrations and forces produced by the drill head as a result of interaction of the drill with one or more simulation objects (e.g., teeth), will produce data (based on the motion and forces experienced by the sensing elements) that can be used to track the movement / trajectory of the tool (e.g., for presentation of the tool’s movement in a 3D model rendered on a user interface application). As noted, additional IMU devices may be deployed in the simulation system 400. Tracking movement of the tool and / or the simulation structure (be it the head of the manakin, or some other body part for other types of medical procedure simulations and training) can be facilitated, or performed independently, by optical sensors (e.g., CDC or CMOS based sensors) that track the movement and positions of the tools and other objects (including, for example, the simulation teeth) of the simulation system.Moreover, in some embodiments, one or more IMU devices may be incorporated within CNC probes, possibly in conjunction with other types of sensors (optical sensors, piezoelectric sensors, and so on) included in such CNC probes.

[0089] Thus, the proposed solution using IMU’s can be used to implement a 3D modeling system for tracking the real-time movement of dental tools, teeth, and gums, using IMUs that are disposed on or within the tools being manipulated / actuated by the trainee, and at one or more locations on or within the manakin’s head. For example, IMUs can be mounted on both the tool handle and on the upper and lower sets of teeth. The IMUs on the tool handle areAttorney Reference No.: 7551-0010W001aligned with the tool tip, allowing for accurate vector tracking of its movements. Each set of the teeth (such as the teeth shown in the dental assemblies of FIGS. 1 and 2) may be equipped with its own IMU, along with a homing system and vibration sensors to enhance precision. To improve accuracy of the 3D model, the system may require the user, in some embodiments, to "home" (calibrate) the system by touching each set of teeth with the tool, with the vibration sensors detecting contact between the operation tool and the teeth. The proposed implementation is built for future expandability, allowing for updates and advanced analytics. Both the tool and head-mounted units can be connected via separate USB cables, providing power and data transfer capabilities. Alternatively, wireless connectivity (e.g., via a short range protocol, such as Bluetooth Low Energy, WiFi, etc.) can be used. To ensure precise tracking, users may also be prompted to select the specific tool handles and tips they are using. This ensures that the system can account for the exact configuration of the tools, leading to more accurate modeling and analysis in dental applications.

[0090] FIG. 5A is a schematic diagram of the platform architecture of an example simulation system 500. While the example system 500 depicts and discusses implementations that rely on inertial sensors (e.g., IMU devices and / or individual accelerometers), this architecture can similarly be used with other sensors, alone or in combination with the inertial sensors. Briefly, the example system includes several interconnected components / modules configured to model and track dental procedures accurately. These components include a dummy denture set 510, which is a physical model of a person’s mouth, equipped with sensors to provide sensor data, including inertial data (motion forces) to determine 3D positional data, a tool head 520, which in the example of FIG. 5 A is a dental tool with integrated sensors to track the tool’s position and orientation in real-time, and a computing system 530 (which may be similar to the computing system 150 of FIG. 1), which is a custom-built processing unit responsible for analyzing sensor data, running collision detection processes, rendering 3D models, etc. The architecture of the example system 500 supports real-time operation, with data collected / measured using the integrated sensors transmitted through wired links (e.g., via USB-based links) or wireless links (WiFi, Bluetooth Low Energy links, etc.) to the computing system 530, where it is processed and visualized.

[0091] As noted, the denture set is a physical model representing the patient’s teeth and gums. In the example of FIG. 5A, the model includes IMU devices (which may be similar toAttorney Reference No.: 7551-0010W001the IMU devices described in relation to FIG. 4) 512a-b embedded on the upper and lower halves of the dentures (which may be similar to the dentures depicted and described in relation to FIGS. 1-4) to track head movements and / or to compute vibrations and forces imparted on the simulation objects (teeth, and mouth parts) by operation of the tool 520. In the example of FIG. 5A, the IMU devices are 6-axis IMU’s. These IMUs provide data on the orientation and movement of the head, which aids compensating for shifts during tool operation. Additionally, in some embodiments, accelerometers, such as accelerometers 514a and 514b, can be strategically placed around the denture set to detect vibrations and refine the position of the denture model in 3D space, ensuring that the denture model remains accurate even under minor movements or vibrations. Additional sensors may be included (on surfaces of the simulation structure, or embedded with the simulation structure), such as vibration sensors, optical sensors, and so on, to measure additional measurable properties. The denture set is connected to the computing system 530 (and optionally to the tool 520 to facilitate interactive operation between these components of the system 500) via a wired connection, implemented, for example, using a USB interfacing port (such as the interfacing port 518), optionally connected to a multiprotocol USB converter FT232H module to convert data signals and packets for various different data protocols). Alternatively and / or additionally, the denture set may be connected to the computing system 530 or via wireless connections (not shown in FIG. 5A), ensuring high-speed data transfer and minimizing latency. The communication protocol is optimized for real-time operation, with data packets being transmitted and processed at a high frequency to maintain the accuracy and responsiveness of the system.

[0092] The tool head 520, which could be selected from various available dental instruments (or in the case of medical or surgery training, can be selected from various available surgery instruments or tools such as different scalpels), may be equipped with, for example, two IMUs 522 and 524 positioned to provide a highly accurate vector of the tool in 3D space (thus allowing accurate 3D tracking of the tool / instrument). These IMUs continuously track the tool’s orientation and movement, allowing the system to model its interaction with the denture set in real-time. In the example embodiments of FIG. 5A, each of the IMU devices deployed in the tool 520 may be configured to measure 9 degrees of freedom (DoF), and to capture accelerometer, gyroscope, and magnetometer data, to determine the tool’s orientation and movement vector in 3D space. In the example of FIG. 5A, the dual IMU setup of the tool 520 is configured to provide redundancy and improve theAttorney Reference No.: 7551-0010W001accuracy of the tool’s vector, especially during rapid movements. Fewer or additional IMU’s, with configurations different from those described in relation to the IMU’s 522 and 524, may be used. The tool 520 is connected to the computing system 530 via a wired connection (implemented, similarly to the wired connection for the denture set 510, using a USB interfacing port, such as the interfacing port 528, optionally connected to a multiprotocol USB converter FT232H module to convert data signals and packets for various different data protocols), or via wireless connections (not shown in FIG. 5A).

[0093] The computing system (device) 530 controls some of the central functions of the system 500, including receiving data from the tool 520 and the denture set 510, processing the data using custom processes / algorithms, rendering a real-time 3D model to facilitate the training or actual non-simulated medical / dental procedures, performing evaluations of the techniques practiced by a trainee or practitioner, controlling at least some of the operations involving the denture set 510 and the tool 520 (such as configuring the sensors deployed in each of these components, etc.). The computing system 530 may also run a collision detection process, more particularly discussed below, to identify an active tooth or body part operated on, and to record various characteristics and statistics associated with such interactions (such as the duration of the interactions). The system’s architecture is optimized for low latency to ensure real-time feedback for the user. It is noted that the 3D tracking and rendering can be achieved with other types of sensors, including the various sensors described herein (e.g., infrared sensors embedded in the simulation structure, vibrations sensors, optical sensors, pressure sensors, etc.)

[0094] The computing system 530 may implement software processes executing on processors of computing system 530, with such processors including one or more CPU’s, one or more graphics processing units (GPU’s), one or more application processing units (APU’s), and other types of processors and circuitries. The software modules of the computing system 530 may include, for example, a 3D modeling unit and rendering engine adapted to construct and render and display the denture 510 and the tool 520 in real-time, and a collision detection system (also referred to as a physics engine) 534 adapted to identify when and where the tool collides with one or more teeth, and / or a user interface module 536 adapted to highlight the currently active tooth and provide real-time feedback on the operation. The computing system 530 may also include a data processing unit 532 to control data flows and operations within the computing system 530, as well as to perform at leastAttorney Reference No.: 7551-0010W001some of the processing required to, for example, perform the 3D modeling, determine interactions between the dental / medical tool and the simulation objects, etc. Further details regarding the computing system and its various modules (including the 3D modeling unit and the physics engine) are provide below with respect to FIG. 6. As further illustrated in FIG.5A, a user interface device 540 can include a display device on which a real-time rendering of a composite 3D model of the tool interacting with the simulation objects is presented. The real-time rendering is continuously updated to render the motion and trajectory of the tool relative to the simulation objects (e.g., the teeth in the example of FIG. 5A).

[0095] The collision detection system is a custom-built physics engine tailored to simulate dental procedures (the collision detection system can be adapted to simulate and assess physical contact between a medical tool and other body parts). The engine models the physical properties of both the tool and the denture, including their respective geometries and materials. The collision detection system works by analyzing the intersection of the tool’s vector with the 3D model of the denture, determining which tooth is being contacted. The system is configured for continuous real-time operation, updating the 3D model and collision data as new sensor data is received. The physics engine processes this data in a short period of time (e.g., milliseconds), ensuring that the user receives immediate feedback. To achieve this, the system can use parallel processing and optimized processes / algorithms to minimize latency. The tool-to-tooth interaction is modeled using a combination of vector mathematics and collision detection algorithms. The system calculates the position of the tool relative to the denture set and identifies which tooth is in the path of the tool. Special procedures / algorithms are employed to differentiate between teeth that are close together, ensuring accurate detection even in complex areas of the mouth.

[0096] FIG. 5B includes images of an example tool 560 and a denture set 550 that may be similar to the schematically illustrated tool 520 and the denture set 510, respectively, of FIG.5A. The tool 560 includes two IMU’s 562 and 564 disposed on (in some embodiments may be embedded in) the body of the tool 560. The denture set 550 includes an IMU 552 that may be disposed on a top surface of the upper part of the denture set 550, or alternatively may be embedded with the denture set, and an accelerometer 554 embedded in the lower part of the denture set. The IMU 552 and the accelerometer 554 may be similar to the various IMU’s and accelerometers used in conjunction with the denture set 510 of FIG. 5A. Additional sensors (IMU, accelerometers, vibration sensors, etc.) may be included with the denture setAttorney Reference No.: 7551-0010W001550 and / or the tool 560 of FIG. 5B. As also shown in FIG. 5B, the denture set 550 may also include a homing device (e.g., a button actuatable by a user) to help calibrate the sensors of the denture set (e.g., to record and correlate sensor measurements sensed by the sensors when the homing device is actuated). The denture set 550 may be connected to a remote computing system via a wired connection extending from a port 558. Alternatively or additionally, communication with the computing system may be established through a wireless link. The tool 560 may communicate with the remote computing system in a manner similar to that of the denture set 550.

[0097] FIG. 6 is a flow diagram showing the various data processing operations that are implemented for the example architecture of the simulation system / platform 500 of FIG. 5A. It is noted that similar data flows can be implemented with respect to the example systems described in relation to FIGS. 1-4. As illustrated, motion and force measurements are made at blocks 610, 612, and 614 by the sensors (in this case IMU device sensors) of the tool (such as the tool 520 of FIG. 5A) and the denture set (such as the top and bottom parts of denture set 510 of FIG. 5A). The IMU’s generate measurement data (conforming to predefined data formats and structures) that is transmitted to a data collection unit 630 at the computing system (such as the computing system 530 of FIG. 5A) via wired or wireless communication links marked as links 620, 622, and 624. As illustrated in FIG. 6, in some embodiments, the denture set may also include accelerometers (and / or other types of sensor devices) to detect force components, such as vibrations, caused from contact between the tool and the denture set, with the data produced by the accelerometers facilitating refinement of positional and motion data generated by the IMU’s Such supplemental data can be transmitted to the data collection unit 630 via the links 622 and 624, or via separate independent links. As further illustrated in FIG. 6, in some embodiments, the data flow 600 may also be configured to collect and transmit homing activity data, in which the user is prompted to contact one or more of the teeth in the denture sets. That data collection, represented by operation block 626, can then be used (e.g., by a data processing unit 640), in combination with sensor data generated and transmitted by the various deployed sensors, to home / calibrate the system so that events in which the tool touched particular teeth can be associated or correlated with measurement data generated by the various sensors. The prompts provided to the user can also include specific instructions on orientation and relative force at which the tool is to contact the particular teeth.Attorney Reference No.: 7551-0010W001

[0098] Having collected the data at the data collection unit 630, at least some of the data is transferred to the data processing unit 640 to for processing that includes, for example, determining positions (relative and absolute) of the various simulation objects (e.g., teeth, the tool, etc.), determining motion and trajectory of the various simulation objects, evaluating performance of the trainee based, for example, on the determined or estimated motion and trajectory information, the computed strength of forces exerted by the tool and / or experienced by the simulation objects (teeth or other body parts), etc. It is noted that, in some embodiments, at least some of the processing may be performed at remote or distributed processing nodes, including at processing units that may be included at the simulation structure (e.g., at one or more of the various sensor devices; such sensor devices may be part of an loT network).

[0099] As depicted in FIG. 6, some of the data processing operations may be performed at dedicated units (implemented as software and / or hardware-based units) that include a physics engine 650 and a 3D modeling unit 652. As noted, the collision detection engine is configured to model the physical properties of both the tool and the denture set, and analyze the interactions of the tool’s vector, as may be determined by data collected by the various sensors deployed on the tool (including motion data determined from measurements of the IMU device), with the 3D model of the denture set. For example, data from the tool’s IMU devices is used to track the linear and rotational motion of the tool, taking into account the tool’s known dimensions and shape, relative to the positions of the various simulation objects (e.g., teeth) as determined by the simulation objects’ known dimensions / shapes and the measurement data collected by the sensors deployed in the simulation structure.Measurement data corresponding to detected vibrations and forces can be combined with the motion data to accurately determine which of the simulation objects / parts (i.e., the teeth in the example of FIG. 6) is contacted by the tool, the duration of such contact, whether the contact is stable or unstable, the direction and level of forces applied to the various simulation objects, and so on. The physics engine produces output that is representative of these various determined features (e.g., represented as data vectors or matrices), with that output data provided to at least the 3D modeling unit 652. The physics engine’s output data may also be provided other modules, implemented by or with the data processing unit 640, such as an evaluation module, , which may be based at least partly on a machine learning implementation, to assess the performance of the trainee or practitioner. The 3D modeling unit 652 generates, in some embodiments, an adjustable composite 3D model representationAttorney Reference No.: 7551-0010W001to represent the positions of the simulation structure (with the simulation objects, such as the teeth) and of the tool. In various examples, such a composite 3D representation is generated by updating a previously determined composite 3D model (e.g., the immediately preceding composite 3D model) with, for example, the output data generated by the physics engine 650 (e.g., through matrix and vector transformation operations). The resultant data of the current updated composite 3D model is transmitted to a 3D Tenderer that produces the graphical rendering of the 3D that can be presented on a graphical user interface device (such as the dental manakin UI 540).

[0100] Accordingly, in various embodiments, the plurality of sensors of the simulation system may include a first set of inertial measurement units (IMUs) mounted on or within the operation tool manipulated by the trainee, and a second set of IMU’s disposed proximate to at least some of the plurality of simulation parts in the simulation structure. The IMU’s of the first set can be aligned with a tip of the operation tool, and configured to measure displacement of the tool, with resultant measured displacement (e.g., motion or trajectory) data being used to derive, by the computing device, a vector movement representative of the displacement of the tool. The IMU’s of the second set of IMU’s can be configured to measure movement and orientation data representative of movement and orientation of at least some of the plurality of simulation parts in the simulation structure, with the movement and orientation data being configured to be communicated (e.g., through wired or wireless communication protocols) to the computing device to adjust a 3D model representative of a state of the simulation structure and / or the tool.

[0101] The computing device configured to determine the dynamic physical state of at least some of the one or more simulation parts and the operation tool may be configured to implement a physics engine modeling physical properties of the one or more simulation parts in the simulation structure and of the operation tool, with the physics engine being configured to analyze interactions between the operation tool and at least one of the one or more simulation parts in the procedure area, based on the modeled physical properties and measurement data detected by the first and second sets of IMU’s, to represent the effect of the interactions on a 3D model of the one or more simulation parts and of the operation tool. The physics engine may be configured to continually adjust the 3D model of the one or more simulation parts and the operation tool in response to receipt of updated data detected by the first and second sets of IMU’s. In some examples, the physics engine can further beAttorney Reference No.: 7551-0010W001configured to model tool-to-tooth interactions caused by application of force by the trainee to the at least one simulation part via the operation tool, and to derive, based on data collected by the first and second sets of IMU’s, and analyzed using the tool-to-tooth model, updates to a state of the 3D model representation of the one or more simulation parts.

[0102] The proposed medical / dental simulation platforms described herein also include implementations of a monitoring framework configured to monitor and track the procedure being performed, whether by a trainee or by a practitioner (dentist or doctor) using the simulation platform to assist with real-patient procedures. IMU’s and optical sensors deployed on the tool and around a patient’s body part being operated on can be used to help model and render on a user-interface device aspects of the procedures performed, such as presenting on a UI device a real-time rendering of the tool and patient’s body parts, and the interactions therebetween. Such rendering may be a virtual reality rendering, or an augmented reality rendering (e.g., using graphically constructed rendering to supplement the scene captured by the optical sensors).

[0103] In example embodiments of the monitoring and tracking framework, the dental / medical simulation platform can be configured according to a button-based dental tracking approach. Under this approach, the solution includes placing, in some embodiments, a button next to each tooth beneath the silicone lining of the dental manikin’s inner mouth. Here, trainees would press the button to indicate which tooth they are working on, with visual feedback displayed on a screen or through an LED illuminated next to the selected tooth. Additionally, vibration sensors strategically placed around the mouth would track the time spent on each tooth, as well as other measurable properties / characteristics. Such vibration sensors need to be sensitive enough to distinguish between different teeth, but robust enough to filter out noise and unrelated vibrations. Fine-tuning the sensor placement and calibration is important for accurate data collection.

[0104] In some embodiments, the monitoring and tracking implementations may include a more streamlined approach that includes an entirely digital user interface (UI) where trainees can select the tooth they are working on, while the system automatically logs data using the vibration sensors. The latter approach removes the need for direct physical interaction between the trainee and the simulation parts (in this case, the simulation teeth) and enhances data accuracy by reducing reliance on user actions, thereby improving the system’s reliability. In embodiments involving a digital UI, vibration sensors can autonomously trackAttorney Reference No.: 7551-0010W001and record time spent on pre-selected tooth, and the visual feedback would be presented digitally through the UI (digital or physical), providing a more user-friendly and dependable solution. More particularly, and with reference to FIG. 7, a flow diagram of an example workflow 700 for the proposed streamlined approach is shown. In a first step / process 710 of the workflow, a trainee presses a button or uses the UI to select the tooth the trainee will be working on. In a second step / process 720 of the workflow, the simulation system monitors which button(s) has been pressed, and collects vibration data. When a tool is detected working on a tooth, the system correlates the vibration data with the selected tooth and records various procedure and performance parameters, such as the time spent working on the tooth. In a third step or process 730 of the workflow, visual feedback indicators provided via a display device (e.g., the UI device 540 of FIG. 5A) or LED display ensures the trainee knows which tooth is being tracked. In a fourth step or process 740 of the workflow, procedure and performance parameters and metrics (such as the time spent on each tooth) is logged, and session data is stored centrally for future review. In a fifth step or process 750 of the workflow, at the conclusion of the training session, the system generates a detailed report for both the trainee and instructor, showing, among other things, time spent on each tooth and any activity gaps.

[0105] The monitoring and tracking framework (also referred to as RG ToothTap) described in relation to FIG. 7 provides a reliable and non-intrusive implementation that records the various performance parameters and metrics, thus avoiding manual recording or rough estimates, which may be inaccurate or prone to human error. Under this framework, for a dental simulation structure (a manakin with a denture set) each tooth has a corresponding button placed beneath a silicone lining of the dental manikin’s mouth.Trainees press a button or use a digital UI to indicate on which tooth they are working.Vibration sensors placed around the mouth can detect a handpiece’s (e.g., the tool’s) vibrations, correlating them with the tooth being worked on to automatically track time.Feedback is provided to the trainee through either screen-based notifications or by illuminating the specific tooth being worked on. Time data, and other procedure parameters and metrics, are collected and logged in real time, with reports generated for instructors to review trainee performance.

[0106] For this implementation, the system (which may be similar, at least in some respects, to the systems and implementations of FIGS. 1-6) includes tactile buttons positionedAttorney Reference No.: 7551-0010W001underneath each tooth’s corresponding location. Such buttons may be wired to a control unit, or alternatively may have wireless communication circuitry (e.g., wireless transceivers), with the pressing of a button causing the logging of a start and end times for that particular tooth. Multiple low-profile vibration sensors may be placed around the manikin's jaw (upper and lower) to detect the use of dental instruments. A control unit (e.g., one implemented using a computing system such as the computing systems 150 or 530), powered by one or more microcontrollers, processes signals from the buttons and vibration sensors, and logs data in a time-stamped format. A display interface provides visual feedback to the trainee on the selected tooth, and whether the time (and / or other parameters) is being recorded correctly. The implementation may also include LED indicators, placed inside the manikin, to illuminate the specific tooth being worked on, providing the trainee with immediate feedback on their selection. As part of this implementation, the platform may include several softwarebased components, including a user interface (UI), which may include a touchscreen or software interface to allow trainees to select which tooth they are working on, without physically pressing buttons. Such an interface could also display real-time information on the selected tooth and time elapsed. Additional software-based modules that may be included with this implementation include a data logging module to log all interactions (button presses, vibration data and other sensor data collected) and create a time-stamped record for each tooth, and a reporting module to compile the data into a format easily reviewed by instructors, including time spent on each tooth, efficiency trends, and areas for improvement.

[0107] This particular time-tracking implementation is a cost-effective and non-intrusive solution for monitoring dental trainee performance. By combining manual inputs and automated vibration-based tracking, the system can significantly improve the accuracy of time measurements during practice sessions. This implementation provides a strong foundation for improving the objectivity and reliability of dental training evaluations.

[0108] Thus, in some embodiments, the plurality of sensors of the simulation system may include one or more simulation part selection items, actuatable by the trainee, that are respectively associated with individual ones of the one or more simulation parts, with the one or more simulation part selection items being selectable by the trainee to indicate an intended simulation part (e.g., a simulation tooth) on which the trainee is to perform the simulated medical or dental procedure. The plurality of simulation sensors may further include one or more vibration sensors to detect vibrations caused by performance of the simulated medicalAttorney Reference No.: 7551-0010W001or dental procedure. In various embodiments, each of the one or more simulation parts selection items may include one or more of, for example, a physical actuatable switch, and / or a graphical selection item.

[0109] The proposed medical / dental simulation platforms described herein can further include implementations that provide an enhanced and realistic training experience that uses drillable and replaceable teeth / dentures. Such replaceable teeth, structured to have shapes and dimensions that capture complex dental scenarios, are combined with a monitoring assembly that tracks and determines the quality of workmanship and techniques exercised by the trainees to thus more closely emulate real-life dental experiences.

[0110] More particularly, FIG. 8 includes a diagram 800 of an example simulation part in original and altered forms. The simulation part includes a drillable replaceable tooth 810 constructed from material with material properties that emulate behavior of a real human tooth when operated upon by a dental tool (such as a dental drill). The tooth 810 is coupled to a sensor array assembly 820, that in some embodiments can be removably attached to the tooth 810. That is, the replaceable tooth 810 may be a modular element that can be attached to the sensor array assembly 820, and subsequently removed and replaced with a new replaceable tooth after the tooth has been degraded to an unusable condition through drilling and carving performed by trainees. In some embodiments, the sensor array assembly may be secured to the denture assembly (such as the assemblies 120 or 202 in FIGS. 1 and 2) in sockets defined in the assemblies into which the replaceable teeth may be mounted. The sensor array assembly 820 may include a fastening mechanism (not shown), such as hooks or threads matching complementary threads near a base 812 of the tooth 810. In various embodiments, the tooth and sensor assembly array may constitute an integrated structure that is attachable to the denture assembly using an attachment mechanism disposed on the tooth and sensor array assembly. Different types of attachment mechanisms may be used in connection with an integral tooth and sensor array device, or with a tooth structure attachable to a sensor array assembly.

[0111] In some embodiments, the sensor array assembly 820 includes an IR based detection mechanism configured to detect and estimate structural changes to the replaceable tooth resulting from degradations to the tooth. In such embodiments, the detection mechanism may include at least one illumination source, such as infrared (IR) light emitting diode (LED) 822, and one or more illumination detectors, such as infrared photoreceptors (sensors) that includeAttorney Reference No.: 7551-0010W001an IR photoreceptors 824, that are disposed in the sensor array assembly 820. In the example of FIG. 8, the illumination source (the at least one IR LED 822) emits, during operation, infrared light into the body of the tooth 810. A portion of the internal body of the replaceable tooth 810 may be hollow or sufficiently porous to allow the emissions from the illumination source to permeate through the body, causing an internal reflection pattern, such as a pattern 830, to form. Some other portion of the emission (e.g., of the emitted IR light) may be partially absorbed by the different materials of the tooth and / or leak out of the tooth.

[0112] As the surface and structure of the tooth model changes over time, the proportion of light that reflects, gets absorbed, and / or leaks, changes. A relationship between the characteristics and patterns of the emissions can thus be used to estimate or predict the changed structure of the replaceable tooth, e.g., based on a machine language model optimized with training data correlating structural properties (including shapes and dimensions) of replaceable teeth to detected emissions and properties of the emissions, or based on approximated modeling of relationships between detected light and structures of the teeth. Accordingly, by measuring the light received by the sensors as a result of radiation emissions (with the emissions having known properties such as pulsating frequency, wavelength, etc., and / or using known properties of the material and initial shape of the replaceable tooth or other simulation part), the shape of the tooth, including its external contours and the tooth’s internal profile, can be estimated or predicted. As shown in FIG. 8, the tooth 810 is altered to include a cavity 840, resulting in a subsequent reflection pattern 850 that is different from the original reflection pattern 830 (assuming the illumination source characteristics have remained substantially fixed). That subsequent reflection pattern will result in a different detection pattern by the sensors (such as by the IR photoreceptor 824), from which an estimated structure of the altered tooth can be derived.

[0113] FIG. 8 also include an example processing pipeline showing the processing that may be performed on signals measured by the sensor array assembly 820. The pipeline may be implemented at a remote computing system (such as the computing system 150 or 530 of FIGS. 1 and 5, respectively) and / or locally at a processing unit included with the assembly 820 or located in the simulation structure (e.g., the manakin). In the example of FIG. 8, the processing pipeline includes a signal processing section 860 (which may be implemented locally at a processing unit located at the manakin, and / or remotely at a computing system) to perform signal processing on measurement data. Such signal processing may includeAttorney Reference No.: 7551-0010W001filtering signals to remove noise, converting signals into appropriate data representations, aligning data records, etc. Resulting data records produced by the signal processing sections can be transmitted to a machine learning section 870. As noted, the ML section may have been trained to model optimized relationships between input data (e.g., detected optical patterns, materials used for the replaceable tooth, characteristic of the illuminating emissions, etc.) and the structure (internal and / or external shape and dimensions) of the replaceable tooth. In some embodiments, the example pipeline depicted in FIG. 8 may also include a tooth deformation modelling section 880 in communication with the machine learning section 870. For example, the machine learning section 870 may generate, based on input data received from the signal processing section 860, output data representative of some predicted features of the structure of the replaceable tooth. That predicted data can then be fed into the modelling section 880, which implements a structure modeling of the tooth (e.g., to allow a 3D rendering of the altered replaceable tooth) that relies, in part, on the predicted data generated by the ML section 880.

[0114] Thus, in various embodiments using a simulation structure that includes a manikin with a physical model of a head part including mouth in which simulation teeth are disposed, for at least one tooth from the one or more simulation teeth, the physical model of the head part further includes respective one or more infrared (IR) LEDs and respective one or more IR sensors embedded underneath the at least one tooth. In such embodiments, the computing device is further configured to controllably cause emissions of the one or more IR LEDs into an internal structure of the at least one tooth during performance of the dental procedure, and determine changes to the structure (internal and / or external) of the at least one tooth, caused by the dental procedure, based on changes of characteristics of reflected IR signals from surfaces of the structure of the at least one tooth, resulting from the changes to the internal and / or external structure of the at least one tooth.

[0115] Embodiments of the simulation framework described herein also include implementations to manufacture replaceable teeth, including to implement automatic, complex dentition generation to create more realistic simulated scenarios for trainees. A key challenge for dentistry training (and other training domains) is that dentition models are always relatively “perfect” - teeth are nicely arranged, non-overlapping, nicely symmetric, etc. This does not reflect the reality of more complex dental cases. The proposed framework allows for specifying complex dentitions (in terms of teeth placement and arrangements,Attorney Reference No.: 7551-0010W001sizes, etc.), and auto-producing models for those dentitions, thus significantly increasing the diversity of training that can be conducted. In one example implementation, a grid-like userinterface system is used to allow for complex teeth coordinates to be specified. From this tooth-contour specification, the rest of the dentition can be automatically modelled and fabricated. To simplify the fabrication process, templates for a select number of dentitions can be developed, and replaceable dentitions parts can be manufactured according to selected ones of the developed dentitions. In some examples, replaceable teeth, manufactured by specifying coordinates and contours for complex dentition, can further be customized by using attachment mechanisms (e.g., rotatable and displaceable attachment joints and adapters) in the dental simulator that allow the manufactured complex dentitions to be attached to the head part at different angles and arrangements that create complex orthodontist configurations (crowded and / or skewed teeth, overbites, etc.) on which the trainees can train.

[0116] The simulation framework allows for customization of specific dental orthodontal scenarios (involving various defects, malocclusion, crooked teeth, etc.) via, in some examples, a third-party computer- supported interface (web-based or API) with dentalcustomization applications (or applications for other medical simulation domains, such as surgery for various body organs / parts, as well as applications to customize tools and training scenarios in non-medical fields). Once customized, the simulation parts (be it teeth, or sets of teeth, or parts to simulate training in other domains and fields) are communicated to a manufacturer, which may be the same third-party supporting the customization interface and applications, to manufacture the customized simulation parts (at requested quantities). It is noted that this customization framework produces simulation parts that can seamlessly be integrated to the simulation platform (e.g., a dental simulator) in which the various sensors (and other tracking devices) are embedded into the platform, and with which tracking apparatus and procedures generally do not need (or if needed, only minimally) to be modified to operate with the customized simulation parts. If modification of, for example, the tracking apparatus and procedures (whether implemented by hardware, software, or both) are needed (e.g., to define the characteristics of simulation teeth and orthodontist configuration in the dental simulator in order to properly account for or weigh signals received by the sensors), such modification can be made based on the specifications of the customized simulation parts, as provided by the customer (e.g., through the grid-based graphical interface that allows the user to customize teeth of specific dimensions and contours, and to specify theAttorney Reference No.: 7551-0010W001overall orthodontist configuration of the teeth), or through data representation produced by the remote party in response to the specification provided by the customer. Thus, in some embodiments, the dental / medical simulation framework, a method for simulation platform development is disclosed that includes receiving, via a grid-like graphic user-interface, specifications regarding dimensions and contours of one or more customized replaceable simulation teeth (which may be, in some embodiments, free of tracking or sensing circuitry), and generating, based on the specification, manufacturing instructions to produce the one or more customized replaceable simulation teeth. The one or more simulation teeth are attachable to a dental simulation system, for training dental practitioners, comprising embedded sensors that track motion of a dental trainee and / or a dental tool operating on at least one of the one or more customized replaceable simulation teeth. Similar methods can be used for developing simulation platforms for other medical or non-medical fields / domains.

[0117] Another feature to enhance the usability of the simulation framework described herein is through the use of a foldable simulation structure that facilitates storage and deployment of the simulation structure (e.g., to facilitate placing and fitting a manakin on a dentist chair). FIG. 9A is an image 900 of a manakin 910 in a deployed, unfolded position. The manakin 910 includes a pivotable head 920 that is configured to rotate about a neck joint 930, located between a torso part 940 and the pivotable head 920, for placement within an interior space defined in the torso part 940. In some embodiments, the neck joint 930 includes multiple interlocking vertebrae that restrain rotation about two axes while allowing the neck to fold about the third axis into the interior space within the torso part 940. Laying the torso on the chair and unfolding the neck provides realistic human proportions and deformations, to prevent or inhibit a dental trainee from developing habits that cannot translate to live-patient situations.

[0118] In FIG. 9A, the interior space is covered by a pivotable lid 950 that can be pivoted about a pivot located at a tab 952 of the pivotable lid 950 from a covered position to an uncovered position. During the folding operation, the lid 950 is opened to expose the unoccupied interior space, and the pivotable head 920 is rotated into the unoccupied interior space which receives the pivotable head for transportation and storage of the manakin 910. In some embodiments, the manakin 910 may be dimensioned and shaped to be mountable on different dental chairs to occupy positions, on the respective dental chairs, that simulate the real working environment of a dental office, enabling ergonomic practice. FIGS. 9B-FAttorney Reference No.: 7551-0010W001include additional images of the manakin 910, or parts thereof, in various folded or configurations.

[0119] The simulation framework can be configured to create a realistic experience for trainees by, for example, manufacturing drillable replaceable teeth with different unique configurations to emulate realistic dental and orthodontic situations. As noted, such replaceable teeth can include removable and replaceable teeth of different models, and for a variety of practice tasks, such as cavity preparation, extractions, or restorations. Another example of creating realistic dental and medical situations for the simulation framework is to construct simulation structures (e.g., manikins) with realistic body parts (e.g., a realistic head part) to provide the trainee with a more realistic experience of the procedures being practiced. For example, in some implementations, the head part of the manikin head may include a controllable jaw. Existing manikin heads for dental / medical training have a fixed mouth opening. In reality, however, patients have very different mouth shapes and sizes. The proposed head element of the simulation structure includes a controllable, lockable jaw, that allows for the mouth to be opened and fixed at various degrees (around the jaw). This allows training simulation on a diverse set of simulation configurations, representing a diverse population, but also allows for the impact the trainee’s practiced procedures on, for example, the patient’s smile to be assessed (by closing the mouth on the controllable jaw - a key subjective evaluation criteria for patients and, as such, important to train for). The controllable jaw may be manually controlled, e.g., with the trainee, or someone else, applying force at various locations (referred to as pressure / articulation points) of the jaw to displace displaceable elements of the jaw, or may be electrically controlled using an electrical motor to actuate displaceable elements based on control signals that are generated automatically, e.g., in response to occurrence of certain events, or in response to various cues from the trainee or a third-party. The actuatable jaw can thus be locked into different positions / configurations. Accordingly, in embodiments using a simulations structure that includes a manikin with a physical model of a head part having a mouth in which simulation teeth are disposed, the head part may include a controllable, lockable jaw, with the controllable jaw including moveable parts that move to place the jaw in one of multiple structural configurations upon application of force at one or more pressure points to actuate movement of one or more of the moveable parts.Attorney Reference No.: 7551-0010W001

[0120] Additional features that may be added to the head structure to make the training experience for dental trainees more realistic include, for example, adding a tongue to the mouth area of the head structure, manufacturing multi-layered teeth to more accurately represent the anatomy of the tooth (e.g., separate layers, possibly using different materials, for the enamel, the dentin, the pulp, etc.), and so on. Such multi-layered simulation teeth can be manufactured using, for example, 3D-printers, or through other manufacturing processes. As noted, in some embodiments, the teeth (or other simulation organs or tissue) may be disposable teeth configured to be decoupled from the simulation structure, and to be replaced with unused disposable replacement teeth (that can be manufactured by authorized vendors, as discussed above) after the current simulation teeth to be replaced have become too eroded to continue being used in training sessions.

[0121] Thus, in embodiments using a simulation structure that includes a manikin with a physical model of a head part including a mouth in which simulation teeth are disposed, at least one tooth from the simulation teeth may be a customized multi-layered tooth, shaped and dimensioned according to grid-based specifications specifying coordinates of features of the at least one tooth, with layers of the multi-layered at least one tooth being composed from respective materials having different material characteristics representing different anatomical layers of an actual tooth.

[0122] In some embodiments, the head part of the simulation structure may be equipped with pressure sensors (load cells, force transducers) embedded in the jaw part of the head structure (whether the jaw is a controllable displaceable jaw, or a locked unmovable jaw). One of the key challenges of manual task learning is understanding how much pressure should be applied through any tool. This is especially important in dentistry where this pressure has an important impact on patient comfort (and dentist fatigue). When the trainee applies pressure onto a tooth, the pivot around the jaw transfers that force onto the load cell near the jaw, returning a force value. Feedback about this force can be provided back to the trainee in real time, so that the trainee can dynamically adjust his / her practice (technique). This feature also requires no augmentation of dentist tools being used by the trainee. Thus, in such embodiments, the plurality of sensors in the simulation structure can include multiple pressure sensors, each embedded at respective locations in the simulation structure and configured to detect forces applied to the simulation structure through contact between the operation tool and the one or more of the simulation parts. In situations where the simulationAttorney Reference No.: 7551-0010W001structure is a dental simulation structure, the multiple pressure sensors can include multiple load cells embedded in a jaw and / or skull parts of a head part of the dental simulator, with each of the multiple load cells being configured to gauge a level of force measured at the respective one of the each of the multiple load cells. In such situations, the computing device is further configured to process the force levels measured by the multiple load cells, and to generate feedback data, provided to a user-interface, representative of performance of the dental procedure and an associated comfort level that a live patient would feel for similar measured force levels.

[0123] Providing a realistic experience for trainees using the simulation systems described herein is also achieve by constructing, for dental training purposes, head models that have detailed anatomical structures, including, for example, soft tissue, teeth, and jaws, to mimic real patient scenarios. In some examples, the simulation structure (manakin) is configured to be mountable on a dental chair. FIGS. 10A-Q include images of various head designs and manakin assemblies that can be used with the dental simulation systems described herein. FIG. 11 includes a photo of a practitioner practicing on a manakin of a simulation system such as any of the simulation systems described herein.

[0124] FIG. 12 is a flowchart of an example procedure 1200 for simulating a medical or dental procedure. The procedure 1200 includes tracking 1210 mechanical displacements with a plurality of sensors distributed in a procedure area of a simulation structure comprising one or more simulation parts emulating anatomical parts (of a human or animal), with the mechanical displacements being caused through application of an operation tool handled by a trainee to at least one simulation part from the one or more simulation parts in the simulation area in which the trainee is performing the medical or dental procedure. The procedure 1200 further includes determining 1220, at a computing device, based on the tracked mechanical displacements, a dynamic physical state of at least some of the one or more simulation parts and the operation tool, resulting from performance of the medical or dental procedure in the simulation area. The procedure 1200 also includes determining 1230 at the computing device, based at least in part on the determined dynamic physical state, procedure performance feedback data representative of the procedure’s performance by the trainee.

[0125] In various example, determining the physical state can include one or more of, for example, i) tracking linear and angular movements of the at least some of the one or more simulation parts, the operation tool, a body part of the trainee, and / or the simulation structure,Attorney Reference No.: 7551-0010W001and / or ii) determining forces applied on the at least some of the one or more simulation parts and / or the operation tool resulting from interaction of the operation tool with the at least one simulation part.

[0126] In various embodiments, the simulation structure can include a manikin with a physical model of a head part having a mouth, with the one or more simulation parts including simulation teeth constituting a training simulation denture set, and with the operation tool includes a dental tool handled by a trainee to perform a simulated dental procedure on at least one tooth from the simulation teeth. In such embodiments, for at least one tooth from the simulation teeth, the physical model of the head part further may include respective one or more infrared (IR) LEDs and respective one or more IR sensors embedded underneath the at least one tooth, with the procedure further including controllably causing emissions of the one or more IR LEDs into a structure of the at least one tooth during performance of the dental procedure, and determining changes to the structure of the at least one tooth, caused by the dental procedure, based on changes of characteristics of reflected IR signals from surfaces of the internal structure of the at least one tooth, resulting from the changes to the structure of the at least one tooth.

[0127] In some examples, the plurality of sensors can include a first set of inertial measurement units (IMUs) mounted on or within the operation tool manipulated by the trainee, and a second set of IMU’s disposed proximate to at least some of the one or more simulation parts in the simulation structure. In such examples, the procedure may further include measuring, by the IMU’s of the second set, movement and orientation data representative of movement and orientation of at least some of the one or more simulation parts in the simulation structure, with the movement and orientation data being configured to be communicated to the computing device to adjust a 3D model representative of a state of the simulation structure. Determining the dynamic physical state may include implementing a physics engine modeling physical properties of the one or more simulation parts in the simulation structure and of the operation tool, with the physics engine being configured to analyze interactions between the operation tool and at least one of the one or more simulation parts in the procedure area, based on the modeled physical properties and measurement data detected by the first and second sets of IMU’s, to represent the effect of the interactions on a 3D model of the one or more simulation parts and of the operation tool.Attorney Reference No.: 7551-0010W001

[0128] In example embodiments, the plurality of sensors may include multiple vibration sensors, each embedded at respective locations in the simulation structure and configured to detect vibration caused through contact of the operation tool with one of the one or more simulation parts of the simulation structure, with the procedure further including deriving contact location of the operation tool based on amplitude, timing, frequency, and / or peak offset characteristics of vibrations signals measured by at least some of the multiple vibration sensors. In such example embodiments, deriving contact location of the operation tool may include deriving the contact location based on one or more of, for example, a multilateration or triangulation procedure applied to the vibration signals measured by the at least some of the multiple vibration sensors, and / or a machine learning (ML) vibration-to-location model that generates an output label representative of the contact locations according to the measured vibration signals inputted to an ML system implementing the ML vibration-to-location model.

[0129] In some examples, determining a trainee’s procedure performance feedback data may include evaluating, using a machine learning system implementing a performance evaluation model, based on tracking data of motion of the operation tool and limbs of the trainee, performance of the trainee, with the performance evaluation model being optimized based on training data comprising training samples with training motion tracking data for a plurality of previously evaluated practitioners, and respective output labels representing the evaluation of the performance of the plurality of previously evaluated practitioners upon performing procedures resulting in the respective training motion tracking data.

[0130] In various embodiments, the plurality of sensors can include multiple optical sensors configured to capture positioning information associated with one or more trackable objects disposed on at least the operation tool. In such embodiments, the procedure can further include tracking, based on the captured positioning information, spatial location and motion of the operation tool. The one or more trackable object may include one or more infrared (IR) LEDs each illuminating cycles of IR emissions at a respective pre-determined cycle frequency, with the multiple optical sensors including multiple macro-lens infrared (IR) cameras configured to capture the IR emissions from the one or more IR LEDs. The procedure can further include deriving the spatial location and motion of the operation tool according to relative positioning and strength of captured IR signals from the IR emissions illuminated by the one or more IR LEDs.Attorney Reference No.: 7551-0010W001

[0131] Implementing the proposed framework and performing the various techniques and operations described herein may be facilitated by a controller device(s) (e.g., a processorbased computing device). Such a controller device may include a processor-based device such as a computing device, and so forth, that typically includes a central processor unit or a processing core. The device may also include one or more dedicated learning machines (e.g., neural networks) that may be part of the CPU or processing core. In addition to the CPU, the system includes main memory, cache memory and bus interface circuits. The controller device may include a mass storage element, such as a hard drive (solid state hard drive, or other types of hard drive), or flash drive associated with the computer system. The controller device may further include a keyboard, or keypad, or some other user input interface, and a monitor, e.g., an LCD (liquid crystal display) monitor, that may be placed where a user can access them

[0132] The controller (computing) device may be configured to control a simulation structure and / or control sensors deployed throughout the simulation structure, and analyze simulation data received for the simulation structure (e.g., a dental structure in a model head) and render visual feedback data (e.g., a 3D rendering of movement tracking data for different simulation parts and objects of the simulation structure) to allow a trainee to hone his / her skills through the proposed simulation systems described herein. The storage device may thus include a computer program product that when executed on the controller device (which, as noted, may be a processor-based device) causes the processor-based device to perform operations to facilitate the implementation of procedures and operations described herein. The controller device may further include peripheral devices to enable input / output functionality. Such peripheral devices may include, for example, flash drive (e.g., a removable flash drive), or a network connection (e.g., implemented using a USB port and / or a wireless transceiver), for downloading related content to the connected system. Such peripheral devices may also be used for downloading software containing computer instructions to enable general operation of the respective system / device. Alternatively and / or additionally, in some embodiments, special purpose logic circuitry, e.g., an FPGA (field programmable gate array), an ASIC (application- specific integrated circuit), a DSP processor, a graphics processing unit (GPU), application processing unit (APU), etc., may be used in the implementations of the controller device. Other modules that may be included with the controller device may include a user interface to provide or receive input and output data. The controller device may include an operating system.Attorney Reference No.: 7551-0010W001

[0133] In some embodiments, artificial intelligence (Al) systems (e.g., trainable machine learning (ML) systems) may be used to facilitate the implementations of the systems and processes described herein. AI / ML systems may be realized using different types of ML architectures, configurations, and / or implementation approaches. For example, neural networks used within the proposed frameworks may include convolutional neural network (CNN), feed-forward neural networks, recurrent neural networks (RNN), etc. Feed-forward networks include one or more layers of nodes (“neurons” or “learning elements”) with connections to one or more portions of the input data. In a feedforward network, the connectivity of the inputs and layers of nodes is such that input data and intermediate data propagate in a forward direction towards the network’s output. There are typically no feedback loops or cycles in the configuration / structure of the feed-forward network.Convolutional layers allow a network to efficiently learn features by applying the same learned transformation(s) to subsections of the data. Other examples of learning engine approaches / architectures that may be used include generating an auto-encoder and using a dense layer of the network to correlate with probability for a future event through a support vector machine, constructing a regression or classification neural network model that indicates a specific output from data (based on training reflective of correlation between similar records and the output that is to be identified), vector transformation ML systems (e.g., to transform data into an embedding space representation), etc. The various learning processes implemented through use of the neural networks may be configured or programmed using TensorFlow (an open-source software library used for machine learning applications such as neural networks). Other programming platforms that can be employed include keras (an open-source neural network library) building blocks, NumPy (an open-source programming library useful for realizing modules to process arrays) building blocks, etc.

[0134] Computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the term “machine -readable medium” refers to any non-transitory computer program product, apparatus, and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a non-transitory machine-readable medium that receives machine instructions as a machine-readable signal.Attorney Reference No.: 7551-0010W001

[0135] In some embodiments, any suitable computer readable media can be used for storing instructions for performing the processes / operations / procedures described herein. For example, in some embodiments computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (such as hard disks, floppy disks, etc.), optical media (such as compact discs, digital video discs, Blu-ray discs, etc.), semiconductor media (such as flash memory, electrically programmable read only memory (EPROM), electrically erasable programmable read only Memory (EEPROM), etc.), any suitable media that is not fleeting or not devoid of any semblance of permanence during transmission, and / or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and / or any suitable intangible media.

[0136] Although particular embodiments have been disclosed herein in detail, this has been done by way of example for purposes of illustration only, and is not intended to be limiting with respect to the scope of the appended claims, which follow. Features of the disclosed embodiments can be combined, rearranged, etc., within the scope of the invention to produce more embodiments. Some other aspects, advantages, and modifications are considered to be within the scope of the claims provided below. The claims presented are representative of at least some of the embodiments and features disclosed herein. Other unclaimed embodiments and features are also contemplated.

Claims

1. Attorney Reference No.: 7551-0010W001CLAIMS WHAT IS CLAIMED IS:

1. A system for simulating a medical or dental procedure, the system comprising: a simulation structure with one or more simulation parts emulating anatomical parts; a plurality of sensors distributed in a procedure area in the simulation structure at which a simulation of a medical or dental procedure is being performed, at least some of the plurality of sensors configured to track mechanical displacements in the procedure area caused through application of an operation tool handled by a user to at least one simulation part from the one or more simulation parts in the procedure area; anda computing device comprising one or more processor-based devices coupled to one or more memory storage devices, the computing device configured to:determine, based on the tracked mechanical displacements, a dynamic physical state of at least some of the one or more simulation parts and the operation tool, resulting from performance of the medical or dental procedure in the procedure area; and determine, based at least in part on the determined dynamic physical state, procedure performance feedback data representative of the procedure’s performance by the user.

2. The system of claim 1, wherein the computing device configured to determine the physical state is configured to perform one or more of: i) track linear and angular movements of the at least some of the one or more simulation parts, the operation tool, a body part of the user, and / or the simulation structure, or ii) determine forces applied on the at least some of the one or more simulation parts and / or the operation tool resulting from interaction of the operation tool with the at least one simulation part.

3. The system of claim 1, wherein the tracked mechanical displacements comprise one or more of: detected vibrations, and / or detected spatial movements of the at least some of the one or more simulation parts and the operation tool.

4. The system of claim 1, wherein the simulation structure comprises a manikin with a physical model of a head part having a mouth, wherein the one or more simulation partsAttorney Reference No.: 7551-0010W001comprises simulation teeth constituting a training simulation denture set, and wherein the operation tool includes a dental tool handled by the user to perform a simulated dental procedure on at least one tooth from the simulation teeth.

5. The system of claim 4, wherein for at least one tooth from the simulation teeth, the physical model of the head part further comprises respective one or more infrared (IR) LEDs and respective one or more IR sensors embedded underneath the at least one tooth, and wherein the computing device is further configured to:controllably cause emissions of the one or more IR LEDs into a structure of the at least one tooth during performance of the dental procedure; anddetermine changes to the structure of the at least one tooth, caused by the dental procedure, based on changes of characteristics of reflected IR signals from surfaces of the internal structure of the at least one tooth, resulting from the changes to the structure of the at least one tooth.

6. The system of claim 4, wherein at least one tooth from the simulation teeth is a customized multi-layered tooth, shaped and dimensioned according to grid-based specifications specifying coordinates of features of the at least one tooth, wherein layers of the multi-layered at least one tooth are composed from respective materials having different material characteristics representing different anatomical layers of an actual tooth.

7. The system of claim 4, wherein the head part includes a controllable, lockable jaw, wherein the controllable jaw includes moveable parts that move to place the jaw in one of multiple structural configurations upon application of force at one or more pressure points actuating movement of one or more of the moveable parts.

8. The system of claim 1, wherein the simulation structure comprises a manikin with a physical model of one or more body parts, wherein the one or more simulation parts comprises simulation organs and blood vessels associated with the respective one or more body parts, and wherein the operation tool includes a surgical tool handled by the user to perform a simulated medical procedure on at least one of the simulation organs and / or blood vessels.Attorney Reference No.: 7551-0010W0019. The system of claim 1, wherein the plurality of sensors comprises one or more computer numerical control (CNC) probes, with each of the CNC probes placed proximate to at least one of the one or more simulation parts in the simulation structure.

10. The system of claim 9, wherein the each of CNC probes is placed directly underneath a simulation tooth structure.

11. The system of claim 9, wherein at least some of the one or more CNC probes is configured to detect linear and angular vibrations.

12. The system of claim 1, wherein the plurality of sensors comprises:one or more simulation part selection items, actuatable by the user, that are respectively associated with individual ones from the one or more simulation parts, wherein the one or more simulation part selection items are selectable by the user to indicate an intended simulation part on which the user is to perform the simulated medical or dental procedure; andone or more vibration sensors to detect vibrations caused by performance of the simulated medical or dental procedure.

13. The system of claim 12, wherein each of the one or more simulation part selection items comprises one or more of: a physical actuatable switch, or a graphical selection item.

14. The system of claim 1, wherein the plurality of sensors comprises:a first set of inertial measurement units (IMUs) mounted on or within the operation tool manipulated by the user, and a second set of IMU’s disposed proximate to at least some of the one or more simulation parts in the simulation structure.

15. The system of claim 14, wherein the IMU’s of the first set are aligned with a tip of the operation tool, and configured to measure displacement of the tool, wherein resultant measured displacement data is used to derive, by the computing device, a vector movement representative of the displacement of the tool.Attorney Reference No.: 7551-0010W00116. The system of claim 14, wherein the IMU’s of the second set are configured to measure movement and orientation data representative of movement and orientation of at least some of the one or more simulation parts in the simulation structure, wherein the movement and orientation data is configured to be communicated to the computing device to adjust a 3D model representative of a state of the simulation structure.

17. The system of claim 14, wherein the computing device configured to determine the dynamic physical state of at least some of the one or more simulation parts and the operation tool is configured to implement a physics engine modeling physical properties of the one or more simulation parts in the simulation structure and of the operation tool, wherein the physics engine is configured to analyze interactions between the operation tool and at least one of the one or more simulation parts in the procedure area, based on the modeled physical properties and measurement data detected by the first and second sets of IMU’s, to represent the effect of the interactions on a 3D model of the one or more simulation parts and of the operation tool.

18. The system of claim 17, wherein the physics engine is configured to continually adjust the 3D model of the one or more simulation parts and the operation tool in response to receipt of updated data detected by the first and second sets of IMU’s.

19. The system of claim 17, wherein the physics engine is further configured to model tool-to-tooth interactions caused by application of force by the user to the at least one simulation part via the operation tool, and to derive, based on data collected by the first and second sets of IMU’s and analyzed using the tool-to-tooth model, updates to a state of the 3D model representation of the one or more simulation parts.

20. The system of claim 1, wherein the plurality of sensors comprises: multiple optical sensors configured to capture positioning information associated with one or more trackable objects disposed on at least the operation tool, wherein the computing device is further configured to track, based on the captured positioning information, spatial location and motion of the operation tool.Attorney Reference No.: 7551-0010W00121. The system of claim 20, wherein the one or more trackable object comprise one or more infrared (IR) LEDs each illuminating cycles of IR emissions at a respective predetermined cycle frequency, wherein the multiple optical sensors comprise multiple macrolens infrared (IR) cameras configured to each capture at least some of the IR emissions from the one or more IR LEDs, and wherein the computing device is configured to derive the spatial location and motion of the operation tool according to relative positioning and strength of captured IR signals from the IR emissions illuminated by the one or more IR LEDs.

22. The system of claim 20, wherein the one or more trackable object comprise one or more visible objects, wherein the multiple optical sensors comprise multiple visible range cameras, and wherein the computing device is configured to:identify the one or more visible objects appearing in image data captured by each of the multiple visible-range cameras, andderive the spatial location and motion of the operation tool and / or a body part of the user according to relative locations of the one or more visible objects appearing in the captured image data from the multiple visible range cameras.

23. The system of claim 1, wherein the plurality of sensors comprises: multiple vibration sensors, each embedded at respective locations in the simulation structure and configured to detect vibration caused through contact of the operation tool with one of the one or more simulation parts of the simulation structure, wherein the computing device is further configured to derive contact location of the operation tool based on amplitude, timing, frequency, and / or peak offset characteristics of vibrations signals measured by at least some of the multiple vibration sensors.

24. The system of claim 23, wherein the multiple vibration sensors comprise multiple piezoelectric sensors embedded in a palette area of a dental simulation structure, and wherein the computing device configured to derive contact location of the operation tool is configured to derive the contact location based on one or more of: a multilateration or triangulation procedure applied to the vibration signals measured by the multiple piezoelectric sensors, or a machine learning (ML) vibration-to-location model that generates an output labelAttorney Reference No.: 7551-0010W001representative of the contact locations according to the measured vibration signals inputted to an ML system implementing the ML vibration-to-location model.

25. The system of claim 1, wherein the plurality of sensors comprises: multiple pressure sensors, each embedded at respective locations in the simulation structure and configured to detect forces applied to the simulation structure through contact between the operation tool and the one or more of the simulation parts.

26. The system of claim 25, wherein the multiple pressure sensors include multiple load cells embedded in a jaw and / or skull parts of a head part of a dental simulator, each of the multiple load cells configured to gauge a level of force measured at the respective one of the each of the multiple load cells, and wherein the computing device is further configured to process the force levels measured by the multiple load cells, and to generate feedback data, provided to a user-interface, representative of performance of the dental procedure and an associated comfort level that a live patient would feel for similar measured force levels.

27. The system of claim 1, wherein the computing device is further configured to perform one or more of:track timing information associated with the operation procedure;derive, based on the determined dynamic physical state, updates for a 3D model of the simulation structure resulting from the application of forces corresponding to interactions between the operation tool and the at least some of the one or more simulation parts; or render a graphical 3D representation of the 3D model on a display device.

28. The system of claim 1, wherein the computing device configured to determine procedure performance feedback data is configured to:evaluate, using a machine learning system implementing a performance evaluation model, based on tracking data of motion of the operation tool and limbs of the user, performance of the user, wherein the performance evaluation model is optimized based on training data comprising training samples with training motion tracking data for a plurality of previously evaluated practitioners, and respective output labels representing the evaluation of the performance of the plurality of previously evaluated practitioners upon performing procedures resulting in the respective training motion tracking data.Attorney Reference No.: 7551-0010W00129. A method for simulating a medical or dental procedure, the method comprising: tracking mechanical displacements with a plurality of sensors distributed in a procedure area of a simulation structure comprising one or more simulation parts emulating anatomical parts, wherein the mechanical displacements are caused through application of an operation tool handled by a user to at least one simulation part from the one or more simulation parts in the procedure area in which the user is performing the medical or dental procedure;determining, at a computing device, based on the tracked mechanical displacements, a dynamic physical state of at least some of the one or more simulation parts and the operation tool, resulting from performance of the medical or dental procedure in the procedure area; anddetermining at the computing device, based at least in part on the determined dynamic physical state, procedure performance feedback data representative of the procedure’s performance by the user.

30. The method of claim 29, wherein determining the physical state comprises one or more of:i) tracking linear and angular movements of the at least some of the one or more simulation parts, the operation tool, a body part of the user, and / or the simulation structure; or ii) determining forces applied on the at least some of the one or more simulation parts and / or the operation tool resulting from interaction of the operation tool with the at least one simulation part.

31. The method of claim 29, wherein the simulation structure comprises a manikin with a physical model of a head part having a mouth, wherein the one or more simulation parts comprises simulation teeth constituting a training simulation denture set, and wherein the operation tool includes a dental tool handled by the user to perform a simulated dental procedure on at least one tooth from the simulation teeth.

32. The method of claim 31, wherein for at least one tooth from the simulation teeth, the physical model of the head part further comprises respective one or more infrared (IR)Attorney Reference No.: 7551-0010W001LEDs and respective one or more IR sensors embedded underneath the at least one tooth, and wherein the method further comprises:controllably causing emissions of the one or more IR LEDs into a structure of the at least one tooth during performance of the dental procedure; anddetermining changes to the structure of the at least one tooth, caused by the dental procedure, based on changes of characteristics of reflected IR signals from surfaces of the internal structure of the at least one tooth, resulting from the changes to the structure of the at least one tooth.

33. The method of claim 29, wherein the plurality of sensors comprises a first set of inertial measurement units (IMUs) mounted on or within the operation tool manipulated by the user, and a second set of IMU’s disposed proximate to at least some of the one or more simulation parts in the simulation structure.

34. The method of claim 33, further comprising:measuring, by the IMU’s of the second set, movement and orientation data representative of movement and orientation of at least some of the one or more simulation parts in the simulation structure, wherein the movement and orientation data is configured to be communicated to the computing device to adjust a 3D model representative of a state of the simulation structure.

35. The method of claim 33, wherein determining the dynamic physical state comprises:implementing a physics engine modeling physical properties of the one or more simulation parts in the simulation structure and of the operation tool, wherein the physics engine is configured to analyze interactions between the operation tool and at least one of the one or more simulation parts in the procedure area, based on the modeled physical properties and measurement data detected by the first and second sets of IMU’s, to represent the effect of the interactions on a 3D model of the one or more simulation parts and of the operation tool.

36. The method of claim 29, wherein the plurality of sensors comprises multiple vibration sensors, each embedded at respective locations in the simulation structure andAttorney Reference No.: 7551-0010W001configured to detect vibration caused through contact of the operation tool with one of the one or more simulation parts of the simulation structure, and wherein the method further comprises:deriving contact location of the operation tool based on amplitude, timing, frequency, and / or peak offset characteristics of vibrations signals measured by at least some of the multiple vibration sensors.

37. The method of claim 36, wherein deriving contact location of the operation tool comprises:deriving the contact location based on one or more of: a multilateration or triangulation procedure applied to the vibration signals measured by the multiple piezoelectric sensors, or a machine learning (ML) vibration-to-location model that generates an output label representative of the contact locations according to the measured vibration signals inputted to an ML system implementing the ML vibration-to-location model.

38. The method of claim 29, wherein determining the procedure performance feedback data comprises:evaluating, using a machine learning system implementing a performance evaluation model, based on tracking data of motion of the operation tool and limbs of the user, performance of the user, wherein the performance evaluation model is optimized based on training data comprising training samples with training motion tracking data for a plurality of previously evaluated practitioners, and respective output labels representing the evaluation of the performance of the plurality of previously evaluated practitioners upon performing procedures resulting in the respective training motion tracking data.

39. The method of claim 29, wherein the plurality of sensors comprises multiple optical sensors configured to capture positioning information associated with one or more trackable objects disposed on at least the operation tool;wherein the method further comprises:tracking, based on the captured positioning information, spatial location and motion of the operation tool.Attorney Reference No.: 7551-0010W00140. The method of claim 39, wherein the one or more trackable object comprise one or more infrared (IR) LEDs each illuminating cycles of IR emissions at a respective predetermined cycle frequency, wherein the multiple optical sensors comprise multiple macrolens infrared (IR) cameras configured to each capture at least some of the IR emissions from the one or more IR LEDs, and wherein the method further comprises:deriving the spatial location and motion of the operation tool according to relative positioning and strength of captured IR signals from the IR emissions illuminated by the one or more IR LEDs.

Images