Surgical simulator systems and methods

Abstract

A surgical simulator includes a haptic arm capable of simulating forces generated during a surgery from an interaction between a surgical tool and operated tissue. The simulator also includes a visual display capable of displaying a three-dimensional image simulating a physics-based computer model of the surgical tool and tissue. The haptic arm controls the movement and orientation of the simulation tool in the three-dimensional image and provides haptic feedback forces to simulate forces experienced during surgery. Methods for simulating surgery and training a user of the simulator are also described.

Description

Technical Field

[0001] This invention relates to the field of surgical simulation, and particularly to surgical simulator systems and surgical methods with tactile force feedback. Background Technology

[0002] In many parts of the world, people suffer from blindness. The effects of blindness are particularly devastating compared to other types of disability, and can be economically devastating. Blind people are unable to work, care for themselves, or care for others. In most cases, they lack access to adequate medical care. In many cases, vision loss due to cataracts occurs gradually over many years, leading to a reduced quality of life, decreased household disposable income, and increased dependence on family caregivers (often for school-aged children). Overall health is affected because of the increased risk of injury, the inability to see injuries to properly care for them (such as cuts and abrasions that can become infected), and a reduced ability to maintain adequate nutrition. Many blind people die from blindness.

[0003] Millions of people are blind, as blindness is often caused by cataracts, which can be surgically removed from the eye to restore vision. Tragically, in many parts of the world, there are simply no resources to provide this treatment. In developing countries, resources for the blind are scarce or nonexistent, often lacking white cane policies (to provide awareness and safety for the blind) and other disability legislation, leaving families to bear the burden alone. As a result, cataract blindness is also associated with extreme poverty and an increased risk of death.

[0004] The prevalence of cataracts also makes blindness a anticipated disability in developing countries, particularly among the poor. It should be understood as part of the aging process and accepted. Information available about cataracts and cataract surgery is often lacking or inaccurate. Patients going to traditional therapists or untrained surgeons often receive misinformation and poor treatment. Surgical treatment is frightening, and many would rather remain blind than undergo surgery. As a result, blind patients in developing countries often believe their only option is to live with blindness, yearning to escape the dark world they are forced to live in, yet feeling powerless to change it.

[0005] Lenticule extraction (PE) cataract surgery and conventional extracapsular cataract extraction (ECCE) and its variations are two commonly used major cataract surgery techniques. While PE cataract surgery is considered the gold standard for cataract removal, it requires expensive equipment and uninterrupted power. The overall cost and maintenance of PE equipment and supplies make it prohibitively expensive for areas with insufficient infrastructure. PE also has a higher rate of intraoperative complications when performed on patients with advanced cataracts.

[0006] A popular variation of ECCE surgery that can be performed without sutures is called Mini-Incision Cataract Surgery (MSICS). The unique advantage of MSICS over ECCE is the smaller incision used to remove the cataract's natural lens and implant an intraocular lens (IOL). The smaller incision also creates a self-closing, sutureless procedure. This translates to shorter healing time, significantly less astigmatism, reduced intraoperative risks, and a shorter overall postoperative recovery. Apart from the surgical microscope, neither ECCE nor MSICS relies on any electric machinery.

[0007] MSICS and ECCE are widely practiced outside of North America and Europe. When comparing different surgical techniques used worldwide, MSICS is often the preferred technique for large-volume cataract surgery. MSICS can be used to safely and cost-effectively restore vision in developing countries where most blind people live, with the same quality expected in the high-tech world. While there are many technical variations, the basic idea of ​​MSICS revolves around properly creating and utilizing a channel that is large enough to deliver dense cataracts, yet stable enough to self-close and have minimal impact on the curvature of the cornea.

[0008] While MSICS is an effective alternative for addressing cataract blindness in developing countries, there is a shortage of skilled ophthalmologists. For example, in some sub-Saharan African countries, there is only one ophthalmologist per million people on average. To address the global burden of cataract blindness, there is an urgent need to train a large number of surgeons in MSICS techniques. Today, even in developing countries, surgeons can provide PE as a care procedure for eligible patients. By increasing the number and skill level of surgeons capable of performing cataract surgery, the global rate of cataract blindness can be significantly reduced.

[0009] When surgical errors occur or patients with complex conditions are treated, it is also crucial for surgeons to learn complication management. For example, early care for patients with ocular trauma presenting in the emergency room can determine their visual recovery outcome. Surgeons need to be well-trained to handle such situations, which, although infrequent, can have a significant impact on results.

[0010] High-quality, efficient simulation-based training can meet a large portion of training needs without posing risks to patients. Therefore, there is a need for a surgical simulator system and method that allows users to master manual small incision cataract surgery (MSICS), phacoemulsification (PE), cataract surgery complication management, and ocular trauma management. Summary of the Invention

[0011] The purpose of this invention is to provide a surgical simulator system and method that provides users with visual, tactile, and audio cues to simulate eye surgery.

[0012] Another object of the present invention is to provide a simulation system and method that mimics tissue, the use of surgical instruments, and the interaction between the surgical instruments and the simulated tissue, and allows users to practice and become proficient in surgery.

[0013] Another object of the present invention is to provide a simulation system and method that progressively incorporates a comprehensive array of real patient factors and surgical complications, thereby allowing trainees to experience the realistic sensations of surgery, including the many variables and errors that may occur.

[0014] Another object of the present invention is to provide a simulation system and method that allows for the monitoring and evaluation of trainees’ performance for feedback, corrective teaching, scoring and development.

[0015] Another object of the present invention is to provide a simulation system and method that uses a mesh model to construct and visually represent the model using rasterization or ray tracing, while running a physical model that affects properties such as collision detection and tissue deformation in parallel. Attached Figure Description

[0016] A further understanding of the invention can be obtained by referring to the embodiments set forth in the accompanying drawings. Although the illustrated embodiments are merely examples of systems, methods, and apparatus for carrying out the invention, in general, the organization and operation of the invention, together with its further objectives and advantages, can be more readily understood by referring to the accompanying drawings and the following description. Similar reference numerals generally refer to similar features (e.g., elements with similar functions and / or structures).

[0017] The accompanying drawings are not necessarily drawn to scale; in some cases, various aspects of the protected subject matter disclosed herein may be exaggerated or enlarged in the drawings to aid in understanding different features. Furthermore, the drawings are not intended to limit the scope of the invention, which is specifically set forth in the appended claims or subsequent modifications, and are merely for illustrative and explanatory purposes.

[0018] Figure 1a This is a schematic diagram of a simulated surgical environment according to the present invention.

[0019] Figures 1b-1c An exemplary simulated microscope according to the present invention is depicted.

[0020] Figures 1d-1e Alternative embodiments of the simulated surgical environment according to the present invention are described.

[0021] Figure 2 It is a diagram depicting the interaction between the simulated surgical environment and computer components according to the present invention.

[0022] Figure 3a This is a diagram illustrating the hardware components of the simulator according to the present invention.

[0023] Figure 3b This is a diagram illustrating the software components of the simulator according to the present invention.

[0024] Figure 3c This is a diagram illustrating the computer components of the simulator according to the present invention.

[0025] Figure 3d This is a diagram illustrating an exemplary connection between the simulator, central server, and instructor client according to the present invention.

[0026] Figures 4a to 4f A simulated physical eye model according to the present invention is described.

[0027] Figures 5a to 5c A simulated image visible through a simulated microscope according to the invention is depicted.

[0028] Figure 6a This is a table of exemplary tools that can be simulated by the tactile right arm according to the present invention.

[0029] Figure 6b This is a table of exemplary tools that can be simulated by the tactile left arm according to the present invention.

[0030] Figure 7a and Figure 7b The steps for performing MSIC surgery and the tools that can be simulated by the haptic arm according to the invention are described.

[0031] Figure 8 An exemplary simulator user interface according to the present invention is depicted.

[0032] Figure 9 An exemplary simulator user interface according to the present invention is depicted.

[0033] Figure 10 An exemplary foot pedal unit according to the present invention is depicted. Detailed Implementation

[0034] The invention can be more readily understood by referring to the following detailed description of embodiments thereof. However, the technology, systems, and operational structures according to the invention can be embodied in various forms and modes, some of which may differ significantly from those in the disclosed embodiments. Furthermore, unless explicitly stated otherwise, the features and elements disclosed herein can be combined to form various combinations without exclusivity. Therefore, the specific structural and functional details disclosed herein are merely representative. However, in this regard, they are considered to provide the best embodiments for the purposes of disclosure and to provide the basis for the claims that define the scope of the invention herein. It should be noted that, as used in the specification and appended claims, the singular forms “a,” “an,” and “the” include plural indicators unless the context clearly indicates otherwise.

[0035] When a numerical range is provided, it should be understood that, unless the context explicitly states otherwise, each interval value between the upper and lower limits of the range, up to one-tenth of the lower limit unit, is also specifically disclosed. Every smaller range between any specified value or interval value within the specified range and any other specified value or interval value within that specified range is included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range, and each range in which neither upper nor lower limit is included, or both are included, is also included in this invention, but is subject to any explicitly excluded limit value within the specified range. Where a specified range includes one or two limit values, ranges excluding one or both of those included limit values ​​are also included in this invention.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. While any methods and materials similar to or equivalent to those described herein may be used in the practice or testing of this invention, some potential and preferred methods and materials are described hereafter. All publications mentioned herein are incorporated by reference to disclose and describe methods and / or materials associated with the publication to which they are cited. It is understood that, in the event of any conflict, this disclosure shall supersede any disclosure of the incorporated publications.

[0037] This invention is described in the context of ocular surgeries such as MSICS or PE. However, the invention can be used to simulate other types of surgical procedures performed on the human eye or other parts of the human body. The invention can also be used to simulate veterinary surgeries. The invention can also be used to evaluate the skills required to perform routine or specific types of surgeries; for example, it can be used by medical students considering a surgical career (including careers in surgical specialties) or by surgeons practicing shortly before performing surgery.

[0038] This invention can stem the global surge in cataract blindness by significantly increasing the number of highly skilled ophthalmic surgeons. The surgical simulator of this invention can be used to provide large-scale MSICS training. The training provided is comprehensive, trainee advancement is performance-based, and successful completion of training will allow trainees to become part of a global network of cataract surgeons. Training can be geared towards individuals with no prior knowledge or experience in eye care. The success of the training program is based on performance where each trainee demonstrates the necessary language, intelligence, motor skills, and depth perception required to master the surgical procedures performed. Data collected during recruitment can be compared with performance during training to refine recruitment tests and establish standardized criteria.

[0039] Using the surgical simulator system and method of this invention, users can systematically and efficiently learn through a series of surgical challenges and variations in a relatively short time without endangering the patient. Upon successful completion of simulator-based training, trainees can return to their home country's surgical facilities to begin on-site surgical training under the supervision of a supervising surgeon. On-site surgical training typically takes several years to complete and often leaves gaps in experience. However, due to the advanced simulation experience, users of this invention can rapidly develop into independent, high-quality practitioners. The transition to independent surgical care is expected to take approximately 1-6 months.

[0040] The surgical simulator and method of this invention are progressively introduced by simulating a comprehensive array of realistic patient factors and surgical complications associated with each step, allowing trainees to experience the true feel of surgery, including the many variables, complications, and errors that may occur during live surgery. Simulator-assisted training provides a safe and effective way to prepare for live surgical experience. The training experience presents various surgical scenarios to develop confidence and surgical intuition for live surgery without risk to the patient. Trainee performance on the simulator can be monitored and evaluated for feedback, corrective teaching, scoring, and development. Each trainee can practice and learn until the desired level of proficiency is demonstrated on the simulator.

[0041] For MSICS or PE surgery to be successful, the following five tasks must be completed sequentially during the simulation: (1) complete patient preparation, (2) create a self-sealing channel to the eye, (3) remove the cataract, (4) insert an artificial lens (“IOL”), and (5) restore conditions for optimizing the healing process.

[0042] An ocular surgery simulator comprises hardware and software to visually and tactilely represent the eye. The simulator provides the user with visual, tactile, and audio cues to simulate realistic MSICS / EP / complication management / ocular trauma surgery. Parameters based on tactile and visual presentations can be derived from on-site surgical force data. Alternatively, the ocular surgery model presented in the simulator can be based on subjective assessments by a professional surgeon and / or objective validation of the model. Tactile presentation can provide the trainee with forces on the instrument that are nearly identical to those experienced during on-site surgery. Visual presentation can reproduce images in realistic quality on a non-fantasy stereoscopic display to mimic what a surgeon would see in a binocular surgical microscope during on-site surgery. Modeling can include: (1) tool-tissue interactions, (2) tissue-tissue interactions, (3) connections between tissues, (4) tissue anatomy, (5) changes in tissue properties, (6) intraocular pressure, (7) fluid injection and aspiration, (8) fluid diffusion, (9) sound playback in response to events, and (10) model patient head movement.

[0043] An eye surgery simulator may include the following simulation elements: (a) a simulator with a haptic arm, (b) a physics-based computer model, and (c) a visual image generator. An eye surgery simulator may also include an instructor / student operator station.

[0044] Simulator with haptic arm refer to Figure 1a and Figure 2 The simulator (101) may include a bed (103) from which a simulated model (105) of the patient's head extends. The simulator (101) may also include a simulated microscope (107) through which the user can perceive the simulation. A tactile right arm (109) and a tactile left arm (111) may be connected and controlled by a right tactile mechanism (113) and a left tactile mechanism (115), respectively. The simulator (101) also includes a touchscreen (117) and a simulated suction syringe (119).

[0045] Figures 1d-1eAnother compact version of the simulator (401) is depicted. The simulator (401) may include a truncated bed frame (403) from which a simulated patient's head (405) extends. The simulator (403) may also include a simulated microscope (407), left and right tactile arms (409, 411), and left and right tactile mechanisms (413, 415) that can be connected to and control the tactile arms (409, 411). The simulator (401) may also include one or more legs (425) for positioning the simulator (401) on a horizontal surface, such as a tabletop (not shown). The legs (425) may be extended and / or retracted to ensure that the simulator (401) is flush with the ground and / or to position the simulator at an ergonomic height for the surgeon using the simulator. A desktop simulator may be placed on a height-adjustable platform to achieve similar functionality.

[0046] Surgeons' hands and fingers are too large to directly manipulate the small and delicate tissues of the eye. Therefore, surgeons use certain instruments or tools to perform ophthalmic surgery. In this invention, those instruments are simulated by a tactile right arm (109) and a tactile left arm (111). The tactile arms (109, 111) are provided in the work area and held in the user's right and left hands to simulate surgery. For example, the tactile arms (109, 111) are used to perform actions on a virtual eye while viewing through the display of a simulated microscope (107). The user can select instruments for simulation and move the instruments into the simulator's operating area under the simulated microscope (107). The user can use these instruments to simulate various surgical tasks.

[0047] The simulator (101) may also include a camera with a field of view that includes the operating area. Figures 1b-1d Alternative embodiments of a simulated microscope (207) are depicted, which may have a camera (217) facing downward toward the operating area. The camera (217) may capture still images (including video) of the tactile arms (109, 111) and / or the user's hand as the user positions and manipulates the tactile right arm (109) and tactile left arm (111) during simulated surgery. Additional applications of these systems may provide features for remote training.

[0048] When working on the virtual eye model, haptic arms (109, 111) provide tactile realism in the form of force feedback. The haptic arms (109, 111) are motion control devices that provide a realistic sense of force to the user's fingers, hands, and arms used to grip the instrument as it interacts with the virtual eye. The virtual eye is programmed to accurately simulate the reactions or behaviors caused by the interaction between the eye and the selected instrument. For example, to simulate pressing on the eye and increasing pressure in the eye by pressing it with Colibri clamps, one of the haptic arms (109, 111) can simulate Colibri clamps, which can be used to restrict eye movement in the display of a simulated microscope (107). Resistance from the corresponding haptic mechanism (113, 115) can be increased to simulate the increase in eye stiffness felt by the Colibri clamps or another tool (as said tool collides with the eye). Similarly, simulating interaction with a crescent blade results in cutting the eye tissue of the virtual eye according to the blade's edge, angle, force, and movement characteristics. The simulator can also simulate the interaction between two or more tools.

[0049] The tactile arms (109, 111) provide simultaneous and bi-handed operation to represent tools that can be used in actual surgery. Preferably, the tactile arms (109, 111) are representative of actual surgical instruments. In one embodiment, the tactile arms (109, 111) allow for alteration of the handle representing an actual surgical instrument. In another embodiment, the tactile arms (109, 111) include a permanently mounted handle that is representative of an actual surgical instrument but may not be an exact replica. In either case, the instrument seen under the simulated microscope (107) can vary depending on the type of instrument.

[0050] The simulator can also simulate other tools, such as the Simcoe syringe. Figures 6a to 6b A list of exemplary tools that can be simulated using a simulator is provided. Each tool can have three translational degrees of freedom. These translations focus on the tactile point at the tip of the tool. Three passive rotational degrees of freedom are also provided and measured. Their rotations are centered on the same point of interest. Tactile sensation is based on the use of admittance control, utilizing a force sensor as input.

[0051] Surgical simulators may also include foot pedal units that can be used to perform various functions. The foot pedal units can be connected to the surgical simulator's computer hardware via wired or wireless connections. Figure 10An exemplary foot pedal unit (1100) is shown. The foot pedal unit (1100) may include a foot pedal (1200) having an upper surface (1220) adapted for contact with a user's foot. The foot pedal unit (1100) may also include one or more switches (1240, 1250, 1260, 1270). Figure 10 As shown, each switch (1240, 1250, 1260, 1270) can be located on the side of the foot pedal (1200). The switches (1240, 1250, 1260, 1270) can also be adapted for user foot contact. The functions performed by the foot pedal (1200) and switches (1240, 1250, 1260, 1270) can be preset or programmed by, for example, the user. Depressing the foot pedal (1200) and / or the switches (1240, 1250, 1260, 1270) causes the foot pedal (1200) and / or the switches (1240, 1250, 1260, 1270) to move from a first position to a second position; for example, this can switch one of the tools depicted in a surgical simulation from a first tool (e.g., a scalpel) to a second tool (e.g., forceps). For example, pressing the foot pedal (1200) and / or the switch (1240, 1250, 1260, 1270) can induce a simulation of liquid injection or aspiration. Further, pressing the foot pedal (1200) and / or the switch (1240, 1250, 1260, 1270) can adjust the physical configuration of the surgical simulator, such as raising or lowering the simulator's height above the ground, raising or lowering the height of the simulated microscope, changing the level of simulated ultrasonic energy transmitted by virtual tools within the simulator, and / or controlling the level of simulated fluid pressure for flushing or vacuum pressure for aspiration. The foot pedal can also be programmed to provide options for proceeding to the next step of the simulated surgical procedure. Additionally, a second or third foot pedal can be added to perform one or more of the above functions.

[0052] As mentioned above, tactile presentation provides the user with forces on the instrument that are almost identical to those experienced during actual surgery. Table 1 below lists the surgical steps that can be simulated by the simulator, the instruments that can be simulated for each step, guidance on instrument movement during surgery, and the maximum and minimum forces for each surgical step.

[0053] Table 1

[0054] In Table 1, all forces are expressed in grams (g), and the direction of movement is indicated as follows: H - horizontal, AP - forward-backward, V - vertical. Fx represents the force in the x-plane specifying movement from left to right, Fy represents the force in the y-plane specifying movement up and down, and Fz represents the force in the z-plane specifying movement in and out. The primary force during puncture is Fz inward, while during the "cutting" action using the crescent blade, the Fx (right or left) force controls the movement.

[0055] The force setting in the simulator reproduces the level of force, giving the operator a realistic feel for the process. The minimum force establishes an upper limit on electronic and frictional noise in the robotic mechanism; exceeding this limit, the surgeon can no longer properly experience surgical forces. The maximum force sets the standard for the motion and stiffness of the robotic mechanism. Force curve characteristics provide a target baseline for testing the realism of interactions between living tissue and the simulator.

[0056] The most critical step in MSICS surgery is creating the scleral canal. It is also the most difficult step to learn. During the back-and-forth "swinging" motion of the crescent blade in creating the scleral canal pocket, the average force (Fx) is 31.9 g, with a maximum of 48.8 g and a minimum of 21.7 g. In the same motion, the surgeon also moves upward and inward along the contour of the sclera. During the scleral canal pocket step, the average "swinging" force (Fy) of the crescent blade (which is the upward force) is 47.0 g (max 63.6 g, min 35.4 g). In the same step, the average force (Fz) is 45.6 g (max 62.5 g, min 24.4 g). All three degrees of freedom exhibit significant forces in this movement. It is important for the surgeon to recognize that when the blade swings left and right, it also propels inward and upward along the contour of the sclera, all with similar average forces.

[0057] The highest force encountered during the simulation was when making a crescent-shaped cut to the right or left during the scleral canal extension step. The average Fx of the crescent-shaped cut during the right or left "cutting" action in the scleral canal extension step was 91.3 g (max 115.8 g, min 61.9 g). The ophthalmic surgery simulator reproduced forces ranging from 0 to a total of at least 115.8 g. The y-force value for this step varied depending on whether the surgeon was cutting to the right or left. When cutting to the right, Fy was directed downwards because left-handed surgeons followed the contour of the sphere. When cutting to the left, Fy was directed upwards because right-handed surgeons extended the canal to the left while simultaneously grasping the outer canal with Colibri forceps. Regardless of the cutting direction, the z-force encountered during the crescent-shaped cut was directed outwards. When cutting to the right, the downward-directed force Fy had an average value of 31.3 g (max 50.2 g, min 12.0 g). When cutting to the left, Fy is an upward force with an average value of 60.8 g (max 87.1 g, min 41.0 g). During the crescent-shaped "cutting" motion, Fz (Fz max) is an outward force with an average value of 66.2 g (max 95.4 g, min 43.5 g).

[0058] The puncture incision force during corneal entry is primarily a z-force (i.e., the puncture site). During the formation of the puncture incision or puncture site, the average force is 23.4 g (max 55.6 g, min 13.3 g) applied inwards using a 15-degree puncture blade in the Fz direction.

[0059] Although it is often taught to "float" in the center of the channel to cut the keratomy knife to the right or left, novice surgeons will recognize that significant Fx forces can still be encountered, up to an average of 55.6 g, especially in the distal range of motion. When using a 3.0 mm keratomy knife to enter the anterior chamber (i.e., through the main incision AC), the Fz max (inward) averages 22.9 g (max 52.0 g, min 2.1 g). When "cutting" with the keratomy knife to open the internal wound, the Fx (right or left) averages 55.6 g (max 82.2 g, min 35.4 g).

[0060] The capsulorhexis provides canister-opening force in the y and z degrees of freedom. No significant force is applied during each cut of the anterior capsule. The actual cutting stroke of the capsulorhexis has minimal force—the surgeon cannot feel the capsulorhexis cutting the anterior capsule. However, significant repositioning forces (Fy upward and Fz outward) are provided immediately after each cutting stroke. Fy, the upward force, averages 22.5 g (max 42.0 g, min 10.9 g) for the capsulorhexis step, while Fz, the outward force, averages 23.6 g (max 48.0 g, min 13.6 g).

[0061] The average force (Fz, outward direction) during lever compression of the lens is 35.1 g (max 66.3 g, min 7.8 g). The Sinskey force is minimal when the IOL is manipulated, emphasizing the minimum force required for proper IOL manipulation under viscoelastic control. When the IOL is manipulated with the Sinskey hook at the 9 o'clock position, the maximum force in any degree of freedom is negligible, averaging 4.7 g (max 7.4 g, min 1.6 g). The basic model simulating the eye can be further adjusted based on surgeon feedback and / or can be independent of force measurements taken during on-site surgery.

[0062] Physics-based computer models Figure 2 A diagram illustrating an exemplary data flow between simulator components during simulation is provided. A physical modeling application (202) can model the tissue and fluid properties of the simulated eye, as well as the interaction between the tool and the eye, including the forces exerted by the tool on the eye. Information about the visual appearance of the eye and tool can be processed by a visual image processing application (204) and delivered to, for example, a graphics card (206) for 3D model rendering. A 3D image of the eye is transmitted to a simulated microscope (107) and can be viewed by the user during the simulation.

[0063] The tactile control unit (208) receives simulation modeling information from the physical modeling application. Regarding the position and orientation of the tactile arms (109, 111), the tactile control unit (208) further receives surgical input information (216) from the tactile arms (109, 111). The tactile control unit (208) controls the amount of force and resistance applied by the tactile mechanisms (113, 115) to the tactile arms (109, 111).

[0064] The simulator may also include a simulator interface application (210) for managing the simulation. The simulator interface application (210) may allow the instructor to assign surgical plans to the user and / or monitor, evaluate, provide feedback and report on the user's simulator operation.

[0065] Figure 3aA diagram illustrating the interrelationships of the simulator's hardware components. The haptic component (2.1) may include a gimbal mechanism (2.1.1.1, 2.1.2.1), motors and actuators (2.1.1.2, 2.1.2.2), and a head interface (2.1.1.3, 2.1.2.3). The 3-D visual display (2.2) of the simulated microscope (107) may include one or more LCDs (2.2.1), optics (2.2.2), and a microscope housing (2.2.3). Figures 1b-1d A microscope housing (223) is depicted. A camera (217) may be located at the bottom (224) of the housing (223). The camera (217) may be positioned such that its field of view includes at least a portion of the tactile arms (109, 111) and / or at least a portion of the user's hand when the simulated surgery is performed.

[0066] The computer hardware (2.3) used by the simulator, sending information to the simulator, or receiving information from the simulator may include a real-time PC (2.3.1), a graphics PC (2.3.2), a tablet PC (2.3.3), a database server (2.3.4), and / or a teaching station (2.3.5). The computer hardware (2.3) may be used to record images (including video) captured by a camera (217). Supplementally or alternatively, the images captured by the camera (217) may be displayed on a screen, such as on the screen of a teaching station (2.3.5), during and / or after the simulated surgery is performed by the user.

[0067] A simulated head (2.4.1) shown in Figure 1 by reference element (105) and a stretcher / patient bed (2.4.2) shown in Figure 1 by reference element (103) may also be provided.

[0068] Visual image generator Simulators can represent visual graphical models of body parts suitable for surgery, such as the eye, to simulate real MSIC surgery. The physically-based model of the eye can be programmed to simulate the eye's behavior in response to surgical actions. The visual 3D eye model can change in real time based on these actions to provide an experience of working with a real eye. The virtual eye can have customizable parameters that allow not only changes to how the eye appears (e.g., the color of the iris, skin, and sclera) but also changes to other anatomical and technical parameters such as the shape of the eye and / or the type of cataract, allowing users to practice on a wide range of patient conditions that surgeons might encounter.

[0069] Figures 4a to 4f Describe the physical model. For example... Figures 4a to 4fAs shown, mesh models can be used to construct and display vision using rasterization, while simultaneously running physical models that affect properties such as collision detection and tissue deformation. Eye models can include all structures involved in eye surgery. Eye models can include high detail of the cornea and limbus to achieve a realistic image from their surfaces. Figure 4d The eye is shown in outline from one side, which can cover the height details of the corneal limbus. Figure 4e It shows the main channel through which it enters the eye, while Figure 4f Explain the main cut lines.

[0070] The simulator contains two visual displays that the user can use to work, including (1) a microscope and (2) an external display. Figures 5a to 5c This describes simulated images observed under a microscope. The simulated objects are imagined to move and deform with physical realism. Images of the visual model are displayed to the trainee through a 3D visual display similar to a stereomicroscope. Using the microscope, the trainee sees a virtual eye model and interacts with it using haptic arms (representing the physical form of surgical instruments) to perform tasks, while simultaneously observing through the microscope eyepiece.

[0071] The visual display can show a stereoscopic 3D image of the eye as seen under a 5X magnification operating microscope, showing everything within a 34 mm–35 mm circular surgical field of view. The image of the field of view is preferably surrounded by a black ring approximately 5 mm wide, resulting in a total visual image diameter of approximately 4 cm. Additional microscope zoom settings required by the surgeon can also be provided on the simulator. Controls for changing the zoom level of the simulated eye can be driven, for example, by controls displayed on a touchpad display or by one or more foot pedals. In a preferred embodiment, rasterization can be used to render the visual effect of the eye model. In other embodiments, ray tracing, or a combination of rasterization and ray tracing, can be used.

[0072] The external display allows trainees and instructors to communicate with the simulator. Communication includes, but is not limited to, selecting surgical tasks for practice, reviewing task-related information on the screen before starting, viewing feedback content (multimedia – text, audio, video, animation, etc.), and, for example, displaying microscopic observations in parallel while performing the task. The external display may include a control panel through which learners and / or instructors practice simulation control.

[0073] Figure 8An exemplary image (800) within a microscope view according to the invention is depicted. The image (800) may include one or more visual cues and / or information related to the simulated surgery, which can help guide the trainee and instruct them to perform surgical techniques to a sufficiently proficient level. The image (800) may include a three-dimensional rendering of the eye (810) and one or more three-dimensional renderings of surgical instruments (820) controlled by haptic arms (109, 111). The image (800) may include a guide line (830) indicating the path along which an incision should be made, or the desired path for any instrument. The guide line (830) may be black, white, or a color such as yellow. The guide line (830) may be any type of line, including solid lines, dotted lines, or dashed lines.

[0074] The guide line (830) may be displayed when the surgical instrument (820) is moved by the user and / or when a portion of the surgical instrument (820) is displayed in the image (800), or when the surgical instrument (820) contacts virtual tissue. Alternatively, the guide line (830) may be displayed in the image (800) but deleted when the surgical instrument (820) is moved by the user and / or when a portion of the surgical instrument (820) is displayed in the image (800). Alternatively, the guide line (830) may be generated but not displayed. Alternatively, one or more colors of the guide line (830) may be matched to one or more colors of the eye to make it imperceptible to the user.

[0075] When the guide line (830) and one or more surgical instruments (820) are simultaneously displayed in the image (800), the guide line (830) may be displayed "above" (i.e., in front of) the surgical instruments (820), so that the surgical instruments (820) are visible regardless of their position. Alternatively, the guide line (830) may be displayed "below" (i.e., behind) the surgical instruments (820), such that when the surgical instruments (820) are above a portion of the guide line (830), that portion of the guide line is not displayed. Alternatively, as the surgical instruments (820) pass "above" the guide line (830), the color, weight, and / or other characteristics of all or one or more portions of the guide line (830) may be changed. For example, if the guide line (830) is a solid black line, the portion of the guide line (830) covered by the surgical instrument (820) can become a dashed line and / or can become gray when the surgical instrument (820) passes over the guide line (830).

[0076] As the trainee uses the displayed tools (820) to perform a virtual incision, the image (800) can display a cut line (840) indicating the cut path. The cut line (840) can be black, white, or a single color. If a guide line (830) is displayed, the color of the cut line (840) can be the same as or different from the color of the guide line (830). The cut line (840) can be displayed during the incision, immediately after the incision is completed, or at a later time, including during a review of the task with the trainer.

[0077] If the incision line (840) deviates from the guide line (830) by a predetermined acceptable distance, the system can provide feedback to the trainee that the incision is not sufficiently similar to the guide line (830) during or after the incision is performed. For example, the deviation can be measured by the distance between the nearest point on the guide line (830) and the incision line (840) performed by the surgical instrument (820) at a specific time. Different acceptable deviation values ​​can also be set for each side of the guide line (830). For example, the acceptable deviation value might be as high as one centimeter on one side of the guide line (830), but only three millimeters on the other side.

[0078] Feedback indicating to the user that the deviation exceeds an acceptable distance may include, for example, vibration of one or more tool handles, tactile force on the tool handle in the direction in which the tool needs to be moved to reduce the deviation, audible sounds, and / or visual indications on the display. If the deviation exceeds a certain threshold, the system may also simulate noise emitted by a simulated patient.

[0079] In addition, such as Figure 8 As shown, one or more measuring instruments may be displayed in the image (800). These measuring instruments may provide information about surgical performance and / or the state of the simulated eye. For example, a measuring instrument (850) may indicate the intraocular pressure (IOP) of the eye and may display changes in IOP as the simulated surgery progresses. If the trainee applies pressure to the eye with an instrument (810), the measuring instrument (850) may indicate an increase in IOP. These one or more measuring instruments may be displayed above and / or near the eye (below, above, or beside).

[0080] For example Figure 8 and Figure 9 As shown, a two-dimensional view (860) illustrating the surgical instrument (920) and the eye (910) can be provided in the image (800). The two-dimensional view (860) can be displayed in a window and / or can be displayed on top of the eye (810) (as shown). Figure 8The two-dimensional view (860) can show the angle of the surgical instrument (920) relative to the eye (910), and the relative distance between the tip of the surgical instrument (920) and the eye (910). The two-dimensional view (860) can help the trainee measure the orientation of the surgical instruments (820, 920) relative to the eye (910), and the spatial relationship between the surgical instruments (820, 920) and the eye (910).

[0081] The angle of the surgical instrument (920) can be measured, for example, as the angle between: a line tangent to the point on the surface of the eye (910) closest to the tip of the surgical instrument (920); and the angular direction of the portion of the surgical instrument (920) closest to the eye (910). Additionally or alternatively, the angle of the surgical instrument (920) can be measured relative to a reference plane generated by the system.

[0082] A two-dimensional view (860) can show the range of preferred or desired angles for the surgical instruments (820, 920). For example... Figure 9 As shown, the range of preferred or desired angles can be indicated by one or more range lines (970). The range lines (970) can be any type of line, including solid lines, dotted lines, or dashed lines. The area (980) of the two-dimensional view (860) between the range lines (970) and / or the area of ​​the two-dimensional view (860) outside the range lines (970) can be highlighted with color.

[0083] The highlighting color in and / or outside the range lines of region (980) can be applied or changed depending on whether the angle of the surgical instruments (820, 920) is within the preferred range. For example, if the angle of the surgical instruments (820, 920) is between the range lines (970), region (980) can be highlighted in green. If the angle of the surgical instruments (820, 920) deviates from the preferred range, the region (980) and / or the area outside the range lines (970) can be highlighted with a different color (e.g., red), thereby providing the trainee with an indication that the angle of the surgical instruments (820, 920) should be adjusted to the preferred range. Alternatively, if the angle of the surgical instruments (820, 920) extends beyond the preferred angle or angle range, the entire two-dimensional view (860) or another sub-section of the two-dimensional view (860) can be highlighted with a color (e.g., red).

[0084] The two-dimensional view (860) may also include markings (990) on the outer surface of the eye (910) indicating the preferred or desired location where the surgical instruments (820, 920) should contact the eye (e.g., where an incision should be made). The system can measure the number of times the surgical instruments (820, 920) are brought within a predetermined distance from the eye (810, 910) and / or the markings (990) before contact is made with the eye (810, 910). The system may require this number to be below a predetermined value for a successful surgery / procedure. For example, if a trainee brings the tip of a virtual scalpel within two centimeters of the surface of the eye (810, 910) more than three times before initiating an incision, the system may indicate that the trainee has made an error and / or that their procedure has not succeeded. The trainee may be allowed to continue trying, or may be required to start the surgery from scratch.

[0085] This system typically provides indications when a trainee deviates from preset standards or requirements, or when an error occurs. These indications can be visual, auditory, and / or tactile. For example, one or more parts of a surgical instrument (820, 920) (e.g., handles, the portion of the instrument near the tip) may change color (e.g., red, yellow). The type and / or change of indication may depend on the severity of the error.

[0086] While the above examples relate to MSICS surgery, the system according to the invention can be used to train users to perform any type of surgery. The system can also be used to quickly test a surgeon or other physician's manual dexterity and spatial or depth perception required to perform surgery, either immediately before or at any time prior to the procedure. For example, an older surgeon may not realize their dexterity has deteriorated or their vision has impaired their depth perception until after they begin surgery or the surgery fails. The system according to the invention can quickly test physicians to determine if they should not perform surgery at a particular time. For example, a physician planning to perform MSICS can perform a simulated MSICS, a simulated subset of the steps required to perform MSICS, or one or more other simulated exercises that can measure the physician's performance. For example, a physician might need to guide a simulated scalpel to a specific location on a simulated patient surface at a specific angle (or within a given range of angles). The physician might only be allowed to attempt to place the scalpel within a predetermined distance on the patient surface at that location and angle a certain number of times to pass the test. Medical students can use similar tests to consider whether to specialize in a medical field requiring high dexterity before deciding which field they intend to pursue. Similar tests can also be used for people in other fields. For example, the system could be used by people in the field of jewelry making.

[0087] Software and computer components Figure 3bThe software components of the simulator are described, including simulating (1) the channel, (2) capsulotomy (or cataract surgery) and lens replacement with intact suspensory ligaments, (3) managing anterior chamber dynamics, (4) capsulotomy (or cataract surgery) and lens replacement with incomplete suspensory ligaments, and (5) vitreous loss management. Each of those components has visual, analog, and tactile subcomponents.

[0088] Simulator software can be programmed to support the entire MSICS procedure, as well as complication management and some modules in PE, on an ocular surgery simulator. The software is built around a physics-based model of the eye, surgical instruments, and the forces experienced during actual surgery. Specifically, the software can include a graphical eye model, a tactile model, and courseware / user interface. The tactile and visual responses to interaction with the model can be virtually indistinguishable from those experienced during actual surgery. This model can reproduce precise tool-tissue contact, realistic tissue resistance, stiffness, flexibility, and texture, providing a realistic feel for operating instruments with the simulator head. The simulator provides sufficient degrees of freedom of movement to allow for accurate simulation and force feedback.

[0089] As used herein, the term "computer" can refer to a standalone computer that may contain one or more processors and may include memory, or the term "computer" can refer to one or more computer processors. Figure 3c As shown, the simulator consists of multiple computers, including a simulator computer, a courseware or "Simulator Interface Application" (SIA) computer, and an application programming interface (API). The simulator computer or "graphics and / or real-time PC" stores and runs simulator software (3-D eye and physical model) and related code. The simulator computer (300) includes components of the eye model (302), an instrument model including cues and guide lines (304), statistical algorithms for evaluation (306), event generation for time-critical responses (308), and real-time video (310).

[0090] The courseware (tablet PC) computer (350) runs an "Emulator Interface Application" (SIA). The courseware computer (350) may include components for an simulator control panel (352), report generation (354, 380), task evaluation saving to a data server (356), voice recognition (358), task content (360), and task evaluation (362). Trainee results may be saved to a trainee results database (382). The instructor station (370) includes a video display (372) and may include a computer (not shown). The courseware computer (350) and / or the instructor station (370) may be remotely configured with the simulator, for example, located in different rooms, different buildings, or different countries. The primary function of the SIA is to communicate with the simulator to provide training. The SIA allows monitoring, interaction, and management of a specific student currently practicing on the simulator, as well as recording that student's performance data. To accomplish this, as described below, SIA interacts with a simulator-based learning system (SBLS) to allow students to view active assignments (subject to prerequisites) and download specific assignment files and content / learning resources corresponding to the assignments selected by the student. When an assignment is completed, SIA uploads the student's performance data at the end of each attempt and phase. The list of active assignments is updated when a student reaches the proficiency standard on an assignment. Regarding SBLS, SIA may include similar modules such as: assignment database, learning experience logic, assignment information screen, simulator communication, performance recording, SBLS interface, video encoding, streaming, and storage, reporting, and data / code updates. The simulator communication module allows SIA to communicate with the simulator, managing assignment performance using the simulator API. As described below, such communication is based on standard assignments received from SBLS. To track each assignment's performance, trainees verify access to identify them and store their performance data in their respective records.

[0091] The simulation application programming interface (API) (middleware layer) connects the simulator computer to the SIA. The middleware layer (330) may include components for loading the eye model (332), selecting instruments (334), receiving statistics (336), and receiving events (338). The API can be stored and run on a graphical PC or tablet PC. An interface is provided for Ethernet communication between the courseware and the simulator to start the simulation, trigger events, obtain measurements, and other necessary communications. Communication requirements between the SIA and simulator middleware include: job parameters – eye model construction and status, performance data – event-based, performance data – general (in progress), event triggering, audio commands, video feed, calibration and pre-session checks, performance logs, and instructions, orientation perspective, and real-time feedback. Key components for communication between the SIA and simulator include: 1. Surgical conditions : a. The start point of the surgery (a stage within the MSICS procedure).

[0092] b. End point of surgery (stage within the MSICS procedure).

[0093] c. Common anatomical changes (e.g., iris color, skin color, anterior chamber depth, etc.).

[0094] d. Surgical changes (right / left eye, intraocular pressure, cataract type).

[0095] e. Select intraoperative challenges (pupil constriction during lens capsule incision or increased IOP after nucleus removal).

[0096] 2. Training conditions : a. One or more specific learning objectives of the task (e.g., to give the scleral sulcus the desired shape and size).

[0097] b. Performance parameters and metrics (e.g., parameters related to the size and shape of the incision).

[0098] c. Evaluation criteria corresponding to the metric (e.g., range, not exceeding or below, event (yes / no) etc.).

[0099] d. Scoring logic – scores and / or weights and conditions associated with each metric to calculate one or more scores.

[0100] e. Selected training tools – guidance / direction view / real-time feedback alerts / auditory cues (set to communication if these training tools are available or unavailable during the process).

[0101] 3. Homework performance data : a. Receive data on job performance parameters, metrics, alarms, triggers, etc.

[0102] b. Receive video.

[0103] 4. Commands : a. Start the job (transferring and loading job files).

[0104] b. End / Abort the job (mark the task as finished and ask to start the performance data transfer).

[0105] c. Verbal commands - for limited purposes (e.g., inquiring about instruments (to be loaded, etc.)).

[0106] d. Triggering certain intraoperative challenges (e.g., microscope bulb malfunction, patient complaints of pain, etc.).

[0107] The above is not an exhaustive description but rather a relatively comprehensive viewpoint. There may be some limited additional communication requirements.

[0108] To deliver training in the most effective way, SIA incorporates the following key features. It breaks down the process into smaller units for unit practice. This allows for repeated practice of sections to help focus on specific tasks or task groups. For example, assuming the process is broken down into 10 tasks, one should be able to start with task 5 and stop after completing task 6. When starting task 5, all actions of tasks 1-4 have been completed, and an updated eye state should be loaded for this purpose. SIA also collects data on performance metrics (data on performance activities generated on the simulator) to compare them to standards, thus providing formative and summative feedback. Formative feedback helps improve learning. For example, applying more force than required to a task results in poor performance, or the incision size is much larger than required. Summative feedback indicates that performance is evaluated against the goals. Summative feedback also includes evaluating performance during attempts. SIA sets up scenarios similar to real life to properly prepare trainees for effective management. These can be scenarios that include intraoperative challenges, such as excessive bleeding in response to actions, weak lens suspensory ligaments supporting the capsule, and other challenges that arise even when performing tasks accurately. Errors or suboptimal performance in earlier tasks can later lead to intraoperative challenges during surgery. To present such scenarios without requiring learners / trainers to generate them every time they need to practice, SIA preserves the state of the eye, allowing trainees to begin practice from that state.

[0109] like Figure 3d As shown, SIA (500) is part of the simulator and manages the communication and data exchange between each learner and each job and the eye surgery simulator, as well as their simulator job performance. In a preferred embodiment, the simulator is connected to a central server (510) via an intranet. A server or network application called a simulator-based learning system (SBLS) (520) is located on the central server (510) and is used to manage the delivery of training to many trainees. SBLS (520) manages the learning management of the simulator for all modes of instruction. It includes the management of multiple learners, simulators, training delivery methods, etc. For example, SBLS (520) can manage more than 50-100 simulators. When simulators are operated simultaneously, a large amount of rich data in the form of video is generated and exchanged between the simulator and the centralized server application (SBLS (520)). SIA (500) communicates with the simulation API ( Figure 3cThe SBLS (520) manages the delivery of assignments to each trainee, identifies trainees, controls access to assignments, records performance data from assignment attempts, and provides other training management features of this type. Communication between the SIA (500) and SBLS (520) includes: an interface that allows users to verify and synchronize the SBLS (520) to maintain a local copy of the assignment database (static files and code related to assignment evaluation and scoring), transfer performance data, videos, etc. to the SBLS (520) for updating trainee records on the SBLS (520), and instructor-related functional communication.

[0110] SBLS (520) may contain many modules, including those listed below. Preferably, a copy of selected modules or some code from SBLS (520) is maintained on SIA (500). This ensures that simulator practice can begin or continue without any dependence on SBLS (520). Therefore, SIA (500) may contain the same modules listed below as SBLS (520). SIA (500) communicates with SBLS (520) for example, the following data: user verification, trainee status information (past records, permitted assignments, access to assignment-related content), data synchronized with SBLS (520) (uploading trainee performance data stored on the local system to SBLS (520) on the network), communication with the instructor client (530), video transfer, downloading and updating the latest version of SIA (500), and uploading reports and issues from simulator calibration and testing. Central server (510) and / or instructor client (530) may be remotely configured with one or more simulators.

[0111] Instructor / Student Operation Modules can be used to manage various users (e.g., superusers, administrators, instructors (trainers), trainees) and provide access control. The same or different modules can be used to generate assignments that can be assigned to learners for practice and evaluation on the simulator. Trainees are guided through a set of "assignments" designed to provide sufficient practice on individual parts of the surgery and then the entire procedure. The complexity and difficulty level of each such assignment increases as the trainee progresses. Assignments expose trainees to various cataract cases and complications. Assignments are generated based on several conditions, training tools, performance parameters, triggers / alarms, scoring, prerequisites, assignment metadata selection, etc.

[0112] Each practice unit on the eye surgery simulator can be referred to as a job in this document. Each job can have a start point and an end point (within the surgical procedure), both indicating the state of the eye and relevant parameters (e.g., color, size, cataract type, etc.). A job can be part of a surgery where some tasks have already been completed (e.g., an eye with a scleral sulcus). Jobs can also include complications to be presented. SBLS can support a large database of surgical jobs, using a user-friendly interface and a series of steps to collect and set all data to define a typical job.

[0113] Each task includes all the information required by the simulator to: (1) begin the simulation (using surgical conditions selected from training conditions, i.e., appropriate eye model variables, patient parameters, standard / custom starting points during the procedure); (2) define access to training tools – providing guidance, orientation, and feedback (in real time); applying additional step-by-step settings and limits for errors and alarms, possible complications, and corresponding actions; (3) determine the end of the simulation based on certain conditions in a particular task, i.e., the triggering of visually threatening complications (Level 2), the trainee terminating the task, the completion of the task, instructor intervention, etc.; (4) allow communication of alarms and performance metrics – in real time and at the end of the task (facilitating error-related feedback, trial scoring calculations, providing detailed scoring classifications for each parameter, etc.); (5) instructor intervention to change / trigger certain modifiable conditions; and / or (6) confirm the instruments used for each part of the simulation task. Each task also includes reference data corresponding to specified performance parameters that enable evaluation. Corresponding to each level are specified values, ranges, or settings, with which parameters are compared to assign scores or pass / fail statuses.

[0114] The assignment database can store assignments and related data that can be communicated with the simulator using, for example, XML messages / commands. Assignment files may include one or more of the following details: (1) relevant performance metrics expected to be monitored and / or recorded in real time and after the assignment attempt has been completed; (2) instructions, directional perspectives, and real-time feedback requirements (including audio feedback and other auditory requirements); (3) scenarios of complications that the instructor can trigger; (4) event triggers; (5) parameters (metrics) connected to feedback remediation content resources; (6) a reference start point name and data file corresponding to a custom start point (saved to follow the trainee's or trainer's performance); and (7) assignment metadata: task group / task / subtask / movement codes skillfully defined according to MSICS, titles, descriptions, title-level, task groups, etc.

[0115] The simulated course can include several modules that allow learners to complete a set of approximately 30-40 learning objectives. Each module can have about 8-10 unique assignments, some of which have 2-3 minor variations. Within these variations, the base assignment file can be influenced to customize the metrics used for scoring, and alerts and triggers can be changed to emphasize certain specific learning objectives related to the assignment. For example, as one variation, an assignment related to "generating a scleral sulcus" could emphasize that the trainee achieves the correct sulcus shape, and its metric could have a higher weight in scoring. Other collected metrics, such as sulcus length and depth, can carry low or no weight. Therefore, the weight applied to each metric for scoring can be varied accordingly to provide appropriate emphasis corresponding to one or more learning objectives.

[0116] Job information screens can include a set of screens that provide information (including images, videos, animations, etc.) for each job. These screens may have links that can load further details in pop-up windows. They will present standard job information such as job descriptions, steps performed, surgeon's clues, best practices followed, videos of expert performances, and videos from actual surgeries.

[0117] Learning experience logic can evaluate each unique task, determine task completion based on proficiency requirements, and determine the process or sequence that makes tasks accessible to users. Many features can determine a trainee's experience on the simulator. One important one is "personalization." One feature related to personalization is managing the trainee's progress through a task list. This can be computer-controlled or trainer-controlled.

[0118] Trainees can be expected to progress through a predetermined sequence of tasks in a computer-controlled mode. While trainees can view the task list, only those tasks they are qualified to access in this predefined order are displayed as active tasks. "Active" tasks represent those that can be practiced. "Inactive" tasks become "active" when the trainee completes prerequisite tasks. The trainer can, based on their personal judgment, grant each trainee access to any task (in any order), or make access to each task / not available to them. Furthermore, the trainer can be allowed to change selection conditions called modifiable conditions (specifications provided within the task). Modifiable conditions refer to any surgical simulation conditions that the trainer can change to enhance the training experience.

[0119] Open assignments are available to trainers in trainer-controlled mode. They can be configured with default simulation conditions. These default conditions represent the most basic settings for simulator practice. Trainers can modify these conditions (among others) and change the starting point before trainees begin attempting open assignments. The purpose of open assignments can be to allow trainees to familiarize themselves with the simulator settings and assignment conditions before beginning formative or summative evaluations. Standard parameters and performance videos can be recorded for each attempt. However, the system may not evaluate or score the attempt. Trainers can input their attempt feedback records into the system.

[0120] A computer-controlled mode can also be provided, where pre-set algorithms manage access to assignments in the assignment library and progress across assignments. However, trainers can have the ability to rewrite trainees' settings for selecting assignments and allowing access. In this mode, trainees can continue practicing assignments without close monitoring and access control by the instructor. However, instructors can be asked to review performance data and provide feedback on a real-time basis or after attempts.

[0121] The Free Play feature within the computer-controlled mode allows for a predetermined number of attempts corresponding to each task as free play. The purpose of Free Play mode can be to allow trainees to familiarize themselves with the simulator settings and task conditions before beginning formative or summative evaluations. However, performance data can still be recorded for monitoring simulator usage.

[0122] Each assignment may require both formative and summative evaluation. For evaluation, several parameters can be monitored and stored during and after the assignment. Summative evaluation may require the use of selective metrics and comparison against established performance criteria to assign scores and / or categorize attempts as successful / failed. A composite score can be calculated from these metrics. It may be required to benchmark performance and to facilitate comparison with other attempts by the same user (or other users using the same criteria). Simplified examples confirming this requirement are listed in the table below:

[0123] A task can be considered complete when consistent performance across a series of attempts confirms the level of proficiency required for the desired level. Completion requirements can also be enforced by imposing a certain amount of practice in the form of the minimum number of attempts required, the minimum number of successful attempts, and the minimum number of consecutive successful attempts.

[0124] Free time may include a preset number of attempts allowed per trainee for each task before considering performance data from the attempts for scoring and progress. This allows them to familiarize themselves with and adapt to the task at hand before being monitored and evaluated. Such requirements for each task can be incorporated into the progress and / or evaluation logic. Sample completion requirements for typical tasks may include: (1) a minimum number of attempts required (e.g., 80 to ensure sufficient practice); (2) a minimum number of successful attempts (e.g., 40 to ensure a certain minimum number of successes); (3) a minimum number of consecutive successful attempts (e.g., 37 to ensure consistency in successful performance); and / or (4) free time (e.g., 15 to allow familiarization with the task without any pressure associated with performance review or evaluation). According to one embodiment, no scoring may be assigned and no feedback may be provided during free time. However, performance data may still be recorded for monitoring purposes.

[0125] A simplified example of the recorded attempts is shown in the table below:

[0126] The performance record database can be used to store logs, scores, metrics, videos, etc., of all learner attempts for each assignment. Every attempt by a trainee can be recorded and stored. The generated data can be stored on SIA / SBLS. This can include video recordings, time taken for each attempt, performance parameter values, scores, attempt counts (success, failure, abandonment), and the count of the last consecutive successful attempts. This data can be used for reporting, analysis, and also within the logic of progress across assignments, as shown above. SIA can immediately transfer the records / data of each trainee's assignment attempts to SBLS after each attempt. Furthermore, other data such as login time, logout time, attempt time, and session length can be collected and stored for each trainee.

[0127] This system allows the transmission of simulator images (including simulated images displayed in a microscope view and / or images or videos captured by a camera (217)) to a remote computer or monitor during simulation. Additionally or alternatively, the system may also allow the export of collected performance data (including video recordings) to a cloud system for long-term storage. The system allows the export of collected performance data, including video recordings. Records of original 3D model parameters can be stored at predetermined stages of the assignment or as needed. Such snapshots of the original 3D model can be exported from SBLS for review and use by the courseware design group. When an assignment is generated, it can be used to establish a unique eye model state.

[0128] The Instructor Module allows instructors to monitor and manage simulator practice sessions from the same location as the simulator or remotely from the simulator. Instructors can utilize a desktop / portable PC, preferably a tablet, connected to the SBLS. The Instructor Module has the following functionalities for accessing the SBLS and managing training deliverables: (1) viewing a dashboard of simulator lab activities – simulator and current user activity; (2) monitoring assignments using video data streams, where trainers can view a set of video streams together or select one and switch between views; (3) obtaining real-time and event-based updates / information; (4) viewing reports; (5) adding subjective comments to each assignment attempted by the trainee; (6) adding or subtracting points from assignment attempts based on subjective evaluation and observation; (7) reviewing the trainee's performance history; and / or (8) rewriting the learning experience logic to allow trainees to progress or prevent progress due to subjective evaluation or other such reasons. Instructors can also log in from the simulator tablet PC to access these features and also use them to perform assignments, just like the trainees.

[0129] Reports can be generated and viewed. Each report's display can be filtered, the view changed, and it can be categorized and customized. Reports can include: • A group of trainees report on the status of their assigned tasks.

[0130] • Individual trainee progress reports for each assignment and across all assignments.

[0131] • Training activity report (trainees or groups – selected time period / day).

[0132] • Learning Analysis (Charts and Tables) - Assignments (Individual, Group, and Comparative) – Here are some examples: o Trends – Success, Failure, Giving Up

[0133] o The average time for success (other states) or all attempts.

[0134] o The number of attempts to become proficient.

[0135] o Performance trend – Selected metrics within the task (one or more selected metrics).

[0136] o Comments from the instructor on the assignment.

[0137] • The number of attempts for each task and the time taken for each attempt.

[0138] • The number and order of failed attempts during the task (failure indicates that the task was terminated due to errors in the visual perception of the threat).

[0139] • The number of failed attempts according to the simulation level (2 or 3).

[0140] • The number and order of attempts abandoned during the task.

[0141] • The number of abandoned attempts according to the level.

[0142] • The number and order of attempts successfully completed during each task.

[0143] • List the error numbers and descriptions of the threat visuals for each job, categorized by the attempt date.

[0144] • A report is generated after every 10 attempts, based on the selected metric for the job and date (refer to the metric table in the interface above).

[0145] Simulation software, including core simulation and simulator API code, can be updated periodically, for example, to address issues and upgrade functionality. Furthermore, to enable simulator use independent of SBLS, certain code and data can be maintained locally on SIA. This can include data such as job libraries and code such as logic for job progression. When updates are available, a data / code update module can be allowed to push these to each simulator periodically or as needed. Regular checks can be run to confirm the latest installed versions and update the simulator as necessary.

[0146] The table below provides use case scenarios, showcasing the functionality and features of connecting various activities with simulators and related components, including simulator-based learning systems (SBLS):

[0147] Verbal commands can be incorporated into the simulator to increase the realism of instrument changes and intraoperative management. Verbal commands requesting instrument changes can be given, and the instrument vision under the microscope can be adjusted accordingly. If the handles are different, trainees may need to manually change them; the handles are designed to feel like selecting the actual instrument being used. Voice recognition is also provided as part of the courseware to support trainees in selecting instruments via voice commands and other commands.

[0148] In some scenarios, there may be interactions with surgical assistant nurses, requiring the surgeon to communicate with others. The simulator may be limited to what can be seen in the microscope and the feel of the user's hands. Any other interactions or higher-level communication will be handled by the courseware, and the simulator will provide an interface to the courseware to make any changes to the simulated scene based on any interactions performed at a higher level. For example, the simulator may indicate that the microscope bulb has burned out. In this case, the microscope bulb has burned out, and the microscope field of view has darkened. The surgeon's response is to ask the nurse to replace the microscope bulb, and then restore normal light. The sequence between the simulator and the courseware is as follows: (1) The courseware sends a command to the simulator that the bulb should burn out, (2) The simulator responds by darkening the view appropriately, (3) The courseware recognizes any sequence of events required to replace the bulb, such as a voice command to tell the nurse to replace the bulb, (4) The courseware sends a command to the simulator that the bulb should work normally again, and (5) The simulator restores light in the microscope view.

[0149] It can also provide heart monitor sounds. Heart rate can be correlated with events in the simulator or set from the courseware. In addition, patient voices, requests, and other sounds are also included, provided the courseware can be set to play sounds under certain events or directly.

[0150] Although the invention has been described in detail with reference to embodiments for the purpose of fully disclosing the invention, these embodiments are merely exemplary and not intended to limit or represent an exhaustive enumeration of all aspects of the invention. Many changes can be made to these details, and the invention can be embodied in other forms without departing from the spirit, essential characteristics, and principles of the invention. Furthermore, any benefit, advantage, solution to a problem, and any element that may allow or promote any benefit, advantage, or solution is not to be construed as critical, necessary, or indispensable to the invention. The scope of the invention is limited only by the appended claims.

Claims

1. A surgical simulator, comprising: A simulated microscope, which includes a display and a camera; The first tactile arm, representing the actual surgical tool; A first tactile mechanism in mechanical communication with a first tactile arm, wherein the tactile arm and the tactile mechanism are configured to provide real-time tactile force feedback to the user of the system during simulated surgery; A tactile control unit, configured to send a signal indicating the level of tactile force feedback to a first tactile mechanism; and A first computer, which is in electronic communication with a tactile control unit and includes a software module having a physical model of a human body part and configured to calculate tactile force feedback to be provided by a first tactile arm and a first tactile mechanism; The display is configured to show a three-dimensional simulated image of a human body part and a three-dimensional image of a first simulated surgical tool corresponding to an actual surgical tool, based on the current orientation and position of the first tactile arm under the simulated microscope.

2. The surgical simulator according to claim 1, wherein, The camera has a downward field of view.

3. The surgical simulator according to claim 2, wherein, Part of the first haptic arm is within the camera's field of view.

4. The surgical simulator according to claim 3, wherein, The system is configured to record an image of a portion of the first tactile arm.

5. The surgical simulator according to claim 3, wherein, Images captured by the camera are transmitted to a remote computer.

6. The surgical simulator according to claim 3, wherein, Images captured by the camera are displayed on the screen.

7. The surgical simulator according to claim 1, wherein, Part of the first haptic arm is within the camera's field of view.

8. The surgical simulator according to claim 7, wherein, The system is configured to record an image of a portion of the first tactile arm.

9. The surgical simulator according to claim 7, wherein, Images captured by the camera are transmitted to a remote computer.

10. The surgical simulator according to claim 7, wherein, Images captured by the camera are displayed on the screen.

11. The surgical simulator according to claim 1, further comprising a foot pedal unit.

12. The surgical simulator according to claim 11, wherein, The foot pedal controller includes a foot pedal and a switch.

13. The surgical simulator according to claim 11, wherein, The visual settings displayed on the monitor are modified according to the position of the foot pedal.

14. The surgical simulator according to claim 13, wherein, The visual settings displayed on the monitor were modified by simulating fluid injection.

15. The surgical simulator according to claim 13, wherein, The visual settings displayed on the monitor are modified by simulating fluid suction.

16. The surgical simulator according to claim 11, wherein, The visual settings displayed on the screen are modified according to the position of the switch.

17. The surgical simulator according to claim 16, wherein, The visual settings displayed on the screen were modified by simulating fluid injection.

18. The surgical simulator according to claim 16, wherein, The visual settings displayed on the screen are modified by simulating fluid suction.

Images