Virtual reality imaging system

Abstract

The invention relates to a device (10) for viewing a simulated image (100) of an internal target zone (Z) of a human body during a simulation of a medical imaging examination, the device (10) comprising an anatomical console (12) comprising an interaction surface (26) with at least one interaction zone (28), a dummy probe (14) with an interaction head (16) configured to cooperate with the interaction surface (26), a virtual scene (101) with a virtual screen (102), and a virtual reality system (18). The virtual screen of the virtual scene displays the simulated image (100) of all or part of the internal target zone in real time, when the interaction head of the dummy probe is cooperating with the at least one interaction zone of the interaction surface.

Description

[0001] Description

[0002] Title: VIRTUAL REALITY IMAGING SYSTEM

[0003] FIELD OF THE INVENTION

[0004] The present invention relates to the field of training future surgeons. The present invention is therefore in the field of tools, methods and materials for education and teaching, in particular the field of surgical training consoles.

[0005] TECHNOLOGICAL BACKGROUND

[0006] To date, most surgical training is conducted in real-life conditions, on patients, through surgical companionship. This method requires significant human resources, presents significant material constraints, and can generate significant stress for the student, which can lead to difficulties with concentration and / or memorization.

[0007] Certain surgeries or even certain non-surgical examinations also require knowing how to generate, read and interpret a medical imaging image such as an ultrasound, for example.

[0008] Some surgeries are performed in conjunction with an ultrasound or colonoscopy to guide the surgeon during the procedure. Therefore, it is also necessary to practice generating this imaging, while learning to read it in real time so that it can be relied upon during surgery.

[0009] The aim of the present invention is therefore to provide a safe, practical, accurate, realistic, easy-to-use and readily available training device, enabling training to generate a realistic medical image in order to be able to interpret it in real time. The device must be able to allow, where appropriate, the simultaneous use of a surgical tool and an imaging device in a realistic manner. The realism of surgical simulation and the generation of medical images depends largely on the possibility of applying realistic constraints to the movement of the manipulation tools held by the user. Applying these constraints is therefore a technical challenge.

[0010] SUMMARY

[0011] This objective is achieved, in accordance with the invention, by means of a device for viewing a simulated image of an internal target area of ​​a human body during a simulation of a medical imaging examination, the device comprising: an anatomical console forming a base, the anatomical console comprising an interaction surface comprising at least one interaction area, a dummy probe comprising an interaction head, the interaction head being configured to cooperate with the interaction surface, a virtual scene presenting a virtual screen, a virtual reality system comprising:

[0012] • a screen allowing the visualization of the virtual scene,

[0013] • a localization and orientation system, configured to locate the anatomical console and the dummy probe in real space,

[0014] • a control unit configured to generate the virtual scene and associate all or part of F at least one internal target zone with F at least one interaction zone,

[0015] This device is characterized in that the virtual screen of the virtual scene displays the simulated image of all or part of the internal target area in real time, when the interaction head of the dummy probe cooperates with at least one interaction area of ​​the interaction surface.

[0016] Thus, this solution allows to achieve the above-mentioned objective. In particular, it allows to simulate, in real time, a realistic and interactive medical imaging, covering a very large number of internal areas of a patient, without the need to change the anatomical console.

[0017] The machining device according to the invention may comprise one or more of the following characteristics, taken in isolation from one another or in combination with one another: - the virtual screen of the virtual scene may display, in real time, any modification of all or part of the internal target zone,

[0018] - the virtual scene may further present a virtual probe forming a virtual image of the dummy probe, the virtual probe being associated with the dummy probe by the virtual reality system so that each movement of the dummy probe in real space is reproduced by the virtual probe in the virtual scene,

[0019] - the virtual reality system may include a mobile calibration tool, the calibration tool forming all or part of a handle of the dummy probe,

[0020] - the interaction head of the dummy probe may comprise at least one interface element configured to generate predetermined haptic feedback when the dummy probe is in contact with at least one interaction zone,

[0021] - the interaction surface may comprise a first interaction zone and a second interaction zone, the interaction head being configured to: generate a first specific haptic feedback when the dummy probe is in contact with the first interaction zone, and generate a second specific haptic feedback when the dummy probe is in contact with the second interaction zone,

[0022] - the dummy probe can be a dummy ultrasound probe and the medical examination is an ultrasound,

[0023] - the dummy probe can be a dummy endoscopy probe and the medical examination is an endoscopy,

[0024] - the device may comprise a dummy surgical tool configured to be connected to the control unit, the virtual scene comprising a virtual surgical tool forming a virtual image of the dummy surgical tool, the virtual surgical tool being associated with the dummy surgical tool by the virtual reality system so that each movement of the dummy surgical tool in real space is reproduced by the virtual surgical tool in the virtual scene,

[0025] - the device may comprise a movable haptic arm connected to the control unit and configured to fix the dummy surgical tool, so as to generate a predetermined haptic feedback when the dummy surgical tool cooperates with the interaction surface of the anatomical console. DESCRIPTION OF THE FIGURES

[0026] The invention will be better understood, and other aims, details, characteristics and advantages thereof will appear more clearly on reading the detailed explanatory description which follows, of embodiments of the invention given as purely illustrative and non-limiting examples, with reference to the appended schematic drawings.In these drawings: Figure 1 is a perspective view of a device according to a first embodiment of the present invention, Figures 2a and 2b are, respectively, a perspective view and a sectional view of a dummy probe according to the present invention, Figures 3a and 3b are detailed views of the virtual scene generated by the device according to the invention, Figure 4 is a perspective view of a device according to a second embodiment of the present invention, including a movable haptic arm, Figure 5a is a perspective view of the embodiment of Figure 4, Figure 5b is a perspective view of a device according to a third embodiment of the present invention, also including a movable haptic arm.

[0027] DETAILED DESCRIPTION

[0028] Imaging

[0029] The present invention allows a trainee practitioner to practice performing a medical imaging examination. The imaging aspect of the present invention relies solely on the observation of one (or more) simulated image(s) 100.

[0030] The present invention thus relates to a device 10 for viewing, by at least one user, a simulated image 100 of at least one internal target zone Z of a human body during a simulation of a medical imaging examination.

[0031] The internal target zone Z can, for example, be the inside of an intestinal tract, various regions of the abdomen (epigastric region, umbilical region, etc.), a specific organ, muscle or a set of muscles, etc. This simulation of a medical imaging examination can thus be, for example, a simulation of an ultrasound, colonoscopy or fibroscopy.

[0032] As visible in Figure 1, the device 10 according to the present invention comprises: an anatomical console 12 forming a base, a dummy probe 14 comprising an interaction head 16, a virtual reality system 18 comprising:

[0033] ■ a 20 screen,

[0034] ■ a location and orientation system 22,

[0035] ■ a 24 control unit.

[0036] In summary, the device 10 is configured so that a user manipulating the dummy probe 14 in cooperation with the anatomical console 12 can see all or part of the internal target area Z appear on the screen 20. By moving the dummy probe relative to the anatomical console 12, the user can observe different parts of the internal target area Z, exactly as during an ultrasound examination, for example. The internal target area Z is made visible to the user exclusively by means of at least one simulated image 100, preferably several simulated images 100.

[0037] Preferably, the anatomical console 12 is an autonomous entity intended to be placed on a flat surface, such as the floor or a table, representing and / or reconstituting in 3D all or part of an external anatomical part of a patient P. More particularly, the anatomical console 12 comprises an interaction surface 26 representing and / or reconstituting in 3D all or part of an external anatomical part of a patient P. Preferably, the anatomical console 12 and more particularly the interaction surface 26 is designed so as to allow tactile feedback and even, depending on the embodiments, haptic feedback, when the user interacts with it. The interaction surface 26 preferably has at least one rigidity characteristic similar to that of a human anatomical wall.

[0038] In the present application, the notion of “haptic feedback (or signal)” is understood as a signal actively generated by the device 10 according to the present invention. It is to be differentiated from the notion of “tactile feedback” which is a simple passive feedback, generated automatically by the interaction between the user and the device 10 in reaction to the manipulation of animate or inanimate objects.

[0039] The anatomical console 12 may in particular comprise one or more elements made of silicone, for example. The anatomical console 12, and more particularly the interaction surface 26; thus makes it possible, depending on the embodiments, to give the user the illusion of interacting with an external anatomical part of a patient P. In the case of an obstetric ultrasound simulation, for example, the interaction surface has a rounded convex shape so as to represent the belly of a person pregnant for the desired number of months for the simulation. In the embodiment shown in Figures 4 and 5a, the anatomical console 12 represents the neck of a patient P, in the context of a simulation of the ablation of a benign thyroid nodule by the use of an antenna which produces microwaves at its end.

[0040] In the embodiment shown in Figure 1, the interaction surface 26 is cut into a series of interaction zones 28 having variable dimensions and shapes. Depending on the embodiment, the interaction surface 26 is cut into a series of interaction zones 28 having similar dimensions and shapes. The interaction surface 26 may be a flat, concave and / or convex surface. It may be a surface external to the anatomical console 12 (open surface) located on the surface of the anatomical console 12 or a surface internal to the anatomical console 12 (closed surface) accessible through an opening provided in the anatomical console 12, for example the interior of a tube or a canal (see Figure 5b). The interaction surface 26 may comprise an external silicone coating.

[0041] Each interaction zone 28 is associated with a simulated image 100 of all or part of the internal target zone Z. According to the embodiments, each interaction zone 28 can be associated with several simulated images 100. These simulated images 100 are organized in a precise sequence and are presented successively to the user, so as to simulate a movement in all or part of the internal target zone Z (for example, in the case of an obstetric ultrasound, the movement of a fetus). Thus, in certain embodiments, when the interaction head 16 of the dummy probe 14 cooperates over an extended period with the interaction zone 28, several simulated images 100 scroll on the screen 20.

[0042] The simulated images 100 are not derived from real data read by a real probe. Each simulated image 100 is generated, by the virtual reality system 18, using 3D models representing the nodules, fluids and organs of the internal target area Z. These models are then simplified into basic mathematical curves (instead of triangles).

[0043] Preferably, in order to prevent the user from feeling dizzy and / or sick, the virtual reality system 18 maintains the simulated image update rate from 100 to 60 images per second, preferably 65 simulated images per second.

[0044] The approaches available on the market do not meet this requirement, only meeting approximately 40 to 50 images per second, because the calculation software associated with these approaches takes into account too much data and calculations that are not essential in the context of the present invention. In the context of the present invention, the usual logical process in mathematics has therefore been optimized in order to achieve 100 to 60 images per second, preferably 65 simulated images per second.

[0045] This simulation speed allows, in the context of an ultrasound simulation, to increase the realism of the simulation and to simulate a realistic physical propagation through the anatomy and to process it to visualize it in real time in the virtual scene 101 (see below).

[0046] The dummy probe 14 has a shape similar to a real medical probe, so as to maximize the realism of the simulation and to offer the best training conditions to the user. More particularly, as shown in FIGS. 2a and 2b, the dummy probe 14 comprises an interaction head 16 configured to cooperate with the interaction surface 26. This cooperation may be cooperation by contact, by friction, by translation or for example by sliding. In the context of an ultrasound simulation, for example, the interaction head 16 is configured to cooperate by sliding with the interaction surface 26 (external, open surface), the user being led to slide the interaction head 16 of the dummy probe 14 on the interaction surface 26 of the anatomical console.In the context of a colonoscopy simulation, for example, the interaction head 16 is configured to cooperate by translation with the interaction surface 26 (internal, closed surface), the user being led to advance the interaction head 16 of the dummy probe 14 along the interaction surface 26 of the anatomical console 12.

[0047] In some embodiments, the dummy probe 14 is a dummy ultrasound probe and the simulated medical examination is an ultrasound. In some embodiments, the dummy probe 14 is a dummy endoscopy or colonoscopy probe and the simulated medical examination is an endoscopy or colonoscopy.

[0048] In order to maximize the realism of the simulation, for example in certain embodiments of ultrasound examinations (see Figures 2a and 2b), the interaction head 16 of the dummy probe 14 comprises at least one interface element 29a, 29b configured to generate a predetermined haptic feedback when the dummy probe 14 is in contact with the at least one interaction zone 28 of the interaction surface 26. More particularly, as illustrated in Figures 2a and 2b, the interaction head 16 of the dummy probe 14 represents an ultrasound probe and it comprises two metal balls 29a each associated with a spring 29b so as to make the user feel a sensation close to that of a real probe when manipulating it on the skin of a real patient P with a gel (sliding). The springs 29b are constrained in a channel which ensures that the force of each spring 29b is always directed towards the corresponding ball 29a.Between the springs 29b and the metal balls 29a is a Nylon interface piece whose contact surface with the balls 29a is a series of empty and full fringes of material, so as to limit the contact surface and therefore the friction to promote the rolling of the balls 29a. The Nylon interface piece is also constrained by a channel and allows a homogeneous pressure which encompasses the balls 29a and pushes them against the lower wall of the interaction head 16. At rest, this constrains the balls 29a and prevents their rotation and translation.When the interaction head 16 of the dummy probe 14 is pressed against the interaction surface 26 of the anatomical console 12 and a translational movement tangent to the interaction surface 26 of the anatomical console 12 is applied to the dummy probe 14, the balls 29a are no longer in contact with the wall of the interaction head 16 of the dummy probe 14 and have more space by having only the nylon interface piece in contact with the dummy probe 14. They therefore slide much better. The silicone coating of the interaction surface 26 of the anatomical console 12 having a slightly “viscous” surface, the balls grip this surface and slide on the interface piece to imitate reality.

[0049] For the sake of realism, in certain embodiments, the interaction surface 26 comprises a first interaction zone 28 and a second interaction zone 28. In these cases the interaction head 16 can be configured to: generate a first specific haptic feedback when the dummy probe 14 is in contact with the first interaction zone 28, and generate a second specific haptic feedback when the dummy probe 14 is in contact with the second interaction zone 28.

[0050] This could be used to simulate a nodule under the skin or a change in the stiffness or elasticity of the skin of a patient P with a tumor or a scar, for example.

[0051] The screen 20 of the virtual reality system 18 allows a user to view the simulation. More particularly, the screen 20 allows one (or more) user(s) to view a virtual scene 101 which will be described below. The screen 20 may be a fixed element in real space or a mobile element in real space, for example, configured to be worn by the user during the simulation. The device 10 may comprise several screens 20, allowing several users to view the simulation, or even to participate in it. The different screens 20 may thus be mobile or fixed. More precisely, as visible in FIGS. 3a and 3b, the screen 20 may be a screen placed on a surface near or at a distance from the anatomical console 12. In another embodiment, the screen 20 may be a virtual reality headset, adjustable to the user and capable of providing audio feedback (see FIG. 1).More particularly, it may be an HP reverb® headset having two screens with a resolution of 2160x2160 pixels, a Meta Quest 3® headset or an Apple Vision Pro headset, for example®. Each screen has a display frequency of 90 Hz. The virtual reality system 18 also comprises a localization and orientation system 22, configured to locate the anatomical console 12 and the dummy probe 14 in real space. The localization and orientation system 22 thus comprises, in a manner known per se, at least one mobile calibration tool Ki. This mobile calibration tool Ki may take the form of a conventional controller as for example illustrated in FIGS. 4a and 4b, but it may also take a different form. Preferably, the calibration tool Ki forms a handle 30 of the dummy probe 14 (see FIGS. 4, 5a and 5b).

[0052] When the screen 20 is a virtual reality headset, it can scan its environment and is capable of identifying its position relative to this environment (real space). In a manner known per se and as already mentioned, each virtual reality headset is preferably associated with at least one calibration tool Ki. The position of the at least one calibration tool Ki relative to the virtual reality headset is identified by the headset (a camera on the headset can for example identify constellations of IR light on one face of the calibration tool). This makes it possible to have the position of the at least one calibration tool Ki (and therefore of the dummy probe 14 when the latter is associated with the calibration tool) relative to real space.

[0053] The anatomical console 12 further comprises at least one calibration base K2 for the at least one calibration tool Ki. This calibration base K2 comprises an imprint, in which the at least one calibration tool Ki can be placed in a unique and repeatable manner. This makes it possible to know the position of the anatomical console 12, the screen 20 and the dummy probe 14 relative to real space and to study each of their movements in real time.

[0054] In the case where the screen 20 is a mobile device configured to be worn by the user (a virtual reality headset), the virtual reality system 18 further makes it possible to locate the user relative to the anatomical console 12.

[0055] As mentioned above, the virtual reality system 18 further comprises a control unit 24. This control unit 24 is notably configured to: generate the virtual scene 101, and associate all or part of the internal target zone Z with the at least one interaction zone 28 of the interaction surface 26.

[0056] The control unit 24 may be integrated into the anatomical console 12 (as illustrated in the figures) or may be an attached element, such as a computer, connected, wired or wirelessly, to the anatomical console 12.

[0057] The control unit 24 is connected to the screen 20 (wired or wireless) and is configured to align the virtual scene 101 (virtual reality) with the real space (physical reality of the user manipulating the platform of the device 10 according to the present invention).

[0058] The virtual scene 101 is a virtual space displayed to the user on the screen 20. If the screen is a virtual reality headset, the virtual scene 101 is immersive. Depending on the embodiments, the virtual scene 101 may comprise different fixed or mobile virtual elements, in order to increase the realism of the simulation for the user. Thus, the virtual scene 101 may comprise: a virtual probe 114 forming a virtual image of the dummy probe 14, and / or a virtual anatomical wall 126 forming a virtual image of the interaction surface 26, and / or a virtual hand 150 of the user, associated with a virtual handle 130 of the dummy probe 114, and / or an operating room or medical office setting, one or more constituent elements of the internal target zone Z (for example all or part of organs, muscle groups, a tumor, a bone, etc.

[0059] The virtual probe 114 is associated with the dummy probe 14 by the virtual reality system 18 so that each movement of the dummy probe 14 in real space is reproduced by the virtual probe 114 in the virtual scene. The virtual anatomical wall 126 is associated with the interaction surface 26 by the virtual reality system 18 so that each movement of the interaction surface in reality is reproduced by the virtual anatomical wall in the virtual scene.

[0060] In all cases, the virtual scene 101 has a virtual screen 102 intended to display4for the user, each simulated image 100 generated by the virtual reality system 28. This virtual screen 102 is preferably in the form of a two-dimensional screen or flat screen. It is the image of a 2D screen in the 3D virtual reality of the virtual scene 101. This virtual screen 102 can, for example, appear in the conventional form of a monitor, it then forms a virtual monitor. More particularly, when the device 10 is running and the simulation is launched, the virtual screen 102 of the virtual scene 101 displays each simulated image 100 of at least one internal target zone Z in real time, as the interaction head 16 of the dummy probe 14 cooperates with each interaction zone 28 of the interaction surface 26 of the anatomical console 12.

[0061] In this way, when the interaction head 16 of the dummy probe 14 cooperates with the at least one interaction zone 28, the user sees the at least one simulated image 100 associated with this interaction zone 28 displayed. Each interaction zone of the interaction surface 26 is thus associated, by the control unit, with the entirety or a predetermined part of the internal target zone Z. The control unit 24 thus allows the creation of a unique correspondence link (bijection) between: an interaction zone 28 of the real space, a series of simulated images 100 comprising at least one simulated image 100 representing all or part of the internal target zone Z.

[0062] In summary, each interaction zone 28 corresponds to a specific part of the internal target zone Z, and allows the display, in real time, by interaction (cooperation) with the interaction head 16, of all the simulated images 101 corresponding to this specific part of the internal target zone Z, on the virtual screen 102 of the virtual scene 101.

[0063] Surgical Intervention In some embodiments that seek to simulate a surgical procedure, during medical imaging simulation, for example, the removal of a benign thyroid nodule by the use of an antenna that produces microwaves at its tip. This type of surgical operation is conducted in parallel with ultrasound monitoring.

[0064] In these embodiments, the device 10 comprises a dummy surgical tool 32 configured to be connected to the control unit 24. This dummy surgical tool has shape, weight and mobility characteristics similar to a real surgical tool.

[0065] Preferably, the device 10 comprises a movable haptic arm 34 connected to the control unit 24 and configured to fix the dummy surgical tool 32. This fixing of the dummy surgical tool 32 to the movable haptic arm 34 makes it possible to generate a predetermined haptic feedback when the dummy surgical tool 32 cooperates with the interaction surface 26 of the anatomical console 12.

[0066] The mobile haptic arm 34 makes it possible to know the position and relative orientation of an object attached thereto (see below). The position and orientation of this object are then obtained relative to the control unit 24 and therefore to the entire virtual reality system 18. Furthermore, since the position of the mobile calibration tool Ki relative to the control unit 24 is known, the position and orientation of the object connected to the mobile haptic arm 34 can then be known. Preferably, the mobile haptic arm 34 is connected to the anatomical console 12 by means of a robot 36 fixed to the anatomical console 12 (see FIGS. 4, 5a, 5b). As seen in FIGS. 4, 5a, 5b, the haptic arm 34 has, opposite the robot 36, a free end 38 configured to connect at least one dummy surgical tool 32.

[0067] The movable haptic arm 34 is movable according to at least six degrees of freedom obtained by means of various elbows and rotating parts cooperating with each other so as to form joints Ji, , h, L, Js, JÔ. More precisely, and as visible in Figures 4, 5a, 5b, the first three joints (distal joints) are actuable by the user while the last three joints (proximal joints) are passive. In order to maximize the realism of the simulation, the dummy surgical tool 32, once connected to the movable haptic arm 34, must have its tip (or free end) positioned where the haptic feedback would occur in reality, i.e. at the haptic point of the haptic arm. This haptic point is designated as “HIP”.The haptic arm 34 simulates the force feedback related to the collision or interactions in the virtual world of the tip (or end) of the dummy surgical tool 32 manipulated with an element of the virtual environment. This is the point at which the interactions and collisions are calculated to be able to simulate them without creating an uncomfortable and disturbing haptic lag or inconsistency for the user. Taking this haptic point HIP into account makes it possible to simulate the penetration of the body of a patient P, for example by the needle of a syringe, by simulating the stress exerted by the body of the patient P on the needle.

[0068] All joints are tracked by position sensors to determine their respective angular positions and rotations, but not all of them benefit from haptic feedback. The joints that do not benefit from haptic feedback are referenced J4, J5 and JÔ. Indeed, since the three distal joints are large, it is possible and easy to equip them with a motor that can limit their mobility if necessary.

[0069] Depending on what the virtual reality generated by the control unit is intended to represent to the user, the mobility parameters of the movable haptic arm 34 vary and the haptic signal generated by the control unit and transmitted by the movable haptic arm 34 to the user also varies.

[0070] Regardless of their shape and simulated function, all of the dummy surgical tools 32 connect to the free end 38 of the mobile haptic arm 34 via a Plug and Play connection. In the present application, the concept of “plug and play” describes a simple action, involving only a limited number of gestures, preferably only one. A “plug and play” connection thus describes a connection that is made with a single gesture.

[0071] In this embodiment including a dummy surgical tool 32, the virtual scene 101 comprises a virtual surgical tool 132 forming a virtual image of the dummy surgical tool 32. The virtual surgical tool 132 is thus associated with the dummy surgical tool 32 by the virtual reality system 18 so that each movement of the dummy surgical tool 32 in real space is reproduced by the virtual surgical tool 132 in the virtual scene 101 (see figure 3b).

[0072] In this embodiment, the virtual reality system 18 is configured to implement a calibration between the screen 20 and the anatomical console 12 by triangulation. More specifically, the virtual reality system 18 allows a real and virtual interaction between the movements of the dummy surgical tool 32 and the interactions with the interaction surface 26 and the simulated images 100 displayed on the virtual screen 102 of the virtual scene 101. Indeed, depending on the movements of the dummy surgical tool 32 and the interactions with the interaction surface 26, the simulated images 100 may comprise a representation of the dummy surgical tool 32.

[0073] Furthermore, the virtual reality system 18 allows, in the virtual scene 101, an interaction between the virtual surgical tool 132 and at least one of the constituent elements of the internal target zone Z.If this or these constituent element(s) are not directly visible in the virtual scene, but only visible via the two-dimensional screen 102 of the virtual scene 101, the virtual reality system 18 nevertheless knows the position of each constituent element of the internal target zone Z and allows, depending on the movements and actions of the dummy surgical tool 32 connected to the mobile haptic arm 34 and therefore of the virtual surgical tool 132: via the mobile haptic arm 34, to generate specific haptic feedback resulting from the simulated interaction between the virtual surgical tool 132 and at least one constituent element of the internal target zone Z to the user, via the two-dimensional screen 102 of the virtual scene 101 or directly via the virtual scene 101, to generate visual feedback to the user.

[0074] The objective is to allow the user to, at the same time: move (with a first hand) the dummy probe 14 (comprising the calibration tool Ki) on the interaction surface 26 of the anatomical console 12 in order to view one or more simulated images 100 of all or starting from the internal target zone Z of a virtual patient P on the remote virtual screen 102 (next to the virtual patient P), and with the other hand, move the dummy surgical tool 32 fixed on the mobile haptic arm 34 along and / or around the interaction surface 26 of the anatomical console 12, in order to be able, on the one hand, to view on the simulated image(s) 100 given by the dummy probe 14, the tip of the dummy surgical tool 32, and on the other hand, of the dummy probe 14 on an interesting plane of the anatomy (therefore of the interaction surface 26) with the dummy surgical tool 32 while avoiding anatomical areas at risk.

[0075] In another case, the objective F is to allow a user to, at the same time: move (with a first hand) a dummy endoscopic camera (dummy probe 14 which includes a calibration tool Ki) inside the anatomical console 12 by a closed interaction surface 26 which leads inside the body (inside the back at the level of the spine for example) in order to view one or more simulated image(s) 100 of the inside of the back (muscles, yellow ligaments, vertebrae and intervertebral discs) on a remote virtual screen 102 (next to the patient P), and with the other hand, to move a replica of an electric milling machine fixed on a mobile haptic arm 34 inside the back in order, on the one hand, to find the end of the motorized milling machine on the remote view of the camera, and on the other hand to position himself correctly at the anatomical level in order to be able to mill a part of the vertebra.

[0076] Imaging process

[0077] The device 10 according to the present invention makes it possible to implement a method for visualizing medical imaging based on the visualization (by a user) of one or more simulated image(s) 100 of the internal target zone Z of a virtual patient P.

[0078] The one or more simulated images of the internal target area Z are based on models stored in the memory of the control unit 24 and are then recreated based on the positioning of the dummy probe 14.More specifically, the steps performed by the device 10 to display a series of simulated images 100 are as follows: modeling, in the form of a 3D model, all the different internal organs, tissues and tumors of the internal target area Z; preprocessing the 3D models in a better format to increase the execution speed (as explained above); gathering all the necessary data (preprocessed models, surgical data) and sending them to a graphics card of the control unit 24 for processing; using the position of the dummy probe 14 as an origin for Ray Tracing (see below); defining an imaging plane in the virtual scene 101, projecting the results of the Ray Tracing into the ultrasound plane of the virtual scene 101; adding post-processing effects to give a realistic impression of a medical image, for example from an ultrasound.

[0079] The imaging plane may be a coplanar plane with an extension axis of the dummy probe 14 (in the case of ultrasound) or a plane orthogonal to the extension axis of the dummy probe 14 (in the case of endoscopy).

[0080] Ray tracing is a technique that allows the emulation of the path of light and its interactions with the environment, in other words its alteration with respect to physical phenomena and objects; notably reflection and refraction effects, but also shadows and other more complex optical phenomena.

[0081] Surgical simulation process

[0082] In embodiments in which the device 10 also comprises a movable haptic arm 34 connected to the control unit 24 and configured to attach the dummy surgical tool 32, the purpose of the device 10 is to enable a user to perform a surgery simulation, guided by the medical imaging simulation. The device 10 therefore enables the simulated medical imaging to react to the simulated operations performed by the user during the surgical simulation.

[0083] Thus, in addition to using the position of the dummy probe 14 as an origin for Ray Tracing, the virtual reality system 18 uses the position of the dummy surgical tool 32 and the actions performed on it by the user to simulate, if necessary, an interaction between at least one constituent element of the internal target zone Z and the virtual surgical tool 132 and generate the corresponding simulated virtual image(s) 100.

[0084] The technical difficulty is thus, for the device 10, not only to realistically display in the two-dimensional screen 102 of the virtual scene 101 a medical imaging visual, but also to modify the state of the 3D models according to the use of the dummy surgical tool 32 (and therefore the virtual surgical tool 132) held by the user. This display is carried out in a non-animated manner (in real time) which allows the user of a surgical simulation session, starting from the same anatomy to have a different result depending on the duration and intensity of interaction (the user chooses when to activate / deactivate the dummy / virtual surgical tool 32, 132), and also the position of the dummy / virtual surgical tool 32, 132 relative to the constituent element of the internal target zone Z targeted by the surgical simulation.

[0085] Thus, the steps performed by the device 10 to display a series of simulated images 100 during a surgical simulation are as follows: model, in the form of a 3D model, all the different internal organs, tissues and tumors of the internal target area Z; preprocess the 3D models in a better format to increase the execution speed (as explained above); gather all the necessary data (preprocessed models, surgical data specific to the actions planned by the dummy surgical tool 32 and send them to a graphics card of the control unit 24 for processing; use the position of the dummy probe 14 as an origin for Ray Tracing (see below);defining an imaging plane in the virtual scene 101, calculating the position of the dummy surgical tool 132, tracking and measuring the actions performed on the dummy surgical tool, modifying the state of the 3D models based on the use of the dummy surgical tool 32, projecting the results of the Ray Tracing into the ultrasound plane of the virtual scene 101; adding post-processing effects to give a realistic impression of a medical image, for example from an ultrasound.;

[0086] Thus, the virtual screen 102 of the virtual scene 101 displays, in real time, any modification of all or part of the internal target zone Z.

[0087] For example, in the case of a simulation of the ablation of a benign tumor (nodule) using a microwave heating antenna (the dummy surgical tool 32 is a reconstruction of a heating antenna).

[0088] The device 10 realistically displays an ultrasound visual on the two-dimensional screen 102 and the virtual reality system 18 modifies the state of the 3D models depending on the use of the antenna which is held in one hand by the user. The other hand manipulates the dummy probe 14. The display occurs in real time, allowing the user of the surgical simulation session to start from the same anatomy and obtain a different sequence depending on the exposure duration (the user chooses when to activate / deactivate the antenna), the exposure power (chosen by the user) and also the position of the antenna relative to the nodule.

[0089] The particularity of ultrasound is linked to the physics of the movement of ultrasound in different materials. What is liquid is visually black (water, blood) because it absorbs melts. What is solid is white (bones, organ walls, etc.) because solid elements amplify the signal. More generally, it is a gradient between black and white. There are also shadow and overexposure phenomena linked to the change of materials. When the nodule burns, by microwave, it becomes whiter and when it cools it becomes black. If it is completely burned, it remains completely black. In this operation, it is important to burn only 80% of the nodule.

[0090] The fact that the user is free to impact the nodule differently each time means that the data to be processed in real time by the device 10 is much more numerous and more complex than if we were just viewing a “film” in VR.

Claims

CLAIMS 1. Device (10) for viewing a simulated image (100) of an internal target area (Z) of a human body during a simulation of a medical imaging examination, the device (10) comprising: an anatomical console (12) forming a base, the anatomical console (12) comprising an interaction surface (26) comprising at least one interaction area (28), a dummy probe (14) comprising an interaction head (16), the interaction head (16) being configured to cooperate with the interaction surface (26), a virtual scene (101) having a virtual screen (102), a virtual reality system (18) comprising: ■ a screen (20) allowing the visualization of the virtual scene (18), ■ a localization and orientation system (22), configured to locate the anatomical console (12) and the dummy probe (14) in real space, ■ a control unit (24) configured to generate the virtual scene (101) and associate all or part of the at least one internal target zone (Z) with the at least one interaction zone (28), characterized in that the virtual screen (102) of the virtual scene (101) displays the simulated image (100) of all or part of the internal target zone (Z) in real time, when the interaction head (16) of the dummy probe (14) cooperates with the at least one interaction zone (28) of the interaction surface (26).

2. Device (10) according to any one of the preceding claims, characterized in that the virtual screen (102) of the virtual scene (101) displays, in real time, any modification of all or part of the internal target zone (Z).

3. Device (10) according to any one of the preceding claims, characterized in that the virtual scene (101) further has a virtual probe (114) forming a virtual image of the dummy probe (14), the virtual probe (114) being associated with the dummy probe (14) by the virtual reality system (18) so that each movement of the dummy probe (14) in real space is reproduced by the virtual probe (114) in the virtual scene (101).

4. Device (10) according to any one of the preceding claims, characterized in that the virtual reality system (18) comprises a mobile calibration tool (Ki), the calibration tool forming all or part of a handle (30) of the dummy probe (14).

5. Device (10) according to any one of the preceding claims, characterized in that the interaction head (16) of the dummy probe (14) comprises at least one interface element (29) configured to generate a predetermined haptic feedback when the dummy probe (14) is in contact with the at least one interaction zone (28).

6. Device (10) according to the preceding claim, characterized in that the interaction surface (26) comprises a first interaction zone (28) and a second interaction zone (28), the interaction head (16) being configured to: generate a first specific haptic feedback when the dummy probe (14) is in contact with the first interaction zone (28), and generate a second specific haptic feedback when the dummy probe (14) is in contact with the second interaction zone (28).

7. Device (10) according to any one of the preceding claims, characterized in that the dummy probe (14) is a dummy ultrasound probe and the medical examination is an ultrasound.

8. Device (10) according to any one of the preceding claims, characterized in that the dummy probe (14) is a dummy endoscopy probe and the medical examination is an endoscopy.

9. Device (10) according to any one of the preceding claims, characterized in that the device (10) comprises a dummy surgical tool (32) configured to be connected to the control unit (24), the virtual scene (101) comprising a virtual surgical tool (132) forming a virtual image (100) of the dummy surgical tool (32), the virtual surgical tool (132) being associated with the dummy surgical tool (32) by the virtual reality system (18) so that each movement of the dummy surgical tool (32) in real space is reproduced by the virtual surgical tool (132) in the virtual scene (101).

10. Device (10) according to the preceding claim, characterized in that the device (10) comprises a movable haptic arm (34) connected to the control unit (24) and configured to fix the dummy surgical tool (32), so as to generate a predetermined haptic feedback when the dummy surgical tool (32) cooperates with the interaction surface (26) of the anatomical console (12).

Images