Device and method for navigating a surgical instrument in an organ of a patient

Abstract

The invention relates to a device (1) and associated method (40) for navigating a surgical instrument in an organ of a patient by visualization, in an immersive virtual reality environment, of a region of an enriched 3D geometric model of the organ, the region being defined around a current virtual position of an end of the surgical instrument in the enriched 3D geometric model of the organ, the current virtual position corresponding to a current real position of localization of the end of the surgical instrument in the organ in a real reference frame.

Description

DEVICE AND METHOD FOR NAVIGATING A SURGICAL INSTRUMENT WITHIN A PATIENT'S ORGAN FIELD OF INVENTION

[0001] The present invention relates to the application of virtual reality in the medical field. More particularly, the present invention relates to a device and a method for navigating a surgical instrument within a patient's organ, via real-time visualization in an immersive virtual reality environment of a region of an enhanced 3D geometric model of the organ. STATE OF THE ART

[0002] Navigating a surgical instrument within a patient's organ is a complex and delicate operation requiring meticulous precision and extensive expertise from the interventional surgeon / physician, as exemplified by catheter ablations for treating cardiac arrhythmias. To navigate the surgical instrument within the patient's organ, the surgeon may proceed blindly, be guided by X-rays and a navigation system (e.g., 3D electromagnetic), or, more commonly, navigate using a camera attached to the surgical instrument.

[0003] However, these navigation approaches suffer from poor organ visibility (e.g., very limited image quality), the inaccessibility of certain organ areas, and other constraints that make precise localization of the surgical instrument relative to the patient's organ difficult, consequently increasing surgical risks that threaten the patient's health and limit the effectiveness of the intervention. Organ inaccessibility is not limited to geometric constraints: some organs, such as the heart, are inaccessible to conventional sensors (e.g., optical, magnetic). Furthermore, the use of direct imaging techniques, such as endoscopy, is practically impossible due to the constant blood flow, which prevents any direct visualization of the organ's interior.Beyond the reduced visibility due to the presence of blood, the introduction of a fiberscope into the heart remains an extremely invasive procedure, presenting a high risk of cardiac trauma.

[0004] The development of virtual reality (VR) models has enabled the creation of immersive, interactive virtual environments in which visualization, immersive navigation, and interaction with patient organs have become possible, especially compared to holograms, where interaction remains limited. Consequently, surgeons can now immerse themselves in a VR environment where they can interact with a 3D model of the patient's organ.

[0005] However, existing navigation systems do not support dynamic synchronization between a potentially moving organ and its corresponding virtual model, and do not provide real-time dynamic calibration, precise anatomical tracking, or surgical instrument-centric navigation interaction.

[0006] The question now arises of how to navigate surgical instruments within a patient's organ using a virtual reality (VR) environment, either in real time during the procedure or during its simulation in an environment replicating real-life conditions, such as a pulsating heart undergoing movements that simulate respiration. Current systems suffer from limitations related to the simultaneous localization of the surgical instrument within the organ and in the VR environment, as well as the spatial resolution of the organ's internal structures. Overcoming these challenges remains crucial in the field of surgery, particularly for improving surgical procedures in terms of time efficiency, manipulation precision, and targeting accuracy, and for making navigation intuitive.

[0007] The present invention aims primarily to solve the following challenges: firstly, to overcome the difficulties related to the precise navigation of the surgical instrument (e.g., catheter) in the patient's organ; secondly, to enable visualization in a real-time immersive virtual reality environment of the patient's organ and the exact position of the surgical instrument relative to the organ for optimal navigation ensuring high-precision surgical intervention; thirdly, to address specific constraints of navigation in a moving organ (e.g., a beating heart), including the direct inaccessibility of the organ, the impossibility of attaching sensors to it, and the dynamic deformations due to the physiological movements of the organ. SUMMARY

[0008] The present invention relates to a device for navigating a surgical instrument within a patient's organ, via visualization in an immersive virtual reality environment of a region of an enhanced 3D geometric model of said organ, said region being defined around a current virtual position of an end of said surgical instrument within said enhanced 3D geometric model of said organ, said current virtual position corresponding to a current real-world location of the end of said surgical instrument within said organ (i.e., a current real-world location of the end of said surgical instrument within said organ in a real-world reference frame), said device comprising: at least one input interface configured to receive: • a 3D geometric model of the patient's organ previously generated from at least one set of images, said 3D geometric model comprising a plurality of polygons forming a mesh representative of a structure of said organ, • the actual current location position of said tip of the surgical instrument in said organ (i.e., the actual current location position of said tip of the surgical instrument in said organ in the actual reference frame), said actual current location position being obtained via at least one position sensor, at least one processor configured to: • calculate a TM / E transformation between a reference frame associated with the 3D geometric model and a reference frame of the immersive virtual reality environment, • project (e.g., realign) the 3D geometric model of the patient's organ into the reference frame of the immersive virtual reality environment by applying the TM / E transformation, • For each polygon in the 3D geometric model, associate at least one visualization feature and at least one interaction feature to obtain an enriched 3D geometric model of said organ, • calculate a TI / E transformation between a reference frame of the current real-world location (i.e., the real reference frame) and the reference frame of the immersive virtual reality environment, and • project (e.g., recalibrate) said current real location position in the frame of reference of the immersive virtual reality environment by applying the TI / E transformation, so as to obtain the current virtual position of the tip of the surgical instrument in the frame of reference of the immersive virtual reality environment, • generate (e.g., obtain) at least one interaction scenario between said enriched 3D geometric model and the current virtual position of the tip of the surgical instrument in the immersive virtual reality environment, at least one output interface configured to provide F at least one interaction scenario between said enriched 3D geometric model and the current virtual position of the tip of the surgical instrument in the immersive virtual reality environment, said scenario including a current view of said region of said enriched 3D geometric model of said organ defined around said current virtual position of the tip of said surgical instrument in the immersive virtual reality environment.

[0009] Advantageously, the present invention provides valuable assistance in navigating a surgical instrument within a patient's organ by visualizing, in a virtual reality environment, a virtual representation of the organ and the surgical instrument. This assistance is particularly useful both during actual surgery on a patient and during surgical simulations for training purposes, which can be performed on a physical reproduction of the patient's organ, for example, using 3D printing. Furthermore, the invention relates to a device for generating scenarios in an immersive virtual reality environment (e.g.An interactive digital simulation that recreates a three-dimensional (3D) space so that users feel physically present in this virtual environment and can interact with this space as if they were actually in it, thanks to the use of various equipment and technologies. These scenarios stage a representation. A particularly precise and accurate representation of a patient's organ (e.g., an enhanced 3D geometric model) is created, allowing the user to interact with the organ's representation (e.g., manipulate the organ, zoom in on specific organ structures). Furthermore, these scenarios also incorporate location data for a surgical instrument intended to interact with the patient's organ. The scenarios thus function to faithfully simulate a real-world interaction (e.g., occurring in the real world, such as during surgery) between the patient's organ and a surgical instrument within an immersive virtual reality environment.

[0010] The simulation of these interactions has many applications, particularly before surgery, for example for planning or training, and during surgery for navigation assistance during surgical operations.

[0011] According to the invention, the patient's organ (e.g., in the real world) can be an organ of the patient's body (e.g., the heart), an artificial organ, or a 3D print of the enhanced 3D geometric model (i.e., a 3D geometric model printed using a 3D printer) of said patient's organ.

[0012] Advantageously, the device of the invention enables precise and efficient navigation of a surgical instrument within the patient's organ during surgery or training on an artificial organ, thanks to the visualization of an enhanced 3D geometric model of the patient's organ within an immersive virtual reality environment. With this device, it is possible to track the movement of the surgical instrument within the patient's organ in real time by visualizing the current virtual position of the instrument's tip in the reference frame of the immersive virtual reality environment, corresponding to the actual current position of said instrument tip within the patient's organ. It is important to note that the device advantageously allows navigation within a patient's specific organ, such as the heart, during a surgical procedure (e.g.(interventional cardiology procedure or catheter ablation of an arrhythmia), even when the catheter is subject to cyclic pulsatile movement. This is a significant advantage, since, due to blood flow, visual guidance using an endoscope or camera at the catheter tip is impossible under such conditions.

[0013] Furthermore, the generated (e.g., obtained) interaction scenario between the enriched 3D geometric model and the current virtual position of the surgical instrument allows surgeons to interact intuitively with the enriched 3D geometric model of the patient's organ, thus facilitating the planning and / or execution of a surgical procedure.

[0014] In addition, the enhanced 3D geometric model of the organ provides a detailed visual representation of the anatomical structure, enabling surgeons to have a better understanding of the organ's anatomy and a reproductive visualization of the organ's actual structure.

[0015] Furthermore, the device according to the invention allows for obtaining a detailed view of the organ's anatomy via an enhanced 3D geometric model, without requiring direct visual feedback (for example, by means of a camera) from inside the organ. Indeed, the position of the tip of the actual surgical instrument is associated in real time with a corresponding virtual position in the enhanced 3D geometric model of the organ.

[0016] In one embodiment, said current position of the surgical instrument is obtained via at least one sensor configured to detect a local variation of an electromagnetic field in which the organ is positioned.

[0017] In one embodiment, at least one input interface is further configured to receive a position of at least one fiducial marker located on said component, said position being obtained via a motion tracking device, said component having a real orientation in a real reference frame, at least one processor being further configured to: calculate a TF / E transformation between a reference frame of the position of at least one fiducial marker and said reference frame of the immersive virtual reality environment and record (e.g., register) said position of at least one fiducial marker in the immersive virtual reality environment (i.e., in the reference frame of the immersive virtual reality environment), register said enriched 3D geometric model with respect to the recorded position (i.e., registered position) of at least one fiducial marker so as to obtain a registered 3D geometric model (i.e.(enhanced 3D geometric model rejected), in. in which the recalibrated 3D geometric model is positioned in a virtual orientation corresponding to said real orientation of said organ in said real reference frame, update F at least one scenario of interaction with the recalibrated 3D geometric model.

[0018] In one embodiment, the 3D geometric model is pre-generated from at least one image set comprising a plurality of images, each image in the image set including at least one portion of the patient's organ. For example, the images were captured pre-operatively (i.e., before surgery) using at least one imaging modality.

[0019] In one embodiment, said region of the enhanced 3D geometric model of said organ corresponds to the entire organ.

[0020] In one embodiment, the current view has an origin point, the origin point being positioned at the current virtual position of the tip of the surgical instrument in the immersive virtual reality environment (i.e., in the frame of reference of the immersive virtual reality environment). The tip (or “distal tip”) of the surgical instrument is understood to be the terminal portion of the surgical instrument configured to interact with a target environment, such as an area of ​​an organ or tissue in a patient. In the case of a catheter, the tip refers specifically to the portion of the catheter designed to be inserted into the patient's body and / or to perform a specific function (e.g., drug delivery, signal measurement, local intervention, therapy delivery, application of physical energy).

[0021] In other words, the origin point can be likened to the "viewpoint" of a user observing the virtual scene. When the origin point is positioned at the current virtual position of the tip of the surgical instrument (in the enhanced 3D geometric model of the organ), the current view then corresponds to a view of an internal portion of the organ (the real organ).

[0022] In one embodiment, said current view has an origin point, said origin point being positioned in the reference frame of the immersive virtual reality environment, at a position external to said enhanced 3D geometric model of said organ.

[0023] In this scenario, the current view corresponds to a "viewpoint" of a user who would observe the organ as represented in the virtual scene from a remote position and would therefore see the organ in its entirety.

[0024] In one embodiment, at least one output interface F is further configured to display a marker representative of the tip of the surgical instrument at said current virtual position in the reference frame of the immersive virtual reality environment.

[0025] In one embodiment, at least one output interface F is further configured to display a marker representing the tip of the surgical instrument at said current virtual position in the reference frame of the immersive virtual reality environment, said representative marker being a three-dimensional portion located at the tip of the surgical instrument.

[0026] In one embodiment, said device is configured to update the current view of said region of the enhanced 3D geometric model of said organ defined around said current virtual position of the tip of said surgical instrument in the immersive virtual reality environment when the device receives a new current real position of location of said tip of the surgical instrument in said organ.

[0027] Advantageously, this allows the surgical instrument's progress within the organ to be tracked in real time as it is navigated.

[0028] In one embodiment, at least one processor is further configured to: receive a navigation instruction containing a future real position of the surgical instrument tip within the organ, compute a TI / E transformation between a reference frame of the future real position and the reference frame of the immersive virtual reality environment, and record (e.g., recalibrate) the future real position of the surgical instrument tip in the reference frame of the immersive virtual reality environment to obtain an updated virtual position of the tip. the surgical instrument in the reference frame of the immersive virtual reality environment, and at least one output interface is further configured to display an updated view of a region of the enriched 3D geometric model of said organ defined around said updated virtual position of the tip of said surgical instrument in the immersive virtual reality environment.

[0029] In one embodiment, said at least one processor is further configured to navigate the surgical instrument so that the tip of said surgical instrument is positioned at said future actual position in the organ.

[0030] In one embodiment, at least one processor is further configured to: receive coordinates of at least two points of the enhanced 3D geometric model of said organ, calculate at least one characteristic dimension of at least one element of said organ from said coordinates of at least two points, and at least one output interface is further configured to display said characteristic dimension of said organ in the immersive virtual reality environment.

[0031] In one embodiment, at least one output interface is further configured to simultaneously display the current view and an updated view in the immersive virtual reality environment.

[0032] Advantageously, the surgeon can thus have two views simultaneously and compare them to plan the next steps in the procedure.

[0033] In one embodiment, the device further comprises: navigating at least one surgical instrument in the printed 3D geometric model of the organ; at least one processor being further configured to: record (e.g., recalibrate) at least one relative position of at least one surgical instrument with respect to the printed 3D geometric model of the organ; • transmit the relative position of at least one surgical instrument with respect to the 3D printed model of the organ to at least one output interface; F at least one output interface being further configured to display in real time a view of the enriched 3D geometric model, the view of the enriched 3D geometric model corresponding to the relative position of F at least one surgical instrument with respect to the printed enriched 3D geometric model.

[0034] Advantageously, the printed 3D geometric model (e.g., printed 3D model) of the organ can be used to train a user (e.g., student, healthcare professional) to navigate a surgical instrument in the organ of a patient's body (i.e., living organ, e.g., a living organ targeted during surgery), by navigating a surgical instrument in the printed 3D geometric model.

[0035] In one embodiment, said organ is the heart.

[0036] In the embodiment in which the organ is the heart, the 3D geometric model of the patient's organ is generated (beforehand) from at least one set of images acquired in synchronization with a predefined instant of a physiological signal from the patient, said physiological signal being an electrocardiogram, and said predefined instant corresponding to the end of diastole. In this embodiment, the device is further configured to receive a real-time electrocardiogram from the patient. The at least one processor is further configured to receive, at an instant of the end of diastole, the current real-time position of the tip of the surgical instrument in the real frame of reference, and to realign said current real-time position in the frame of reference of the immersive virtual reality environment, by applying the TI / E transformation based on the information acquired at that predefined instant of said physiological signal (e.g., images of the organ and actual current location of the tip of the surgical instrument at the end of diastole).

[0037] In one embodiment, the reception of the current actual position of the tip of the surgical instrument takes place in real time, for example with a reception frequency of less than 1 Hz. In this embodiment, for each new received position, the processor implements the steps of calculating a TME transformation, realigning the 3D geometric model, associating visualization features with the polygons, calculating a TI / E transformation, generating (e.g., obtaining) at least one interaction scenario, and providing said interaction scenario. In other words, these steps are repeated iteratively by the processor to enable real-time tracking of the surgical instrument in the operating room and, in particular, within the organ.

[0038] The present invention further relates to a method for navigating a surgical instrument within a patient's organ, via visualization in an immersive virtual reality environment of a region of an enhanced 3D geometric model of said organ, said region being defined around a current virtual position of an end of said surgical instrument in the enhanced 3D geometric model of said organ, said current virtual position corresponding to a current real-world location of the end of said surgical instrument in said organ, said method comprising: receiving a 3D geometric model of the patient's organ previously generated from at least one set of images, said 3D geometric model comprising a plurality of polygons forming a mesh representative of a structure of said organ, receiving the current real-world location of said end of the surgical instrument in said organ.Given that the current real-time location is obtained via at least one position sensor, calculate a TM / E transformation between a reference frame associated with the 3D geometric model and a reference frame of the immersive virtual reality environment; project (e.g., re-register) the 3D geometric model of the patient's organ into the reference frame of the immersive virtual reality environment by applying the TM / E transformation; for each polygon of the 3D geometric model, associate at least one visualization feature and at least one interaction feature so as to obtain an enriched 3D geometric model of the organ. Calculate a TI / E transformation between a reference frame of the current real location position (i.e., the real reference frame) and the reference frame of the immersive virtual reality environment; project (e.g., recalibrate) said current real location position into the reference frame of the immersive virtual reality environment by applying the TI / E transformation, so as to obtain the current virtual position of the tip of the surgical instrument in the reference frame of the immersive virtual reality environment; generate (e.g., obtain) at least one interaction scenario between the enriched 3D geometric model and the current virtual position of the tip of the surgical instrument in the immersive virtual reality environment, provide F at least one interaction scenario between the enriched 3D geometric model and the current virtual position of the tip of the surgical instrument in the immersive virtual reality environment, said scenario including a current view of said region of the enriched 3D geometric model of said organ defined around said current virtual position of the tip of said surgical instrument in the immersive virtual reality environment.

[0039] The present invention further relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to implement the process described above according to any one of the embodiments.

[0040] The present invention further relates to a computer-readable (non-transient) recording medium comprising instructions which, when executed by a computer, enable the process described above to be implemented according to any one of the embodiments.

[0041] Such a non-transient, computer-readable recording medium may be, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any suitable combination thereof. It should be noted that the following examples, while more specific, are merely illustrative and not exhaustive, readily understood by those skilled in the art: a laptop diskette, a hard drive, a ROM, an EPROM (Erasable Programmable ROM) or a Flash memory, a portable CD-ROM (Compact-Disc ROM). DEFINITIONS

[0042] In the present invention, the terms below are defined as follows:

[0043] The term "processor" should not be interpreted as being limited to computer hardware capable of running software, and generally refers to a processing device, which may include, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). The processor may also encompass one or more graphics processing units (GPUs), whether used for computer graphics and image processing or other functions. Furthermore, the instructions and / or data required to execute the associated and / or resulting functionalities may be stored on any medium readable by the processor, such as, for example, an integrated circuit, a hard drive, a CD (Compact Disc), an optical disc such as a DVD (Digital Versatile Disc), RAM (Random-Access Memory), or ROM (Read-Only Memory).Instructions can be stored in computer hardware, software, firmware, or any combination thereof.

[0044] The terms "adapted" and "configured" are used in this description to broadly encompass initial configuration, subsequent adaptation or supplementation of the existing device, or any combination thereof, whether by hardware or software means (including firmware).

[0045] The term "immersive virtual reality environment" refers to a computer-generated virtual environment in which the user is immersed using technical devices that allow for real-time sensory and / or motor interaction with the environment, thereby inducing a sense of presence. Immersion is generally achieved through the use of virtual reality headsets, motion sensors, spatial audio systems, and / or haptic feedback devices.

[0046] The term "interaction scenario" refers to a structured sequence of events and / or interactions that can occur (i.e., be simulated) within the immersive virtual reality environment, and which the user can then repeat or use as information to continue the intervention or interaction with the real-world environment. An interaction scenario may include guided interactive sequences generated based on the position of the surgical instrument and the anatomical area traversed within the patient's organ, enabling dynamic and adaptive interaction. In other words, an interaction scenario can comprise a predefined and structured sequence of actions and events that can occur, and that the user can experience or repeat, within the immersive virtual reality environment.The interaction scenario can include what the user must do and in what order: interact with objects, trigger reactions, and respond to specific events. The interaction scenario can include a timeframe corresponding to each action that must occur and the associated conditions. Some interactions may depend on the user's profile or real-time behavior. For example, a predefined and structured action can constitute a fragment of an interaction scenario; an action can be triggered by the user (e.g., via a hand gesture by the user, such as pressing a button) or by the navigation device of the present invention and can lead to a subsequent navigation step. DESCRIPTION OF THE FIGURES

[0047] The present invention will be better understood, and other specific features and advantages will become apparent upon reading the following description of particular and non-restrictive embodiments, the description having reference to the accompanying drawings in which:

[0048] Figure 1 is a functional diagram schematically representing a particular mode of a device for navigating a surgical instrument into a patient's organ in accordance with this disclosure;

[0049] Figure 2 is a flowchart showing the steps of a process carried out with the device in Figure 1;

[0050] Figure 3 schematically represents a device integrating the functions of the device in Figure 1;

[0051] Figure 4 schematically represents an example of the application of the device in Figure 1, including the calculated transformations between the reference frames of the surgical instrument, the 3D geometric model, and the enriched 3D geometric model in the virtual reality environment; and

[0052] Figure 5 represents a particular embodiment of the device in Figure 1.

[0053] In the figures, the drawings are not to scale and identical or similar elements are designated by the same references. DETAILED DESCRIPTION

[0054] This description illustrates the principles of this disclosure. It will therefore be appreciated if a person skilled in the art is able to devise various arrangements that, while not explicitly described or shown here, embody the principles of the disclosure and are included within its scope.

[0055] All examples and conditional language cited here are intended for educational purposes to help the reader understand the principles of disclosure and the concepts brought by the inventor to advance the state of the art, and should be interpreted as not being limited to those specifically cited examples and conditions.

[0056] Furthermore, all statements of principles, aspects, and implementation methods of disclosure, along with their specific examples, are intended to encompass their structural and functional equivalents. Moreover, these equivalents are expected to include both currently known and future-developed equivalents—that is, all developed elements that fulfill the same function, regardless of their structure.

[0057] Thus, for example, a person skilled in the art will understand that the schematic diagrams presented here can represent conceptual views of illustrative circuits implementing the principles of disclosure. Similarly, it will be appreciated that all flowcharts, diagrams, and the like represent various processes that can essentially be represented on a computer-readable medium and thus executed by a computer or processor, whether or not that computer or processor is explicitly shown.

[0058] The functions of the various elements illustrated in the figures can be performed using dedicated computer hardware as well as computer hardware capable of running software in conjunction with appropriate software. When performed by a processor, the functions can be carried out by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

[0059] It is understood that the elements illustrated in the figures can be implemented in various forms of computer hardware, software, or combinations thereof. Preferably, these elements are implemented in a combination of computer hardware and software on one or more appropriately programmed multipurpose devices, which may include a processor, memory, and input / output interfaces.

[0060] This disclosure will be described with reference to a particular functional embodiment of a device 1 for navigating a surgical instrument 34 into an organ 31 of a patient 32 as illustrated in Figure 1 and Figure 4.

[0061] Navigation of a surgical instrument inside a patient's organ can be performed in real time, either during the actual surgical procedure on the real organ 31 of a patient's body, or during a simulation / manipulation of an artificial organ or a 3D printing of the enriched 3D geometric model 23 of the patient's organ 31 reproducing the real conditions of the patient's real organ 31.

[0062] According to the invention, organ 31 can correspond to the heart, a lung, a kidney, a blood vessel, the brain, the liver, the bile ducts, the colon, a bone (e.g., femur, clavicle, radius, etc.), a joint (e.g., knee, shoulder, elbow, ankle, etc.) etc. Organ 31 can be a moving organ (e.g., a beating heart, lung, liver). In one embodiment, organ 31 can be a moving organ with modelable cyclic changes in shape and volume; organ 31 can, in particular, undergo cyclic respiratory movements.

[0063] Device 1 is indeed suitable for navigating a surgical instrument 34 within an organ 31 of a patient 32, via visualization in an immersive virtual reality environment 33 of a region 39 of an enhanced 3D geometric model 23 of the organ 31. It should be noted that the present invention can also be implemented in a non-immersive virtual reality environment, i.e., in any interactive 3D simulation, including when viewed on a simple computer screen, without a headset or immersion device. In other words, a virtual reality environment 33 can be used, this term encompassing both a non-immersive and an immersive virtual reality environment.

[0064] According to the invention, region 39 of the enhanced 3D geometric model 23 of the organ 31 corresponds to a volume defined around a current virtual position 36 of an end of the surgical instrument 34 in the enhanced 3D geometric model 23 of the organ 31. The enhanced 3D geometric model 23 of the organ 31 may include enhanced areas corresponding to different elements associated with the organ 31 (e.g., pulmonary veins or atria / ventricles for the heart). The current virtual position 36 corresponds, in turn, to the current real position 22 of the end of the surgical instrument 34 in the real organ 31 (e.g., the patient's organ or a replica of the patient's organ).In other words, the current virtual position 36 is located in the frame of reference of the immersive virtual reality environment, while the current real position 22 is located in the real frame of reference Ri; these two positions correspond in the sense that they translate by the same relative position between a position of the surgical instrument and its associated frame of reference (e.g., the current virtual position 36 and the current real position 22 can be associated with relative coordinates of identical values, each of the two positions having its relative coordinates in the frame of reference associated with it).

[0065] The device 1 includes at least one input interface configured to receive 11 (via module 11): a 3D geometric model 21 of the patient's organ 31 32 previously generated from at least one set of images, and the actual current position 22 of the location of the tip of the surgical instrument 34 in the organ 31.

[0066] In addition, the device 1 includes at least one output interface configured to provide 18 at least one interaction scenario 50 between the enriched 3D geometric model 23 and the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33. The scenario includes a current view of the region 39 of the enriched 3D geometric model 23 of the organ 31 defined around the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33. In one example, providing 18 at least one interaction scenario 50 includes the generation of the current view by perspective projection (e.g., using a perspective projection algorithm).Advantageously, this allows the user to have a realistic and immersive visual representation of region 39 of the enriched 3D geometric model 23, taking into account the current virtual position 36 of the tip of the surgical instrument 34 which corresponds to the current real position 22 of the tip of the surgical instrument 34 in the actual organ 31 of the patient 32.

[0067] Device 1 can be a device, or a physical part of a device, designed, configured, and / or adapted to perform the aforementioned functions and produce the aforementioned effects or results. In alternative implementations, Device 1 is realized as a set of devices or physical parts of devices, whether grouped in the same machine or in different, possibly remote, machines. Device 1 may, for example, have functions distributed across a cloud infrastructure and be available to users as a cloud-based service, or have remote functions accessible via an API.

[0068] Device 1 can be integrated into the same device or set of devices, and intended for the same users.

[0069] In the disclosures that follow, modules should be understood as functional entities rather than as physically distinct hardware components. They can therefore be represented either as a single tangible component or distributed across several such components. Similarly, each of these modules may itself be shared among at least two physical components. Furthermore, modules are implemented as hardware, software, firmware, or any combination thereof. They are preferably incorporated into at least one processor of device 1.

[0070] As previously stated, the device 1 includes at least one input interface configured to receive 11 (i.e., via a receiving module 11): a 3D geometric model 21 of the patient's organ 31 and the current actual position 22 of the tip of the surgical instrument 34 within the organ 31. The 3D geometric model 21 and the current actual position 22 of the tip of the surgical instrument 34 can be stored in one or more local or remote databases 10. The latter can take the form of storage resources available from any suitable storage medium, which may include RAM or EEPROM (Electrically-Erasable Programmable Read-Only Memory), such as Flash memory, possibly within a SSD (Solid-State Disk).In some embodiments, the 3D geometric model 21 and the actual current position 22 of the location of the end of the surgical instrument 34 are received by a communication network.

[0071] In particular, the 3D geometric model 21 corresponds to a three-dimensional mathematical representation of the patient's organ 31 32. In one embodiment, it is generally composed of a set of polygons 38 that form a three-dimensional mesh. Each polygon 38 is defined by a set of points called vertices, connected to each other to form, for example, triangles, quadrilaterals, pentagons, etc.

[0072] The 3D geometric model 21 can be received directly as input to device 1 and be pre-generated, for example, via another device, or alternatively, device 1 can include an additional module for generating the 3D geometric model 21. In this case, the additional model generation module The 3D geometric model 21 can be configured to receive at least one set of images of organ 31 from patient 32, captured via at least one medical imaging modality, based on which the 3D geometric model 21 can be generated. The image set of organ 31 from patient 32 can be specific to patient 32. The image set of organ 31 from patient 32 can be enhanced by artificial intelligence tools so that the 3D geometric model 21 is an accurate morphological reproduction (e.g., a fast and annotated reproduction) of organ 31 from patient 32.

[0073] For example, the image set to generate the 3D geometric model 21 of patient 32 may come from computed tomography (CT), 3D ultrasound (i.e. ultrasound) or magnetic resonance imaging (MRI).

[0074] In one embodiment, the 3D geometric model 21 of the organ 31 of patient 32 is pre-generated from at least one set of images obtained in synchronization with a predefined time of a physiological signal of patient 32. The physiological signal may be an electrocardiogram or a respiratory signal (e.g., capnogram, spirogram, plethysmogram). The predefined time may be the end of diastole or the end of a respiratory cycle.

[0075] In one embodiment, the 3D geometric model 21 of patient 32 can be generated from at least two different imaging modalities, for example, from a set of images from a first imaging modality (e.g., CT) and a set of images from a second imaging modality (e.g., ultrasound). Advantageously, this embodiment makes it possible to obtain a 3D geometric model 21 with different spatial resolutions, thus obtaining a 3D geometric model 21 that combines different structural and anatomical properties made accessible by the different imaging modalities. Furthermore, the generated 3D geometric model 21 of patient 32, including the enriched areas corresponding to different elements associated with organ 31, will allow for the dynamic adaptation of interaction scenarios (and possible navigation aids) according to the clinically relevant regions for the surgical intervention.

[0076] Methods for segmenting the portions of organ 31 visible on medical images captured on patient 32 can be applied, such as, for example, Segmentation methods using thresholding or clustering (such as K-means clustering) or other suitable machine learning approaches (e.g., semi-automatic analysis algorithms, expert systems) can be employed. 3D reconstruction methods can then be implemented to obtain the 3D geometric model, such as contour interpolation, surface smoothing, or volumetric segmentation. Segmentation and / or reconstruction can be based on patient-specific clinical data.

[0077] Optionally, device 1 may include a preprocessing module for the image set(s) used to generate the 3D geometric model 21. This preprocessing module can be adapted to normalize the pixel / voxel intensities of the image(s) for efficient and reliable processing. Optionally, it can resample the image(s) to a predefined spatial resolution. This can improve the efficiency of downstream processing by the additional 3D geometric model generation module 21. Such normalization can be particularly useful when the image set(s) come from different sources, especially different imaging modalities. For example, the image set(s) can be normalized using Z-score normalization, histogram normalization, Min-Max normalization, or atlas normalization.In another example, the preprocessing module can be adapted to perform image processing to make anatomical structures of interest visible on the images (e.g., heart valves).

[0078] In one embodiment, the surgical instrument 34 may include an endoscope or fiberscope comprising a sheath into which a catheter is inserted for guiding at least one tool, such as, for example, a viewing camera, a light, a stent, a biopsy needle, or any other similar device. In other words, the surgical instrument 34 may include a sheath into which a catheter is inserted for guiding at least one tool, such as, for example, a viewing camera, a light, a stent, a biopsy needle, etc.

[0079] The surgical instrument 34 is detectable and designed for insertion via a vascular route, which can itself be modeled in a virtual reality environment to improve visualization and intervention planning. The surgical instrument 34 can be configured to perform complex movements, including advances, retractions, and precise rotations during navigation. Furthermore, it can be designed to withstand periodic movements related to the deformation of the patient's organ 31, for example, in the case of the heart, which is affected by respiratory movements. The surgical instrument 34 may include, at its tip, one or more electrodes capable of capturing a characterizable electrical signal at each contact between the electrode and the patient's organ 31. Advantageously, this enables precise and adaptive navigation, taking into account the patient's physiological dynamics.

[0080] In an embodiment where the organ is the heart, surgical instrument 34 is a catheter used in cardiac surgery (or interventional cardiology), consisting of a thin, flexible, sterile tube inserted into the patient's blood vessels to reach and treat internal structures of the heart without requiring opening the chest. This type of catheter may include, at its tip inserted into the heart, one or more measuring sensors, such as a pressure sensor in the heart chambers, a flow sensor, an intracardiac ECG sensor, or an impedance sensor. It may also include a position sensor allowing real-time tracking of the instrument tip by means of navigation systems, for example, electromagnetic ones.In addition, surgical instrument 34 may include one or more intervention tools, such as an ablation electrode, a biopsy or puncture needle, or any other similar device.

[0081] The actual current position 22 of the location of the tip of the surgical instrument 34 can be obtained via at least one position sensor 37. The position sensor 37 can be, for example, a magnetic, electromagnetic, optical, or ultrasonic sensor.

[0082] The position sensor 37 can be positioned on the surgical instrument 34, in particular on the sheath described above; in one example, at a distal end of the sheath (e.g., according to the invention, the distal end is defined with respect to the surgeon). such as the furthest end of the latter). In another example, the position sensor 37 is placed on one end of the catheter (see Figure 4). In one example, the actual localization position of the surgical instrument 22 is measured by an electromagnetic sensor integrated into the surgical instrument. Advantageously, the position sensor 37 operates without requiring direct contact with the patient's organ 31 32, which remains unvisualized with a camera and inaccessible to direct monitoring.

[0083] The current (real or virtual) position 22 of the location of the tip of the surgical instrument 34 can, for example, be defined by a set of Cartesian coordinates (x, y, z) and / or a set of angular coordinates (0 X , 0 y , 0 Z ) in space.

[0084] In one embodiment, the actual current position 22 can be obtained via at least one positioning sensor 37 (i.e., position sensor 37) configured to detect a local variation in an electromagnetic field in which the organ 31 is positioned. The positioning sensor 37 configured to detect a local variation in an electromagnetic field (i.e., electromagnetic positioning sensor or EM positioning sensor) can be placed in an electromagnetic field generated by an external transmitter or source. When the surgical instrument 34 moves, this disturbs the electromagnetic field around the EM positioning sensor 37. The EM positioning sensor 37 can capture this disturbance or local variation in the electromagnetic field as electromagnetic field variation data.Next, the electromagnetic field variation data can be processed by a data processing system to obtain the actual current location position 22.

[0085] The immersive virtual reality environment 33 and its parameters can be predefined and received as input by module 11. Alternatively, device 1 can include an additional module for defining the immersive virtual reality environment 33 and its parameters. Typically, the parameters of the immersive virtual reality environment 33 include the definition of a reference frame for the immersive virtual reality environment 33 (e.g., positioning of the origin and reference axes), environment boundaries (boundaries to prevent the user from exceeding the intended areas), a scale, and visual properties (ambient light, light sources, background) and sound properties (sound environment, sound sources), physical properties (gravity) etc.

[0086] Device 1 includes at least one processor adapted to perform various operations such as calculations, transformations, and projections (e.g., registrations). In what follows, the at least one processor will be presented in modules so that each operation is assigned to a specific module. The functions assigned to each module do not limit the functionality of the at least one processor, which can be adapted to perform functions other than those specified in the modules presented.

[0087] Device 1 further includes a module 12 for calculating a TM / E transformation between a reference frame RM associated with the 3D geometric model 21 and the reference frame RE of the immersive virtual reality environment 33 (see Figure 4). A reference frame is understood to be a coordinate system defining the position and orientation of an object in space. In other words, the 3D geometric model 21 has a first set of coordinates defining its position and orientation in the reference frame RM associated with said 3D geometric model 21; and the 3D geometric model 21 has a second set of coordinates defining its position and orientation in the reference frame RE of the immersive virtual reality environment 33. The first set of coordinates is, in principle, different from the second set of coordinates.The TM / E transformation allows the conversion of coordinates from one reference frame to another, for example, converting the first set of coordinates of the 3D geometric model 21 in the RM reference frame to the second set of coordinates of the 3D geometric model 21 in the RE reference frame of the immersive virtual reality environment 33. Typically, the TM / E transformation takes the form of a homogeneous 4x4 transformation matrix. This matrix includes information about the rotation and translation necessary to go from the RM reference frame of the 3D geometric model 21 to the RE reference frame of the immersive virtual reality environment 33.

[0088] Device 1 further includes a module 13 configured to project (e.g., register) the 3D geometric model 21 of said patient 32 into the RE frame of reference of the immersive virtual reality environment 33 RE by applying the TM / E transformation. Thus, the 3D geometric model 21 can be positioned in the environment of immersive virtual reality 33 in order to visualize the 3D geometric model 21 in the immersive virtual reality environment 33.

[0089] Device 1 further includes a module 14 for associating at least one visualization feature and associating at least one interaction feature with the 3D geometric model 21 so as to obtain an enriched 3D geometric model 23 of the organ 31, this for each polygon 38 of the 3D geometric model 21.

[0090] For example, a visualization feature (e.g., surface properties influence how polygons interact with light and the surrounding environment) can be at least one of the following features: a color, a transparency, a specularity, an emissivity, a texture (e.g., roughness, vascularization, granularity), a material.

[0091] An interaction feature (e.g., defining how users can interact with polygons 38 in the immersive virtual reality environment 33) can be at least one of the following: a mass (or weight), a selection feature (to select certain polygons), a feature to simulate a section (e.g., visualizing a surgical section in a specific direction / orientation). Furthermore, an interaction feature can allow users to rotate, pivot, translate, resize, and / or modify the enhanced 3D geometric model 23 of the organ 31.

[0092] An interaction feature can be associated with a specific anatomical area. For example, if polygons 38 are associated with a pulmonary vein or an isthmus, specific interaction scenarios can be generated / activated.

[0093] Additionally, polygons 38 can be annotated using metadata generated or received as input by device 1. For example, text or symbols providing contextual information can be attached to polygons 38.

[0094] The association of at least one interaction feature to the 3D geometric model 21 can be automatically triggered in a guided clinical scenario based on the actual current position 22 of the location of the tip of the surgical instrument 34 or of a surgical step reached.

[0095] The interaction features associated with the 3D geometric model 21 can be used during the execution of an interaction scenario 50 between said enriched 3D geometric model 23 and the current virtual position 36 of the tip of the surgical instrument, as described below.

[0096] Device 1 further includes a module 15 for calculating a TI / E transformation between a reference frame Ri of the surgical instrument 34 and the reference frame RE of the immersive virtual reality environment 33. In other words, the current real position 22 of the tip of the surgical instrument 34 has a first set of coordinates defining its position and orientation in the reference frame Ri associated with the surgical instrument 34; and the current real position 22 of the tip of the surgical instrument 34 has a second set of coordinates defining its position and orientation in the reference frame RE of the immersive virtual reality environment 33. The TI / E transformation allows the coordinates to be converted (e.g.(first set of coordinates) of the current real position 22 of the location of the tip of the surgical instrument 34 in the organ 31 in the reference frame Ri of the surgical instrument 34 in coordinates (e.g., second set of coordinates) of the location of the tip of the surgical instrument 34 in the reference frame RE of the immersive virtual reality environment 33. To facilitate understanding of this description, the location coordinates of the tip of the surgical instrument 34 in the reference frame RE of the immersive virtual reality environment 33 are called the "current virtual position 36".

[0097] Device 1 further includes a module 16 for projecting (e.g., recalibrating) the actual current position 22 of the localization in the immersive virtual reality environment 33 by applying the TI / E transformation, so as to obtain the actual virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33. Thus, a correspondence is established between the actual virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33 and the actual current position 22 of the localization of the tip of the surgical instrument 34 in reality. This can be advantageous for visualizing, for example, the actual virtual position 36 of the tip of the surgical instrument 34. in the immersive virtual reality environment 33 simultaneously with the enhanced 3D geometric model 23 of the organ 31. In some cases, it may be advantageous for the surgeon to visualize in real time the current virtual position 36 of the tip of the surgical instrument 34 relative to the enhanced 3D geometric model 23 in order to place the surgical instrument 34 in a precise location and perform a surgical procedure. The registration of the current real position 22 of the tip of the surgical instrument 34 may include a three-dimensional portion (e.g., between 0.5 and 3 cm) located at the tip of the surgical instrument 34, allowing for better visualization in the immersive virtual reality environment 33 of the position and orientation of the distal portion of the surgical instrument 34 with respect to the patient's organ 31 32.For example, in the case of a catheter used for cardiac interventions, this real-time positioning of the surgical instrument tip would allow cardiologists to see in real time the exact position and orientation of the distal portion of the catheter tip inside the patient's heart. This is particularly useful for navigating cardiac chambers and blood vessels, where the catheter's precision and flexibility are crucial for reaching specific areas, such as heart valves or coronary arteries. With this enhanced visualization, cardiologists can precisely adjust the catheter's position and angle, thereby reducing the risk of damage to cardiac tissue and improving the accuracy of the procedure, such as during angioplasty or catheter ablation.

[0098] In one embodiment, module 16 allows the current real position 22 to be recalibrated in the RE reference frame of the immersive virtual reality environment by applying the TI / E transformation on the basis of a predefined instant (e.g., instant T0 of end of diastole, instant of end of a respiratory cycle) of a physiological signal (e.g., electrocardiogram, capnogram, plethysmogram, spirogram).

[0099] In one example, organ 31 of patient 32 is the patient's heart, and the predefined time is T0, which indicates the end of diastole. This T0 can be defined in synchronization with the acquisition time of the image set used to generate the 3D geometric model 21 of organ 31. For example, each image in the image set may have been acquired at the same time (specifically at T0) or synchronized with the patient's electrocardiogram (ECG), for example, at the 70% point of the cardiac cycle. The predefined time may be defined according to the patient's respiratory cycle, for example, at the end of inspiration. The predefined time may be defined, for example, by a physician, such as a radiologist, cardiologist, or electrophysiologist.

[0100] In one embodiment, module 16 allows the current real position 22 to be recalibrated on the basis of one or more predefined times of different physiological signals (e.g., an ECG and a plethysmogram).

[0101] Device 1 further includes a module 17 for generating (e.g., obtaining) at least one interaction scenario 50 between the enriched 3D geometric model 23 and the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33. Obtaining an interaction scenario 50 can be generated or triggered once an anatomical area is reached (e.g., arrival in proximity to the orifice of the left auricle, arrival in proximity to the ostia of the coronary sinus) which allows dynamic assistance to the gesture of the user (e.g., electrophysiologist) during the intervention. According to the invention, an interaction scenario 50 comprises a structured sequence of events (e.g., steps) and / or interactions that can take place (i.e., be simulated) in the immersive virtual reality environment 33, and that the user could then repeat, or use as information, enabling them to continue the intervention.An interaction scenario 50 may include guided interactive sequences, generated according to the position of the surgical instrument and an anatomical area traversed by the organ 31 of the patient 32, which allows for dynamic and adaptive interaction.

[0102] In one embodiment, an interaction scenario 50 can be obtained based on predefined rules. In one example, when a user explores the enhanced 3D geometric model 23 and focuses on a target area (e.g., the mitral valve of the heart), a predefined rule can automatically trigger an interaction scenario comprising a zoom on the target area (e.g., the first action of the interaction scenario) followed by the display of an information window containing anatomical and clinical data relating to the target area (e.g., the second action of the interaction scenario). In another For example, an interaction scenario 50 (e.g., for navigation training) may include displaying in the immersive virtual reality environment an instruction prompting the user to spot an anomaly (e.g., first action of the interaction scenario); if the user correctly identifies the anomaly, a predefined rule validates the user's action (e.g., second action of the interaction scenario) and displays a contextual medical explanation (e.g., third action of the interaction scenario).

[0103] In an example where the organ is the heart and an anatomical area includes a pulmonary vein, an interaction scenario 50 might involve exploring a region of the heart near the pulmonary vein. This scenario would include a series of actions configured to display a colored highlight around a region of the enhanced 3D geometric model 23, trigger an alert if a predefined critical zone is approached, and display a safety zone to prevent unintentional injury. To implement this interaction scenario 50, interaction characteristics are defined in relation to the enhanced 3D geometric model 23, such as displaying a colored highlight around the ostium (e.g., for polygons corresponding to the ostium in the enhanced 3D geometric model), and triggering a visual or audible alert if the critical zone is approached (e.g.,(e.g., for polygons corresponding to the critical area in the enhanced 3D geometric model), a visualization of a safety zone to prevent unintentional injury (e.g., for polygons corresponding to the safety zone in the enhanced 3D geometric model). In an example where the anatomical area corresponds to the left atrium, an interaction feature could be: a visual guidance element towards the pulmonary veins (e.g., an arrow or a light halo) when approaching a vein targeted for ablation, a temporary spatial marker anchored to a given position for easy return if needed.

[0104] For example, an interaction scenario may include a joint analysis interaction of the properties of a local electrical signal from a specific anatomical area of ​​the patient's actual organ (such as duration, amplitude, morphology, fragmentation, and activation time), by referring them to an electrophysiological reference (e.g., a surface ECG or an intracardiac signal).

[0105] For example, an interaction scenario 50 might include a sequence of steps enabling the precise navigation of the surgical instrument to a predefined area of ​​the patient's organ 31, visualized in the immersive virtual reality environment via the enhanced 3D geometric model. One step of the interaction scenario 50 might include real-time visual / auditory / haptic feedback to the user to validate the effectiveness of the intervention; once the user has received this feedback, they can, for example, stimulate the target area in the real environment.

[0106] For example, an interaction scenario 50 might include a switching step, following the reception of a navigation instruction, between an internal and an external view of the enhanced 3D geometric model 23 of organ 31 of patient 32. The navigation instruction could be a voice command. This interaction scenario 50 could, for example, be triggered by receiving a voice command, which advantageously allows the user to navigate easily within organ 31 without having to use their hands to trigger a navigation instruction (e.g., changing views, validating a target area).

[0107] An interaction scenario might include a step of annotating lesion area(s) on the real-time, enriched 3D geometric model; based on these annotations, the user can trigger precise ablations in the patient's actual organ. Advantageously, annotating lesion areas thus facilitates more precise planning and execution of the surgical intervention (e.g., memorizing treatment application sites).

[0108] Advantageously, these scenarios allow users to immerse themselves in interactive simulations where they can explore, interact, and learn in a safe and controlled environment. Multiple scenarios can be generated during a single surgical procedure.

[0109] For example, an interaction scenario can be advantageous for simulating a surgical procedure for the purpose of training a professional, for surgical planning, or for optimizing the use of surgical instruments.34 The interaction scenario can be used advantageously during an operation on a patient to guide the user's actions in real time.

[0110] An interaction scenario can be an exploration, targeting, and / or validation scenario. For example, an exploration scenario might allow the physician to navigate and examine patient 32's organ 31 in detail. A targeting scenario might focus on a specific area of ​​patient 32's organ 31, such as the cavotricuspid isthmus or the antral area of ​​the pulmonary veins. A validation scenario, on the other hand, might be used to evaluate the contact force of the surgical instrument; in one example, the contact force is represented on the enhanced 3D geometric model. An interaction scenario can be contextualized based on patient 32's organ 31 and the current real / virtual position of the surgical instrument tip. [OR I] In one embodiment, the user selects the type of interaction scenario to generate.

[0112] The device 1 further includes a module 18 (i.e., output interface) to provide F at least one interaction scenario 50 between the enriched 3D geometric model 23 and the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33. The interaction scenario 50 may include a current view of the region 39 of the enriched 3D geometric model 23 of the organ 31 defined around the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33.

[0113] Depending on the user's needs, the region 39 of the enhanced 3D geometric model 23 of organ 31 displayed in the scenario can vary. The user can also switch between different views or display multiple views simultaneously (e.g., a first view in the virtual scene and a second view in a floating window displayed within the virtual scene). For example, the region 39 of the enhanced 3D geometric model 23 of organ 31 could correspond to the entire organ 31. This could be useful, for instance, for a surgeon interested in a comprehensive view of the entire organ 31 with which they can interact (e.g., rotate it).

[0114] Alternatively, region 39 of the enriched 3D geometric model 23 of organ 31 may correspond to an external or internal part of organ 31. For example, the surgeon may be interested in visualizing the detail of a specific region 39 of organ 31, such as a potential area where surgery may take place or an area containing an element of interest of organ 31 (tumor, blood vessel, etc.).

[0115] In certain cases, particularly when it is useful to observe / anticipate the progression of the surgical instrument 34 into / towards the organ 31, at least one output interface of the device 1 can be further configured, for example within an exploration interaction scenario, to display a marker representing the tip of the surgical instrument 34 at the current virtual position 36 in the RE frame of reference of the immersive virtual reality environment 33. In other words, the marker representing the tip of the surgical instrument 34 can be a 3D representation (e.g., a 3D model or a digital twin) of the surgical instrument 34 in the immersive virtual reality environment 33; this representative marker can be: a three-dimensional portion (e.g., of approximately 1 cm) located at the end of the surgical instrument 34 allowing better visualization in the immersive virtual reality environment 33 of the position and orientation of the distal portion of the surgical instrument 34 with respect to the organ 31 of the patient 32; or a complete representation of the surgical instrument 34. Advantageously, the display of this marker allows real-time visualization of the orientation in space of the surgical instrument 34 and in particular of its end, which helps the user to guide the surgical gesture more precisely in real space (i.e. real environment).

[0116] The invention is particularly applicable to the navigation of a surgical tool 34 within an organ 31, whether real (e.g., during a surgical operation or an observational operation such as a colonoscopy) or a simple reproduction (for training or planning an operation). In this scenario, it is particularly advantageous for the user to be able to observe, within the immersive virtual reality environment 33, the virtual surgical instrument moving simultaneously or almost simultaneously with the real surgical instrument within the organ. real. Thus, device 1 can be configured to update the current view of region 39 of the enriched 3D geometric model 23 of organ 31 in the immersive virtual reality environment 33 as soon as the device receives a new real current position 22 of the location of the tip of the surgical instrument 34 in the organ 31. Therefore, device 1 can allow an update of the current view of region 39 of the enriched 3D geometric model 23 according to the displacement of the tip of the surgical instrument 34 in the organ 31.

[0117] The invention also allows for the planning of several navigation strategies for the surgical instrument 34. Since the enhanced 3D geometric model 23 is a particularly accurate representation of the patient's organ 32, the user can thus plan the movement of the surgical instrument 34 within the enhanced 3D geometric model 23, while being assured that the corresponding movement in the patient's actual organ 32 will be identical. In the case of an organ that cannot be visualized with a camera and is difficult to access, such as the heart, this planning can be based on an enhanced 3D geometric model dynamically registered with the actual anatomy (i.e., the patient's actual organ 32), without the need for a marker placed on the heart; this ensures that navigation strategies within the enhanced 3D geometric model 23 (e.g., digital twin) remain relevant in the real environment.

[0118] In particular, at least one processor can be configured to: - receive a navigation instruction including a future actual position for locating the tip of the surgical instrument 34 in the organ 31 (e.g., different navigation strategies), - calculate a TI / E transformation between a reference frame Ri of the future real location position and the reference frame RE of the immersive virtual reality environment, and - record (e.g., recalibrate) the future actual location position of the tip of the surgical instrument 34 in the RE reference frame of the immersive virtual reality environment to obtain an updated virtual position of the tip of the surgical instrument 34 in the RE reference frame of the immersive virtual reality environment 33, and at least one output interface can be further configured to display an updated view of a region 39 of the enriched 3D geometric model 23 of the organ 31 defined around the updated virtual position of the tip of the surgical instrument 34 in the immersive virtual reality environment 33.

[0119] For example, the navigation instruction F could be a command to translate, rotate, tilt, orient, and / or pivot the surgical instrument 34 within the organ 31. The received navigation instruction, containing a future real-world position of the tip of the surgical instrument 34 within the organ 31, can thus update the virtual position of the tip of the surgical instrument 34 in the RE frame of the immersive virtual reality environment 33 and update the view of region 39 of the enriched 3D geometric model 23 based on the navigation instruction. Therefore, the updated view corresponds to a view from the future real-world position of the tip of the surgical instrument 34 within the organ 31. This updated view can guide the surgeon in navigating within the organ 31 to choose which direction to navigate next and reach a specific target area.For example, based on this updated view, the surgeon can decide whether or not to navigate to the actual future position. In other words, an interaction scenario might involve displaying an updated view of a region 39 of the enhanced 3D geometric model 23, the updated view corresponding to a future actual position of the tip of the surgical instrument 34 in the organ 31 (the patient's actual organ). This interaction scenario could be generated or triggered, for example, by a voice command as a navigation instruction. For instance, the interaction scenario during the real-time procedure might involve virtually simulating the advancement of a catheter by detaching a "virtual camera" from its tip; it is as if a camera has retrieved a future view of the actual future position of the tip of the surgical instrument 34 in the organ 31.This allows virtual navigation within a target area of ​​the enhanced 3D geometric model 23 in order to explore it before physically accessing it (in the real environment of organ 31). After predicting the most optimized navigation strategy, the processor can then. Navigate the surgical instrument 34 so that its tip is positioned at its future actual position in the organ 31. For example, in the case of a moving organ 31 (e.g., heart), real-time dynamic registration (see Module 20) during surgery is necessary to ensure that the enhanced 3D geometric model 23 remains constantly synchronized and aligned with the position and orientation of the patient's actual organ 31. Real-time dynamic registration can be achieved using spatiotemporal registration algorithms or real-time non-rigid registration algorithms, such as registration based on deformable models (e.g., "Demons algorithm," "B-spline Freeform Deformation," "Adaptive mesh warping," etc.), registration with statistical / machine learning models (e.g., neural networks trained on dynamic image databases) or registration with temporal (spatio-temporal) constraints.

[0120] Various characteristic dimension measurements can also be performed on the enriched 3D geometric model 23. To do this, at least one processor can be configured to: receive coordinates of at least two points of the enriched 3D geometric model 23 of the organ 31, calculate at least one characteristic dimension of at least one element of the organ 31 from the coordinates of the at least two points, and at least one output interface can be further configured to display the characteristic dimension of the organ 31 in the immersive virtual reality environment 33.

[0121] The points (two or more) of the enhanced 3D geometric model 23 can be selected by the surgeon (or operator) using a VR controller, a VR hand tracking tool, a laser pointer, voice commands, or a graphical interface tool. Thus, the surgeon can choose the points of interest from the enhanced 3D geometric model 23. The coordinates of at least two points of the enhanced 3D geometric model 23 can be Cartesian coordinates (x, y, z) expressed in the coordinate system of the RE reference frame of the immersive virtual reality environment 33. Based on the selected points of the enhanced 3D geometric model 23, the The processor can calculate a characteristic dimension of an element of organ 31 (e.g., pulmonary veins, cavotricuspid isthmus, aortic annulus, atria of the heart). It will be clear to a person skilled in the art that the elements of organ 31 correspond to enriched regions in the enriched 3D geometric model 23.

[0122] An element of organ 31 can be any structure that is part of, located on, and / or within organ 31. For example, an element of organ 31 can be a heart valve, a blood vessel, or a heart chamber.

[0123] A characteristic dimension can be, for example: a length: e.g., the length of a blood vessel or the length of a segment of organ 31; a width: e.g., the distance between the furthest edges of the two cardiac ventricles; a thickness: e.g., the thickness of a blood vessel or of cardiac muscle; a volume: e.g., the volume of a cardiac cavity; an angle: e.g., the angle formed between the longitudinal axis of the heart and the axis of the arterial trunk to evaluate the position and orientation of the heart in the thoracic cavity; a circumference: e.g., the circumference of the aortic valve.

[0124] Advantageously, a characteristic dimension can, for example, help the surgeon to define a target area for surgery or to define a surgical intervention protocol.

[0125] In one embodiment, the organ 31 of patient 32 is an organ 31 of a body of patient 32, an artificial organ 31 or a 3D print of the enhanced 3D geometric model 23 of the organ 31 of patient 32.

[0126] For example, if organ 31 of patient 32 is an organ 31 of a body of patient 32, this can allow the surgeon to navigate the surgical instrument 34 into the organ 31 of a body of patient 32 and simultaneously see a view of the inside of the enriched 3D geometric model 23 of organ 31 of patient 32 which corresponds to the actual current position 22 of the tip of the surgical instrument 34 in organ 31. Thus, the enriched 3D geometric model 23 can allow a detailed view of the internal structure of organ 31 of a patient's body 32 in real time.

[0127] In another example, where organ 31 of patient 32 is a 3D print of the enhanced 3D geometric model 23 of organ 31 of patient 32 (i.e., a printed 3D model), organ 31 of patient 32 can be used for training purposes. Advantageously, the printed 3D model is patient-specific. A student can practice navigating organ 31 of patient 32 by navigating a surgical instrument 34 within the printed 3D model. In one example, the printed 3D model can be pulsatile and undergo movements simulating respiration.

[0128] On the other hand, if the organ 31 of patient 32 is an artificial organ 31 or a 3D-printed model of the organ 31 of patient 32, the surgeon can use it to plan a surgical procedure. For example, the surgeon can navigate a surgical instrument 34 inside the organ 31 of patient 32. During navigation, the surgeon can record, using at least one processor, points of interest for the tip of the surgical instrument 34. Then, during the surgical procedure, the surgeon can navigate the surgical instrument 34 to the recorded points of interest. This can improve navigation by performing a faster and more efficient surgical procedure, potentially reducing the risk to the health of patient 32.

[0129] The device 1 can also receive, via module 11, a position of at least one fiduciary marker 35 (optionally located on the organ 31), the position being obtained via a motion tracking device, and the organ 31 having a real orientation in the real reference frame Ri.

[0130] A fiduciary marker 35 can be a visual or physical landmark used as a fixed reference point for navigation. For example, a fiduciary marker 35 can be an optical marker (e.g., a QR code detectable by a camera), an infrared marker trackable by an infrared sensor, or a physical marker such as a reflective sphere. A fiduciary marker 35 can be an external reference point not located on the patient's organ 31 32.

[0131] Device 1 may further include a module 19 for calculating a TF / E transformation between an RF reference frame of the position of a fiduciary marker 35 and the RE reference frame of the immersive virtual reality environment 33 and for recalibrating the position of the fiduciary marker 35 in the immersive virtual reality environment 33.

[0132] Module 19 can operate in parallel with modules 12, 13, 14 and / or 15, or perform its function independently, at any time in relation to the functions performed by the other modules.

[0133] The device 1 may further include a module 20 for registering the enriched 3D geometric model 23 with respect to the registered position of the fiducial marker 35 so as to obtain a registered enriched 3D geometric model positioned in a virtual orientation corresponding to the actual orientation of the organ 31 in the real reference frame Ri. The registration can be performed using one or more reference points, such as patches on the rib cage or fluoroscopic contours, as well as hybrid methods of rigid registration and ICP (Iterative Closest Point). The registration can also use the current real position 22 of the tip of the surgical instrument 34. This registration can be performed continuously throughout a surgical procedure on the organ 31 of the patient 32; it is important to note that this is advantageous in the case of an organ that is physically inaccessible for placing markers on it (e.g., heart) which allows indirect access to this organ.

[0134] The enhanced 3D geometric model 23 can thus be dynamically realigned in real time, based on changes in the organ's position in the real reference frame Ri. This is essential when the organ is not visible with a camera, is mobile (e.g., due to respiratory and / or cardiac movements), and cannot be continuously observed, as is the case for the heart. This allows for precise correspondence with the patient's anatomical reality without requiring a direct sensor placed on the organ, ensuring the device's reliability 1.

[0135] Module 20 can operate in parallel with module(s) 15 and / or 16, or execute its functions independently, at any time relative to the functions performed by the other modules.

[0136] Device 1 may also include a module 21 to update the interaction scenario 50 with the recalibrated, enhanced 3D geometric model. For example, module 21 may operate after module 17. For instance, for a moving organ (e.g., heart), this update advantageously maintains continuous, real-time synchronization between the enhanced 3D geometric model and the corresponding real organ of the moving patient in the real frame of reference, the real organ itself being inaccessible during the operation for anatomical examination.

[0137] The fiduciary marker 35 can optionally be placed on organ 31 (in the actual environment of organ 31 associated with the actual reference frame), and its position can be obtained by a motion tracking device. For example, the fiduciary marker 35 could be a patch placed on the patient's rib cage, or a fluoroscopic outline.

[0138] In one example, the real environment of organ 31 is the operating room. The real environment of organ 31 can be associated with the real reference frame Ri. The use of a fiducial marker 35 (to which an RF reference frame of the position of F can be associated with at least one fiducial marker) is important for accurately determining the position and orientation of organ 31 in the RE reference frame of the immersive virtual reality environment 33 relative to the position and orientation of the enriched 3D geometric model 23 in the RM reference frame of the 3D geometric model 21. The enriched 3D geometric model 23 can thus be registered with respect to the recorded position of the fiducial marker 35 so as to obtain a registered 3D geometric model, in which the registered enriched 3D geometric model is positioned in a virtual orientation corresponding to the real orientation of organ 31 in the real reference frame Ri.Next, interaction scenario 50 can be updated with the recalibrated enriched 3D geometric model. Advantageously, updating interaction scenario 50 with the recalibrated enriched 3D geometric model as described above provides a current view of region 39 of the enriched 3D geometric model 23 of organ 31, which corresponds to the actual orientation of organ 31 in a real reference frame.

[0139] In other words, one or more fiduciary markers 35 may be located on the patient's organ or on a part of the patient's body (e.g., chest, rib cage) thoracic, back, abdomen) enabling real-time synchronization between the patient's actual organ and the organ's enhanced 3D geometric model 23. For example, to synchronize the patient's actual organ (e.g., the heart during an operation) with the organ's enhanced 3D geometric model 23 in real time, one or more fiducial markers 35 can be used. These fiducial markers 35 allow the organ's movements to be tracked, even when it is not visible with a camera, inaccessible, or moving due to respiratory and cardiac movements. Advantageously, the enhanced 3D geometric model 23 is thus constantly updated to reflect the patient's actual anatomy. The heart, for example, undergoes external movements due to the periodic and traceable movements of the diaphragms during the respiratory cycle, and its position can vary relative to neighboring structures depending on the patient's position.Furthermore, the heart undergoes periodic changes in shape and volume that can be modeled during the cardiac cycle (e.g., systole, diastole). This is particularly useful in complex surgical / interventional situations where precision is crucial. In addition, this approach allows for continuous tracking of the movements and shape changes of the patient's organ, providing a better understanding and more precise control of the medical intervention, thus ensuring its safety and efficacy.

[0140] Device 1 may further include a speech recognition module 110 to interpret voice commands (e.g., navigation instructions) during surgery, to enable the user to control an interaction scenario, change a current view, interact with the enriched 3D geometric model or validate a navigation step, reducing the user's manual interaction with Device 1 and thus making Device 1 easier to use.

[0141] The device 1 may further include an interface module 120 (“middleware”) to retrieve (e.g., receive) position data (e.g., the current real position 22 of the tip of the surgical instrument 34 in the organ 31 in the real reference frame) emitted by a third-party motion capture system (e.g., position sensor 37 and associated modules: acquisition and processing modules). data, communication modules), and convert this position data into a reference frame (e.g., the reference frame of the immersive virtual reality environment, the reference frame associated with the 3D geometric model) usable by device 1 without requiring modification of the motion capture system (see Figure 5). The third-party motion capture system (i.e., motion tracking system) can be present in an operating environment (e.g., operating room, navigation training room). The motion capture system includes a system control unit, a sensor control unit, and a magnetic field generator.

[0142] In other words, the 120 interface module can perform the following steps: - retrieve the raw position / orientation data (i.e. position data) emitted by the third-party motion capture system; - dynamically convert the raw position / orientation data into the reference frame of the immersive virtual reality (VR) environment (e.g., calculate a TI / E transformation between a reference frame Ri of the current real position 22 and the reference frame RE of the immersive virtual reality environment 33, and recalibrate the current real position 22 in the reference frame RE of the immersive virtual reality environment 33 by applying the TI / E transformation, so as to obtain the current virtual position 36 of the tip of the surgical instrument 34 in the reference frame RE of the immersive virtual reality environment 33); - synchronize the current virtual position 36 of the end of the surgical instrument 34 with the recalibrated enriched 3D geometric model 23 (or the recalibrated enriched 3D geometric model) and the current interactive scenario(s).

[0143] Advantageously, the interface module 120 provides device 1 with an interoperability advantage, allowing its use in the operating environment without modification of existing hardware (e.g., surgical instrument 34, position sensor 37). Furthermore, this interface module 120 facilitates quick and easy integration of device 1, ensures seamless operation of device 1, and enables automatic control of the enhanced 3D geometric model 23 from heterogeneous systems.

[0144] Device 1 may also include a recording module 130 for the navigation steps performed by the other modules of Device 1 and intraoperative data from the immersive virtual reality environment (i.e., a navigation history recording module). The recording module 130 allows for the intraoperative recording of a navigation history including the path taken by the surgical instrument, variations in the enhanced 3D geometric model 23, the different views displayed, model annotations, and interaction scenarios with the model.Advantageously, this navigation history can be used for postoperative review for medical analysis, for planning subsequent navigation, for personalized patient simulation, for self-adaptation of the device through machine learning, for user training on navigation (educational use), and for training a machine learning model (e.g., for pattern detection, for optimizing interaction scenarios). Navigation history data can be stored locally or in a secure remote database.

[0145] The speech recognition module 110, the interface module 120 and the recording module 130 can operate in parallel with the other modules of device 1, or perform their function(s) independently, at any time in relation to the functions performed by the other modules.

[0146] Device 1 can, for example, perform the following method 40 (Figure 2) for navigating a surgical instrument 34 in an organ 31 of a patient 32, via visualization in an immersive virtual reality environment 33 of a region 39 of an enriched 3D geometric model 23 of the organ 31, the region 39 being defined around a current virtual position 36 of an end of the surgical instrument 34 in the enriched 3D geometric model 23 of the organ 31, the current virtual position 36 corresponding to a current real position 22 of the location of the end of the surgical instrument 34 in the organ 31, the method 40 comprising: o receiving the 3D geometric model 21 of the organ 31 of the patient 32 previously generated from at least one set of images (step 41), o receiving the current real position 22 of the location of the end of the surgical instrument 34 in organ 31 (step 41), o Calculate a TM / E transformation between a reference frame RM associated with the 3D geometric model and a reference frame RE of the immersive virtual reality environment 33 (step 42), o Project (e.g., register) the 3D geometric model 21 of the organ 31 of patient 32 into the reference frame RE of the immersive virtual reality environment 33 by applying the TM / E transformation (step 43), o For each polygon 38 of the 3D geometric model 21, associate at least one visualization feature and at least one interaction feature so as to obtain an enriched 3D geometric model 23 of the organ 31 (step 44), o Calculate a TI / E transformation between a reference frame Ri of the current real position 22 of localization and the reference frame RE of the immersive virtual reality environment 33 (step 45), o Project (e.g., recalibrate) said current real position 22 of location in the RE reference frame of the immersive virtual reality environment 33 by applying the TI E transformation, so as to obtain the current virtual position 36 of the tip of the surgical instrument 34 in the RE reference frame of the immersive virtual reality environment 33 (step 46), o generate at least one interaction scenario 50 between the enriched 3D geometric model 23 and the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33 (step 47), o provide F at least one interaction scenario 50, the scenario comprising a current view of the region 39 of the enriched 3D geometric model 23 of the organ 31 defined around the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33 (step 48).

[0147] A particular device 9, visible in Figure 3, implements the device 1 described above. It corresponds, for example, to a workstation, a laptop, a tablet, a smartphone, or a "head-mounted display" (HMD) such as a virtual reality (VR) headset, an augmented reality (AR) headset, or a mixed reality headset.

[0148] This device 9 is adapted to provide at least one interaction scenario 50 between the enriched 3D geometric model 23 and the current virtual position 36 of the tip of the surgical instrument 34 in the immersive virtual reality environment 33. It comprises the following elements, linked to each other by an address and data bus 95 which also carries a clock signal: - a 91 microprocessor (or CPU); - a graphics card 92 comprising several graphics processing units (or GPUs) 920 and graphics random access memory (GRAM) 921; - a non-volatile memory of type ROM 96; - 97 RAM; - one or more input / output (I / O) devices 94 such as, for example, a keyboard, a mouse, a trackball, a webcam; other methods of inputting commands such as speech recognition are also possible; - a power source 98; and - a radio frequency unit 99.

[0149] In one embodiment, the power supply 98 is external to the device 9.

[0150] Device 9 may also include the speech recognition module 110, the interface module 120 and the recording module 130.

[0151] Device 9 also includes a display device 93, a type of display screen directly connected to the graphics card 92, to display synthesized images calculated and composed within the graphics card. Using a dedicated bus to connect the display device 93 to the graphics card 92 offers the advantage of significantly higher data transmission rates, thus reducing latency for displaying the images composed by the graphics card. The display device 93 can be a screen, an augmented reality headset, a virtual reality headset, or a mixed reality headset.

[0152] In one embodiment, a display device is external to the device 9 and is connected to it by a cable or wirelessly to transmit display signals. Device 9, for example via the graphics card 92, includes a transmission or connection interface suitable for transmitting a display signal to an external display device such as an LCD or plasma screen or a video projector. In this respect, the radio frequency unit 99 can be used for wireless transmissions.

[0153] It should be noted that the word "register" used below in the description of memories 97 and 921 can refer, in each of the aforementioned memories, to a small-capacity memory area (holding a few binary data points) as well as a large-capacity memory area (allowing the storage of an entire program or the calculation or display of all or part of the data representing the data). Similarly, the registers represented for RAM 97 and GRAM 921 can be arranged and configured in any way, and each of them does not necessarily correspond to adjacent memory locations and can be distributed differently (this notably covers the case where a register comprises several smaller registers).

[0154] Upon power-up, the microprocessor 91 loads and executes the program instructions contained in the RAM 97.

[0155] As a person skilled in the art will understand, the presence of the 92 graphics card is not mandatory, and can be replaced by complete processing by the central processing unit and / or simpler visualization implementations.

[0156] In one embodiment, the device 1 can be implemented differently from standalone software, and a device or set of devices comprising only parts of the device 9 can be operated via an API call or a cloud interface.

Claims

DEMANDS 1. Device (1) for navigating a surgical instrument (34) within an organ (31) of a patient (32), via visualization in an immersive virtual reality environment (33) of a region (39) of an enhanced 3D geometric model (23) of said organ (31), said region (39) being defined around a current virtual position (36) of an end of said surgical instrument (34) in said enhanced 3D geometric model (23) of said organ (31), said current virtual position (36) corresponding to a current real position (22) of the end of said surgical instrument (34) within said organ (31) in a real reference frame (Ri), said device (1) comprising: at least one input interface configured to receive: • a 3D geometric model (21) of the organ (31) of said patient (32) previously generated from at least one set of images, said 3D geometric model (21) comprising a plurality of polygons (38) forming a mesh representative of a structure of said organ (31), • the actual current position (22) of the location of said end of the surgical instrument (34) in said organ (31) in the actual reference frame (Ri), said actual current position (22) being obtained via at least one position sensor (37), at least one processor configured to: • calculate a TM / E transformation between a reference frame associated with the 3D geometric model (21) and a reference frame (RE) of the immersive virtual reality environment (33), • realign the 3D geometric model (21) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TM / E transformation, • for each polygon (38) of the 3D geometric model (21), associate at least one visualization feature and associate at least one interaction feature so as to obtain an enriched 3D geometric model (23) of said organ (31), • calculate a TI / E transformation between a reference frame (Ri) of the current real position (22) and the reference frame (RE) of the immersive virtual reality environment (33), and • realign said current real position (22) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TI / E transformation, so as to obtain the current virtual position (36) of the tip of the surgical instrument (34) in the reference frame (RE) of the immersive virtual reality environment (33), • obtain at least one interaction scenario (50) between said enriched 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33), at least one output interface configured to provide F at least one interaction scenario (50) between said enriched 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33), said scenario including a current view of said region (39) of said enriched 3D geometric model (23) of said organ (31) defined around said current virtual position (36) of the tip of said surgical instrument (34) in the immersive virtual reality environment (33).

2. Device according to claim 1, wherein the current position of the surgical instrument (34) is obtained via at least one sensor configured to detect a local variation of an electromagnetic field in which the organ (31) is positioned.

3. Device according to claim 1, wherein: F at least one input interface is further configured to receive a position of at least one fiduciary marker (35) located on said component (31), said position being obtained via a motion tracking device, said component (31) having a real orientation in the real reference frame (Ri); and F at least one processor being further configured to: o calculate a TF / E transformation between a reference frame (RF) of the position of F at least one fiduciary marker (35) and said reference frame (RE) of the immersive virtual reality environment (33) and re-register said position of F at least one fiduciary marker (35) in the reference frame (RE) of the immersive virtual reality environment (33), o re-register said enriched 3D geometric model (23) with respect to the re-registered position of F at least one fiduciary marker (35) so as to obtain a re-registered enriched 3D geometric model, in which the re-registered enriched 3D geometric model is positioned in a virtual orientation corresponding to said real orientation of said organ (31) in said real reference frame (Ri), o update F at least one interaction scenario (50) with the re-registered enriched 3D geometric model.

4. Device according to any one of claims 1 to 3, wherein said 3D geometric model (21) is pre-generated from at least one image set comprising a plurality of images, each image of said image set including at least a portion of said organ (31) of the patient (32), said images being pre-operatively captured with at least one imaging modality.

5. Device according to any one of claims 1 to 4, wherein said region (39) of the enriched 3D geometric model (23) of said organ (31) corresponds to the entire organ (31).

6. Device according to any one of claims 1 to 5, wherein said current view has an origin point, said origin point being positioned at the current virtual position (36) of the end of the surgical instrument (34) in the immersive virtual reality environment (33).

7. Device according to any one of claims 1 to 6, wherein said current view has an origin point, said origin point being positioned in the reference frame (RE) of the immersive virtual reality environment (33), at a position external to said enhanced 3D geometric model (23) of said organ (31).

8. Device according to any one of claims 1 to 7, wherein F at least one output interface is further configured to display a marker representative of the end of the surgical instrument (34) at said current virtual position (36) in the reference frame (RE) of the immersive virtual reality environment (33), said representative marker being a three-dimensional portion located at the end of the surgical instrument (34).

9. Device according to any one of claims 1 to 8, wherein said device is configured to update the current view of said region (39) of the enriched 3D geometric model (23) of said organ (31) defined around said current virtual position (36) of the tip of said surgical instrument (34) in the immersive virtual reality environment (33) when the device receives a new current real position (22) of location of said tip of the surgical instrument (34) in said organ (31).

10. Device according to any one of claims 1 to 9, wherein: at least one processor is further configured to: receive a navigation instruction comprising a future real position of location of said tip of the surgical instrument (34) in the organ (31), calculate a TIF / E transformation between a reference frame (Ri) of the future real position of location and said reference frame (RE) of the immersive virtual reality environment (33), and re-register said future real position of location of the tip of the surgical instrument (34) in the reference frame (RE) of the immersive virtual reality environment (33) to obtain an updated virtual position of the tip of the surgical instrument (34) in the reference frame (RE) of the immersive virtual reality environment (33),and at least one output interface is further configured to display an updated view of a region (39) of the enhanced 3D geometric model (23) of said component (31) defined around said updated virtual position of the end of said component, surgical instrument (34) in the immersive virtual reality environment (33).

11. Device according to claim 10, wherein said at least one processor is further configured to navigate the surgical instrument (34) so ​​that the end of said surgical instrument (34) is positioned at said future actual position in the organ (31).

12. Device according to any one of claims 1 to 11, wherein: at least one processor is further configured to: receive coordinates of at least two points of the enhanced 3D geometric model (23) of said organ (31), calculate at least one characteristic dimension of at least one element of said organ (31) from said coordinates of at least two points, and at least one output interface is further configured to display said characteristic dimension of said organ (31) in the immersive virtual reality environment (33).

13. Device according to any one of claims 1 to 12, wherein at least one output interface is further configured to simultaneously display the current view and an updated view in the immersive virtual reality environment (33).

14. Device according to any one of claims 1 to 13, wherein said organ (31) of the patient (32) is an organ (31) of a body of the patient, an artificial organ (31) or a 3D print of the enhanced 3D geometric model (23) of said organ (31) of the patient (32).

15. Device according to any one of claims 1 to 14, wherein said organ (31) is the heart.

16. Device according to claim 15, wherein: - said 3D geometric model (21) of the organ (31) of said patient (32) is previously generated from at least one set of images obtained in synchronization with a predefined instant of a physiological signal from said patient, said physiological signal being an electrocardiogram and said predefined instant being the end of diastole; - said at least one processor is further configured to realign said current real position (22) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TI / E transformation on the basis of said predefined instant of said physiological signal.

17. A method (40) for navigating a surgical instrument (34) within an organ (31) of a patient (32), said organ (31) being an artificial organ or a 3D print of an enhanced 3D geometric model (23), via visualization in an immersive virtual reality environment (33) of a region (39) of said enhanced 3D geometric model (23) of said organ (31), said region (39) being defined around a current virtual position (36) of an end of said surgical instrument (34) within the enhanced 3D geometric model (23) of said organ (31), said current virtual position (36) corresponding to a current real position (22) of the end of said surgical instrument (34) within said organ (31), said method (40) comprising: receiving (41) a 3D geometric model (21) of the organ (31) of said patient (32) previously generated from at least a set of images,said 3D geometric model (21) comprising a plurality of polygons (38) forming a mesh representative of a structure of said organ (31), to receive (41) the current real position (22) of location of said end of the surgical instrument (34) in said organ (31) in a real reference frame (Ri), said current real position (22) of location being obtained via at least one position sensor (37), to calculate (42) a TM / E transformation between a reference frame associated with said 3D geometric model (21) and a reference frame (RE) of the immersive virtual reality environment (33), o to realign (43) the 3D geometric model (21) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TM / E transformation, o for each polygon (38) of the 3D geometric model (21), associate 44 at least one visualization feature and associate at least one interaction feature so as to obtain an enriched 3D geometric model (23) of said organ (31), o calculate (45) a TP / E transformation between a reference frame of the current real position (22) and the reference frame (RE) of the immersive virtual reality environment (33), o to realign (46) said current real position (22) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TP / E transformation, so as to obtain the current virtual position (36) of the tip of the surgical instrument (34) in the reference frame (RE) of the immersive virtual reality environment (33),to obtain (47) at least one interaction scenario (50) between the enhanced 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33), to provide (48) at least one interaction scenario (50) between the enhanced 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33), said scenario including a current view of said region (39) of the enhanced 3D geometric model (23) of said organ (31) defined around said current virtual position (36) of the tip of said surgical instrument (34) in the immersive virtual reality environment (33).

18. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to implement a method (40) for navigating a surgical instrument (34) into an organ (31) of a patient (32), via visualization in a virtual reality environment immersive (33) of a region (39) of an enhanced 3D geometric model (23) of said organ (31), said region (39) being defined around a current virtual position (36) of an end of said surgical instrument (34) in the enhanced 3D geometric model (23) of said organ (31), said current virtual position (36) corresponding to a current real position (22) of the end of said surgical instrument (34) in said organ (31), said method (40) comprising: o receiving (41) a 3D geometric model (21) of the organ (31) of said patient (32) previously generated from at least one set of images, said 3D geometric model (21) comprising a plurality of polygons (38) forming a mesh representative of a structure of said organ (31), o receiving (41) the current real position (22) of the end of said surgical instrument (34) in said organ (31) in a real reference frame (Ri),said current real position (22) of localization being obtained via at least one position sensor (37), o calculate (42) a TM / E transformation between a reference frame associated with said 3D geometric model (21) and a reference frame (RE) of the immersive virtual reality environment (33), o re-register (43) the 3D geometric model (21) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TM / E transformation, o for each polygon (38) of the 3D geometric model (21), associate 44 at least one visualization feature and associate at least one interaction feature so as to obtain an enriched 3D geometric model (23) of said organ (31), o calculate (45) a TP / E transformation between a reference frame of the current real position (22) and the reference frame (RE) of the immersive virtual reality environment (33),to realign (46) said current real position (22) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TP / E transformation, so as to obtain the current virtual position (36), of the tip of the surgical instrument (34) in the reference frame (RE) of the immersive virtual reality environment (33), o obtain (47) at least one interaction scenario (50) between the enriched 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33), o provide (48) F at least one interaction scenario (50) between the enriched 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33), said scenario including a current view of said region (39) of the enriched 3D geometric model (23) of said organ (31) defined around said current virtual position (36) of the tip of said surgical instrument (34) in the immersive virtual reality environment (33).

19. A computer-readable recording medium comprising instructions which, when executed by a computer, enable the implementation of a method (40) for navigating a surgical instrument (34) in an organ (31) of a patient (32), via visualization in an immersive virtual reality environment (33) of a region (39) of an enhanced 3D geometric model (23) of said organ (31), said region (39) being defined around a current virtual position (36) of an end of said surgical instrument (34) in the enhanced 3D geometric model (23) of said organ (31), said current virtual position (36) corresponding to a current real position (22) of the end of said surgical instrument (34) in said organ (31) in a real reference frame (Ri), said method (40) comprising: receiving (41) a 3D geometric model (21) of the organ (31) of said patient (32) previously generated from at least one set of images,said 3D geometric model (21) comprising a plurality of polygons (38) forming a mesh representative of a structure of said organ (31), receive (41) the current real position (22) of location of said end of the surgical instrument (34) in said organ (31) in a real reference frame (Ri), said current real position (22) of location being obtained via at least one position sensor (37), calculate (42) a TM / E transformation between a reference frame associated with said 3D geometric model (21) and a reference frame (RE) of the immersive virtual reality environment (33), register (43) the 3D geometric model (21) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TM / E transformation, for each polygon (38) of the 3D geometric model (21), associate 44 at least one visualization feature and associate at least one interaction feature so as to obtain an enriched 3D geometric model (23) of said organ (31),calculate (45) a TP / E transformation between a reference frame of the current real position (22) and the reference frame (RE) of the immersive virtual reality environment (33), realign (46) said current real position (22) in the reference frame (RE) of the immersive virtual reality environment (33) by applying the TP / E transformation, so as to obtain the current virtual position (36) of the tip of the surgical instrument (34) in the reference frame (RE) of the immersive virtual reality environment (33), obtain (47) at least one interaction scenario (50) between the enriched 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33),provide (48) F at least one interaction scenario (50) between the enhanced 3D geometric model (23) and the current virtual position (36) of the tip of the surgical instrument (34) in the immersive virtual reality environment (33), said scenario including a current view of said region (39) of the enhanced 3D geometric model (23) of said organ (31) defined around said current virtual position (36), of the tip of said surgical instrument (34) in the immersive virtual reality environment (33).

Images