Surgical planification method

Abstract

The present invention relates to a surgical planification method for visualization and manipulation by a user of 3D visual representations of anatomical body parts of a patient, who will undergo surgery, and 3D visual representations of at least one selected synthetic device among an ensemble of predefined synthetic devices, wherein said at least one synthetic device is configured to be positioned in proximity of at least one anatomical element of interest comprised in said anatomical body parts during said surgery.

Description

SURGICAL PLANIFICATION METHODFIELD OF INVENTION

[0001] The present invention relates to field of image processing for the planification of surgery, notably but not restricted to orthopedic surgery.BACKGROUND OF INVENTION

[0002] In the field of medical imaging, particularly in the domain of orthopedic surgery, there is an escalating demand for additional insights into tissue quality, particularly concerning the bones slated for treatment. As orthopedic procedures become increasingly complex, surgeons require detailed information regarding the quality and integrity of the bone structures they are operating on. This need arises from the imperative to ensure optimal surgical outcomes and minimize the risk of post-operative complications. By incorporating advanced imaging techniques and analytical tools into the visualization and manipulation interfaces, this invention seeks to address this crucial requirement.

[0003] The present invention may also be applied in surgeries other that orthopedic surgery. Indeed, the state of art planning software for other types of surgeries offer an analysis of the medical images through segmentation, which is often required for the generation of automatic measurements. They also offer the possibility to display within the image a virtual representation of a physical object to be used during surgery. However, additional information about tissue quality is desirable, and can be offered through volume rendering, which need to be combined smartly with the other components.

[0004] The proposed platform not only provides comprehensive 3D representations of anatomical structures and synthetic devices but also offers valuable insights into tissue quality, empowering surgeons with the information needed to make informed decisions and execute surgical interventions with precision and confidence.SUMMARY

[0005] This invention thus relates to a surgical planification method for visualization and manipulation by a user of 3D visual representations of anatomical body parts of a patient, who will undergo surgery, and 3D visual representations of at least one selected synthetic device among an ensemble of predefined synthetic devices, wherein said at least one synthetic device is configured to be positioned in proximity of at least one anatomical element of interest comprised in said anatomical body parts during said surgery, said method comprising: obtaining at least: o a 3D surface model of said at least one anatomical element of interest and a segmentation mask of said at least one anatomical element of interest, wherein said 3D surface model being a surface rendering obtained from segmentation of said at least one anatomical element of interest on at least one 3D medical image of said patient comprising a representation of said anatomical body parts; wherein said segmentation mask is obtained from said segmentation; o a 3D volume model of said anatomical body parts obtained using a raybased volume rendering technique on said at least one 3D medical image of the patient; o an object 3D surface model for each predefined synthetic device comprised in the ensemble of predefined synthetic devices; obtaining an element 3D volume model of said at least one anatomical element of interest by applying said segmentation mask to select from the 3D volume model the voxels associated to said at least one anatomical element of interest; displaying a graphical user interface on a visual display device of a system; displaying, on the graphical user interface, a first visualization panel comprising a hybrid rendering of at least one portion of said 3D volume model and said 3D surface model of said at least one anatomical element of interest;displaying on the graphical user interface, a second visualization panel comprising at least one device selection component configured to select at least one synthetic device from the ensemble of predefined synthetic devices; displaying upon selection of the at least one synthetic device by the user, on the graphical user interface, a third visualization panel comprising: o a third visualization panel configured to display said element 3D volume model and a selected object 3D surface model, said selected object 3D surface model being the object 3D surface model associated to the selected at least one synthetic device; o at least one displacement component configured to displace, in the third visualization panel, the selected object 3D surface model with respect to the element 3D volume model, so as to allow the user to find an optimal positioning for the at least one selected synthetic device with respect to the said at least one anatomical element of interest.

[0006] According to other advantageous aspects of the invention, the method comprises one or more of the features described in the following embodiments, taken alone or in any possible combination.

[0007] According to one embodiment, the first visualization panel further comprises at least one toggling component configured to toggle off said 3D surface model from the first visualization panel.

[0008] According to one embodiment, the method further comprises obtaining a masked 3D volume model by applying said segmentation mask to the 3D volume model to remove from the 3D volume model the voxels associated to said at least one anatomical element of interest, and wherein the at least one portion of said 3D volume model displayed in the first visualization panel is said masked 3D volume.

[0009] According to one embodiment, the graphical user interface further comprises at least one slice panel, comprising at least a portion of a selected slice of said 3D medical image, (in the axial, sagittal or coronal plane), and at least one first region taggingcomponent configured to identify pixels represented in said at least a portion of the selected slice that are associated to the at least one anatomical element of interest.

[0010] According to one embodiment, when at least one synthetic device has been selected and positioned within the anatomical body parts, said at least one slice panel further comprises a second tagging component configured to identify pixels represented in said at least a portion of the selected slice that are associated to the at least one synthetic device.

[0011] According to one embodiment, at least one displacement component is configured to rotate and translate the selected object 3D surface model.

[0012] According to one embodiment, the predefined synthetic devices are at least one among: implants, prosthesis or surgical devices.

[0013] According to one embodiment, the ensemble of predefined synthetic devices consists of a single synthetic device, and the selection component may consist of validating the use of the single synthetic device. In this case, the method advantageously allows the planification of the surgery with said single synthetic device. When the surgery is planned by the user according to this embodiment, the selection component may be a button confirming the creation of the representations in the other panels.

[0014] According to one embodiment, the at least one anatomical element of interest is a joint comprising at least two bones and wherein the masked 3D volume model comprises voxels associated with soft tissues surrounding said joint.

[0015] According to one embodiment, each object 3D surface model of a predefined synthetic device is obtained by surface rendering of an object file (STL OBJ or any other type of suitable file format for an object file) representing said predefined synthetic device.

[0016] According to one embodiment, the object 3D surface model is a simplified representation of the selected object 3D surface model, such as a cylinder representing a biopsy probe. In one example, the object 3D surface model may be a 3D surface model of a geometrical 3D shape such as a cube, a cone, a cylinder and the like.

[0017] According to one embodiment, the third visualization panel further comprises a clipping component configured to update the displaying of the third hybrid visualization panel by: remove the selected object 3D surface model; visualizing a clipped element 3D volume model obtained by removing from the element 3D volume model the voxels overlaying between the element 3D volume model and the selected object 3D surface model, which is used as clipping object with respect to which the element 3D volume model is configured to be clipped.

[0018] According to one embodiment, the method further comprises displaying on the graphical user interface, upon selection of the user, a fourth visualization panel comprising a surface visualization panel comprising the 3D surface model of said at least one anatomical element of interest and the at least one selected object 3D surface model.

[0019] According to one embodiment, the method further comprises computing anatomical measurements (sizes, angles or any other geometrical measurements between anatomical portions and / or anatomical landmarks) using morphological analysis algorithms, and displaying, on one of the visualization panels, graphical elements representative of said anatomical measurements (lines, landmarks / icons or any other graphical representation suitable for said representation).

[0020] According to one embodiment, when the at least one anatomical element of interest is a joint and the synthetic device is an implant, the third visualization panel further comprises a clipping component configured to clip the element 3D volume model with respect to a clipping object positioned with respect to the selected object 3D surface model, said clipping object being positioned with respect to the selected object 3D surface model according to a surgical cutting shape to be applied to the at least one of the bones of the joint in order to position the selected synthetic device during the surgery.

[0021] According to one embodiment, where at least one overlapping index is computed to represent an overlap between at least one object 3D surface model and at least one anatomical element of interest, wherein said segmentation mask is used as an input tocalculate said overlapping index, and wherein a representation of said overlapping index is displayed in at least one visualization panel.

[0022] According to one embodiment, the visualization panels are visualized simultaneously or sequentially, the wording “first visualization panel”, “second visualization panel” and “third visualization panel” being not necessarily related to a viewing order in the case of sequential display.

[0023] The present invention also relates to a system for visualization and manipulation of 3D visual representations of anatomical body parts of a patient, who will undergo surgery, and 3D visual representations at least one synthetic device, the system comprising: a visual display device; a user interface, and at least one computer processor coupled to the visual display device and the user interface programmed to perform the method according to any one of the embodiments cited above.

[0024] In addition, the disclosure relates to a computer program comprising software code adapted to perform a method for visualization and manipulation compliant with any of the above execution modes when the program is executed by a processor.

[0025] The present disclosure further pertains to a non-transitory program storage device, readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method for visualization and manipulation, compliant with the present disclosure.

[0026] Such a non-transitory program storage device can be, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor device, or any suitable combination of the foregoing. It is to be appreciated that the following, while providing more specific examples, is merely an illustrative and not exhaustive listing as readily appreciated by one of ordinary skill in the art: a portable computer diskette, a hard disk, a ROM, an EPROM (Erasable Programmable ROM) or a Flash memory, a portable CD-ROM (Compact-Disc ROM).DEFINITIONS

[0027] In the present invention, the following terms have the following meanings:

[0028] The terms “adapted” and “configured” are used in the present disclosure as broadly encompassing initial configuration, later adaptation or complementation of the present device, or any combination thereof alike, whether effected through material or software means (including firmware).

[0029] The term “processor” should not be construed to be restricted to hardware capable of executing software, and refers in a general way to a processing device, which can for example include a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). The processor may also encompass one or more Graphics Processing Units (GPU), whether exploited for computer graphics and image processing or other functions. Additionally, the instructions and / or data enabling to perform associated and / or resulting functionalities may be stored on any processor- readable medium such as, e.g., an integrated circuit, a hard disk, a CD (Compact Disc), an optical disc such as a DVD (Digital Versatile Disc), a RAM (Random- Access Memory) or a ROM (Read-Only Memory). Instructions may be notably stored in hardware, software, firmware or in any combination thereof.BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description of particular and non- restrictive illustrative embodiments, the description making reference to the annexed drawings wherein:

[0031] Figure 1 is a block diagram representing schematically a particular mode of a system for power consumption management including a device for selecting data samples, compliant with the present disclosure;

[0032] Figure 2 is a representation of the GUI according to one embodiment of the present invention concerning first visualization panel;

[0033] Figure 3 is a representation of the GUI according to one embodiment of the present invention concerning first visualization panel;

[0034] Figure 4 is a representation of the GUI according to one embodiment of the present invention, concerning first visualization panel wherein the 3D surface model have been toggled off;

[0035] Figure 5 is a representation of the GUI according to one embodiment of the present invention concerning first visualization panel wherein the at least one portion of the 3D volume model displayed is said masked 3D volume;

[0036] Figure 6 is a representation of the GUI according to one embodiment of the present invention concerning first visualization panel wherein masked 3D volume is cropped;

[0037] Figure 7 is a representation of the GUI according to one embodiment of the present invention concerning first visualization panel wherein masked 3D volume is cropped at a different level than Figure 6;

[0038] Figure 8 is a representation of the GUI according to one embodiment of the present invention concerning second visualization panel;

[0039] Figure 9 is a representation of the GUI according to one embodiment of the present invention concerning third visualization panel for a first bone of a joint;

[0040] Figure 10 is a representation of the GUI according to one embodiment of the present invention concerning third visualization panel, comprising multiple displacement components;

[0041] Figure 11 is a representation of the GUI according to one embodiment of the present invention concerning third visualization panel for a second bone of a joint;

[0042] Figure 12 is a representation of the GUI according to one embodiment of the present invention concerning third visualization panel, comprising multiple displacement components;

[0043] Figure 13 is a representation of the GUI according to one embodiment of the present invention concerning third visualization panel showing the effect of the use of the clipping component;

[0044] Figure 14 is a representation of the GUI according to one embodiment of the present invention concerning third visualization panel wherein the rendering of the bone structure is modified between the upper and lower image using a modulation visualization component;

[0045] Figure 15 is a representation of the GUI according to one embodiment of the present invention concerning fourth visualization panel;

[0046] Figure 16 is a flow chart showing successive steps executed with the device for predicting of figure 1 ;

[0047] On the figures, the drawings are not to scale, and identical or similar elements are designated by the same references.ILLUSTRATIVE EMBODIMENTS

[0048] The present description illustrates the principles of the present disclosure. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the disclosure and are included within its scope.

[0049] All examples and conditional language recited herein are intended for educational purposes to aid the reader in understanding the principles of the disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions.

[0050] Moreover, all statements herein reciting principles, aspects, and embodiments of the disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that suchequivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

[0051] Thus, for example, it will be appreciated by those skilled in the art that the block diagrams presented herein may represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable media and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0052] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared.

[0053] It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software or combinations thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory and input / output interfaces.

[0054] The present disclosure will be described in reference to a particular functional embodiment of a system 1 for visualization and manipulation by a user of 3D visual representations of anatomical body parts of a patient, who will undergo surgery, and 3D visual representations of at least one selected synthetic device among an ensemble of predefined synthetic devices, as illustrated on Figure 1.

[0055] The system 1 is adapted to produce a graphical user interface via a visual display device 60 wherein the displaying of a graph panel 100 is updated in real time on the base of the interaction of a user with said graphical user interface by mean of a user interface 50. The system 1 is notably configured to for visualization and manipulation by a user of 3D visual representations of anatomical body parts of a patient, who will undergo surgery,and 3D visual representations of at least one selected synthetic device among an ensemble of predefined synthetic devices, via said graphical user interface.

[0056] The system 1 comprises said user interface 50, via which information can be entered by a user. The user interface 50 includes any means appropriate for entering, information or instructions, notably visual, tactile and / or audio capacities that can encompass any or several of the following means as well known by a person skilled in the art: a keyboard, a trackball, a mouse, a controller, a touchpad, a touchscreen, a voice recognition system and the like.

[0057] Though the presently described devices 1 is versatile and provided with several functions that can be carried out alternatively or in any cumulative way, other implementations within the scope of the present disclosure include devices having only parts of the present functionalities.

[0058] The devices 1 is advantageously an apparatus, or a physical part of an apparatus, designed, configured and / or adapted for performing the mentioned functions and produce the mentioned effects or results. In alternative implementations, any of the device 1 is embodied as a set of apparatus or physical parts of apparatus, whether grouped in a same machine or in different, possibly remote, machines. The device 1 may e.g. have functions distributed over a cloud infrastructure and be available to users as a cloud-based service, or have remote functions accessible through an API.

[0059] In what follows, the modules are to be understood as functional entities rather than material, physically distinct, components. They can consequently be embodied either as grouped together in a same tangible and concrete component, or distributed into several such components. Also, each of those modules is possibly itself shared between at least two physical components. In addition, the modules are implemented in hardware, software, firmware, or any mixed form thereof as well. They are preferably embodied within at least one processor of the device 1.

[0060] The system 1 comprises a module 11 for receiving the 3D surface model 21, the segmentation mask of said at least one anatomical element of interest, the 3D volume model 22 and the object 3D surface model 23 (and / or at least one 3D medical image ofsaid patient), stored in one or more local or remote database(s) 40. The latter can take the form of storage resources available from any kind of appropriate storage means, which can be notably a RAM or an EEPROM (Electrically-Erasable Programmable Read-Only Memory) such as a Flash memory, possibly within an SSD (Solid-State Disk). Alternatively, the 3D surface model 21, the 3D volume model 22 and the object 3D surface model 23 are received from a communication network.

[0061] Alternatively, according to one embodiment, the processor(s) of the present system is configured to obtain the 3D surface model 21, the 3D volume model 22, segmentation mask and the object 3D surface model 23.

[0062] The 3D surface model 21 is a surface rendering obtained from segmentation of said at least one anatomical element of interest on at least one 3D medical image of said patient comprising a representation of said anatomical body part. The segmentation mask is obtained from said segmentation.

[0063] The 3D volume model 22 of said anatomical body part is obtained using a raybased volume rendering technique (such as a ray casting or ray tracing technique) on said at least one 3D medical image of the patient.

[0064] At least one object 3D surface model 23 is received for each predefined synthetic device comprised in the ensemble of predefined synthetic devices.

[0065] The system 1 comprises a module 12 for obtaining an element 3D volume model 24 of said at least one anatomical element of interest by applying said segmentation mask to select from the 3D volume model the voxels associated to said at least one anatomical element of interest.

[0066] The system 1 comprises a module 13 for displaying the graphical user interface 100 on a visual display device 60.

[0067] The system 1 comprises a module 14 for displaying, on the graphical user interface 100, a first visualization panel 101 comprising a hybrid rendering of at least one portion of said 3D volume model 22 and said 3D surface model 21 of said at least one anatomical element of interest, as shown in Figure 2.

[0068] According to one embodiment, each object 3D surface model 23 of a predefined synthetic device is obtained by surface rendering of an object file (STL OBJ or any other type of suitable file format for an object file) representing said predefined synthetic device.

[0069] According to one embodiment, the graphical user interface 100 further comprises at least one slice panel 105, comprising at least a portion of a selected slice of said 3D medical image, (in the axial, sagittal or coronal plane), and at least one first region tagging component configured to identify pixels represented in said at least a portion of the selected slice that are associated to the at least one anatomical element of interest, as shown in Figure 2.

[0070] According to one embodiment, the first visualization panel 101 further comprises at least one toggling component configured to toggle off said 3D surface model 21 from the first visualization panel 101. An example, wherein the anatomical element of interest is a joint of two bones, is shown in Figures 3 and 4. Indeed, a first bone of the joint (i.e., humerus) is removed in Figure 3 then the second bone is removed in Figure 4.

[0071] According to one embodiment, the system (at least one processor) is further configured to obtain a masked 3D volume model 25 which is visible in Figure 5. The masked 3D volume model 25 is obtained by applying said segmentation mask to the 3D volume model 22 to remove from the 3D volume model 22 the voxels associated to said at least one anatomical element of interest, and wherein the at least one portion of said 3D volume model displayed in the first visualization panel 101 is said masked 3D volume.

[0072] According to one embodiment, illustrated in Figure 5, the at least one anatomical element of interest is a joint comprising at least two bones and the masked 3D volume model comprises voxels associated with soft tissues surrounding said joint.

[0073] The first visualization panel 101 may further comprise a cropper component 107, configured to select on portion of the masked 3D volume model 25, as shown in Figure 6 where more of the soft tissues are represented, with respect to Figure 5. In Figure 7 the cropper component 107 has been again slide along the vertical axis in order to visualize a smaller portion of the masked 3D volume model 25. The cropper component 107 maybe a tool or widget used by the user to interactively select or “crop” a subset of the masked 3D volume model 25 so as to inspect or analyze a specific region of interest. The cropper component 107 work in the same way on the 3D volume model 22.

[0074] The system 1 comprises a module 15 for displaying, on the graphical user interface 100, a second visualization panel 102 comprising at least one device selection component 201 configured to select at least one synthetic device from the ensemble of predefined synthetic devices, as shown in Figure 8.

[0075] According to one embodiment, the predefined synthetic devices are at least one among: implants, prosthesis or surgical devices. In the illustrative example of Figure 8, the predefined synthetic devices are implants for replacing a joint (i.e., shoulder joint).

[0076] The system 1 comprises a module 16 for displaying upon selection by the user of the at least one synthetic device, on the graphical user interface 100, a third visualization panel 103 as shown in Figures 9 and 11. The third visualization panel 103 comprises: o a third (hybrid) visualization panel configured to display said element 3D volume model 22 and a selected object 3D surface model, said selected object 3D surface model being the object 3D surface model 23 associated to the selected at least one synthetic device; o at least one displacement component 202 configured to displace, in the third visualization panel 103, the selected object 3D surface model with respect to the element 3D volume model 24, so to allow the user to find an optimal positioning for the selected at least one synthetic device with respect to the said at least one anatomical element of interest.

[0077] According to one embodiment, when at least one synthetic device has been selected and positioned within the anatomical body parts, the at least one slice panel 105 further comprises a second tagging component configured to identify pixels represented in the at least a portion of the selected slice that are associated to the at least one synthetic device (represented in green in the slice panel 105 in Fig. 9 and 11).

[0078] According to one embodiment, at least one displacement component 202 is configured to rotate and translate the selected object 3D surface model, as shown in Figures 10, 11, 12 and 13.

[0079] According to one embodiment, the third visualization panel 103 further comprises a clipping component configured to update the displaying of the third hybrid visualization panel by:• remove the selected object 3D surface model;• visualizing a clipped element 3D volume model obtained by removing from the element 3D volume model 24 the voxels overlaying between the element 3D volume model 24 and the selected object 3D surface model, which is used as clipping object.

[0080] To do so, the clipping component may be configured to, before removing the selected object 3D surface model, determine a set of voxels of the element 3D volume model 24 overlaid by the element 3D volume model. Removing, from the element 3D volume model 24, the voxels overlaying between the element 3D volume model and the selected object 3D surface model implying removing, from the element 3D volume model 24, said set of voxels.

[0081] According to one embodiment, when the at least one anatomical element of interest is a joint and the synthetic device is an implant, the third visualization panel 103 further comprises a clipping component configured to clip the element 3D volume model 24 with respect to the clipping object positioned with respect to the selected object 3D surface model, said clipping object being positioned with respect to the selected object 3D surface model according to a surgical cutting shape to be applied to the at least one of the bones of the joint in order to position the selected synthetic device during the surgery.

[0082] According to one example, illustrated in Figure 12 and 13, the clipping object is a plane aligned with a portion of implant in order to mimic the cutting plane that the surgeon will have to perform on the bone on order to correctly position the implant. The application of this clipping component on the element 3D volume model advantageously allows to visualize the details of the bone structure where the implant should be fixed. Sothat the surgeon will know in advance if the bone is of poor quality, which jeopardize the positioning of the implant, and therefore plan in advance to use a different implant or if to position it differently, for example.

[0083] According to one embodiment, the visualization panels comprising a 3D volume rendering model (notably, first and third visualization panel) comprises a modulation visualization component 203, as shown in Figure 14. By moving the slider 203 the user may modify the visualization colors of the element 3D volume model 24 so that the user may advantageously choose the color setting allowing to better visualize bone inner structure.

[0084] According to one embodiment, the method further comprises displaying on the graphical user interface 100, upon selection of the user, a fourth visualization panel 104 comprising a surface visualization panel comprising the 3D surface model 21 of said at least one anatomical element of interest and the at least one selected object 3D surface model.

[0085] According to one embodiment, the method further comprises computing anatomical measurements (sizes, angles or any other geometrical measurements between anatomical portions and / or anatomical landmarks) using morphological analysis algorithms, and displaying, on one of the visualization panels, graphical elements representative of said anatomical measurements (lines, landmarks / icons or any other graphical representation suitable for said representation), as illustrated in Figures 2, 3 and 15.

[0086] According to one embodiment, at least one overlapping index is computed to represent an overlap between at least one object 3D surface model 23 and at least one anatomical element of interest, wherein said segmentation mask is used as an input to calculate said overlapping index, and wherein a representation of said overlapping index is displayed in at least one visualization panel.

[0087] According to one embodiment, the visualization panels can be visualized simultaneously or sequentially, the wording “first visualization panel”, “second visualization panel” and “third visualization panel” being not necessarily related to aviewing order in the case of sequential display. The second visualization panel 102 comprising at least one device selection component 201 may be, for example, represented in a smaller window / panel near to first visualization panel 101 comprising the anatomical representation. Alternatively the three panels could be represented side by side.

[0088] In its automatic actions, the device 1 may for example execute the following process (Figure 16):- receiving a 3D surface model 21 of said at least one anatomical element of interest and a segmentation mask of said at least one anatomical element of interest, a 3D volume model 22 of said anatomical body parts and an object 3D surface model 23 for each predefined synthetic device (step 41),- obtaining an element 3D volume model 24 of said at least one anatomical element of interest by applying said segmentation mask (step 42),- displaying a graphical user interface on a visual display device 60 (step 43),- displaying, on the graphical user interface 100, a first visualization panel 101 comprising a hybrid rendering of at least one portion of said 3D volume model 22 and said 3D surface model 21 of said at least one anatomical element of interest (step 44);- displaying, on the graphical user interface 100, a second visualization panel 102 comprising at least one device selection component 201 configured to select at least one synthetic device from the ensemble of predefined synthetic devices (step 45);- displaying upon selection by the user of the at least one synthetic device, on the graphical user interface 100 a third visualization panel 103 (step 46).

[0089] A particular system 1 corresponds for example to a workstation, a laptop, a tablet, a smartphone, or a head-mounted display (HMD).

[0090] Orthopedic surgery planning

[0091] The device 1 may be used by surgeons in the context of orthopedic surgery planning. In this field, the 3D surface model 21 may represent a bone or an ensemble of bones forming a joint, the 3D volume model 22 may represent the soft tissues around saidbone or joint (muscles, tendons...), the first visualization panel 101 may display both a hybrid rendering of the 3D surface model 21 and the 3D volume model 22, the object 3D surface model 23 may represent an orthopedic implant, the element 3D volume model 24 may be a volume rendering of the bone or joint and the third visualization panel 103 may display an hybrid rendering of the element 3D volume model 24 together with the object 3D surface model 23.

[0092] This configuration advantageously lets the user visualize the soft tissues surrounding the bone / joint and takes this information into account to select the best implant. The volume rendering of the bone enables the visualization of the bone’s quality. Using the clipping object, the surgeon can visualize the “inside” of the bones and figure out if the implant can be inserted / screwed at the target placement. Various measurements can be added to the process so as to ease the decision making of the surgeon. They can comprise prepositioning measurements co-displayed on the first panel including distances and angles characterizing the preoperative anatomy. They can also include post positioning measurements, where the same measurements can be computed after the simulation of the implant’ s placement by the surgeon.

[0093] Vascular surgery planning

[0094] The device may be used by surgeons in the context of vascular surgery planning. In this field, the 3D surface model 21 may be a blood vessel or an ensemble of vessels forming a vascular network, the 3D volume model 22 may represent the soft tissues around said vessels (muscles, other organs...), the first visualization panel 101 may display both a hybrid rendering of the 3D surface model 21 and the 3D volume model 22, the object 3D surface model 23 may be a representation of a vascular implant such as a stent or a coil, the element 3D volume model 24 may be a volume rendering of the vessels and the third visualization panel 103 may display an hybrid rendering of the element 3D volume model 24 together with the object 3D surface model 23.

[0095] This configuration advantageously lets the user visualize the soft tissues surrounding the vessels and takes this information into account to select the best implant. Then the volume rendering of the vessels enables the visualization of the lumen and theassessment of plaque and calcifications which are visible through volume rendering. The surgeon can visualize the “inside” of the vessels and figure out if the implant can be inserted at the target placement. Various measurements can be added to the process so as to ease the decision making of the surgeon. They can comprise prepositioning measurements co-displayed on the first panel including vessels’ diameters and lengths, characterizing the preoperative anatomy. They could also include automatic quantification of said plaque and calcifications, together with visual elements to highlight them.

[0096] Brain surgery planning

[0097] The device might be used by surgeons in the context of brain surgery planning. In this field, the 3D surface model 21 may represent a specific region of the brain such as a specific lobe or a brain tumor, the 3D volume model 22 may represent the other regions of the brain and / or other head’ s anatomical elements , the first visualization panel 101 may display both a hybrid rendering of the 3D surface model 21 and the 3D volume model 22, the anatomical elements may be rendered with high transparency so as to visualize the 3D surface model 21 through them, the object 3D surface model 23 is a representation of a brain implant or a brain stimulation device or a probe or a surgical device, the element 3D volume model 24 is a volume rendering of the specific region of the brain, the third visualization panel 103 displays an hybrid rendering of the element 3D volume model 24 together with the object 3D surface model.

[0098] This configuration advantageously lets the user visualize the localization of the specific region in the context of the global anatomy of the head. Then the volume rendering of the specific region enables to focus on evaluating the quality of this region without being visually overflown by information of the other regions.

[0099] Advantageously the first visualization panel 101 may display the visualization of several 3D surface models 21 of several specific regions, and let the user select at least one specific region to be selected in the second visualization panel 102.

Claims

CLAIMS1. A surgical planification method for visualization and manipulation by a user of 3D visual representations of anatomical body parts of a patient, who will undergo surgery, and 3D visual representations of at least one selected synthetic device among an ensemble of predefined synthetic devices, wherein said at least one synthetic device is configured to be positioned in proximity of at least one anatomical element of interest comprised in said anatomical body parts during said surgery, said method comprising: obtaining at least: o a 3D surface model (21) of said at least one anatomical element of interest and a segmentation mask of said at least one anatomical element of interest, wherein said 3D surface model (21) is a surface rendering obtained from segmentation of said at least one anatomical element of interest on at least one 3D medical image of said patient comprising a representation of said anatomical body parts; wherein said segmentation mask is obtained from said segmentation; o a 3D volume model (22) of said anatomical body parts obtained using a raybased volume rendering technique on said at least one 3D medical image of the patient; o an object 3D surface model (23) for each predefined synthetic device comprised in the ensemble of predefined synthetic devices; obtaining an element 3D volume model (24) of said at least one anatomical element of interest by applying said segmentation mask to select from the 3D volume model (22) the voxels associated to said at least one anatomical element of interest; displaying a graphical user interface (100) on a visual display device (60) of a system (1); displaying, on the graphical user interface ( 100), a first visualization panel (101) comprising a hybrid rendering of at least one portion of said 3D volume model(22) and said 3D surface model (21) of said at least one anatomical element of interest; displaying, on the graphical user interface (100), a second visualization panel (102) comprising at least one device selection component (201) configured to select at least one synthetic device from the ensemble of predefined synthetic devices; displaying upon selection of the at least one synthetic device by the user, on the graphical user interface (100), a third visualization panel (103) comprising: o a third visualization panel configured to display said element 3D volume model (24) and a selected object 3D surface model, said selected object 3D surface model being the object 3D surface model (23) associated to the selected at least one synthetic device; o at least one displacement component (202) configured to displace, in the third visualization panel, the selected object 3D surface model with respect to the element 3D volume model (24), so as to allow the user to find an optimal positioning for the at least one selected synthetic device with respect to the said at least one anatomical element of interest.

2. The method according to claim 1, wherein the first visualization panel (101) further comprises at least one toggling component configured to toggle off said 3D surface model (21) from the first visualization panel (101).

3. The method according to claim 1 or 2, further comprises obtaining a masked 3D volume model (25) by applying said segmentation mask to the 3D volume model (22) to remove from the 3D volume model (22) the voxels associated to said at least one anatomical element of interest, and wherein the at least one portion of said 3D volume model displayed in the first visualization panel (101) is said masked 3D volume.

4. The method according to any one of claims 1 to 3, wherein graphical user interface (100) further comprises at least one slice panel (105), comprising at least a portion of a selected slice of said 3D medical image and at least one first region taggingcomponent configured to identify pixels represented in said at least a portion of the selected slice that are associated to the at least one anatomical element of interest.

5. The method according to any of claims 1 to 4, wherein, when at least one synthetic device has been selected and positioned within the anatomical body parts, said at least one slice panel (105) further comprises a second tagging component configured to identify pixels represented in said at least a portion of the selected slice that are associated to the at least one synthetic device.

6. The method according to any of claims 1 to 5, wherein at least one displacement component (202) is configured to rotate and translate the selected object 3D surface model.

7. The method according to any of claims 1 to 6, wherein the predefined synthetic devices are at least one among: implants, prosthesis or surgical devices.

8. The method according to any of claims 1 to 7, wherein the at least one anatomical element of interest is a joint comprising at least two bones and wherein the masked 3D volume model comprises voxels associated with soft tissues surrounding said joint.

9. The method according to any of claims 1 to 8, wherein each object 3D surface model (23) of a predefined synthetic device is obtained by surface rendering of an object file representing said predefined synthetic device.

10. The method according to any of claims 1 to 9, wherein the third visualization panel (103) further comprises a clipping component configured to update the displaying of the third visualization panel (103) by:• remove the selected object 3D surface model;• visualizing a clipped element 3D volume model obtained by removing from the element 3D volume model (24) the voxels overlaying between the element 3D volume model (24) and the selected object 3D surface model, which is used as clipping object with respect to which the element 3D volume model is configured to be clipped.

11. The method according to any of claims 1 to 10, further comprising displaying on the graphical user interface (100), upon selection of the user, a fourth visualization panel (104) comprising a surface visualization panel comprising the 3D surface model (21) of said at least one anatomical element of interest and the at least one selected object 3D surface model.

12. The method according to any of claims 1 to 11, further comprising computing anatomical measurements using morphological analysis algorithms, and displaying, on one of the visualization panels, graphical elements representative of said anatomical measurements.

13. The method according to any of claims 1 to 12, when the at least one anatomical element of interest is a joint and the synthetic device is an implant, the third visualization panel (103) further comprises a clipping component configured to clip the element 3D volume model with respect to a clipping object positioned with respect to the selected object 3D surface model, said clipping object being positioned with respect to the selected object 3D surface model according to a surgical cutting shape to be applied to the at least one of the bones of the joint in order to position the selected synthetic device during the surgery.

14. The method according to any of claims 1 to 13, wherein at least one overlapping index is computed to represent an overlap between at least one object 3D surface model (23) and at least one anatomical element of interest, wherein said segmentation mask is used as an input to calculate said overlapping index, and wherein a representation of said overlapping index is displayed in at least one visualization panel.

15. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of any of claims 1 to 14.

16. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method any of claims 1 to 14.

17. A system (1) for visualization and manipulation of 3D visual representations of anatomical body parts of a patient, who will undergo surgery, and 3D visual representations at least one synthetic device, the system comprising: a visual display device (60); - a user interface (50), and at least one computer processor (10) coupled to the visual display device (60) and the user interface (50) programmed to perform the method according to any one of claims 1 to 14.

Images