Real-time support system for performing at least one bone tunnel by arthroscopy

Abstract

The invention relates to a real-time support system (1) for performing at least one bone tunnel (10) by arthroscopy in a joint of a patient, comprising an imaging device (2) capable of acquiring two-dimensional images of portions of the joint of the patient, a first tracking device (3) and a programmable device (6).

Description

[0001] REAL-TIME AID SYSTEM FOR THE ACHIEVEMENT OF AT LEAST ONE

[0002] BONE TUNNEL BY ARTHROSCOPY

[0003] FIELD OF THE INVENTION

[0004] The present invention relates to assistance in arthroscopy surgery. More particularly, the invention relates to a real-time assistance system for creating at least one bone tunnel in a patient's joint during an operation.

[0005] STATE OF THE ART

[0006] Knee ligament reconstruction surgeries are procedures that consist of replacing one or more ligaments in the knee joint, connecting the femur to the tibia, with a strip of tendon tissue, which will constitute the neoligament, taken from the patient (autograft), or more rarely, taken post-mortem from donors (allograft). The surgeon will then create a bone tunnel in the femur and a bone tunnel in the tibia and position these tunnels in the insertions of the damaged ligament, then pass the graft through these tunnels before fixing it.

[0007] There are two main types of ligaments. First, there are intra-articular ligaments, which connect the femur to the tibia by passing through the knee joint space, a space defined and limited by the joint capsule. The other category of ligament concerns the extra-articular ligaments, located on the periphery of the knee and which connect the knee to the tibia without passing through the joint space.

[0008] Intra-articular ligaments are easily identifiable and individualizable ligaments. They can be observed using an arthroscope, i.e. by introducing a mini camera into the joint space through a small incision (arthroscopic port).

[0009] Conversely, the extra-articular ligaments are located at the periphery of the joint and are "stuck" to the joint capsule. These ligaments are therefore not observable under arthroscopy; they are difficult to identify and individualize. During surgery, locating these extra-articular ligaments, and in particular their bony insertions, in order to reconstruct them requires large incisions and a long dissection step. Some very thin extra-articular ligaments stuck to the joint capsule, such as the anterolateral ligament or the medial patellofemoral ligament, are almost impossible to identify by dissection.

[0010] The placement of bone tunnels in the native insertion of the ligaments is therefore not always possible for certain structures or in certain cases when this insertion has been altered during the trauma (such as for example the femoral insertion of the anterior cruciate ligament - ACL, which is often damaged when the latter ruptures).

[0011] Furthermore, it is often necessary to position several bone tunnels during the same operation. Indeed, medical imaging studies in recent years have demonstrated that isolated rupture of the anterior cruciate ligament (ACL) is in fact rare and is accompanied in the vast majority of cases by an injury to a very specific extra-articular ligament, the anterolateral ligament (ALL). Reconstruction of the ACL and ALL is now indicated for the majority of patients. Also, in rarer cases of severe knee sprain, several intra- and extra-articular ligaments are injured and must be reconstructed.

[0012] Currently, mechanical aiming devices assist in the positioning of bone tunnels during knee surgery for intra-articular ligament reconstructions. The principle is based on the placement of a guide pin on which the tunnel is then drilled using a cannulated drill bit.

[0013] In a first aiming technique called "IN OUT", the femoral pin is placed from the inside to the outside of the joint. In this technique, the exit of the tunnel on the external part of the bone is not controlled and is likely to cause iatrogenic lesions. Furthermore, this technique does not allow the positioning of grafts for the reconstruction of extra-articular ligaments such as the anterolateral ligament.

[0014] In a second aiming technique called "OUT IN", the pin is inserted from the outside to the inside of the joint. This technique allows aiming with the knee at 90 degrees of flexion. For these OUT IN aiming devices, the entry point on the outside of the bone is made percutaneously (i.e. through the skin) and the area is chosen after palpation of the bony relief (here, the lateral epicondyle). The positioning of this entry point by palpation therefore remains unreliable and difficult to reproduce.

[0015] There are also computer-assisted aiming systems that provide additional information to the surgeon by using an imaging technique, such as fluoroscopy, in real time during the procedure and by projecting points acquired using a probe on the patient's knee in real time onto this imaging. However, intraoperative fluoroscopy is an irradiating process. Also, this aiming system only takes into account the positioning of the tunnels intra-articularly and does not allow precise positioning of the extra-articular orifice or precise positioning of the extra-articular grafts.

[0016] There is therefore no system in the state of the art that can meet the new requirements of knee ligament reconstruction, i.e. a system that is compatible and consistent with arthroscopic surgery, does not generate significant additional operating time or significant additional cost, is precise and suitable for reconstructions of two ligaments at the same time, for example an intra-articular ligament and an extra-articular ligament.

[0017] SUMMARY

[0018] The invention relates to a system for assisting in the creation of at least one bone tunnel by arthroscopy in a patient's joint using a surgical drilling instrument, comprising an imaging device or a measuring device, a tracking device and a programmable device, said programmable device being suitable for:

[0019] - obtaining a preoperative three-dimensional anatomical model specific to the patient, said model comprising a representation of at least one portion of interest of the patient's joint and a representation of anatomical structures of interest,

[0020] - acquiring tracking information of the current position and orientation of the surgical drilling instrument during manipulation of said surgical drilling instrument by an operator using the tracking device; - determining a partial intraoperative three-dimensional anatomical model, comprising a representation of the at least one portion of interest of the joint, by:

[0021] - acquiring, using the imaging device, a stream of intraoperative two-dimensional images comprising said at least one portion of interest of the joint, and acquiring tracking information of the current position and orientation of the imaging device using the tracking device; wherein said intraoperative partial three-dimensional anatomical model is obtained from the stream of intraoperative two-dimensional images and said tracking information of the current position and orientation, or

[0022] - acquiring, using the measuring device, a point cloud representative of the at least one portion of interest of the joint, and acquiring tracking information of the current position and orientation of said measuring device; wherein said intraoperative partial three-dimensional anatomical model is obtained from said point cloud;

[0023] - align the preoperative three-dimensional anatomical model with the intraoperative partial three-dimensional anatomical model;

[0024] - calculate, on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model, and the current position and orientation of the surgical drilling instrument:

[0025] * a current localization area, in a current intraoperative two-dimensional image, of a piercing end of the surgical piercing instrument,

[0026] * a first projection including a current estimate of a location area of ​​a first orifice of a bone tunnel to be made,

[0027] * a second projection including a current estimate of a location area of ​​a second orifice of the bone tunnel to be created.

[0028] In one embodiment, the invention relates to a real-time assistance system for creating at least one bone tunnel by arthroscopy in a patient's joint using a surgical drilling instrument, comprising an imaging device capable of acquiring two-dimensional images of portions of the patient's joint, a tracking device and a programmable device, said programmable device being adapted to: - obtain a preoperative three-dimensional anatomical model specific to the patient, said model comprising a representation of at least one portion of interest of the patient's joint and a representation of anatomical structures of interest,

[0029] - acquiring, using the imaging device, a stream of intraoperative two-dimensional images, said intraoperative two-dimensional images comprising said portion of interest of the joint and acquiring tracking information of the current position and orientation of the imaging device using the tracking device;

[0030] - acquiring tracking information of the current position and orientation of the surgical drilling instrument during manipulation of the surgical drilling instrument by an operator using the tracking device;

[0031] - determine a partial intraoperative three-dimensional anatomical model from the intraoperative two-dimensional image stream and / or a point cloud representative of the portion of interest of the joint obtained using a measuring device;

[0032] - align the preoperative three-dimensional anatomical model with the intraoperative partial three-dimensional anatomical model;

[0033] - calculate, on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model, and the current position and orientation of the surgical drilling instrument:

[0034] * a current localization area, in a current intraoperative two-dimensional image, of a piercing end of the surgical piercing instrument,

[0035] * a first projection including a current estimate of a location area of ​​a first orifice of a bone tunnel to be made,

[0036] * a second projection including a current estimate of a location area of ​​a second orifice of the bone tunnel to be created.

[0037] Thanks to the invention, a surgeon performing an operation requiring the creation of at least one bone tunnel can know in real time, depending on the position and orientation of his surgical drilling instrument, the corresponding positions of the entry zone and the exit zone of the bone tunnel to be created. The surgeon can thus adjust the position and orientation of the surgical drilling instrument in order to target a precise entry zone, or an exit zone, of the bone tunnel.

[0038] In one embodiment, the registration of the preoperative three-dimensional anatomical model on the intraoperative partial three-dimensional anatomical model is obtained via a 3D / 3D registration technique.

[0039] In one embodiment, said intraoperative partial three-dimensional anatomical model comprises a representation of the portion of interest of the joint.

[0040] In one embodiment, the programmable device is further configured to calculate:

[0041] - in the first projection, first projections of a first part of the representation of the anatomical structures of interest during an intervention phase, and / or

[0042] - in the second projection, second projections of a second part of the representation of the anatomical structures of interest.

[0043] Thus, it is possible for a surgeon, during an intervention, to know the position in real time of the entry zone and the exit zone of the bone tunnel to be created in relation to the anatomical structures of interest and to adjust the position and / or the orientation of his surgical drilling instrument so as to position the entry zone and the exit zone of the bone tunnel in relation to these anatomical structures of interest so as not to damage some of these anatomical structures of interest during the drilling of the bone tunnel. By anatomical structures of interest, we mean anatomical landmarks that help during the intervention to position his working instruments such as the surgical drilling instrument to create a bone tunnel.For example, the anatomical structures of interest include bony contours of the femur, tibia, patella and fibular head, a representation of the cartilaginous contours of the femur, tibia, a complete or partial representation of the anterior cruciate ligament, the posterior cruciate ligament, the lateral collateral ligament, the medial collateral ligament, and the patellofemoral ligament. In addition, the anatomical structures of interest include bony reliefs, cartilaginous boundaries, tendon or ligament structures. In other words, the anatomical structures of interest include anatomical landmarks and any so-called noble structure that should not be damaged during an intervention.

[0044] In one embodiment, the real-time help system further comprises a display unit and further adapted to display on the display unit:

[0045] * an overlay of the current intraoperative two-dimensional image and the current location area of ​​the piercing tip of the surgical piercing instrument,

[0046] * a first superposition of the first projection and the first projections of the first part of the representation of the anatomical structures of interest,

[0047] * a second superposition of the second projection and the second projections of the second part of the representation of the anatomical structures of interest.

[0048] Thus, a surgeon during an operation can visualize in real time the position of the entry zone and the exit zone and adjust the position / orientation of his surgical drilling instrument so as to avoid any damage to the anatomical structures of interest.

[0049] In one embodiment, the real-time assistance system is further adapted to, prior to obtaining the preoperative three-dimensional anatomical model, obtain and store a plurality of preoperative two-dimensional and / or three-dimensional images comprising the portion of interest of the joint.

[0050] Since knee interventions are often preceded by examinations using magnetic resonance imaging, as is typical in a patient's healthcare journey, the results of such examinations can be used to advantage to generate the three-dimensional model.

[0051] In one embodiment, the real-time assistance system is further adapted to calculate an intersection between the current estimate of the location area of ​​the first orifice, respectively of the second orifice, current with the first projections of the first part, respectively the second projections of the second part of the representation of the anatomical structures of interest. Thus, the real-time assistance system makes it possible to detect, as a function of the position and orientation of the surgical drilling instrument, any possibility of intersection of the corresponding bone tunnel with this position and this orientation.

[0052] In one embodiment, the real-time help system is further adapted, when the intersection with the first, respectively second, projections comprises a portion of the first projections of the first part of the representation of the anatomical structures of interest or a portion of the second projections of the second part of the representation of the anatomical structures of interest, to display an information message on the first, respectively, the second overlay.

[0053] Displaying an information message helps the surgeon adjust the current position and orientation of his surgical drilling instrument in real time, thus reducing the risk of error during surgery.

[0054] In one embodiment, the real-time assistance system is further adapted to, after a first bone tunnel has been created in the patient's joint, calculate on the first, respectively second, current projection a first, respectively second, pattern representing the location of the first bone tunnel. In other words, after a first bone tunnel has been created in the patient's joint, the system is further configured to calculate a three-dimensional representation of the first bone tunnel and to calculate, on the first, respectively second, current projection, a first, respectively second, two-dimensional view of said three-dimensional representation of the first bone tunnel.

[0055] Thus, the location of the first bone tunnel created is known and can be used by the surgeon to find his way around his workspace in order to create a second bone tunnel without the latter interfering with the first bone tunnel.

[0056] In one embodiment, the real-time assistance system is further adapted to display, on the first projection, the first pattern (i.e. the first two-dimensional view of said three-dimensional representation of the first bone tunnel) and, on the second projection, the second pattern (i.e. the second two-dimensional view of said three-dimensional representation of the first bone tunnel). In one embodiment, the real-time assistance system is further adapted to calculate a three-dimensional representation of the first bone tunnel produced.

[0057] The display of the calculated three-dimensional representation thus makes it possible to check, for example, whether the first bone tunnel created is indeed created in the bone over its entire length and whether its walls are thick enough to avoid the risk of fracture of the first bone tunnel when placing means of fixation of an intratunnel graft (such as interference screws). The fracture of a tunnel is known as "cortical wall blow out". Thus, if the surgeon knows that he has shaved the cortex, he can choose to use an extratunnel fixation means, such as a button, as a precaution.

[0058] In one embodiment, the real-time assistance system is further adapted to calculate an intersection between the current estimate of the location area of ​​the first, respectively of the second, orifice and, on the one hand the first pattern (i.e. the first two-dimensional view), on the other hand, the second pattern (i.e. the second two-dimensional view).

[0059] This allows, after creating a first bone tunnel in the patient's joint, if the surgeon has to create a second bone tunnel, to know its position in real time in relation to the position of the first bone tunnel already created.

[0060] In one embodiment, the real-time help system is further adapted when the intersection with the first (i.e. the first two-dimensional view), respectively second pattern (i.e. the second two-dimensional view), comprises at least a portion of the first pattern (i.e. the first two-dimensional view), respectively a portion of the second pattern (i.e. the second two-dimensional view), to display an information message on the first, respectively second, overlay.

[0061] Displaying an information message helps the surgeon adjust the current position and orientation of his surgical drilling instrument in real time.

[0062] Another aspect of the invention relates to a computer-implemented method for assisting in the creation of at least one bone tunnel by arthroscopy in a joint of a patient using a surgical drilling instrument, said method comprising: - obtaining a preoperative three-dimensional anatomical model specific to the patient, said model comprising a representation of at least one portion of interest of the patient's joint and a representation of anatomical structures of interest,

[0063] - acquiring tracking information of the current position and orientation of the surgical drilling instrument during manipulation of said surgical drilling instrument by an operator using the tracking device;

[0064] - determining a partial three-dimensional intraoperative anatomical model, comprising a representation of said at least one portion of interest of the joint, by:

[0065] - acquiring, using an imaging device, a stream of intraoperative two-dimensional images of portions of interest of the patient's joint, said intraoperative two-dimensional images comprising said at least one portion of interest of the joint, and acquiring tracking information of the current position and orientation of said imaging device using a tracking device; wherein said intraoperative partial three-dimensional anatomical model is obtained from the stream of intraoperative two-dimensional images and said tracking information of the current position and orientation, or

[0066] - acquiring, using a measuring device, a point cloud representative of said at least one portion of interest of the joint, and acquiring tracking information of the current position and orientation of said measuring device; wherein said intraoperative partial three-dimensional anatomical model is obtained from said point cloud;

[0067] - align the preoperative three-dimensional anatomical model with the intraoperative partial three-dimensional anatomical model;

[0068] - calculate, on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model, and the current position and orientation of the surgical drilling instrument:

[0069] * a current localization area, in a current intraoperative two-dimensional image, of a piercing end of the surgical piercing instrument,

[0070] * a first projection including a current estimate of a location area of ​​a first orifice of a bone tunnel to be made,

[0071] * a second projection comprising a current estimate of a location area of ​​a second orifice of the bone tunnel to be produced. Another aspect of the invention relates to a computer program product comprising instructions which cause the system described above to execute the steps of the method according to the present invention.

[0072] In one embodiment, the invention relates to a computer program product comprising instructions for implementing the following steps of a method for real-time assistance in the creation of at least one bone tunnel by arthroscopy in a joint of a patient during execution of the program by a processor of a programmable device:

[0073] - obtaining a preoperative three-dimensional anatomical model specific to the patient, said model comprising a representation of the patient's joint and a representation of anatomical structures of interest,

[0074] - acquiring, using an imaging device, a stream of intraoperative two-dimensional images, said intraoperative two-dimensional images comprising a portion of interest of the joint,

[0075] - acquiring tracking information of the position and orientation of a surgical drilling instrument during manipulation of the surgical drilling instrument by an operator using a tracking device,

[0076] - determine a partial intraoperative three-dimensional anatomical model from the intraoperative two-dimensional image stream and / or a point cloud representative of the portion of interest of the joint obtained using a measuring device,

[0077] - align the preoperative three-dimensional anatomical model with the intraoperative partial three-dimensional anatomical model;

[0078] - calculate, on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model and the current position and orientation of the surgical drilling instrument:

[0079] * a current localization area, in a current intraoperative two-dimensional image of a piercing tip of the surgical piercing instrument,

[0080] * a first projection including a current estimate of a location area of ​​a first orifice of a bone tunnel to be made,

[0081] * a second projection comprising a current estimate of a location area of ​​a second orifice of the bone tunnel to be made. In one embodiment, the invention relates to a method for real-time assistance in the creation of at least one bone tunnel by arthroscopy in a joint of a patient, comprising the following steps:

[0082] - obtaining a preoperative three-dimensional anatomical model specific to the patient, said model comprising a representation of the patient's joint and a representation of anatomical structures of interest,

[0083] - acquiring, using an imaging device, a stream of intraoperative two-dimensional images, said intraoperative two-dimensional images comprising a portion of interest of the joint,

[0084] - acquiring, using a tracking device, tracking information on the position and orientation of a surgical drilling instrument during manipulation of said surgical drilling instrument by an operator,

[0085] - determine a partial intraoperative three-dimensional anatomical model from the intraoperative two-dimensional image stream and / or a point cloud representative of the portion of interest of the joint obtained using a measuring device,

[0086] - align the preoperative three-dimensional anatomical model with the preoperative partial three-dimensional anatomical model;

[0087] - calculate, on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model, and the current position and orientation of the surgical drilling instrument:

[0088] * a current location area, in the two-dimensional image, of a piercing end of the surgical piercing instrument,

[0089] * a first projection including a current estimate of a location area of ​​a first orifice of a bone tunnel to be made,

[0090] * a second projection including a current estimate of a location area of ​​a second orifice of the bone tunnel to be created.

[0091] Another aspect of the invention relates to a computer-readable (non-transitory) medium, on which the computer program described above is recorded. DETAILED DESCRIPTION

[0092] The present invention relates to a real-time assistance system 1 for creating at least one bone tunnel by arthroscopy in a patient's joint. In the present description, the joint is one of the patient's knees. By bone tunnel is meant a tunnel made in a patient's bone, in particular the femur and the tibia, and intended to receive a neoligament.

[0093] The placement of a knee neoligament involves drilling the femoral and / or tibial ends for the passage of the neoligament, these drillings constituting the aforementioned bone tunnel.

[0094] DESCRIPTION OF FIGURES

[0095] Figure 1 is a representation of a real-time assistance system for creating a bone tunnel according to one or more embodiments of the invention.

[0096] Figure 2 represents an example of a preoperative three-dimensional model of a portion of interest in a patient's joint.

[0097] Figure 3 represents a flowchart of the steps carried out during a three-dimensional model reconstruction algorithm according to one or more embodiments.

[0098] Figure 4 represents a first type of correction carried out during step E20 of the flowchart in Figure 3.

[0099] Figure 5 represents a second type of correction made during step E20 of the flowchart in Figure 3.

[0100] Figure 6 shows a superposition of an intraoperative partial three-dimensional model and a preoperative three-dimensional model according to one or more embodiments of the invention.

[0101] Figure 7 shows a partitioning of a knee according to one or more embodiments of the invention. The sub-figures of Figure 8 are different examples of two-dimensional projections of parts of the knee and in different directions. Figure 8h and Figure 8i are schematic representations of a femoral epiphysis and a tibial epiphysis.

[0102] Figure 9 depicts a two-dimensional projection of a portion of interest of a patient's knee with multiple projections of anatomical structures of interest according to one or more embodiments.

[0103] Figure 10 depicts another two-dimensional projection of a portion of interest of a patient's knee with multiple projections of anatomical structures of interest according to one or more embodiments.

[0104] Figure 11 depicts an overlay of an intraoperative two-dimensional image with a common location area of ​​an end of a surgical piercing instrument according to one or more embodiments.

[0105] Figure 12a represents a superposition of a first orifice of a bone tunnel to be produced on a first two-dimensional projection according to one or more embodiments.

[0106] Figure 12b is another example of superimposing an intraoperative two-dimensional image with a common location area of ​​an end of a surgical piercing instrument according to one or more embodiments such as the superimposing of Figure 11.

[0107] Figure 12c represents a superposition of a second orifice of the bone tunnel to be produced from figure 12a on a second two-dimensional projection according to one or more embodiments.

[0108] Figure 13a shows an overlay of an alert message on the first two-dimensional projection of Figure 9.

[0109] Figure 13b is another example of overlaying an intraoperative two-dimensional image with a common location area of ​​an end of a surgical piercing instrument such as the overlay of Figure 11, according to one or more embodiments. Figure 13c shows an overlay of another alert message on the other two-dimensional projection of Figure 10 according to one or more embodiments.

[0110] Figure 14 represents an overlay of a positive message for the creation of a bone tunnel according to one or more embodiments.

[0111] Figure 15 represents a superposition of a representation of a bone tunnel produced on a two-dimensional projection according to one or more embodiments.

[0112] Figure 16 represents a superposition of a representation of a first bone tunnel already made and a provisional representation of a second bone tunnel to be made during the same intervention according to one or more embodiments.

[0113] Figure 17 is a flowchart of the steps carried out during a real-time assistance method for the creation by arthroscopy of at least one bone tunnel according to one or more embodiments.

[0114] ILLUSTRATIVE EMBODIMENTS OF THE INVENTION

[0115] The real-time assistance system 1, shown in FIG. 1, comprises an imaging device 2 capable of acquiring two-dimensional images of portions of the patient's joint, a tracking device 3 and a programmable device 6.

[0116] The imaging device 2 is, for example, an arthroscope. An arthroscope is a thin tube equipped with a miniaturized lens connected to a camera for viewing the inside of a joint. The arthroscope is inserted into the joint after incision. The diameter of the arthroscope is of the order of a few millimeters. During an arthroscopic intervention on the patient's joint, the imaging device 2 acquires a preferably continuous stream of two-dimensional images of all or part of the patient's joint.

[0117] Furthermore, during an arthroscopic procedure to create one or more bone tunnels, a surgeon uses a surgical drilling instrument 5 such as a surgical motor, comprising a drilling end. The drilling end makes it possible to create tunnels in the bone structures. For example, the end of the surgical drilling instrument 5 is a drill or a pin. The tracking device 3 comprises, for example, one or more cameras making it possible to locate and track in real time the position in space of the different instruments used during the procedure, such as the imaging device 2 and the surgical drilling instrument 5. On the imaging device 2 and the surgical drilling instrument 5 are fixed one or more markers configured to emit, receive, or reflect electromagnetic radiation, so as to be located by the tracking device 3.Advantageously, one or more passive markers may be attached to the imaging device 2 and the surgical drilling instrument 5, so as to be visible on images acquired by the tracking device 3. For example, the passive markers are QR codes, AR tags or 3D markers. Furthermore, the system 1 is configured to receive measurements obtained using a measuring device, for example a contact probe. This measuring device makes it possible to obtain a point cloud representative of the contact surface, in the case of the present invention of the surface of the portion of interest of the joint. The measuring device may be mounted on a robotic arm or be coupled with a marker rigidly attached to the probe itself. If a marker is used, it may be located in space by the tracking device 3 or another equivalent tracking device.Alternatively, the measuring device may be a depth camera (for example: “ToF or Time of Flight” cameras, “EiDAR”, Kinect camera, RGB-D camera, etc.) configured to acquire images, in grayscale or color, which indicate the distance between the camera and the objects in the scene. These images make it possible to obtain a 3D point cloud representative of the objects in the scene. In an alternative embodiment, the system 1 may further comprise said measuring device.

[0118] The position in space of the imaging device 2 and the drilling instrument 5 relative to the tracking device 3 can then be known by triangulation.

[0119] Advantageously, if the geometry of each tracked instrument such as the imaging device 2 and the drilling instrument 5 is known, the tracking device 3 directly recognizes them without providing them with markers and their respective position and orientation relative to the tracking device is known.

[0120] For example, the tracking device 3 is a Hololens™ type system for locating, using cameras, among other things, the imaging device 2 and the surgical drilling instrument 5 in an operating room. Thus, as soon as the imaging device 2 or the surgical drilling instrument 5 are in the field of vision of the cameras, the latter are located and their positions are calculated and recorded.

[0121] The programmable device 6 is a processing device which may be, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). The programmable device 6 may also comprise one or more graphics processors (GPUs).

[0122] The programmable device 6 is configured to execute instructions stored on a computer-readable medium, such as an integrated circuit, a hard disk, a CD, a DVD, RAM memory or ROM memory.

[0123] According to one embodiment, the real-time assistance system 1 is adapted to assist in the creation of one or more bone tunnels during an arthroscopic intervention in a knee of a patient requiring a graft of one or more ligaments.

[0124] Prior to the intervention, during a preoperative phase, a preoperative three-dimensional model 7 of a portion of interest of the knee 8 is obtained as described below. Such a preoperative three-dimensional model 7 is for example shown in Figure 2. The portion of interest of the knee 8 includes the areas of the knee where the graft of the ligament(s) is to be performed.

[0125] Data are previously obtained by magnetic resonance imaging (MRI). The data include different slices of the portion of interest of the knee 8. By "slice" is meant a two-dimensional image. The different slices are slices in planes parallel respectively to the sagittal, axial and coronal planes as defined in the field of magnetic resonance imaging. The MRI technique can be two-dimensional, i.e. based on sequential acquisition triplets, or three-dimensional, i.e. based on volume acquisitions. Alternatively, other techniques allowing the acquisition of several slices of areas of interest can be used, such as computational axial tomography.

[0126] A preoperative programmable device generates, by applying a reconstruction algorithm to the data, the preoperative three-dimensional model 7.

[0127] The magnetic resonance imaging technique has the advantage of providing data relating to anatomical structures not accessible by techniques such as radiography. Thus, the three-dimensional model 7 may comprise, for example, in addition to a representation of the bony contours of the femur, the tibia, the patella and the head of the fibula, a representation of the cartilaginous contours of the femur, the tibia, a complete or partial representation of the anterior cruciate ligament, the posterior cruciate ligament, the external lateral ligament, the internal lateral ligament, the patellofemoral ligament.

[0128] Advantageously, the reconstruction algorithm comprises two successive steps E10 and E20. Figure 3 represents a flowchart of the different steps of the reconstruction algorithm.

[0129] In a first step E10, the data acquired by MRI, for a plurality of subjects, are segmented and labeled, so as to generate a training database. A deep neural network of the CNN type (Convolutional Neural Network) can be trained on said training database so as to generate as output a segmentation of the anatomical structures when the network receives as input at least one 3D image or a collection of 2D slices, representing the portion of interest of the knee 8 of the patient (i.e., data acquired by MRI). According to a first option, a convolutional artificial neural network of the RCNN type (Region-based Convolutional Neural Network), such as the YOLACT++ architecture, can be used for example. The operation of such a network is for example described in the article “YOLACT++ Better Real-Time Instance Segmentation”.The model implemented by this type of artificial network, once trained, performs the prediction, in other words, the detection, segmentation and classification of structures present in each slice. Following the predictions, the slices are first merged by cutting plane (axial, sagittal or coronal) using a "slice matching" algorithm to generate three initial 3D models. Then, an interplane fusion of these three initial 3D models is carried out using a 3D / 3D registration. The preliminary 3D model generated is then obtained by refinement with a filtering algorithm and by elimination of outliers.

[0130] Other types of artificial neural networks can be used, such as RCNN (Fast, Faster, Mask RCNN), Computer Vision, U-net, MeshCNN, SDU-Net. As a second option, a 3D CNN+ type neural network (i.e., a type of convolutional neural network used for analyzing three-dimensional data) can be used.

[0131] Once option 1 or 2 has been carried out, a preliminary three-dimensional model can then be reconstructed, for example, by using techniques described in patents FR2920565B1 or US7123255.

[0132] The preliminary three-dimensional model may contain errors (e.g., poorly reconstructed anatomical structures) or present incomplete elements (e.g., consequences of the trauma suffered by the patient).

[0133] The second step E20 of the reconstruction algorithm aims to correct the preliminary three-dimensional model, based on lessons learned from the literature related to knee anatomy, as will be explained below.

[0134] A first type of correction carried out during the second stage is the repositioning of a structure misplaced during the generation of the preliminary three-dimensional model.

[0135] Figure 4, left, shows the femoral insertion structure of the anterior cruciate ligament circled. In this figure 4, this femoral insertion structure is malpositioned.

[0136] Thus, during the second step E20, this femoral insertion structure is repositioned to the correct location, noted H, as shown in Figure 4, right. The correct location is for example known from the literature on knee anatomy. Thus, the repositioning from the incorrect position to the correct position is for example performed manually by a designer in view of this knowledge. In another example, from geometric models described in the literature mentioned above (such as a quantified distance, a quantified radius of curvature), a curve or surface reconstruction algorithm will be applied to the incorrectly positioned element. In yet another example, the repositioning is performed using a neural network trained on a database of 3D models from step E10. For illustration purposes, a network such as the 3D CNN network can be used.

[0137] A second type of correction carried out during the second stage is the completion of incomplete elements.

[0138] Figure 5, left, illustrates, inside the oval shape K, a representation of the external lateral ligament in the preliminary three-dimensional model. This representation is incomplete. Thus, in the second step, this representation will be completed as illustrated in Figure 5, right.

[0139] Similar methods to those used to implement the first type of correction can be used to implement this second type of correction.

[0140] H will now be described how the real-time assistance system 1 works in the intraoperative phase, that is, during the arthroscopic intervention by a surgeon in the knee of the patient requiring a graft of one or more ligaments.

[0141] During the procedure, the surgeon manipulates a surgical drilling instrument 5 of the surgical motor type equipped with a drill or a pin. The surgical drilling instrument 5 is used to initially identify areas of skin to make incisions at the appropriate locations in the patient's knee in order to create the bone tunnel(s).

[0142] One aspect of the invention relates to the ability of the real-time assistance system 1 to enable visualization in real time, and before performing the surgical procedure, when the surgeon manipulates the surgical drilling instrument 5 in order to create a bone tunnel 10, a first orifice 11a representing the entrance to the bone tunnel and a second orifice 11b representing the exit from the bone tunnel. The first orifice 11a and the second orifice 11b are determined, as will be described later, as a function of the position and orientation, in real time, of the end of the surgical drilling instrument 5. By "entrance" of the bone tunnel 10, is meant the surface area of ​​the bone structure through which the end of the surgical drilling instrument 5 enters. By "exit" of the bone tunnel, is meant the surface area of ​​the bone structure through which the end of the surgical drilling instrument 5 exits.Ultimately, the system allows to see in real time the position that the entry and exit orifices (respectively orifices 11a and 11b) would occupy if the surgeon continued the drilling operation with the drilling instrument along a trajectory defined by the position and orientation of the instrument.

[0143] Thus, by visualizing in real time the location of the first orifice 11a and the second orifice 11b, the surgeon will be able to adjust the position and orientation in space, relative to the knee, of the surgical drilling instrument 5, to correctly position the bone tunnel 10 relative to anatomical structures of interest which are known anatomical landmarks, and without damaging “noble” anatomical structures, such as neighboring tendon or ligament structures.

[0144] Prior to the intervention, the programmable device 6 received the preoperative three-dimensional model 7 generated by the preoperative programmable device. The preoperative three-dimensional model 7 is for example stored in a memory associated with the programmable device 6 or in a database accessible by the programmable device 6.

[0145] The position of the patient's knee relative to the surgical piercing instrument 5 and the imaging device 2 must be known at all times.

[0146] To do this, for example, markers, such as pins, can be positioned in the femur or tibia to create a reference frame R. The reference frame R is used to know the position of the knee to be operated on relative to the surgical drilling instrument 5. The tracking device 3 makes it possible to capture information on the position and orientation of the markers attached to the patient. The reference frame R makes it possible to carry out the registration of the preoperative three-dimensional model 7 on the patient's knee. Thus, the real-time position of the patient's knee in the operating room space is known.

[0147] Alternatively, since the imaging device 2 acquires a stream of images, the position of the knee in the space of the operating room can be known from the stream of images. Indeed, it is possible to obtain, from the stream of images, a current partial three-dimensional model and to re-register the preoperative three-dimensional model 7 on the current partial three-dimensional model. In particular, the distance between the imaging device 2 and the anatomical structures of which the latter produces the image can be known. Thus, the position of the knee in space can be known after re-registering.

[0148] The programmable device 6 of the real-time assistance system 1 can be configured to control all of the equipment used during the intervention. Thus, the programmable device 6 is configured to receive signals from the tracking device 3, and calculate, on the basis of these signals, the position and orientation in real time in the space of the operating room of the elements of interest tracked by the tracking device 3 (i.e., the patient's femur and tibia, the imaging device 2, the surgical drilling instrument 5 and optionally the measuring device).

[0149] A partial intraoperative three-dimensional model 9 of the portion of interest of the knee 8 is generated by the programmable device 6. The term “partial” refers to the fact that the model concerns only all or part of the portion of interest of the knee and not the entire anatomical structure of the knee. It is recalled that the portion of interest of the knee 8 includes the areas of the knee where the graft of the ligament(s) is to be performed.

[0150] Different methods can be used to generate the intraoperative partial three-dimensional model 9.

[0151] In one embodiment, an operator (who may be the surgeon or any other qualified person assisting with the procedure) palpates points on the surface of the patient's knee using a probe (i.e., the measuring device), on which a marker is attached. The marker can be located. The marker attached to the probe makes it possible to know the position in space of each point of the cloud of points obtained. Alternatively, a probe mounted on a robotic arm can be used. In an alternative embodiment, a depth camera (i.e., the measuring device) can be used to acquire a cloud of points representative of the surface of the portion of interest of the knee 8. In all cases, the measuring device makes it possible to acquire the cloud of points representative of the portion of interest of the joint, in particular representative of the surface of the portion of interest of the joint.An algorithm then generates at least one three-dimensional surface from this point cloud, corresponding for example to a part of the femoral condyle in three dimensions. A “bone morphing” algorithm can for example be used. This (these) algorithm(s) can be executed by the programmable device 6 in order to generate the intraoperative partial three-dimensional model 9. Alternatively, an auxiliary programmable device can execute the algorithm(s) and send the generated intraoperative partial three-dimensional model 9 to the programmable device 6.

[0152] In another example, the two-dimensional images acquired by the imaging device 2 can be used to generate the intraoperative partial three-dimensional model 9 in real time using depth maps generated from the image stream recorded by the imaging device 2 and a SLAM (simultaneous localization and mapping) algorithm executed by the programmable device 6. Here, the image stream is monocular. Thanks to the tracking device 3, the position in space of the imaging device 2 which generates the video stream from which the partial three-dimensional model is generated is known in real time, thus the three-dimensional position of the intraoperative partial three-dimensional model 9 in space and also the position in space of the preoperative three-dimensional model 7 is known in real time.

[0153] Advantageously, a SLAM type algorithm is used to calculate the current partial three-dimensional model 8. SLAM type algorithms make it possible, from the use of two-dimensional images, acquired by an image sensor, and the position in space of the image sensor, to produce, by means of predictions of depth maps associated with the two-dimensional images, a three-dimensional map of the scene observed in the two-dimensional images. The three-dimensional map obtained can then be located in a reference system associated with the image sensor which acquires the two-dimensional images. Furthermore, this three-dimensional map is dynamic. In other words, it can be obtained in real time, as the image sensor acquires the two-dimensional images of the scene.A description of SLAM-type algorithms can be found in the article “Visual SLAM algorithms: a survey from 2010 to 2016”, T. Taketomi et al., IPSJ Transactions on Computer Vision and Applications (2017).

[0154] Once the intraoperative partial three-dimensional model 9 has been generated, the programmable device 6 executes, in real time, a 3D / 3D registration algorithm making it possible to superimpose in real time the preoperative three-dimensional model 7 on the intraoperative partial three-dimensional model 9. A 3D / 3D registration advantageously makes it possible to obtain a more precise superposition than in the case of a 2D / 3D registration. Such a superposition is represented in FIG. 6. For example, an algorithm minimizing the distance between corresponding points can be used.Thus, a superposition S(t) can be displayed on a display screen visible to the surgeon, so that the latter can visualize in real time, on the basis of the correspondence between the preoperative three-dimensional model 7 and the intraoperative partial three-dimensional model 9, the position of the anatomical structures represented in the preoperative three-dimensional model relative to the actual position of the structures represented in the intraoperative partial three-dimensional model 9. Indeed, precise registration makes it possible to precisely locate the drilling tool and thus better calculate the information making it possible to anticipate the position of the bone tunnel, thus reducing the risks of error for the surgeon.

[0155] For example, the S(t) superposition can be used to locate skin incision areas on the skin.

[0156] Furthermore, once the intraoperative partial three-dimensional model 9 is superimposed on the preoperative three-dimensional model 7, three parts of the patient's knee can be defined on the superposition S(t), for example, by implementing a division algorithm by the programmable device 6. For example, the division algorithm creates in real time three volumes on the intraoperative partial three-dimensional model 9 by: separating the femur and the tibia by a plane passing through the joint space; separating the femur into two compartments (internal and external) at a sagittal plane perpendicular to the plane of the joint space and passing through the apex of the notch. As shown in Figure 7, a first part PGI corresponds to the volume occupied by the external condyle; a second part PG2 corresponds to the volume occupied by the internal condyle; finally, a third part PG3 corresponds to the volume occupied by the patient's tibial epiphysis.

[0157] Furthermore, prior to the operation of creating the bone tunnels by the surgeon, the programmable device 6 executes an algorithm for calculating and generating a plurality of two-dimensional projections of the preoperative three-dimensional model 7. This generation of two-dimensional images makes it possible to represent or display the preoperative three-dimensional model, which is a 3D object, on a two-dimensional support such as a computer screen. By projection, we mean a view of the preoperative three-dimensional model 7. A geometric perspective projection or an orthogonal (or parallel) projection can be calculated and generated.

[0158] For example, the calculation algorithm first separates the femur from the tibia along a plane passing through the joint space. The femoral epiphysis is obtained from which the first part PGI and the second part PG2 are determined after dividing this femoral epiphysis along a sagittal plane passing through the center of the roof of the notch. The tibial epiphysis is also obtained from which the third part PG3 is determined. These three parts (i.e. the three volumes on the intraoperative partial three-dimensional model 9) PGI, PG2 and PG3 are then used to generate said plurality of two-dimensional projections.

[0159] These two-dimensional projections are stored in the memory associated with the programmable device 6 and will be dedicated, as will be described later, to an automatic display determined as a function of the current position at a time t of the different elements of the real-time assistance system. With reference to figures 8a to 8g, these projections are respectively:

[0160] - an external projection (or lateral view) of the external condyle Pi, illustrated in Figure 8a and 8h,

[0161] - an internal projection (or medial view) of the external condyle P2, illustrated in figures 8b and 8h (i.e., image of the section along a sagittal plane passing through the center of the roof of the notch),

[0162] - an external projection (or medial view) of the internal condyle P3, illustrated in figure 8c and 8h,

[0163] - an internal projection (or lateral view) of the internal condyle P4, illustrated in Figure 8d and 8h (i.e., image of the section along a sagittal plane passing through the center of the roof of the notch),

[0164] - an axial projection (or view) of the P5 tibia, from the illustrated in figure 8e and 8i,

[0165] - a projection of the external tibia, or lateral view of the tibia PO, illustrated in Figure 8f and 8i,

[0166] - a projection of the internal tibia, or medial view of the P7 tibia, illustrated in Figure 8g and 8i.

[0167] These different projections contain the corresponding projections of the anatomical structures of interest.

[0168] Figure 9 is an example of the external projection (or lateral view) of the external condyle Pi. It can be observed: the femoral insertion zone A of the popliteal tendon, the femoral insertion zone B of the external lateral ligament, and the insertion zone of the anterolateral ligament C. These different elements represent the projections of the corresponding structures in the preoperative three-dimensional model 7 from step E20.

[0169] Figure 10 is an example of the inner projection (or medial view) of the lateral condyle P2. Here can be seen: the insertion surface D of the anterior cruciate ligament, the inferior cartilaginous limit E, the proximal limit of the intercondylar notch and the roof of the notch.

[0170] Typically, in Figure 9, the insertion area of ​​the anterolateral ligament was not identifiable in the preliminary three-dimensional model obtained during the preoperative T1 phase of magnetic resonance imaging data acquisition and three-dimensional model generation 7. This area was typically reconstructed during the second step (E20) of the reconstruction algorithm aimed at correcting the preliminary three-dimensional model.

[0171] During the intervention, the imaging device 2 of the real-time assistance system 1 captures in real time a continuous stream of two-dimensional images. At each instant t, the surgeon can view on a viewing device 12 such as a screen a current two-dimensional image I(t) of the portion of interest of the knee 8.

[0172] Furthermore, still during the intervention, the programmable device 6 receives in real time a signal from the tracking device 3, on the basis of which it calculates the position and orientation Di(t) in real time of the imaging device 2 relative to the reference frame R. The position and orientation Di(t) comprise for example three coordinates and three angles.

[0173] Also, still during the intervention, the programmable device 6 receives in real time a signal from the tracking device 3, on the basis of which it calculates the position and orientation Ip(t) in real time of the piercing end of the surgical piercing instrument 5. The position and orientation Ip(t) comprise for example three coordinates and three angles.

[0174] Thus, with the real-time assistance system 1, it is possible to superimpose a current location area L of the piercing end of the surgical piercing instrument 5 on the current two-dimensional image I(t). Such a superposition is shown in Figure 11.

[0175] It will now be described how, in another aspect of the invention, the real-time assistance system 1 allows the surgeon to adequately position and orient the surgical drilling instrument 5, before drilling the bone tunnel, so as not to damage anatomical structures to be preserved during drilling.

[0176] A first screening P aand a second projection Pb among the projections Pi to P7 will be displayed on a first auxiliary viewing screen and a second auxiliary viewing screen, according to the current position and orientation of the end of the surgical piercing instrument 5 and / or the imaging device 2. By "current", it is understood current, that is to say corresponding to the current instant t of the intervention.

[0177] At the current time t of the intervention, the current position and orientation of the end of the surgical piercing instrument 5 and / or the imaging device 2 are known thanks to the second tracking device 3 and are collected and recorded by the programmable device 6.

[0178] Depending on the position and / or orientation in one of the first part PGI, the second part PG2 and the third part PG3, the selection of the projections P a and Pb is carried out according to the following rules.

[0179] If the tip of the surgical piercing instrument 5 is located in (or near) the first part PGI (volume occupied by the external condyle), the first projection P a is the outer projection (or lateral view) of the external condyle Pi, and the second projection Pb is the inner projection (or medial view) of the external condyle P2.

[0180] The real-time position of the first part PGI, the second part PG2, and the third part PG3, is determined using real-time knowledge of the position of the patient's knee in the operating room space. Indeed, the first part PGI, the second part PG2, and the third part PG3 are calculated by the division algorithm previously described, applied to the preoperative three-dimensional model 7. The latter is realigned in real time on the intraoperative partial three-dimensional model 9, whose position in the operating room space is known using the real-time position of the patient's knee.

[0181] If the tip of the surgical piercing instrument 5 is located in (or close to) the second part PG2 (volume occupied by the internal condyle), the first projection P ais the outer projection (or medial view) of the internal condyle Pi, and the second projection Pb is the inner projection (or lateral view) of the internal condyle P2.

[0182] If the tip of the surgical piercing instrument 5 is located in (or near) the third part PG3 (volume occupied by the tibial epiphysis) and is oriented downward, the first projection P a is the projection of the external tibia (or lateral view of the tibia) PÔ, and the second projection Pb is the projection of the internal tibia (or medial view of the tibia) P7.

[0183] If the tip of the surgical piercing instrument 5 is located in (or close to) the third part PG3 (volume occupied by the tibial epiphysis) and is oriented upwards, i.e. so as to cross the plane of the joint space), the first projection P ais displayed and corresponds to the axial projection of the tibia P5, and the second projection Pb corresponding to the projection of the internal tibia (or medial view of the tibia) P7.

[0184] These rules have been determined in order to facilitate and make more intuitive the handling of the drilling instrument 5 by ensuring the consistency of the selection of the first projection P a and the second projection Pb with the current position and orientation of the surgical piercing instrument 5. Other rules making it possible to facilitate and make more intuitive the handling of the piercing instrument 5 can be determined by those skilled in the art.

[0185] Advantageously, when the surgeon positions and orients the end of the surgical drilling instrument 5 in order to create a bone tunnel 10, the programmable device 6 executes an algorithm for calculating a current estimate of the current location zone L, in the current intraoperative two-dimensional image I(t), of the drilling end of the surgical drilling instrument 5. In addition, the programmable device 6 executes an algorithm for calculating a current estimate of a location zone of a first orifice 11a of the bone tunnel 10 to be created, in the first projection P a, and a current estimate of a location area of ​​a second orifice 11b of the bone tunnel 10 to be made, in the second projection Pb, for said current position and orientation Ip(t) of the end of the surgical drilling instrument 5. The calculations are carried out on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model with a partial intraoperative three-dimensional anatomical model (on the basis of knowledge of the real-time position in the space of the operating room of the preoperative three-dimensional model - resulting from step E20 - recalibrated in real time on the intraoperative three-dimensional model 9) and of the current position and orientation Ip(t) of the surgical drilling instrument 5. Advantageously, the location area of ​​the first orifice I la of the bone tunnel 10 to be made is superimposed on the first projection P a. Similarly, advantageously, the location zone of the second orifice 11b of the bone tunnel 10 to be produced is superimposed on the second projection Pb. In particular, the programmable device 6 is configured to produce a superposition image to be displayed on the display unit comprising this superposition of the location zone of the first orifice 11a of the bone tunnel 10 on the first projection P a . Similarly, the programmable device 6 is configured to produce a superposition image to be displayed on the display unit comprising this superposition of the location area of ​​the second orifice 11b of the bone tunnel 10 on the second projection Pb.

[0186] The first orifice 11a corresponds for example to the inlet orifice of the bone tunnel 10 and the second orifice 11a then corresponds to the outlet orifice of the bone tunnel 10, or vice versa.

[0187] Figure 12a represents the external projection of the external condyle Pi of Figure 8a, where a circle representative of the location area of ​​the first orifice 11a of the bone tunnel 10 has been superimposed. In Figure 12a, the circle intersects with the femoral insertion area of ​​the popliteal tendon B. Such a position of the first orifice 11a of the bone tunnel 10 is therefore not possible because the latter would damage the femoral insertion area of ​​the popliteal tendon B.

[0188] Figure 12c represents the inner projection of the external condyle P2 (i.e., section) of Figure 8b, where a circle representative of the location area of ​​the second orifice 11b of the bone tunnel 10 has been superimposed. In Figure 12c, the circle intersects with the insertion area of ​​the anterolateral ligament E. Such a position of the second orifice 11b of the bone tunnel 10 is thus not possible because the latter would damage the femoral insertion area of ​​the anterolateral ligament.

[0189] Advantageously, the superposition of the current two-dimensional image I(t) and the current location zone L of the end of the drilling instrument 5 (figure 12b) can be displayed between these two superpositions, by aligning figures 12a, 12b and 12c. Thus, by real-time visualization of the location zone of the first orifice 11a of the bone tunnel 10 to be made and the location zone of the second orifice 11b of the bone tunnel 10 to be made, the surgeon can adaptively move the surgical drilling instrument 5 in order to avoid any intersection with anatomical structures to be preserved represented in the projections.

[0190] Advantageously, the programmable device 6 executes a first algorithm for calculating the intersection of the location zone of the first orifice 11a of the bone tunnel 10 with the projections of the anatomical structures present in the first projection P aThe first intersection calculation algorithm is carried out on the basis of knowledge of the real-time position in the operating room space of the preoperative three-dimensional model 9 (from step E20) recalibrated in real time on the intraoperative three-dimensional model 7.

[0191] Likewise, advantageously, the programmable device 6 executes a second algorithm for calculating the intersection of the location zone of the second orifice 11b of the bone tunnel 10 with the projections of the anatomical structures present in the second projection Pb. The second intersection calculation algorithm is carried out on the basis of knowledge of the position in real time in the space of the operating room of the preoperative three-dimensional model 9 (from step E20) recalibrated in real time on the intraoperative three-dimensional model 7.

[0192] Advantageously, when the first intersection calculation algorithm results in an intersection comprising part of the projections of the anatomical structures present in the first projection P a (ie, detection of an intersection between the current estimate of the localization zone of the first orifice and a part of the projections of the anatomical structures present in the first projection P a ) or when the second intersection calculation algorithm results in an intersection comprising part of the projections of the anatomical structures present in the second projection Pb (i.e., detection of an intersection between the current estimate of the location zone of the second orifice and part of the projections of the anatomical structures present in the second projection Pb), the programmable device 6 can execute a display algorithm, on the first projection P a, or respectively, the second projection Pb, of an information message in order to warn the surgeon of the danger of damage for example.

[0193] Figures 13a and 13c show examples of such superpositions (i.e., superposition images). In Figure 13a, the superposition image further comprises the information message “LLE” meaning that there is a detected intersection of an orifice to be made with the femoral insertion of the external lateral ligament B. In Figure 13c, the superposition image further comprises the information message “cartilage” meaning that there is an intersection between the second orifice 11b to be made and the lower cartilaginous limit.

[0194] Advantageously, when the first intersection calculation algorithm results in an intersection not comprising a noble anatomical structure of interest, and the second intersection calculation algorithm results in an intersection not comprising a noble anatomical structure of interest, the programmable device 6 can execute a display algorithm, on the first projection P a , or respectively, the second projection Pb, of a positive message aimed at warning the surgeon that the positioning and orientation of the surgical instrument 5 are reliable and that he can proceed with drilling the bone tunnel 10.

[0195] Figure 14 shows an example of such an overlay. In Figure 14, the information message “Safe” means that the position and orientation of the tip of the surgical drilling instrument 5 are compatible with safe drilling of the bone tunnel 10.

[0196] In another aspect of the invention, the real-time assistance system 1 also makes it possible to assist the surgeon in creating several bone tunnels during the same operation.

[0197] It is assumed here that a first bone tunnel 10a has been made in the patient's knee and that a second bone tunnel 10b is also to be made in the patient's knee.

[0198] Based on the position and orientation Di(ti) of the tip of the surgical drilling instrument 5 that were used at time ti to make the first bone tunnel, the programmable device 6 executes a calculation algorithm aimed at calculating a three-dimensional representation of the first bone tunnel 10a (already made and present in the bone) as well as the location of this representation in the superposition S(t) (i.e., superposition image calculated and displayed in real time) as follows.

[0199] The calculation algorithm is executed on the basis of knowledge of the real-time position in the operating room space of the preoperative three-dimensional model 9 (from step E20) realigned in real time on the intraoperative three-dimensional model 7, and on the basis of knowledge of the real-time position in the operating room space of the surgical drilling instrument 5. The algorithm thus makes it possible to record the trajectory of the surgical drilling instrument 5 when it penetrates and is moved inside a bone structure. The recorded trajectory can then be displayed in real time. The three-dimensional representation of the first bone tunnel 10a corresponds to the recorded trajectory.

[0200] For example, the three-dimensional representation of the first bone tunnel 10a may be displayed on either the overlay S(t). In another example, a two-dimensional view of the three-dimensional representation of the first bone tunnel 10a may be calculated and displayed as an overlay on one of the first projection P a and the second projection Pb.

[0201] It is assumed that at the current time t, the surgeon is positioning the end of the drilling instrument 5 in order to create the second bone tunnel 10b.

[0202] Advantageously, the programmable device 6 executes a display algorithm, on the first projection P a current and on the second current Pb projection, of the representation of the first bone tunnel 10a already made.

[0203] In other words, the system is configured to generate a two-dimensional view of the three-dimensional representation of the first bone tunnel 10a already made and produce an overlay image which comprises an overlay of the first projection P a . This two-dimensional view can be displayed (or rendered) on the overlay image using a first pattern, for example a dotted or hatched pattern. In the same way, the system is configured to generate a two-dimensional view of the three-dimensional representation of the first bone tunnel 10a already made and to produce an overlay image which comprises an overlay of the second projection Pb. This two-dimensional view can be displayed (or rendered) on the overlay image using a second pattern, identical to or different from the first, for example a dotted or hatched pattern.

[0204] Figure 15 illustrates an example of such a display, where the two-dimensional view of the three-dimensional representation of the first bone tunnel 10a is represented in hatching (i.e. pattern).

[0205] In an alternative example, the representation of the first bone tunnel is displayed as a digital hologram superimposed on the first projection P a current, respectively the second current Pb projection. This allows the surgeon to have a three-dimensional view of his workspace. For example, the surgeon can view the first projection P a common through augmented reality glasses, such as viewing glasses of a Hololens™ system.

[0206] In order to avoid intersections between the first bone tunnel 10a already made and the second bone tunnel 10b to be made, the programmable device 6 advantageously executes, in addition to the first intersection calculation algorithm and the second intersection calculation algorithm, a third intersection calculation algorithm between the trajectory of the second bone tunnel 10b to be made which is calculated in real time from the position in the space of the operating room of the drilling instrument 5 and the position in the space of the pre-operative three-dimensional model 7 and the representation of the first bone tunnel 10a. Similarly, the programmable device 6 advantageously executes a fourth intersection calculation algorithm between the trajectory of the second bone tunnel 10b to be made and the representation of the first bone tunnel 10a.

[0207] Advantageously, when the third intersection calculation algorithm results in a non-empty intersection, i.e. comprising a part of the representation of the first bone tunnel 10a previously produced, or when the fourth intersection calculation algorithm results in a non-empty intersection, i.e. comprising a part of the representation of the first bone tunnel 10a previously produced, the programmable device 6 can execute a display algorithm, on the first projection P a , or respectively, the second projection Pb, of an information message in order to warn the surgeon of the risk of intersection with the first bone tunnel 10a already made.

[0208] Advantageously, when the third intersection calculation algorithm results in an empty intersection, and the fourth intersection calculation algorithm results in an empty intersection, the programmable device 6 can execute a display algorithm, on the first projection P a , or respectively, the second projection Pb, of a positive message aimed at warning the surgeon that the positioning and orientation of the surgical instrument 5 are reliable and that he can proceed with drilling the second bone tunnel 10b.

[0209] Figure 16 illustrates a projection among the first projection P aand the second projection Pb, on which have been superimposed a two-dimensional view of the three-dimensional representation of the first bone tunnel 10a already made and a two-dimensional view of the predicted three-dimensional representation of the second bone tunnel 10b to be made corresponding to the current position and orientation of the end of the drilling instrument 5. Such an illustration of the superposition makes it possible to adjust the position and the current orientation of the end of the drilling instrument 5 in order to avoid any interference between the already existing first bone tunnel 10a and the second bone tunnel 10b to be made. If there is no intersection between the already made first bone tunnel 10a and the trajectory of the second bone tunnel 10b to be made, the two-dimensional view of the three-dimensional representation of the latter is displayed in full, as in FIG. 16.If there is an intersection, an information message is displayed instead of the trajectory of the second bone tunnel 10b.

[0210] Another aspect of the invention relates to a method for real-time assistance in the creation of at least one bone tunnel 10 by arthroscopy in a joint of a patient, comprising the following steps:

[0211] - receiving, El 00, a preoperative three-dimensional anatomical model 7 specific to the patient, said model comprising a representation of the patient's joint and a representation of anatomical structures of interest,

[0212] - acquiring, E200, using an imaging device 2, a stream of intraoperative two-dimensional images, said intraoperative two-dimensional images comprising a portion of interest of the joint, and acquiring tracking information of the current position and orientation Di(t) of the imaging device using the tracking device 3, or alternatively, acquiring, using a measuring device, a point cloud representative of the portion of interest of the joint, and acquiring tracking information of the current position and orientation of said measuring device

[0213] - acquiring, E300, using a tracking device 3 of a surgical drilling instrument 5, tracking information on the position and orientation Ip(t) of said surgical drilling instrument 5 during manipulation of said surgical drilling instrument 5 by an operator,

[0214] - determine, E400, a partial intraoperative three-dimensional anatomical model 9 from the intraoperative two-dimensional image stream (and the tracking information of the current position and orientation Di(t) of the imaging device) or alternatively, from the point cloud representative of the portion of interest of the joint obtained using the measuring device (and the tracking information of the current position and orientation of the measuring device),

[0215] - re-align, E500, the intraoperative partial three-dimensional anatomical model 9 on the preoperative three-dimensional anatomical model 7;

[0216] - calculate, E600, on the basis of a current intraoperative two-dimensional image I(t) including the portion of interest of the joint:

[0217] * a current localization area, in the current intraoperative two-dimensional image I(t), of a drilling end of the surgical drilling instrument 5,

[0218] * a first projection comprising a current estimate of a location zone of a first orifice 11a of a bone tunnel 10 to be produced;

[0219] * a second projection comprising a current estimate of a location zone of a second orifice 11b of the bone tunnel 10 to be produced.

[0220] The different steps of this method are illustrated in the flowchart in Figure 17.

Claims

CLAIMS System (1) for assisting in the creation of at least one bone tunnel (10) by arthroscopy in a patient's joint using a surgical drilling instrument (5), comprising an imaging device (2) or a measuring device, a tracking device (3) and a programmable device (6), said programmable device (6) being suitable for: - obtaining a preoperative three-dimensional anatomical model (7) specific to the patient, said model comprising a representation of at least one portion of interest of the patient's joint and a representation of anatomical structures of interest, - acquiring tracking information of the current position and orientation of the surgical drilling instrument (Ip(t)) during manipulation of said surgical drilling instrument (5) by an operator using the tracking device (3); - determining a partial intraoperative three-dimensional anatomical model (9), comprising a representation of at least one portion of interest of the joint, by: - acquiring, using the imaging device (2), a stream of intraoperative two-dimensional images comprising said at least one portion of interest of the joint, and acquiring tracking information of the current position and orientation (Di(t)) of the imaging device (2) using the tracking device (3); wherein said intraoperative partial three-dimensional anatomical model (9) is obtained from the stream of intraoperative two-dimensional images and said tracking information of the current position and orientation (Di(t)), or - acquiring, using the measuring device, a point cloud representative of the at least one portion of interest of the joint, and acquiring tracking information of the current position and orientation of said measuring device; in which said intraoperative partial three-dimensional anatomical model (9) is obtained from said point cloud; - re-align the preoperative three-dimensional anatomical model (7) on the model partial intraoperative three-dimensional anatomical (9); - calculate, on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model, and the current position and orientation (Ip(t)) of the surgical drilling instrument (5): * a current localization area (L), in a current intraoperative two-dimensional image (I(t)), of a drilling end of the surgical drilling instrument (5), * a first projection (PA) comprising a current estimate of a location zone of a first orifice (l ia) of a bone tunnel (10) to be produced, * a second projection (PB) comprising a current estimate of a location zone of a second orifice (11b) of the bone tunnel (10) to be produced. System (1) according to claim 1, in which the programmable device is further configured to calculate: - in the first projection (PA), first projections of a first part of the representation of the anatomical structures of interest during an intervention phase, and / or - in the second projection (PB), second projections of a second part of the representation of the anatomical structures of interest. System (1) according to claim 2, in which the anatomical structures of interest comprise bone reliefs, cartilaginous limits, tendon or ligament structures. System (1) according to one of the preceding claims, further comprising a display unit and further adapted to display on the display unit: * a superposition of the current intraoperative two-dimensional image (I(t)) and the current localization area (L) of the piercing end of the surgical piercing instrument (5), * a first superposition of the first projection (PA) and the first projections of the first part of the representation of the anatomical structures of interest, * a second superposition of the second projection (PB) and the second projections of the second part of the representation of the anatomical structures of interest.

5. System (1) according to one of the preceding claims, further adapted to, before obtaining the preoperative three-dimensional anatomical model (7), obtain and store a plurality of preoperative two-dimensional images comprising the portion of interest of the joint.

6. System (1) according to one of the preceding claims, further adapted to calculate an intersection between the current estimate of the location zone of the first orifice (11a), respectively of the second orifice (11b), current with the first projections of the first part, respectively the second projections of the second part of the representation of the anatomical structures of interest.

7. System (1) according to claims 2, 4 and 6, further adapted, when the intersection with the first, respectively second, projections, comprises a portion of the first projections of the first part of the representation of the anatomical structures of interest or a portion of the second projections of the second part of the representation of the anatomical structures of interest, to display an information message on the first, respectively, the second superposition.

8. System (1) according to any one of claims 1 to 6, further adapted to, after creation of a first bone tunnel (10a) in the patient's joint, calculate a three-dimensional representation of the first bone tunnel (10a) and to calculate, on the first (PA), respectively second (PB), current projection, a first, respectively second, two-dimensional view of said three-dimensional representation of the first bone tunnel (10a).

9. System (1) according to claim 8, further adapted to display, on the first projection (PA), the first two-dimensional view and, on the second projection (PB), the second two-dimensional view. System (1) according to one of claims 8 to 9, further adapted to calculate an intersection between the current estimate of the location area of ​​the first (11a), respectively of the second (11b), orifice and on the one hand the first two-dimensional view, on the other hand, the second two-dimensional view. System (1) according to claim 10 and claim 4, further adapted to display an information message on the first, respectively second, superposition, when the intersection with the first, respectively the second two-dimensional view, comprises at least a portion of the first two-dimensional view, respectively a portion of the second two-dimensional view. A computer-implemented method for assisting in the creation of at least one bone tunnel (10) by arthroscopy in a joint of a patient using a surgical drilling instrument (5), said method comprising: - obtaining a preoperative three-dimensional anatomical model (7) specific to the patient, said model comprising a representation of at least one portion of interest of the patient's joint and a representation of anatomical structures of interest, - acquiring tracking information of the current position and orientation of the surgical drilling instrument (Ip(t)) during manipulation of said surgical drilling instrument (5) by an operator using the tracking device (3); - determining a partial intraoperative three-dimensional anatomical model (9), comprising a representation of at least one portion of interest of the joint, by: - acquiring, using the imaging device (2), a stream of intraoperative two-dimensional images comprising said at least one portion of interest of the joint, and acquiring tracking information of the current position and orientation (Di(t)) of the imaging device (2) using the tracking device (3); wherein said intraoperative partial three-dimensional anatomical model (9) is obtained from the stream of intraoperative two-dimensional images and said tracking information of the position and orientation current (Di(t)), or - acquiring, using the measuring device, a point cloud representative of said at least one portion of interest of the joint, and acquiring tracking information on the current position and orientation of said measuring device; in which said intraoperative partial three-dimensional anatomical model (9) is obtained from said point cloud, - re-aligning the preoperative three-dimensional anatomical model (7) on the intraoperative partial three-dimensional anatomical model (9); - calculate, on the basis of information obtained with the recalibration of the preoperative three-dimensional anatomical model, and the current position and orientation (Ip(t)) of the surgical drilling instrument (5): * a current localization area (L), in a current intraoperative two-dimensional image (I(t)), of a drilling end of the surgical drilling instrument (5), * a first projection (PA) comprising a current estimate of a location zone of a first orifice (l ia) of a bone tunnel (10) to be produced, * a second projection (PB) comprising a current estimate of a location area of ​​a second orifice (11b) of the bone tunnel (10) to be produced. Computer program product comprising instructions which cause the system (1) according to claim 1 to execute the steps of the method according to claim 12. Computer-readable medium, on which the computer program according to claim 13 is recorded.

Images