Method and device for segmenting and registering pre-operative models of anatomical structures - Patents.com

Abstract

The present invention relates to a method and device (1) for the segmentation of a pre-operative model (21) of a target anatomical structure, taking into account the registration of the pre-operative model with an image of the target anatomical structure exposed during surgery.

Description

[Technical field]

[0001] The present invention relates to the field of image analysis in the medical domain, in particular to analyzing images acquired during surgery that include a target anatomical structure, allowing the registration of a pre-operative model of said target anatomical structure to the image. [Background technology]

[0002] Nowadays, computer and robotic assisted surgery has evolved thanks to the improvements achieved by the use of computer methods and robotic devices to plan and execute surgical interventions. In many cases, the alignment of various references (i.e., coordinate systems), often through the matching of three-dimensional (3D) data sets, is a fundamental step to successfully link surgical planning and execution.

[0003] In orthopedic surgery, several methods have already been developed that involve registration routines that typically use fixed fiducial markers that are surgically implanted directly into the patient's bones. Replacing this invasive routine with a procedure that does not require fixed fiducials for registration is an important step toward minimizing surgical invasiveness.

[0004] The registration process allows the computation of a transformation that links different coordinate systems: registration is the matching of spatial datasets that are defined in different coordinate systems.

[0005] A registration algorithm in biomedical applications is proposed to match a preoperative model of an anatomical structure to a data set acquired by an imaging system during surgery. The preoperative model includes a set of data points that define a 3D model of the anatomical target structure on which the surgical procedure will be performed. It is built before the procedure is performed based on medical images acquired before the procedure, such as CT scan, MRI, PET, etc. However, this model does not include soft tissues or artificial structures around the anatomical structure that may be present during the procedure, which may affect the accuracy of the registration. The present invention provides a solution to filter the preoperative model to obtain an optimal registration of the preoperative model of the target anatomical structure during the procedure with the known coordinate system of the operating room. Summary of the Invention

[0006] The invention therefore relates to a device for segmentation of a pre-operative model of a target anatomical structure, taking into account a registration between the pre-operative model and an image of the target anatomical structure exposed during surgery, said device comprising: at least one input section, at least one 3D image acquired from at least one 3D imaging sensor, the data points of the 3D image representing at least one exposed portion of a target anatomical structure; and a pre-operative model including data points defined in a model reference having at least one first axis; An input configured to receive at least one processor, Initially aligning at least one portion of the pre-operative model to at least one portion of the 3D image; Define a current subset of data points that includes data points from the preoperative model; generating an intermediate subset of data points of the pre-operative model that includes the data points of the current subset of data points and further includes at least one group of further data points of the pre-operative model that are located along a first axis outside a portion of the pre-operative model that corresponds to the current subset of data points; Compute a distance measure that represents the distance between each data point of the intermediate subset of data points and a corresponding data point of the 3D image; Determining whether the distance measure is less than a predefined threshold; In response to determining that the distance measure is less than a predefined threshold, including each data point of the intermediate subset of data points in a new iterative subset; in response to determining that the distance measure is greater than a predefined threshold, discarding each data point of the intermediate subset of data points from the new iterative subset; registering the pre-operative model to at least one portion of the 3D image based on the new iterative subset; repeating the process of generating intermediate subsets of data points, calculating distance measures, determining and including data points as a new iterative subset, and aligning until a termination criterion is met, and the new iterative subset is used as the current subset of data points; A processor configured to: The present invention relates to a device comprising:

[0007] In other words, a pre-operative model of the patient's target anatomical structure is iteratively filtered to reduce noise and improve the accuracy of the registration. For example, the target anatomical structure may be a bone structure, and the noise is caused by the presence of tissue on the bone structure that interferes with the registration process.

[0008] Advantageously, the present invention allows obtaining an optimally selected subset of data points of the pre-operative model through multiple iterations that include aligning the selected subset of data points of the pre-operative model to a 3D image of the surgical target, the points being associated with points in the surgical field represented in the 3D image representative of the surgical target. In other words, through the iterations, the subset of data points selected from the pre-operative model is enriched by adding new points that correspond to data points in the 3D image that belong to the target structure, while points that correspond to surrounding tissues or artificial structures are filtered out.

[0009] Although not limited thereto, the implantation of knee prostheses, especially TKA, using robotic assistance or surgical navigation, is one of the procedures that mainly benefit from the approach proposed by the present invention. Indeed, in computer and robotic assisted TKA, the alignment step is a key link between the planning and execution phases, since it is crucial to achieve the same high geometric accuracy during the execution of the actual surgery as was planned during the preoperative planning.

[0010] Alternatively, the present invention may also be used in a number of situations where an alignment step is required during surgery. The surgery may be performed on any anatomical structure, such as a joint or bone structure, for example, the shoulder, hip, elbow, ankle, tibia, etc.

[0011] According to one embodiment, the device further comprises at least one output adapted to provide a pre-operative model that is aligned to at least one corresponding part of the obtained 3D image when the termination criterion is met. Aligning the pre-operative model to the 3D image of the surgical field allows the estimation of a transformation that aligns the model reference to the target reference in the operating room. Since the pre-operative surgical plan is predefined with the model reference, knowing this transformation (i.e., alignment) allows the translation of actions planned during the pre-operative surgical plan to actions in the operating room. This is of particular interest when performing surgery using a robotic arm. Furthermore, the use of a pre-operative model of bones and its alignment to at least one 3D image of the target in the surgical field allows the knowledge of the transformation between the model reference and the target reference, independent of external markers attached to the patient.

[0012] According to one embodiment, the target anatomical structure has an elongated shape and a first axis in the model reference is aligned with a longitudinal axis of the pre-operative model.

[0013] According to one embodiment, the initial subset of data points of the model is composed of data points from the pre-operative model that fall between the origin coordinate and a predefined first iteration coordinate along a first axis, such that the maximum coordinate value along the first axis of the data points of the current subset of data points is equal to the predefined first iteration coordinate.

[0014] According to one embodiment, in each iteration, an intermediate subset of data points of the pre-operative model is generated from data points of the current subset of data points, and further from at least one group of data points of the pre-operative model that are located between the maximum coordinate value along a first axis of the data points of the current subset of data points and the maximum coordinate value along said first axis + iteration step.

[0015] According to one embodiment, the iteration step is predefined and constant. According to one embodiment, the iteration step is adapted at each iteration.

[0016] According to one embodiment, the at least one processor is configured to select predefined initial iteration coordinates based on information regarding the target anatomy and / or the type of surgery.

[0017] According to one embodiment, the at least one processor is further configured to calculate a registration score representative of the accuracy of the registration between the preoperative model of the target anatomical structure and the current subset of data points, the termination criterion being met when the registration score reaches an optimal value. This advantageously allows the iterations to be stopped when the quality of the registration is considered satisfactory. Indeed, the quality of the registration will be directly linked to the accuracy of the positioning of the surgical instruments used to operate on the target anatomical structure, and therefore the more accurate the registration, the more accurate the positioning of the surgical instruments according to the preoperative surgical plan. Since the target anatomical structure involved in the operation is firmly fixed to the operating table and is therefore rigid with respect to the external environment (i.e. imaging sensor, robotic device, etc.), the registration method of the present invention can be performed only once. If the target anatomical structure is displaced during the operation, the registration process can be repeated.

[0018] According to one embodiment, the registration score is a function of the square root of the mean squared distance between the data points of the current subset of data points and the corresponding data points in the 3D image. According to one embodiment, the distance measure used in the RMSE calculation is the Euclidean distance.

[0019] According to one embodiment, the at least one processor is further configured to calculate a registration score representative of the accuracy of the registration between the current subset of data points and the corresponding data points in the 3D image, and to register the pre-operative model to obtain an optimal registration of the pre-operative model to the target anatomical structure using the current subset of data points that yields the optimal value of the registration score. Alternatively, the processor may be configured to select the registration result that yields the highest registration score during the iterations.

[0020] According to one embodiment, the termination criterion is configured to stop the iterations when, over a given number of iterations, none of the data points of the current subset of data points is associated with a distance from a corresponding data point in the 3D image that is below a predefined threshold.

[0021] According to one embodiment, the at least one processor is further configured to generate a pre-operative model of the target anatomical structure by segmentation of a medical image comprising at least one portion of the target anatomical structure.

[0022] According to one embodiment, the at least one processor is configured to stop generating the intermediate subset of data points, calculating the distance measure, determining and including the data points as a new iterative subset, and repeating the alignment when the maximum coordinate value along the first axis exceeds a predefined final coordinate for all points of the current subset of data points.

[0023] According to one embodiment, the final predefined coordinates are defined based on information about the target anatomy and / or the type of surgery.

[0024] The present invention also provides a computer-implemented method for segmentation of a pre-operative model of a target anatomical structure, given a registration between the pre-operative model and an image of the target anatomical structure exposed during surgery, said method comprising: - receiving at least one 3D image acquired from at least one 3D imaging sensor, wherein the data points of the 3D image represent at least one exposed portion of a target anatomical structure of a patient; receiving a pre-operative model including data points defined with a model reference having at least one first axis; - initially aligning at least one portion of the pre-operative model to at least one portion of the 3D image; - defining a current subset of data points that includes data points from the preoperative model; - generating an intermediate subset of data points of the pre-operative model that includes the data points of the current subset of data points and further includes at least one group of further data points of the pre-operative model that are located along a first axis outside a portion of the pre-operative model that corresponds to the current subset of data points; - calculating a distance measure representative of a distance between each data point of the intermediate subset of data points and a corresponding data point of the 3D image; - determining whether the distance measure is less than a predefined threshold; - in response to determining that the distance measure is less than a predefined threshold, including each data point of the intermediate subset of data points in a new iterative subset; - discarding each data point of the intermediate subset of data points from the new iteration subset and all future iteration subsets in response to determining that the distance measure is greater than a predefined threshold; - registering the pre-operative model to at least one portion of the 3D image based on the new iterative subset; - repeating the generation of intermediate subsets of data points, the calculation of distance measures, the determination and inclusion of data points as a new iterative subset, and the alignment until a termination criterion is met, where the new iterative subset is used as the current subset of data points; The present invention relates to a method comprising the steps of:

[0025] Furthermore, the present disclosure relates to a computer program comprising a software code adapted to perform, when the program is executed by a processor, a method for segmentation of an exposed target anatomical structure complying with any of the above mentioned modes of execution.

[0026] The present disclosure further pertains to a computer readable, non-transitory program storage device tangibly embodying a program of instructions executable by a computer to perform a method for segmenting an exposed target anatomical structure in accordance with the present disclosure.

[0027] Such non-transitory program storage devices may be, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor devices, or any suitable combination of the above. Below are some more specific examples: portable computer diskettes, hard disks, ROMs, EPROMs (erasable programmable ROMs) or flash memories, portable CD-ROMs (small disc ROMs), but it should be noted that this is merely an illustrative and not exhaustive list, as would be readily understood by one skilled in the art.

[0028] definition In the present invention, the following terms have the following meanings:

[0029] The terms "adapted" and "configured" in this disclosure are used broadly to encompass the initial configuration of the device, subsequent adaptation or supplementation, or any combination thereof, whether done through physical or software means (including firmware).

[0030] The term "processor" should not be construed as being limited to hardware capable of executing software, but generally refers to a processing device that may include, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device (PLD). A processor may also encompass one or more graphics processing units (GPUs), whether used for computer graphics and image processing or other functions. Furthermore, instructions and / or data enabling the execution of the associated and / or resulting functions may be stored on any processor-readable medium, such as, for example, an integrated circuit, a hard disk, an optical disk such as a CD (small diskette), a DVD (digital versatile disk), a RAM (random access memory), or a ROM (read only memory). Instructions may be stored in hardware, software, firmware, or any combination thereof, among others.

[0031] "Pre-operative Model" refers to a three-dimensional digital (or virtual) model that is a three-dimensional virtual object. The position and orientation of the model are known by an associated model reference.

[0032] "Preoperative planning" in the context of surgery refers to a list of actions to be performed during the various surgical phases. This surgical plan can be obtained by a simulation program executed before the operation, using radiographic images of the patient's anatomy that is the target of the operation. In the case of knee arthroplasty, for example, the preoperative plan includes the definition of the cutting plane and drilling axis in relation to three-dimensional models of the femur and tibia, respectively.

[0033] "Reference" refers to a coordinate system that uses one or more numbers or coordinates to uniquely determine the location of a point or other geometric element on a manifold, such as Euclidean space.

[0034] "(Image) registration" refers to the process of transforming different data sets into one coordinate system. Image registration involves spatially transforming a "moving" image or images to align them with a "target" image. The reference coordinate system (i.e., reference) in the target image is fixed, while the other data sets are transformed to match the target.

[0035] In the drawings, the figures are not to scale and identical or similar elements are represented by the same reference numbers. [Brief description of the drawings]

[0036] [Figure 1] FIG. 2 is a block diagram that generally represents a particular mode of a device for segmentation according to the present disclosure. [Diagram 2] 1 is a schematic diagram of at least a subset of data points of a 3D image and a pre-operative model after initial registration and before filtering out data points not related to the target anatomical structure, in accordance with the present disclosure; FIG. [Diagram 3] 1 is a schematic diagram of a pre-operative model registered to an optimal subset of data points of a 3D image in accordance with the present disclosure; [Figure 4] Shown from left to right are (a) a schematic of a 3D preoperative model of the target anatomical structure, (b) a schematic of the surgical field with the exposed target anatomical structure, and (c) a schematic of a subset of data points from the preoperative model overlaid on the schematic of the surgical field. [Diagram 5] 1 shows a schematic diagram of (a) a first subset of data points selected from the pre-operative model, and (b) a subset of data points selected from the pre-operative model at the i-th iteration. [Figure 6] The left figure shows a first subset of 3D image data points defined in the first iteration of the method aligned to the 3D pre-operative model, and the right figure shows a subset of data points obtained after multiple iterations of the method aligned to the 3D pre-operative model. [Figure 7](a) shows a schematic perspective view of a point cloud of a pre-operative model registered onto a target anatomical structure represented in a 3D image, whose elements are here represented diagrammatically as lines and contours, and (b) shows a cross-section corresponding to plane AA represented in (a), where the grey points represent points of the 3D image belonging to plane AA and the black segments represent points of the pre-operative model image belonging to plane AA. [Figure 8] 1 shows a perspective view of a subset of data points from a 3D image registered to a 3D pre-operative model. [Figure 9] 2 is a flow chart showing successive steps carried out in the prediction device of FIG. 1; [Figure 10] 2 shows a schematic diagram of an apparatus incorporating the functionality of the segmentation device of FIG. 1; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] The present description illustrates the principles of the present disclosure and it will thus be appreciated that those skilled in the art may devise various arrangements which, although not explicitly described or shown herein, embody the principles of the present disclosure and are included within the scope of the present disclosure.

[0038] All examples and conditional language recited herein are intended for educational purposes to aid in the understanding of concepts contributed by the inventors to the development of the principles and techniques of the present disclosure, and should not be construed as being limited to such specifically recited examples and conditions.

[0039] Moreover, all statements herein reciting principles, aspects, and embodiments of the present disclosure, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, such equivalents are intended to include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.

[0040] Thus, for example, those skilled in the art will appreciate that the block diagrams presented herein may represent conceptual views of illustrative circuitry embodying the principles of the present disclosure. Similarly, it will be appreciated that flow charts, flow diagrams, and the like are substantially represented on a computer-readable medium and thus represent various processes that may be executed by such a computer or processor, whether or not a computer or processor is explicitly depicted.

[0041] The functions of the various elements shown in the figures may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared.

[0042] It should be understood that the elements shown in the figures may be implemented in various forms of hardware, software, or a combination thereof. Preferably, these elements are implemented in a combination of hardware and software on one or more appropriately programmed general-purpose devices, which may include a processor, memory, and input / output interfaces.

[0043] This disclosure describes embodiments of specific features of device 1, as shown in FIG. 1, that identify an optimal subset of points for segmenting an exposed target anatomical structure.

[0044] The device 1 is adapted to generate an optimal subset 31 of such data points for registering the pre-operative model 21 to image data 22 representative of at least one 3D image of the target anatomical structure T acquired during surgery. In particular, during multiple iterations of the method, all points belonging to other structures (i.e. soft tissues such as tendons, cartilage, skin, etc. or external objects such as bone screws, surgical instruments, etc.) are excluded, so that the optimal subset 31 of data points obtained as output mainly comprises points of the one intra-operative model 21 associated with the target anatomical structure T.

[0045] The 3D image 22 may be derived from at least one 3D imaging sensor present in the operating room and positioned to include within its field of view at least a portion of the surgical field including the target anatomical structure T. Indeed, during an orthopedic surgical procedure, the surgeon proceeds to expose the target anatomical structure T, which in this case may be a bone, on which a surgical operation such as machining or drilling has to be performed. The surgical field is essentially the area of ​​the patient where the operation will be performed and will include the exposed target anatomical structure T and surrounding structures B such as tissues (i.e. cartilage, tendons, muscles, skin, or bones that are not targeted during the operation, etc.) and / or artificial structures (i.e. bone screws, surgical instruments, grasping tools, etc.). A schematic diagram of an exemplary surgical field is provided in FIG. 4(b).

[0046] A 3D imaging sensor refers to a sensor for acquiring topological data of a three-dimensional real scene. These topological data are recorded in the form of a point cloud and / or a depth map. Since it is known to a person skilled in the art how to perform registration with both a point cloud or a depth map, in the following the term "data points" is used to refer to both a point cloud or a depth map. Thus, at least one portion of the data points of one 3D image 22 represents at least one exposed portion of the target anatomical structure T of the patient. The other data points are generally associated with surrounding structures B of the target anatomical structure T included within the field of view of the 3D imaging sensor.

[0047] To obtain these topological data, multiple acquisition techniques may be used, for example techniques based on measuring the propagation time of waves, such as ultrasound or light (LIDAR, time of flight), or stereo cameras or sensors, which are a type of camera with two or more lenses, with a separate image sensor or film frame for each lens. This allows the camera to simulate human binocular vision, thus giving it the ability to capture three-dimensional images. Other techniques may be based on light deformation, such as structured light 3D scanners, which project a pattern of light onto an object and look for deformations of the pattern on the object. The advantage of structured light 3D scanners is speed and accuracy. Instead of scanning one point at a time, structured light scanners scan multiple points or the entire field of view in one go. Scanning the entire field of view in a fraction of a second reduces or eliminates the problem of distortion due to motion. Another class of techniques is based on laser scanning, using laser techniques, such as handheld lasers or time-of-flight 3D laser scanners, to sample or scan surfaces. More generally, any technique known to those skilled in the art that provides topological data of a three-dimensional real scene may be used to implement the invention.

[0048] The 3D image(s) 22 may be, inter alia, grayscale or color (RGB-D) images. The 3D image(s) 22 may include numerical data, such as digital data, which may include individual image data in compressed form, for example according to the JPEG (for Joint Photographic Experts Group), JPEG2000, or HEIF (for High Efficiency Image Format) standards, as is well known to those skilled in the art of image compression.

[0049] Since the 3D image(s) 22 are acquired by a 3D imaging sensor, the data points of the 3D image are related to a reference of the sensor, in particular to the reference of the 3D imaging sensor.

[0050] For a given registration run, the 3D image(s) 22 may be derived from a unique 3D imaging sensor used to acquire at least one 3D image of at least a portion of the surgical field including the target anatomical structure T. Alternatively, the 3D image(s) 22 may be derived from two or more 3D imaging sensors, or even from two or more different types of 3D imaging sensors. In this case, data from the multiple sensors may also be combined into a single fused point cloud or depth map.

[0051] The pre-operative model 21 comprises a set of data points defining a 3D model of the anatomical target structure on which the surgical procedure will be performed. The data points of the pre-operative model 21 are defined in a model reference in which a first axis X1 is defined. The pre-operative model 21 may be derived from medical images acquired before surgery. Typically, said medical images are images (or slices) of the patient obtained by medical imaging (CT, MRI, PET, etc.). The 3D pre-operative model 21 can be obtained by a segmentation process of these medical images and subsequent interpolation between the images. The 3D pre-operative model 21 obtained from the segmentation and interpolation of the medical images can be modified to take into account elements that are not visible in the medical images, for example cartilage that is not visible in the CT scan images. In this case, the modifications can be generated from training data or biomechanical simulation data. The 3D pre-operative model 21 can also be generated from statistical models or abacus and patient data that may or may not be associated with the pre-operative medical images. Furthermore, the pre-operative model 21 can be adapted taking into account data acquired during surgery. Alternatively, the 3D pre-operative model 21 can be generated through digitizing 3D points on the exposed bone surface using an optical tracking system (e.g., the 3D imaging sensor itself, or another system present in the operating room).

[0052] While the device 1 described herein is versatile and includes several functions that may be performed alternatively or in any cumulative manner, other implementations within the scope of this disclosure include devices having only a portion of the functions.

[0053] Device 1 is advantageously an apparatus or physical piece of apparatus designed, configured, and / or adapted to perform the aforementioned functions and produce the aforementioned effects or results. In alternative implementations, device 1 is embodied as a set of apparatus or physical piece of apparatus, whether grouped on the same machine or on different, possibly remote, machines. Device 1 may, for example, be distributed on a cloud infrastructure and have functionality available to users as a cloud-based service, or have remote functionality accessible through an API.

[0054] In the following, modules should be understood as functional entities, not as materially, physically distinct components. They can therefore be embodied as grouped together in the same tangible concrete component, or distributed across several such components. Also, each of these modules may itself be shared between at least two physical components. Furthermore, the modules may be implemented in hardware, software, firmware, or any mixed form thereof. They are preferably embodied in at least one processor of the device 1.

[0055] As shown in FIG. 1, the device 1 comprises a module 11 for receiving image data 22 (i.e. 3D image(s)) and a pre-operative model 21, which may be stored in one or more local or remote database(s) 10. The latter may take the form of storage resources available from any kind of suitable storage means, which may in particular be a RAM or an EEPROM (Electrically Erasable Programmable Read Only Memory), such as a flash memory in a SSD (Solid State Disk). In an advantageous embodiment, the pre-operative model 21 has been generated beforehand by a system including a device for generating a 3D pre-operative model. Alternatively, the pre-operative model 21 is received from a communication network.

[0056] The device 1 further optionally comprises a module 12 for pre-processing the received 3D image(s) 22 and possibly the pre-operative model 21. The module 12 may be adapted to standardize the received image data 22, in particular for efficient and reliable processing. The image data 22 can be transformed, for example by image or video decompression. Depending on the various configurations, the module 12 is adapted to perform only some or all of the above functions, in any manner and in any possible combination, suitable for the next processing stage.

[0057] In an advantageous mode, module 12 is configured to pre-process image data 22 to standardize the images, which may improve the efficiency of downstream processing by device 1. Such standardization may be particularly useful when utilizing images generated from different sources, possibly including different imaging systems.

[0058] The device 1 comprises a module 13 whose main purpose is to advantageously filter the data points of the pre-operative model 21 to leave only data points associated with exposed target anatomical structures in the 3D image 22 and to remove data points associated with soft tissues or surgical instruments in the 3D image 22. More specifically, the module 13 is configured to iteratively perform a series of operations including, in particular, the alignment of different subsets of data points of the pre-operative model 21 to the associated data points in the 3D image 22 until a predefined termination criterion is met.

[0059] Since the target anatomical structure T may have an elongated shape, such as a long bone, the first axis X1 may be aligned with the longitudinal axis X1 of the target anatomical structure in the model reference, as shown in FIG. tThe first axis X1 may be defined to be aligned with the preoperative model 21. In general, the equation of the axis X1 (i.e., spatial direction and position in the model reference) may be defined based on predefined information regarding the type of target surgical structure, the type of surgery to be performed, the surgeon's preferences, etc. The first axis may be defined to pass through at least one portion of the preoperative model 21 that corresponds to a portion of the target anatomical structure T that is exposed during surgery. The first axis X1 may be predefined by the device that generated the preoperative model 21, in which case the equation of the first axis X1 is received by the receiving module 11 together with the data points of the preoperative model 21. Alternatively, the module 13 may be configured to define the first axis X1.

[0060] More specifically, the module 13 is configured to perform an initialization step, which may be performed only once for each 3D image 22 received. A first initialization step involves aligning at least one portion of the data points of the pre-operative model 21 to at least one portion of the data points of the 3D image 22. FIG. 2 shows an example of the result of this first alignment of the pre-operative model 21 to at least one portion of the data points of the 3D image 22. In this example, the target anatomical structure is the femur. In this case, the longitudinal axis of the model (i.e. the first axis X1) and the longitudinal axis X2 of the femur in the 3D image 22 are aligned in a 3D manner. t The pre-operative model 21 is not aligned in an optimal manner, as can be seen from the fact that the longitudinal axis X1 of the pre-operative model and the longitudinal axis X2 of the femur in the 3D image 22 do not overlap (i.e. the two axes do not pass through the same set of points). On the other hand, FIG. 3 shows the overlap resulting from the alignment of the pre-operative model 21 with the optimal subset 31 of data points resulting from the iterations performed in module 13. In this case, the alignment is particularly accurate, since there are no or very few outlier data points in the optimal subset 31 of data points, and the longitudinal axis X1 of the pre-operative model and the longitudinal axis X2 of the femur in the 3D image 22 are aligned in an optimal manner, as can be seen from the fact that the longitudinal axis X1 of the pre-operative model and the longitudinal axis X2 of the femur in the 3D image 22 do not overlap (i.e. the two axes do not pass through the same set of points). t The steps performed by module 13 leading to this result are explained in detail in the following paragraphs.

[0061] After said first alignment, the module 13 is configured to perform a second initialization step on the pre-operative model 21 received at the device 1. Said second initialization step consists of selecting a first subset P1 of data points of the aligned pre-operative model 21. The points of the subset P1 of data points are selected from the data points of the aligned 3D pre-operative model 21 such that each selected point is associated with at least one part of the target anatomical structure to be exposed during surgery and not associated with other surrounding structures.

[0062] FIG. 4(a) shows a pre-operative model of a long bone having a longitudinal axis X1, and FIG. 4(b) shows a schematic diagram of a surgical field including a target anatomical structure T and surrounding tissues / objects B represented in a 3D image 22 (e.g., as seen by the human eye or a color camera). FIG. 4(c) shows a schematic diagram of a preliminary registration of a subset of data points of the pre-operative model 21 to the 3D image 22 (including both target and non-target anatomical structures). Since the 3D image 22 also includes data points associated with structures not represented in the pre-operative model 21, the result of the registration is unsatisfactory and therefore the longitudinal axis X1 of the target structure is not represented in the 3D image 22, as shown in FIG. 4(c). t does not overlap or even align with the longitudinal axis of the model (i.e., X1 of the long bone in this example).

[0063] To select, for the first subset P1 of data points, data points of the aligned pre-operative model 21 that mostly relate to at least one exposed portion of the target anatomical structure, the module 13 may be configured to select data points of the pre-operative model 21 that fall between two coordinates defined along the first axis X1. In one example shown in FIG. 5(a), the two coordinates used to define the first subset P1 of data points are the origin coordinate

number

number

number

number

[0064] According to one embodiment, the predefined first iteration coordinates

number

number

[0065] The operations necessary to define the first subset of data points P1 do not need to be repeated any further in the iterations performed by the module 13. The subset of data points undergoing the i-th iteration is referred to in the present disclosure as the current subset of data points P i The subset of data points P1 is therefore considered as the current subset of data points in the first iteration performed by the module 13.

[0066] The current subset of newly defined data points P iGiven, the module 13 is configured to perform at least one of a first, second, third, fourth and fifth action that are iteratively repeated until a termination criterion is met.

[0067] The first operation is to select a subset of data points P i an intermediate subset of data points obtained by adding at least one point or group of data points from the data points of the aligned preoperative model 21 to the data points of

number

[0068] In particular, the intermediate subset of data points in the preoperative model is the current subset of data points P i The data points of the current subset of data points P along the first axis are i may be generated by adding at least one group of further data points of the pre-operative model 21 that are located outside the portion of the pre-operative model 21 corresponding to

[0069] According to an example depicted in FIG. 5, the intermediate subset of data points

number

number

number

number

number

[0070] The iteration step Δx may be predefined and constant, or alternatively may be predefined and decrease with each iteration.

number

[0071] Alternatively, the iteration step Δx may be predefined and constant, or alternatively it may be predefined and increase with each iteration. This embodiment has the advantage of increasing the speed of calculation.

[0072] In one alternative embodiment, the iteration step Δx is calculated by multiplying the alignment score s calculated during the previous iteration. i-1 (i.e., when the module 13 is configured to calculate the alignment score). The length of the step Δx is determined based on the alignment score s i-1 Alternatively, the iteration step Δx is inversely proportional to the current position along the major axis

number

number

[0073] The second operation is to extract the intermediate subset of data points.

number

number

[0074] The third operation then consists of generating a new subset of data points P i+1 is the intermediate subset of data points with distances from the aligned pre-operative model below a predefined threshold

number

number

number

[0075] More precisely, this third operation comprises a step of determining whether the distance measure is less than a predefined threshold. In response to the determination that the distance measure is less than a predefined threshold, each data point of the intermediate subset of data points is assigned to a new iterative subset P i+1 In other cases, in response to a determination that the distance measure is greater than a predefined threshold, each data point of the intermediate subset of data points is added to a new iterative subset P i+1 Therefore, if the termination criterion is not yet met, a new iterative subset P i+1 is used as the current subset of data points in the next iteration i+1, so the new iteration subset P i+1 Any data points discarded from will also be absent from all future iteration subsets.

[0076] Module 13 is a new iterative subset P selected from the preoperative model 21. i+1 to a corresponding portion of the 3D image 22.

[0077] Fig. 6 provides a graphical illustration of the current subset of data points and the progression of the accuracy of the registration of the pre-operative model after multiple iterations. In the left diagram of Fig. 6, an example of a first subset of data points P1 is depicted, which includes a small number of data points that are in fact all associated with 3D image data points related to the target anatomical structure. However, the pre-operative model 21 is initially (i.e., during the first initialization step) registered using data points that may also correspond to data points in the image 22 that are not related to the exposed target anatomical structure T, so that the axes X1 and X2 are aligned. t The quality of the registration is still insufficient, as indicated by the lack of overlap between the 3D images 21 and 22. The right diagram in FIG. 6 shows the result of adding to the subset of data points only those data points of the registered pre-operative model that are associated with data points of the 3D image 22 corresponding to the target anatomical model T and not the surrounding tissue B at each iteration, since the data points are selected based on their distance from the registered model. The surrounding tissue B is represented in this diagram merely to aid in understanding the invention. The surrounding tissue B is not part of the pre-operative model 21, and in this FIG. 6 the selected and represented data points are primarily associated with the target anatomical structure T. FIG. 8 shows the current subset of data points P i 1 shows a perspective view of a preoperative model 21 aligned with the surgical model 20.

[0078] Optionally, the module 13 determines a new subset P of data points at the i iteration. i+1 and a registration score s , which represents the accuracy of registration between corresponding data points in the 3D image 22. i The alignment score s i is a new subset of data points P i+1 The registration score s can be obtained as a function of the square root of the mean squared difference of the 3D distances between the points in the 3D image 22 and the corresponding data points in the 3D image 22 (root mean square error RMSE). i is a new subset of data points P i+1 between corresponding pairs of data points ini+1 can be obtained as a function of the Euclidean distance between known anatomical landmarks contained in and corresponding data points in the 3D image 22.

[0079] Finally, module 13 is configured to perform a fifth operation configured to verify whether a predefined termination criterion is met.

[0080] According to one embodiment, the predefined termination criterion is the alignment score s i For example, the alignment score s i The iterations can be stopped when a threshold value is exceeded or an optimum value is reached.

[0081] According to an alternative embodiment, the termination criterion is configured to stop the iterations when, during a given number of iterations (i.e. one, two or more iterations), no pair of corresponding data points of the intermediate subset of data points and corresponding data points in the 3D image 22 is associated with a distance below a predefined threshold. The termination criterion may also be configured to stop if fewer than a predefined number of data points of the intermediate subset of data points of the model have a distance from the corresponding data point in the 3D image below a predefined threshold.

[0082] According to one embodiment, the termination criterion is to select a new subset of data points P i+1 The maximum coordinate value along the first axis X1 for all points in

number

number

number

[0083] Whenever the termination criterion is not met, the module 13 selects a new subset of data points P i+1 Thus, during the new iteration i+1, the module 13 is configured to start a new subset P i+1 Conversely, when a termination criterion is met, module 13 is configured to stop the iteration (i.e., stop repeating the first, second, third, fourth, and fifth actions).

[0084] Thus, the device 1 advantageously allows to obtain an optimal alignment 32 of the pre-operative model 21 to the exposed target anatomical structure T represented in the 3D image 22. The optimal alignment is obtained by using an optimal subset of data points of the pre-operative model 21 selected through the iterations that satisfy a selected termination criterion. The optimal subset of data points may be the one selected during the last iteration performed before the termination criterion is met, or one of the subsets of data points for which the best alignment score or other evaluation parameter is obtained.

[0085] The right diagram of FIG. 6 shows the superimposition of the pre-operative model 21 on the target anatomical structure T represented in the 3D image 22, obtained when an optimal subset of data points of the pre-operative model is registered with the corresponding data points of the 3D image 22 belonging to the target anatomical structure T. In this case, outlier data points are absent or very few in the optimal subset of data points, and therefore the registration is particularly accurate, with the longitudinal axis X1 of the model and the axis X2 of the target anatomical structure being aligned with each other.t overlap.

[0086] According to one embodiment, the device 1 further comprises a modeling module (not shown) configured to generate a pre-operative model 21 of said target anatomical structure by segmentation of medical images including at least one part of said target anatomical structure T. Said medical images may be images acquired before surgery, such as images obtained from CT, MRI, PET, etc. The modeling module may be configured to perform a segmentation of the target anatomical structure T in these images and then to interpolate between the segmented images.

[0087] The device 1 interacts with a user interface 16, through which a user can input and obtain information. The user interface 16 includes any means suitable for inputting or obtaining data, information or instructions, in particular visual, tactile and / or audio capabilities, which may include any or some of the following means well known to those skilled in the art: screen, keyboard, trackball, touchpad, touchscreen, loudspeaker, voice recognition system. In its automatic behavior, the device 1 may, for example, execute the following process (FIG. 9): - receiving the pre-operative model and image data 22 (step 41); - pre-processing the image data 22 for more efficient and / or reliable processing (step 42); - Aligning the pre-operative model 21 to at least one part of the 3D image 22 (first initialization step 43), - performing an iterative loop on the data points of the preoperative model 21 until a termination criterion is met (steps 44 to 47), in particular o The current subset of data points for the model, P i (Step 44), o Current subset P igenerating an intermediate subset of data points of the pre-operative model (step 45), the intermediate subset including the data points of the pre-operative model 21 and further including at least one group of further data points of the pre-operative model 21; o calculating a distance measure representative of the distance between each data point of the intermediate subset of data points and a corresponding data point of the 3D image 22 (step 46); o A new iterative subset P of data points that includes the data points of the intermediate subset that have a distance from the corresponding point in the 3D image below a predefined threshold. i+1 Generate a new iterative subset of data points P i+1 to the 3D image (step 47); - Output the aligned model 31.

[0088] The process of FIG. 10 may also include a step of aligning the pre-operative model 21 to corresponding data points in the 3D image 22 using the optimal subset of data points (i.e., the current subset of data points for which an optimal alignment score is obtained).

[0089] The particular apparatus 9 seen in Fig. 9 embodies the aforementioned device 1. It corresponds for example to a workstation, a laptop, a tablet, a smartphone or a head mounted display (HMD).

[0090] This device 9 is suitable for segmenting a 3D image and for registering a model of an exposed target anatomical structure to the 3D image. It comprises the following elements connected together by an address and data bus 95, which also carries a clock signal: - a microprocessor 91 (or CPU), - a graphics card 92 comprising several Graphic Processing Units (or GPUs) 920 (GPUs are very suitable for image processing due to their highly parallel structure) and a Graphical Random Access Memory (GRAM) 921; - ROM type non-volatile memory 96, - RAM97, - one or several I / O (Input / Output) devices 94, such as, for example, a keyboard, a mouse, a trackball, a webcam (other modes for introducing commands, such as, for example, voice recognition, are also possible), - Power supply 98, and - Radio Frequency Unit 99.

[0091] According to a variant, the power supply 98 is external to the device 9 .

[0092] The device 9 also comprises a display device 93 of the display screen type connected directly to the graphics card 92 for displaying the composite image calculated and created in the graphics card. The use of a dedicated bus for connecting the display device 93 to the graphics card 92 brings the advantage that the data transfer bit rate is significantly higher and therefore the latency for displaying the image created by the graphics card is reduced. According to a variant, the display device is external to the device 9 and is connected to the device 9 by a cable or wirelessly for transmitting the display signal. The device 9 comprises an interface for transmission or connection adapted to transmit the display signal, for example through the graphics card 92, to external display means, such as for example an LCD or plasma screen or a video projector. In this respect, an RF unit 99 can be used for wireless transmission.

[0093] It should be noted that the term "register" as used below in the description of memories 97 and 921 may refer to a low-capacity memory zone (some binary data) and a high-capacity memory zone (allowing storage of all or part of data representing an entire program or data to be calculated or displayed) in each memory mentioned. Also, the registers represented for RAM 97 and GRAM 921 may be arranged and organized in any manner, and each of them does not necessarily correspond to adjacent memory locations, but may also be distributed in other manners (in particular the situation where one register contains several smaller registers).

[0094] When switched on, the microprocessor 91 loads and executes instructions of a program stored in the RAM 97 .

[0095] As will be appreciated by those skilled in the art, the presence of a graphics card 92 is not required and may be replaced by all CPU processing and / or a simpler visualization implementation.

[0096] In an alternative mode, device 9 may include only the functionality of device 1. Furthermore, device 1 may be implemented in a manner other than as standalone software, and a device or set of devices including only a portion of device 9 may be utilized through API calls or via a cloud interface.

Claims

1. A device (1) for segmentation of a preoperative model (21) of a target anatomical structure, taking into account a registration between the preoperative model and an image of the target anatomical structure exposed during surgery, said device (1) comprising: At least one input unit, at least one 3D image (22) acquired from at least one 3D imaging sensor, wherein data points in the 3D image (22) represent at least one exposed portion of a target anatomical structure; At least one first axis (X １ a preoperative model (21) including data points defined by a model reference having the an input configured to receive at least one processor, First, registering at least one portion of the pre-operative model (21) with at least one portion of the 3D image (22); The current subset of data points (P) includes data points from the preoperative model (21). ｉ ) and The current subset of data points (P ｉ ) data points, and further includes a first axis (X １ ) along the current subset of data points (P ｉ generating an intermediate subset of data points of the preoperative model (21) including at least one group of further data points of the preoperative model (21) located outside the portion of the preoperative model (21) corresponding to the calculating a distance measure representing the distance between each data point of the intermediate subset of data points and a corresponding data point of the 3D image (22); determining whether the distance measure is less than a predefined threshold; In response to determining that the distance measure is less than a predefined threshold, each data point of the intermediate subset of data points is assigned to a new iterative subset (P ｉ＋１ ) and In response to determining that the distance measure is greater than a predefined threshold, each data point of the intermediate subset of data points is assigned to a new iterative subset (P ｉ＋１ ) and The new iterative subset (P ｉ＋１ aligning the preoperative model (21) to at least one portion of the 3D image (22) based on the repeating the generation of intermediate subsets of data points, calculation of distance measures, determination and inclusion of data points as a new iterative subset, and alignment until a termination criterion is met, wherein the new iterative subset (P ｉ＋１ ) is the current subset of data points (P ｉ ) and a processor configured to: A device (1) comprising:

2. The target anatomical structure has an elongated shape and is oriented along a first axis (X １ ) is aligned with the longitudinal axis of the preoperative model (21).

3. A first subset of data points for the model (P ｉ ) is the origin coordinate [Equation 1] and the first axis (X １ ) and the pre-defined first iteration coordinate along the [Equation 2] The first axis (X １ ) maximum coordinate value along [Equation 3] The device of claim 1 , wherein is equal to a predefined first iteration coordinate.

4. At each iteration, the intermediate subset of data points for the preoperative model is calculated based on the current subset of data points (P ｉ ) data points, and furthermore, the current subset of data points (P ｉ ) data points on the first axis (X １ ) maximum coordinate value along [Equation 4] and the first axis (X １ ) maximum coordinate value along the [Equation 5] 4. The device of claim 3, wherein the preoperative model is generated from at least one group of data points of the preoperative model (21) located between

5. The device of claim 1 , wherein the iteration step (Δx) is predefined, constant or adapted at each iteration.

6. The device of claim 1 , wherein the at least one processor is configured to select predefined initial iteration coordinates based on information about a target anatomy and / or a type of surgery.

7. The at least one processor further comprises: a preoperative model (21) of the target anatomy and a current subset of data points (P ｉ ) ｉ ), and the termination criterion is a registration score (s ｉ ) reaches an optimal value.

8. The alignment score (s ｉ ) is the current subset of data points (P ｉ 8. The device of claim 7, wherein the distance is a function of the square root of the mean square distance between the data points of the 3D image (22) and the corresponding data points in the 3D image (22).

9. The termination criterion is the number of iterations over which the current subset of data points (P ｉ 2. The device of claim 1, configured to stop iterations when none of the data points in the 3D image (22) is associated with a distance from a corresponding data point in the 3D image (22) that is below a predefined threshold.

10. The device of claim 1 , wherein the at least one processor is further configured to generate a pre-operative model (21) of the target anatomical structure by segmenting a medical image including at least one portion of the target anatomical structure.

11. The at least one processor selects a current subset of data points (P ｉ ) for all points on the first axis (X １ ) maximum coordinate value along [Equation 6] is the predefined last coordinate [Equation 7] 2. The device of claim 1, configured to stop generating intermediate subsets of data points, calculating distance measures, determining and including data points as new iterative subsets, and repeating the registration when

12. The predefined last coordinate [Equation 8] The device of claim 11 , wherein is defined based on information about the target anatomy and / or type of surgery.

13. 1. A computer-implemented method for segmentation of a pre-operative model of a target anatomical structure (T) given a registration of the pre-operative model with an image of the target anatomical structure exposed during surgery, the method comprising: receiving at least one 3D image (22) acquired from at least one 3D imaging sensor, data points of the 3D image representing at least one exposed portion of a target anatomical structure of a patient; At least one first axis (X １ receiving a preoperative model (21) including data points defined in a model reference having a First, registering at least one portion of the pre-operative model (21) with at least one portion of the 3D image (22); The current subset of data points (P) includes data points from the preoperative model (21). ｉ ) and The current subset of data points (P ｉ ) data points, and further includes a current subset of data points (P ｉ generating an intermediate subset of data points of the preoperative model (21) including at least one group of further data points of the preoperative model (21) that lie outside the portion of the preoperative model (21) corresponding to the calculating a distance measure representing the distance between each data point of the intermediate subset of data points and a corresponding data point of the 3D image (22); determining whether the distance measure is less than a predefined threshold; In response to determining that the distance measure is less than a predefined threshold, each data point of the intermediate subset of data points is assigned to a new iterative subset (P ｉ＋１ ) and In response to determining that the distance measure is greater than a predefined threshold, each data point of the intermediate subset of data points is assigned to a new iterative subset (P ｉ＋１ ) and The new iterative subset (P ｉ＋１ aligning the preoperative model (21) to at least one portion of the 3D image (22) based on the repeating the generation of intermediate subsets of data points, calculation of distance measures, determination and inclusion of data points as a new iterative subset, and alignment until a termination criterion is met, wherein the new iterative subset (P ｉ＋１ ) is the current subset of data points (P ｉ ) and A method comprising:

14. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method of claim 13.

Images