A method for precisely and accurately segmenting a 2D or 3D image

Abstract

The present invention relates to a computer-implemented method for segmenting an image, comprising (S1) Obtaining a 2D / 3D image I of a region of interest; (S2) Initial segmentation of the 2D / 3D image I; (S3) Processing each image Pj, starting from one or more error margins of the initial segmentation of the image I so as to obtain a corresponding image Uj; (S4) Processing the pixels / voxels of the 2D / 3D image I corresponding to the uncertain area of ​​the image Uj; (S5) Determining a difference image Ej between the pixels / voxels Fjn of the mask Fj and the pixels / voxels Pjn of the image Pj; (S7) Processing of assigning each pixel / voxel Ejn in deviation to assign it or not to the object Aj and thus obtain a final segmentation mask Mj including a refined segmentation of the object Aj.

Description

Title of the invention: Method for precise and accurate segmentation of a 2D or 3D image. FIELD OF THE INVENTION

[0001] The invention relates to a method for segmenting a 2D or 3D image, for example of a subject, a patient or a structure. In particular, the 2D or 3D image contains an object to be segmented, for example a bone. STATE OF THE ART

[0002] Image segmentation consists of classifying each pixel or voxel of an image (2D or 3D) as belonging or not to an object, for example a bone.

[0003] Whatever the application, accurate segmentation is desirable and accurate segmentation should generally follow the most visible contour in the image.

[0004] For example, in orthopedic surgery, the segmentation of bones in images can be performed manually by an annotator, using interactive image annotation and processing tools, and / or automatically by segmentation algorithms.

[0005] However, manual methods are inherently subject to human bias and are not very precise, while semi-manual methods are complex and time-consuming. With automatic methods, the gains in terms of practicality, speed, and precision are offset by reduced robustness. These methods are therefore unsatisfactory for obtaining accurate and precise segmentation of objects, for example, in computed tomography (CT) or cone beam computed tomography (CBCT) images. Description of the invention

[0006] The invention proposes to segment in a precise and accurate manner one or more object(s) contained in an image, preferably a medical one.

[0007] To this end, the invention proposes a computer-implemented method for segmenting an image, comprising

[0008] - Obtaining a 2D / 3D image of a region of interest of an element comprising minus one object Aj, j = 1 to J, the 2D / 3D image I comprising N pixels / voxels In, n=l to N;

[0009] - Initial segmentation of the 2D / 3D image I so as to obtain J image Pj of object segmented Aj, each image Pj comprising N pixels / voxels Pjn, each center of which is a probability of belonging to the object Aj;

[0010] - Processing each image Pj, based on one or more error margins of the initial segmentation of image I so as to obtain a corresponding image Uj, image Uj comprising a first zone and a second zone, the first zone comprising only Ujn pixels / voxels that are reliably part of the object, the first zone being totally included in the object Aj; the second zone comprising only Ujn pixels / voxels that are not reliably part of the object Aj; the pixels / voxels not belonging to the first zone nor to the second zone belonging to an uncertain zone;

[0011] - Processing of the pixels / voxels of the 2D / 3D image I corresponding to the area uncertain of the image Uj such that the values ​​of the pixels / voxels corresponding to the uncertain area of ​​the image Uj are defined with respect to a boundary delimiting the object Aj in the 2D / 3D image I in the uncertain area, the boundary being the most visible in the uncertain area, the pixels / voxels processed in the uncertain area and the pixels / voxels of the first and second areas defining a mask Fj;

[0012] - Determination of an image Ej of difference between the pixels / voxels Fjn of the mask Fj and the pixels / voxels Pjn of the image Pj, the non-nuis pixels / voxels Ejn of the gap image Ej being gap pixels / voxels, these pixels / voxels being grouped into M disjoint groups EGjm named "gap", a gap EGjm being a set of gap pixels / voxels Ejn having the same non-zero value, the gap EGjm and therefore the gap pixels / voxels Ejn of the gap EGjm being classified "significant" if the size and / or depth of the set is greater than a threshold EM1N and / or Edepth and "not significant" otherwise;

[0013] - Processing of assigning each pixel / voxel Ejn in deviation to assign it or not to the object Aj and thus obtain a final segmentation mask Mj comprising a refined segmentation of the object Aj, the assignment processing comprising an assignment rule defined as follows, for each pixel / voxel n of each significant gap EGjm, if Ejn=l then Mjn=0, the pixel / voxel Ejn not belonging to the object Aj, if Ejn=-1 then Mjn=l, the pixel / voxel Ejn belonging to the object Aj, the remaining pixels Mjn being identical to the corresponding pixels Fjn of the mask Fj.

[0014] The invention is advantageously complemented by the following features, taken alone or in any technically possible combination thereof:

[0015] - the assignment process includes an assignment rule defined in the following way next: Pixel value Ejn of EGjm Class(es) of EGjm Pixel value Mjn 1 (in FBj, outside PBj) S (Significant) 0 -1 (outside FBj, in PBj) S (Significant) 1

[0016] Mjn = 1 when the pixel / voxel with significant deviation Ejn belongs to the object Aj and Mjn=0 when the pixel / voxel with significant deviation Ejn does not belong to the object Aj, the other remaining pixels / voxels Mjn being identical to the corresponding pixels Fjn of the mask Fj.

[0017] - the method includes a classification step for each pixel / voxel n of each significant EGjm deviation (class S) according to one of the following classes: Inside object Aj (IN) or outside object Aj (OUT) using a previously trained classification model.

[0018] - the classification takes into account at least one subclass assigned to each significant difference, this subclass being used depending on the use case of the final segmentation Mj to discriminate differences that can be assigned to one or the other of the classes depending on the use case.

[0019] - the assignment rule is further defined as follows Pixel value Ejn of EGjm Class(es) of EGjm Pixel value Mjn 1 (inside FBj, outside PBj) S and IN (Inside Aj) Fjn (> 0.5) -1 (outside FBj, in PBj) S and IN (Inside Aj) 1 1 (inside FBj, outside PBj) S and OUT (Outside Aj) 0 -1 (outside FBj, in PBj) S and OUT (Outside Aj) Fjn (< 0.5) or 0

[0020] Mjn = 1 or > 0.5 when the pixel / voxel in deviation Ejn belongs to the object Aj and Mjn = 0 or < 0.5 when the pixel / voxel in deviation Ejn does not belong to the object Aj, the other remaining pixels / voxels Mjn being identical to the corresponding pixels Fjn of the mask Fj-

[0021] - the processing of each image Pj to obtain the image Uj (S3) consists, for all the pixels / voxels n from 1 to N:

[0022] if Pin > Phigh j, where Pin jn is the minimum value of the points in the image Pj in a convolution kernel K in j around the pixel / voxel Pjn of Pj, the pixel / voxel Ujn of the image Uj belongs to the first zone;

[0023] if Pqvt in < PüOW j °ù For Jn is the minimum value of the points of the image Pj in a convolution kernel Kovt j around the pixel / voxel Pjn of Pj or if P in kn> Phigh k for k=l to J where k^j, the pixel / voxel Ujn of the image Uj belongs to the second zone.

[0024] - Phigh j = 1 / 2, Plow j = 1 / 2 and in which the pixels / voxels of the first area have a probability equal to 1, the pixels / voxels of the second zone have a probability equal to 0, the pixels / voxels of the uncertain zone have a probability equal to 1 / 2.

[0025] - the most visible boundary is obtained by processing the 2D / 3D image I in the uncertain zone by means of a graph cutting algorithm, the most visible boundary Fj being the minimal cut of a graph linking neighboring pixels / voxels in the uncertain zone by links having a capacity defined by a function of the image I and possibly the initial segmentation Pj, the minimum cutoff being obtained by means of an algorithm configured to solve a maximum flow problem.

[0026] - the most visible boundary is obtained by processing the 2D / 3D image I in the uncertain zone to obtain a geodesic curve / surface, the geodesic curve / surface being characterized by a function using the magnitude of the gradient of the image I, the most visible boundary being that which is the solution to the dual problem of the Combinatorial Continuous Max Flow (CCMF) algorithm, solved for example by the Primal-Dual Interior Point (PDIP) algorithm, or an Appleton-Talbot Continuous Max Flow (AT-CMF) type partial differential equation solving algorithm.

[0027] - the function / uses the magnitude of the gradient of image I and possibly the magnitude of the gradient of the initial segmentation Pj to discriminate an external boundary of Aj from internal boundary(s) of Aj, for example using the dot product of the normalized gradient vectors of Pjn and In, for example with 0 < a < 1:

[0028] / (In, Pjn) = Il vin II • max(a; W l+cos(vln, vPjn))).

[0029] - the determination of the image Ej (S5) includes a binarization of the mask Fj and of the image Pj consisting of obtaining a binary image FBj where each pixel FBjn is equal to 1 if Fjn>0.5 otherwise 0 and a binary image PBj where each pixel PBjn is equal to 1 if Pjn>0.5 otherwise 0, the binary mask FBj and the binary image PBj being binary versions of the mask Fj and the image Pj where a pixel / voxel of the binary mask FBj and / or of the binary image PBj is equal to 1 for each pixel / voxel greater than 0.5 or 0 for each pixel / voxel less than 0.5, the image Ej comprising pixels / voxels whose levels are 0, +1 or -1, the uncertain pixels / voxels having a value equal to -1 or +1, the gap image Ej being determined from the binary images FBj and PBj.

[0030] - the image Ej of the gap contains pixels / voxels Ejn=l if the pixel In of the image I is included in the binary mask FBj and outside the binary image PBj, Ejn=0 if the pixel In of the image I is included in the mask FBj and in the mask PBj or outside the mask FBj and the mask PBj, Ejn=-1 if the pixel In of the image I is outside the binary mask FBj and included in the image PBj.

[0031] - the method includes a step of determining a curve / surface Cj delimiting the object Aj in final segmentation mask Mj, the curve / surface Cj corresponding to a boundary in the final segmentation mask Mj obtained using a SurfaceNets, Marching Squares / Cubes, Cubical Marching Squares or Flying Edges algorithm.

[0032] - the image I is a 3D image reconstructed from a number K of 2D images of which We obtain the projection parameters, for example K between 1 and 10, and the boundary Fj is then determined on an image which is a linear combination of I and the K 2D images back-projected into the space of I according to their respective projection parameters, for example a weighted average where I has a weight equal to K and the K images each have a weight of 1. DESCRIPTION OF THE FIGURES

[0033] Other features, objectives and advantages of the invention will become apparent from the following description, which is purely illustrative and not limiting, and which should be read in conjunction with the accompanying drawings on which:

[0034] Fig. 1 illustrates steps of a segmentation process according to one embodiment of the invention.

[0035] Fig. 2 illustrates an initial 2D image I of a region of interest comprising three objects A1, A2, A3, the image I comprising N pixels In, n=1 to N.

[0036] Fig. 3 illustrates an initial segmentation of the 2D image I so as to obtain J image Pj of segmented object Aj, each image Pj comprising N pixels Pjn, each value of which is a probability of belonging to the object Aj.

[0037] Fig. 4 illustrates a processing of each image Pj, from one or more margins of error of the initial segmentation of the image I for an image comprising a first zone Z1 and a second zone Z2, the first zone Z1 comprising only pixels which belong safely to the object, the first zone Z1 being totally included in the object; the second zone Z2 comprising only pixels which do not belong safely to the object; the pixels which do not belong to the first zone or the second belong to an uncertain zone Z3.

[0038] Fig. 5 illustrates a processing of the pixels of the initial 2D image I corresponding to the uncertain area of ​​the image so that the values ​​of the pixels corresponding to the uncertain area of ​​the image Uj are defined with respect to the most visible boundary delimiting the object Aj in the 2D / 3D image I in the uncertain area, the pixels processed in the uncertain area and the pixels of the first and second area defining a mask Fj of the most visible boundary and a mask Vj of the relative visibility of this boundary in each pixel / voxel.

[0039] Fig. 6 illustrates a determination of an image Ej of difference between the pixels Fjn of the mask Fj and the pixels Pjn of the image Pj.

[0040] Figure 7 illustrates a treatment of significant deviations in class(es) or even subclass(es).

[0041] Fig. 8 illustrates a processing of assigning each pixel in deviation to assign it or not to the object Aj and thus obtain a final segmentation mask Mj comprising a refined segmentation of the object Aj.

[0042] Across all figures, similar elements relate to similar elements. DETAILED DESCRIPTION OF THE INVENTION General presentation of the process

[0043] The method according to the invention comprises the main steps below enabling the segmentation of objects in 2D or 3D images to be accurate to within one pixel / voxel, or even less, and following the objectively most visible boundary in the initial image if and only if it is determined to be precisely that of the object:

[0044] Step S1: Obtain an initial image I 2D or 3D;

[0045] Step S2: Obtain an initial segmentation of one or more objects, for example of semi-automatically or with a model trained by machine learning;

[0046] Step S3: Automatically determine an uncertain zone relative to each segmented object, for example according to distance and / or probability errors characteristic of the initial segmentation and determined beforehand;

[0047] For each segmented object the steps are implemented.

[0048] Step S4: Determine a segmentation, following the most visible boundary in image I separating in the uncertain area the object from the rest of the image;

[0049] Step S5: Determine and locate groups of connected pixels / voxels in deviation between the initial segmentation (S2) and the most visible boundary (S4), these deviations being located in the uncertain zone (S3), each deviation being classified according to its dimensions as "significant" or "non-significant";

[0050] Step S6 (optional): Classify each significant deviation according to other characteristics that may be a function of the initial image I;

[0051] Step S7: Decide whether to include or exclude each deviation of the object based on this or these classifications;

[0052] Step S8 (optional): Obtain a curve / surface following a contour of the segmented object in sub-pixel / voxel precision. Initial image I (step SI)

[0053] The method comprises a step SI of obtaining an initial 2D or 3D image, denoted I, of a region of interest comprising J object(s) Aj, j=l to J. An object is preferably bone, cartilage, or ligament. However, the method described herein applies to any type of object with visible contours in the image I. This image I is acquired using a known type of imaging system, depending on the type of initial image desired, and will not be further described.

[0054] The initial image I partitions the 2D or 3D physical space where the objects Aj are located into pixels or voxels, each indicating one or more real number(s), this number(s) being influenced by the nature and proportion of the objects Aj present in each pixel or voxel, for example, X-ray absorption, spin relaxation atoms, wave reverberation, etc., depending on the acquisition modality chosen. The initial image I is, for example, an image from radiography, fluoroscopy, MRI, CT or CBCT.

[0055] In addition, in the case of a 3D image, the initial 3D image I can be reconstructed from a small number of lower-dimensional images: - a 3D image reconstructed with a reduced number of ultrasound slices or; - a 3D X-ray absorption image reconstructed with a reduced number of 2D radiographs.

[0056] For example, the number K of images is between 1 and 10.

[0057] This initial image I therefore comprises N pixels / voxels denoted In = un with n=l to N, un being the value of the pixel / voxel.

[0058] Figure 2 illustrates a 2D image I of N=16x6=256 pixels whose values ​​are Influenced primarily by the nature and proportion of each object within each voxel, for example, Ia & [0, 255] for all n [1, N]. The proportion is visible in [Fig. 1] relative to the gray level. The darker the pixel, the more object Aj is present proportionally. Conversely, the lighter the pixel, the less object Aj is present proportionally. When an object fully occupies a pixel / voxel, the pixel / voxel value depends only on the nature of that object and the chosen imaging system. Initial segmentation Pj of an object (step S2)

[0059] In a step S2, the initial image I is processed according to an initial segmentation so as to obtain J image Pj of segmented object Aj. At the end of this initial segmentation, we obtain J image Pj each comprising N pixels / voxels, each value of which is a probability of belonging to the object Aj.

[0060] This initial segmentation can be implemented in several ways and must be robust, that is, it must avoid gross errors. However, its accuracy is limited. Such an initial segmentation thus exhibits one or more error margins characteristic of the initial segmentation method in pixel(s) / voxel(s) value and / or in distance; for example, a method exhibits maximum distance errors of ±2 pixels / voxels, or maximum probability errors of ±0.1, or a combination of the two allows for a better characterization of the maximum errors and results in smaller, on average, uncertain areas at step S3.

[0061] The initial segmentation can be implemented manually, semi-automatically or by means of a neural network trained in image segmentation.

[0062] In addition, the initial segmentation can combine several types of segmentation, for example by averaging the annotations of several methods or annotators, possibly weighted by their performance.

[0063] The annotations may be incomplete or even only point-based and the segmentation obtained by considering for each pixel / voxel In, is for example a probability of belonging to the object of the nearest annotated point, possibly based on distance.

[0064] The initial segmentation can be binary and made non-binary by applying, for example, a convolution operation such as a Gaussian blur.

[0065] Fig. 3 illustrates an example of an initial segmentation: on the left the initial image I comprising three segmented objects A1, A2, A2 and on the right the different resulting images Pj PI, P2, P3 comprising pixels / voxels at 1 or 0 depending on whether they belong to the object Aj or not.

[0066] Partition Uj of the initial segmentation Pj into 3 zones, one of which is uncertain (step S3)

[0067] During a step S3, each image Pj is processed, from one or more error margins of the initial segmentation of the image I so as to obtain a corresponding image Uj comprising N pixels / voxels.

[0068] The objective of this step S3 is to partition each image Pj by defining the following areas, taking into account one or more possible errors in the initial segmentation:

[0069] A first zone comprising only pixels / voxels Ujn of Uj that belong reliably to the object Aj;

[0070] A second area comprising only pixels / voxels Ujn of Uj that do not reliably belong to the object Aj;

[0071] An uncertain area comprising pixels / voxels that do not belong to either the first area or the second area.

[0072] The pixel / voxel values ​​of the first zone are preferably set to 1, those of the second zone set to 0 and those of the uncertain zone set to 1 / 2.

[0073] According to one embodiment, to partition the image Pj automatically, the following procedure is used:

[0074] For all n in [1 .. N]: • Ujn = 1 if: • Pin jn > Phighj (typically > ½) • where P^jn is the minimum value of the points of Pj in a convolution kernel K1Nj around Pjn • Ujn = 0 if: • Pouijn < PLowj (typically < U2) • where Pouijn is the maximum value of the points of Pj in a convolution kernel K0UTj around Pjn * PlN kn > PHIGH k • for all k in [1 .. J] where kj • where PIX k„cst is the minimum value of the points of Pk in a convolution kernel K1Nk around Pkn • Otherwise Ujn = 1 / 2.

[0075] In other words, the partition Uj of Pj includes a first zone totally included in the object Aj (erosion) and the second zone is a zone which is beyond the object (dilation).

[0076] We thus obtain the image Uj where the pixels Ujn for all n in [1 .. N] have a probability among {1, 1 / 2, 0} for each voxel center In of belonging to Aj given the characteristic errors of the initial segmentation and the other segmented objects. Note that these characteristic errors are determined before the initial segmentation of the image I by a characterization of the errors of the initial segmentation method.

[0077] Alternatively, the uncertain zone can also be obtained by simple thresholding operations and / or discrete mathematical morphology, i.e. in the domain {0,1} or even manually.

[0078] Figure 4 illustrates step S3. Considering the probability of 1D points belonging to an object around %=0 following a Gaussian distribution, step S3 allows the points to be partitioned according to different neighborhoods inside (first zone Z1) and outside (second zone Z2), and even according to different directions of the 1D image (K1 and K2 are anisotropic), as well as according to different probability thresholds to determine an uncertain zone that best accounts for a large number of possible characteristic errors for the initial segmentation method. The uncertain zone Z3 is the complementary zone of the first and second zones Z1 and Z2.

[0079] According to one embodiment, and to allow the next step to determine the most visible boundary of the object in an uncertain area that completely separates the first and second areas Z1, Z2, it is preferable to choose convolution kernels x, K0Ut x of size strictly greater than 1 pixel in all dimensions of I and a threshold PLow x < Phigh x-

[0080] In addition, it is possible to treat the non-connected components of Zl, Z2 in the image Uj by integrating them into the uncertain area to allow the next step to determine whether or not there is a boundary connecting them in the image I.

[0081] Most visible boundary Fj in the uncertain zone of Uj (stage S4)

[0082] In a step S4, the uncertain area determined in step S3 is used to detect, in the initial image I, the most visible boundary in this uncertain area. The most visible boundary separates, in the uncertain area, the pixels / voxels belonging to the object Aj from the others.

[0083] In particular, the pixels / voxels corresponding to those of the uncertain area identified in the image Uj are associated with a function measuring the visibility of the object boundary between neighboring pixels / voxels, considering the intensities of the pixels / voxels in the image I and possibly the initial segmentation Pj. This function is used to determine the boundary that is globally the most visible, that is, the one that globally optimizes this function.

[0084] According to one embodiment, a graph cut algorithm is used, the most visible boundary being the minimum cut of a discrete graph connecting neighboring pixels / voxels in the uncertain region by links having a maximum capacity defined by the function / , the minimum cut being obtained by means of an algorithm for solving the dual problem of maximum flux passing through the links of the graph (Graph Cut, GC). In this regard, reference can be made to the document Krcah, Marcel, Gâbor Székely, and Rémi Blanc. "Fully automatic and fast segmentation of the femur bone from 3D-CT images with no shape prior." 2011 IEEE international symposium on biomedical imaging: from nano to macro. IEEE, 2011.

[0085] According to one embodiment, a method for approximating a continuous geodesic curve / surface is used, the geodesic curve / surface being characterized by a metric, the metric being an antimonotonic function of the function, the most visible boundary being the approximate solution of this geodesic curve / surface, the approximate solution being obtained by means of a known algorithm for solving the Continuous Max Flow (CMF) dual problem, for example the Primal-Dual Interior Point (PDIP) algorithm for solving the combinatorial formulation of the problem (Combinatorial Continuous Max Flow, CCMF), or a partial differential equation solving algorithm for solving a physical formulation of the problem, for example Appleton-Talbot Continuous Max Flow (AT-CMF). For these algorithms, reference may be made to the following documents: - Couprie, Camille & Grady, Léo & Talbot, Hugues & Najman, Laurent. (2010). Combinatorial Continuous Maximal Flows. CoRR. abs / 1010.2733; - Appleton, Ben & Talbot, Hugues. (2003). Globally Optimal Surfaces By Continuous Maximal Flows.

[0086] Determining the most visible boundary implements one or more functions / (In, Pjn) that increase as the boundary of Aj becomes more visible in image I. To bring the greatest possible accuracy to the final segmentation, the function / (In, Pjn) must take into account the most local characteristics possible; that is, for each In, only the values ​​of In and / or its neighbors in the image I should be considered. For example, a norm of a discretized gradient of the image I that does not introduce a shift, such as the central difference of In. When the image I includes several values, such as a color image or, more generally, a multispectral image, the visibility of the boundary may depend on one or more of these values.

[0087] In addition, it may be useful to discriminate between internal or external boundaries of the object Aj. To do this, a directional component is introduced into the metric to ensure that, for example, the external boundary of the cortical bone (decreasing intensities from the inside to the outside) is segmented instead of its internal boundary (increasing intensities from the inside to the outside).

[0088] In this case, the function uses the vector gradient of the image I and the vector gradient of the initial segmentation Pj to discriminate an external boundary of Aj from internal boundary(s) to Aj, for example by using the dot product of the normalized gradient vectors of Pjn and In, for example with 0 < a < 1:

[0089] / (In, Pjn) = Il vin II • max(a; W l+cos(vln, vPjn))).

[0090] Such a function is expressed as a function of I and Pj, to be higher on the pixels / voxels where the gradient vector of Pjn is in a direction opposite to the gradient vector of In, thus making these pixels / voxels less "cutting" than pixels with an equally large gradient of In but oriented differently.

[0091] At the end of this step S4, a mask Fj is obtained which includes the pixels / voxels of the first zone and second zone and in the uncertain zone the values ​​adjusted with respect to the most visible boundary.

[0092] In addition, when the PDIP algorithm is used to obtain the most visible boundary, it provides at each iteration a solution v(nu) bounding the desired most visible boundary and generally converging rapidly towards it. The PDIP algorithm also provides a variable X(lambda) quantifying the relative contribution of each pixel / voxel to the visibility of the desired boundary.

[0093] In this case, at the end of step S4, Vj = X, a strictly positive and increasing local visibility indicator of the border Fj with respect to the local visibility of the border at the level of In, is obtained, Vj being advantageously normalized between 0 and 1.

[0094] Figure 5 illustrates in 2D the image I with the object Ai in particular, the corresponding image Ui, the resulting mask Fi, and the local visibility indicator Vj. The mask Fi is obtained by approximating a geodesic; the pixels of Fi corresponding to the uncertain zone Z3 of Ui are then given a value close to 0 (outside AJ) or 1 (in AJ), except for the pixels on either side of the most visible boundary in the image I, which are given a value on either side of Uz. The threshold of Uz allowing the pixels on either side of this boundary to be segmented. The mask Vi is obtained by the PDIP algorithm, with pixels having a value close to 0 except for the pixels on either side of the most visible boundary in the image I, which receive a value that is all the greater the more locally visible the boundary is.

[0095] It is observed that from image I and image Ui, in the uncertain area Z3, the mask Fi segments the pixels on either side of the most visible boundary in the initial image I.

[0096] Gap image, grouping and classification by dimension (step S5)

[0097] In a step S5, a gap image Ej between the pixels / voxels Fjn of the mask Fj and the pixels / voxels Pjn of the image Pj is determined, the unaffected pixels / voxels Ejn of the gap image Ej being gap pixels / voxels.

[0098] The idea here is to compare the pixels / voxels of the mask Fj with those of the image Pj to determine the differences which should be classified as belonging or not belonging to the object Aj.

[0099] Preferably, this gap image Ej is obtained from binary versions of the mask Fj and the image Pj.

[0100] To do this, the binary version FBj of Fj and the binary version PBj of Pj are obtained according to the following rule:

[0101] For all n in [1, ..., N]:

[0102] - FBjn= 1 if Fjn> 0.5 otherwise 0;

[0103] - PBjn = 1 if Pjn > 0.5 otherwise 0.

[0104] Then the gap image between the initial segmentation and the most visible boundary of Aj in the uncertain zone Ej = FBj - PBj is determined where - Ejn = 1 if In is in FBj (=1) but outside PBj (=0) - Ejn = 0 if In is in FBj (=1) and PBj (=1), or outside FBj (=0) and Pj (=0) - Ejn=-1 if In is outside FBj (=0) but in PBj (=1)

[0105] We then obtain, by a known connected component analysis, the set EGj of the coordinates, and optionally other features such as bounding boxes, M disjoint groups of connected pixels / voxels EGjm of value 1 and -1 named gaps, the gap EGjm being classified as "significant" (class S) if the size and / or depth of the set of these pixels / voxels is greater than an EM1Net / or Edepth threshold and "not significant" (class NS) otherwise.

[0106] The pixels / voxels of the non-significant deviations (class NS) will be assigned at step (S7) so as to follow the most visible boundary to correct the lack of precision of the initial segmentation.

[0107] Optional classification of significant deviations (step S6)

[0108] When the discrepancies are significant (class S), it may be advantageous to interpret them to determine whether the most visible boundary is indeed that of the object Aj and to assign them further to one of the following classes: Inside the object Aj (class IN) or outside the object Aj (class OUT).

[0109] Such an interpretation is implemented by means of a classification model, for example, a neural network, trained by deep learning, taking as input a region of interest of I centered on the coordinates or on the bounding box of a significant gap (class S) EGjm, and the same region of interest of Ej.

[0110] For example, it may be useful for surgical planning to assign a subclass that can explain the segmentation gap or influence the surgery, for example, for a bone-type object Aj in a CT image: • belonging to a neighboring bone (e.g., ribs, ossicles, etc.) or a bone growth not relevant to surgery (e.g., an osteophyte, etc.); • an artifact causing values ​​possibly lower than the lower limit of 700 HU (e.g. shadowing, beam hardening, ring artifacts, etc.); • a condition explaining a local under-intensity (e.g. osteoporosis, tumor, geode, etc.); • an artifact causing values ​​possibly exceeding the lower limit of 3000 HU for bones (e.g., streak, ring artifacts, etc.); • calcification of another nearby tissue (e.g., ligament, artery, etc.); • the presence of hardware (e.g., pedicle screw, etc.);

[0111] Certain subclasses such as "geode" or "belonging to a neighboring bone" may by themselves justify assigning a deviation to one of the IN or OUT classes.

[0112] Additionally, to assign a significant deviation (class S) to one of the IN or OUT classes, it may be useful to also consider the use case. For example, the subclass "osteophyte" may be assigned to the OUT class for a final segmentation intended to calculate the size of an implant, but to the IN class for a final segmentation intended to determine the area of ​​bone that a robot should cut.

[0113] Thus, depending on the subclass and optionally the use case, the gap can be integrated in the most appropriate way into the IN or OUT class.

[0114] At the end of this step S6 we obtain ECjm the set of class(es) and possibly subclass(es).

[0115] Fig. 6 illustrates in 2D a binarized segmentation PBi and a binarized boundary FBi whose difference makes the image of the gaps Ei appear in three connected components EGjm, one of which is not significant in size (comprising a single pixel).

[0116] Figure 7 illustrates in 2D an image I comprising two significant deviations classified by a classification model. The significant deviations are those corresponding to a light artifact and an osteophyte. These must then be classified as belonging to the object or not.

[0117] Assignment of each pixel / voxel to Ejn deviation (step S7)

[0118] In a step S7 each pixel / voxel in gap Ejn (see steps S5 and S6) is then assigned to the final segmentation mask Mj comprising a refined segmentation of the object Aj.

[0119] According to a first embodiment, the assignment is implemented according to the assignment rule defined as follows, for each pixel / voxel n of each significant gap EGjm (class S), if Ejn=l then Mjn=0, the pixel / voxel Ejn not belonging to the object Aj, if Ejn=-1 then Mjn=l, the pixel / voxel Ejnp belonging to the object Aj, the remaining pixels Mjn being identical to the corresponding pixels Fjn of the mask Fj to precisely follow the most visible boundary in the absence of gap or when the gaps are not significant (class NS).

[0120] According to a second embodiment, the pixels / voxels in significant deviations (class S) are more finely assigned taking into account their class determined in step S6 (classes IN or OUT).

[0121] The two embodiments can be summarized as follows: either Mj = Fj, then for each gap EGjm of EGj and for each pixel / voxel n of EGjm located in Ej, Fj, Pj and Mj, the assignment is implemented in the following way. Pixel value Ejn of EGjm Class(es) of EGjm Pixel value Mjn 1 (in FBj, outside PBj) S (Significant) 0 -1 (outside FBj, in PBj) S (Significant) 1 1 (inside FBj, outside PBj) S and IN (Inside Aj) Fjn (> 0.5) -1 (outside FBj, in PBj) S and IN (Inside Aj) 1 1 (inside FBj, outside PBj) S and OUT (Outside Aj) 0 -1 (outside FBj, in PBj) S and OUT (Outside Aj) Fjn (< 0.5) or 0

[0122] Thus, according to a first embodiment (only the first two lines of the table above), the pixels / voxels Ejn in deviation are assigned or not to the mask Mj according to their class NS or S and their value +1,-1 only.

[0123] According to a second embodiment, when the pixel / voxel in deviation Ejn is significant (class S) then it is interpreted by a classification model that can take into account the initial image and the use case and its resulting IN or OUT class is taken into account, those that are not significant (class NS) being classified only with respect to their value as in the first embodiment.

[0124] Fig. 8 illustrates in 2D an initial segmentation PBi towards the most visible boundary FBi shown in image I to result in the final segmentation Mb Curve / surface (step S8)

[0125] In addition, in a step S8, a curve / surface Cj delimiting the object Aj in the final image Mj is determined. This is the curve / surface Cj corresponding to a boundary of the image Mj obtained using a known algorithm, for example SurfaceNets, Marching Squares / Cubes, Cubical Marching Squares or Flying Edges.

[0126] Indeed, the geodesic defines the most visible unique boundary of Aj, but the accuracy of this geodesic is limited by the resolution of the pixels / voxels where the geodesic metric is calculated. In particular, the visibility indicator given by the CCMF method brackets this geodesic with values ​​that are attracted to the pixel / voxel centers and their midpoints.

[0127] This curve / surface Cj allows obtaining a contour of the object Aj with an accuracy of the order of ±0.3 pixel / voxel.

Claims

1. Demands A computer-implemented method for segmenting an image, comprising (SI) Obtaining a 2D / 3D image I of a region of interest of an element comprising at least one object Aj, j = 1 to J, the 2D / 3D image I comprising N pixels / voxels In, n=l to N; (S2) Initial segmentation of the 2D / 3D image I so as to obtain J image Pj of segmented object Aj, each image Pj comprising N pixels / voxels Pjn, each center of which is a probability of belonging to the object Aj; (S3) Processing each image Pj, from one or more margins of error of the initial segmentation of the image I so as to obtain a corresponding image Uj, the image Uj comprising a first zone and a second zone, the first zone comprising only pixels / voxels Ujn which belong reliably to the object, the first zone being totally included in the object Aj; the second zone comprising only pixels / voxels Ujn which do not belong reliably to the object Aj; the pixels / voxels not belonging to the first zone nor to the second belonging to an uncertain zone; (S4) Processing of the pixels / voxels of the 2D / 3D image I corresponding to the uncertain area of ​​the image Uj such that the values ​​of the pixels / voxels corresponding to the uncertain area of ​​the image Uj are defined with respect to a boundary delimiting the object Aj in the 2D / 3D image I in the uncertain area, the boundary being the most visible in the uncertain area, the pixels / voxels processed in the uncertain area and the pixels / voxels of the first and second areas defining a mask Fj; (S5) Determination of a gap image Ej between the pixels / voxels Fjn of the mask Fj and the pixels / voxels Pjn of the image Pj, the non-nuisance pixels / voxels Ejn of the gap image Ej being gap pixels / voxels, these pixels / voxels being grouped into M disjoint groups EGjm called "gaps", a gap EGjm being a set of gap pixels / voxels Ejn having the same non-zero value, the gap EGjm and therefore the gap pixels / voxels Ejn of the gap EGjm being classified as "significant" if the size and / or depth

2.

3.

4. of the set is greater than an EM1N and / or Edepth threshold and "not significant" otherwise; (S7) Assignment processing of each pixel / voxel Ejn in deviation to assign it or not to the object Aj and thus obtain a final segmentation mask Mj comprising a refined segmentation of the object Aj, the assignment processing comprising an assignment rule defined as follows, for each pixel / voxel n of each significant deviation EGjm, if Ejn=l then Mjn=0, the pixel / voxel Ejn not belonging to the object Aj, if Ejn=-1 then Mjn=l, the pixel / voxel Ejn belonging to the object Aj, the remaining pixels Mjn being identical to the corresponding pixels Fjn of the mask Fj. Method according to claim 1, comprising a step (S6) of classifying each pixel / voxel n of each significant deviation EGjm (class S) according to one of the following classes: Inside object Aj (IN) or outside object Aj (OUT) by means of a previously trained classification model. A method according to claim 2, wherein the classification takes into account at least one subclass assigned to each significant deviation, this subclass being used according to the use case of the final segmentation Mj to discriminate deviations that can be assigned to one or the other of the classes according to the use case. A method according to any one of claims 1 to 3, wherein the assignment rule is further defined as follows: Pixel value Ejn of EG jm Class(es) of EGjm Pixel value M jn 1 (in FBj, outside PBj) S and IN (Inside Aj) Fjn (>0.5) -1 (outside FBj, in PBj) S and IN (Inside Aj) 1 1 (inside FBj, outside PBj) S and OUT (Outside Aj) 0 -1 (outside FBj, in PBj) S and OUT (Outside Aj) Fjn (< 0.5) or 0 Mjn = 1 or > 0.5 when the pixel / voxel in deviation Ejn belongs to the object Aj and Mjn = 0 or < 0.5 when the pixel / voxel in deviation Ejn does not belong to the object Aj, the other remaining pixels / voxels Mjn being identical to the corresponding pixels Fjn of the mask Fj.

5. A method according to any one of the preceding claims, wherein the processing of each image Pj to obtain the image Uj (S3) consists, for all pixels / voxels n from 1 to N: if Pin jn > Phigh j, where Pin jit is the minimum value of the points of the image Pj in a convolution kernel Kin j around the pixel / voxel Pjn of Pj, the pixel / voxel Ujn of the image Uj belongs to the first zone; if For j / i < P1JW j °where POUT jjn is the minimum value of the points of the image Pj in a convolution kernel Kout j around the pixel / voxel Pjn of Pj or if P in k# > Phigh k for k=1 to J where k / j, the pixel / voxel Ujn of the image Uj belongs to the second zone.

6. A method according to claim 5, wherein Phigh j — 1 / 2, Plow j — 1 / 2 and wherein the pixels / voxels of the first zone have a probability equal to 1, the pixels / voxels of the second zone have a probability equal to 0, the pixels / voxels of the uncertain zone have a probability equal to 1 / 2.

7. A method according to any one of the preceding claims, wherein the most visible boundary (S4) is obtained by processing the 2D / 3D image I in the uncertain area using a graph cutting algorithm, the most visible boundary Fj being the minimum cut of a graph linking neighboring pixels / voxels in the uncertain area by links having a capacity defined by a function of the image I and optionally of the initial segmentation Pj, the minimum cut being obtained using an algorithm configured to solve a maximum flow problem.

8. A method according to any one of the preceding claims, wherein the most visible boundary (S4) is obtained by processing the 2D / 3D image I in the uncertain zone to obtain a geodesic curve / surface, the geodesic curve / surface being characterized by a function using the magnitude of the gradient of the image I, the most visible boundary being that which is the solution to the dual problem of the Combinatorial Continuous Max Flow (CCMF) algorithm, solved for example by the Primal-Dual Interior Point (PDIP) algorithm, or a partial differential equation solving algorithm. of the continuous maximum flow problem of the Appleton-Talbot Continuous Max Flow type, AT-CMF.

9. A method according to any one of claims 7 to 8, wherein the function / uses the magnitude of the gradient of the image I and possibly the magnitude of the gradient of the initial segmentation Pj to discriminate an external boundary of Aj from internal boundary(s) to Aj, for example by using the dot product of the normalized gradient vectors of Pjn and In, for example with 0 < a < 1: / (In, Pjn) = Il vin II • max( a ; W l+cos(vln , vPjn)) ).

10. A method according to any one of the preceding claims, wherein the determination of the image Ej (S5) comprises a binarization of the mask Fj and the image Pj consisting of obtaining a binary image FBj in which each pixel FBjn is equal to 1 if Fjn>0.5 otherwise 0 and a binary image PBj in which each pixel PBjn is equal to 1 if Pjn>0.5 otherwise 0, the binary mask FBj and the binary image PBj being binary versions of the mask Fj and the image Pj where a pixel / voxel of the binary mask FBj and / or the binary image PBj is equal to 1 for each pixel / voxel greater than 0.5 or 0 for each pixel / voxel less than 0.5, the image Ej comprising pixels / voxels whose levels are 0, +1 or -1, the uncertain pixels / voxels having a value equal to -1 or +1, the gap image Ej being determined from the binary images FBj and PBj.

11. A method according to claim 10, wherein the gap image Ej has pixels / voxels Ejn=1 if the pixel In of the image I is included in the binary mask FBj and outside the binary image PBj, Ejn=0 if the pixel In of the image I is included in the mask FBj and in the mask PBj or outside the mask FBj and the mask PBj, Ejn=-1 if the pixel In of the image I is outside the binary mask FBj and included in the image PBj.

12. A method according to any one of the preceding claims, comprising a step (S8) of determining a curve / surface Cj delimiting the object Aj in the final segmentation mask Mj, the curve / surface Cj corresponding to a boundary in the final segmentation mask Mj obtained by means of a SurfaceNets, Marching Squares / Cubes, Cubical Marching Squares or Flying Edges algorithm.

13. A method according to any one of the preceding claims, wherein the image I (SI) is a 3D image reconstructed from a number K of 2D images whose projection parameters are obtained, for example K between 1 and 10, and the boundary (S4) Fj is then determined on an image that is a linear combination of I and the K 2D images back-projected into the space of I according to their respective projection parameters, for example a weighted average where I has a weight equal to K and the K images each have a weight of 1.