Device and method for training a segmentation model, and method and device for segmenting a three-dimensional object
A neural network-based segmentation model using transformed target masks and iterative application with previous image prompts addresses the inefficiencies of current 3D medical image segmentation, achieving accurate and automated results.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- RAIDIUM
- Filing Date
- 2025-10-27
- Publication Date
- 2026-05-07
AI Technical Summary
Current medical image segmentation models are task-specific and inefficient for three-dimensional images, requiring tedious manual input on each slice of a volume, especially when segmenting multiple slices.
A neural network-based segmentation model trained with transformed target masks and iterative application using previous image masks as prompts, allowing automated and efficient 3D segmentation.
Accurate and fast segmentation of 3D medical images by automating the process across multiple slices, reducing manual input and improving efficiency.
Smart Images
Figure FR2025050994_07052026_PF_FP_ABST
Abstract
Description
[0001] Description
[0002] Title of the invention: device and method for training a segmentation model, method and device for segmenting an object in three dimensions.
[0003] Technical Field
[0004] [1] This disclosure falls within the general field of medical imaging systems. More specifically, this disclosure relates to a method for training a segmentation model and a method for segmenting a plurality of medical images captured by a medical imaging device, the plurality of medical images representing a series of two-dimensional slices of a three-dimensional volume.
[0005] Previous technique
[0006] [2] Segmentation is a fundamental task in medical imaging analysis, involving the identification and delineation of regions of interest (ROIs) in various medical images, such as organs, lesions, and tissues. Accurate segmentation is essential for many clinical applications, including disease diagnosis, treatment planning, and monitoring disease progression. Manual segmentation has long been the solution for delineating anatomical structures and pathological regions, but this process is time-consuming, labor-intensive, and often requires a high degree of expertise. Semi-automatic or fully automatic segmentation methods can significantly reduce the time and labor required, improve consistency, and enable the analysis of large-scale datasets.
[0007] [3] Deep learning-based models have shown great promise for medical image segmentation due to their ability to learn complex image features and provide accurate segmentation results across a range of tasks, from segmenting specific anatomical structures to identifying pathological regions. However, a significant limitation of many current medical image segmentation models is their task-specific nature. [4] Recent advances in natural image segmentation have seen the emergence of basic segmentation models, such as SAM (segment anything model) and SEEM (segment everything everywhere with multi-modal prompts all at once), which demonstrate remarkable versatility and performance across a variety of segmentation tasks.These models are used on two-dimensional images to predict segmentation masks on a single image, based on an instruction regarding the area to be segmented, obtained from a user of the model. This instruction can be a single point or two points defining a rectangle around the object (also called a "bounding box"). Such methods can be used for segmenting 2D medical images by refining them.
[0008] [5] However, in radiology, many images are three-dimensional: MRI or CT scans produce images containing numerous images or slices, separated by a few millimeters, which amounts to a 3D volume composed of voxels. Furthermore, it can be useful to obtain 3D segmentation masks of the anatomical structures or lesions in these images, for example, to calculate their volume. To segment in three dimensions from a 2D segmentation model, the two-dimensional model can be applied to each slice separately. This requires positioning an instruction, also called a prompt (a point or a set of two points), on each slice of the volume.This can be tedious and make the process extremely slow in the case of a plurality of slices, especially when the user has to select the object of interest in each slice.
[0009] [6] There is therefore a need to improve the segmentation of objects of interest in three-dimensional images.
[0010] Description of the invention
[0011] [7] To this end, the present invention relates to a method for training a segmentation model based on a neural network, said segmentation model receiving a plurality of input images and an instruction for an area to be segmented in said input images, said method comprising:
[0012] - an encoding of the characteristics of the input images, including a patch decomposition of the input image and a vector representation of said patches,
[0013] - an encoding of said at least one instruction to obtain a vector representation of said at least one instruction,
[0014] - a decoding of a mask representative of the segmented area from the vector representation of said patches and of said at least one vector representation of said at least one instruction, in which said at least one instruction is a mask obtained from at least one transformation of a target mask of said area to be segmented.
[0015] [8] Thus, training the segmentation model can advantageously yield a model that is suitable for segmenting successive images in which the object to be segmented is the same but varies slightly from one image to the next. The model is adapted to refine the result from image to image and therefore produces accurate results compared to traditional point-based or bounding box methods.
[0016] [9] According to some embodiments, said transformation may be one or more transformations chosen from one or more of the following:
[0017] - a rotation,
[0018] - a translation,
[0019] - a dilation,
[0020] - erosion.
[0021]
[0010] To ensure that the model is trained correctly, it is relevant that it be trained using masks that are different from the target mask obtained, while still being related to that mask. This can allow for robust training of the model.
[0022]
[0011] The present invention also relates to a method for segmenting a plurality of medical images captured by a medical imaging device, said plurality of medical images representing a series of two-dimensional slices of a three-dimensional volume, said method comprising
[0023] - a selection of at least one point on a first image selected from said plurality of images, said at least one point being positioned on an object of interest in said image,
[0024] - an application of a segmentation model trained according to any one of the embodiments of the training method according to the present invention, said segmentation model receiving as input said selected image and an instruction consisting of said at least one point selected on said at least one first selected image, to obtain a mask representative of said segmented object of interest,
[0025] - an iterative application of said segmentation model, on at least a subset of said plurality of images, such that said segmentation model receives as input for a current image other than said first image, an instruction consisting of a selection of points obtained from the representative mask of said object of interest obtained for the previously segmented image of said plurality of images.
[0026]
[0012] Since the model is trained by receiving a mask as an instruction, it can take as input the mask obtained for a previous image of the image slice or points from that mask. Indeed, because the mask is slightly different in each image since the object of interest varies from one image to another, the mask of the previous image or points from the mask of the previous image can be considered as a transformation of the target mask. Furthermore, using at least one selection of points obtained from the mask of the previous image automates the segmentation process, preventing a user from having to enter an instruction for each image of the slice to be segmented. The process is thus fast, automatic, and very efficient in terms of segmentation results.
[0027]
[0013] According to certain embodiments, said selection of points obtained from the representative mask of said object of interest is determined to be:
[0028] - the mask of the object of interest obtained for the previously segmented image or,
[0029] - a box encompassing the mask of the object of interest obtained for the previously segmented image or,
[0030] - a point on the mask of the object of interest obtained for the previously segmented image,
[0031] - the barycenter of the mask of the object of interest obtained for the previously segmented image.
[0032]
[0014] According to some embodiments, said first selected image is the image comprising a size of the object of interest greater than a threshold.
[0033]
[0015] This can advantageously improve the performance of the model.
[0034]
[0016] According to some embodiments, said selection of at least one point for said first image includes the selection not of a single point but of a set of points forming a bounding box around said object of interest.
[0035]
[0017] When a mask is not available for a first image, a bounding box for the first image can be selected as an instruction.
[0018] According to certain embodiments, said selection of at least one point for said first image includes the selection of a mask of said object of interest.
[0036]
[0019] According to certain embodiments, the iterative application of said segmentation model is repeated for at least a subset of said plurality of images as long as one or a combination of the following conditions is satisfied:
[0037] - the size of the object of interest in the current image is greater than a size determined based on the average size of said object,
[0038] - the size of the object of interest in the current image is greater than a percentage of the size of said object of interest in said first image,
[0039] - the intersection between the said representative masks of said object in said current image and in said previous image is less than a percentage of the size of said mask of the previous image.
[0040]
[0020] The set of images in the image slice may include images that do not contain the object of interest. This is theoretically the case for most organs of the human body, which are not present in a volume as deep as the depth to which the medical imaging system can reach. It is therefore advantageous to have a process termination condition that allows the process to stop when the size of the organ in an image is small relative to its actual size. The size can be determined as the size of the mask in the image.
[0041]
[0021] According to certain embodiments, the process comprises:
[0042] - for at least one of the iterations, other than the first iteration, called the adjustment iteration, a reception of an adjustment of said segmentation, said adjustment consisting of a selection of at least one point positioned on the segmented object of the previous image
[0043] - for the adjustment iteration, an application of said segmentation model, said segmentation model receiving as input said at least one point selected on the image corresponding to the current iteration, to obtain a mask representative of said segmented object of interest,
[0044] - an iterative application of said segmentation model for iterations following the fitting iteration, such that said segmentation model receives as input a selection of points obtained from the representative mask of said object of interest obtained for the previously segmented image of said plurality of images.
[0022] According to certain embodiments, the method comprises:
[0045] - the reconstruction of said object of interest in three dimensions from the segmentation of said object on said slices of two-dimensional images.
[0046]
[0023] The features, presented individually in this application in connection with certain embodiments of the processes of this application, can be combined with each other according to other embodiments of the present processes.
[0047]
[0024] The present invention also relates to a computer program comprising instructions for executing the steps of the training process according to one of its embodiments when said program is executed by a computer.
[0048]
[0025] The present invention also relates to a computer program comprising instructions for executing the steps of the segmentation process according to one of its embodiments when said program is executed by a computer.
[0049]
[0026] The present invention also relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for executing the steps of the segmentation training process according to one of its embodiments when said program is executed by a computer.
[0050]
[0027] The present invention also relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for executing the steps of the segmentation process according to one of its embodiments when said program is executed by a computer.
[0051]
[0028] In another aspect, the present invention also relates to a training device for a segmentation model based on a neural network, for executing the steps of the process according to the invention, according to any one of its embodiments. Thus, the present invention relates to a training device for a segmentation model based on a neural network, said device comprising one or more processors configured together or separately to
[0052] - encode features of input images, including a patch decomposition of the input image and a vector representation of said patches,
[0053] - encode said at least one instruction to obtain a vector representation of said at least one instruction,
[0054] - decode a mask representative of the segmented area from the vector representation of said patches and of said at least one vector representation of said at least one instruction, in which said at least one instruction is a mask obtained from at least one transformation of a mask of said area to be segmented.
[0055]
[0029] In another aspect, the present invention also relates to a segmentation device based on a neural network for performing the steps of the segmentation process according to the invention, according to any one of its embodiments. Thus, the present invention relates to a segmentation device comprising one or more processors configured together or separately to:
[0056] - select at least one point on a first image selected from said plurality of images, said at least one point being positioned on an object of interest in said image,
[0057] - apply a segmentation model trained according to any of the embodiments of this disclosure, said segmentation model receiving as input said selected image and an instruction consisting of said at least one point selected on said at least one first selected image, to obtain a mask representative of said segmented object of interest
[0058] - iteratively apply said segmentation model, on at least a subset of said plurality of images, such that said segmentation model receives as input for a current image other than said first image, an instruction consisting of a selection of points obtained from the representative mask of said object of interest obtained for the previously segmented image of said plurality of images.
[0059]
[0030] Other features and advantages of the present invention will become apparent from the following description of embodiments of the invention. These embodiments are given by way of illustration and are not intended to be limiting. The following description is illustrated by the accompanying drawings.
[0060] Brief description of the drawings
[0061]
[0031] The attached drawings are schematic and are intended primarily to illustrate the principles of the exposition.
[0062]
[0032] In these drawings, from one figure to another, identical elements (or parts of elements) are identified by the same reference symbols.
[0033] [Fig. 1] Figure 1 shows an example of a medical image with a bounding box
[0063]
[0034] [Fig. 2] Figure 2 shows an example of a medical image with a mask,
[0064]
[0035] [Fig. 3] Figure 3 represents an embodiment of the method for training a segmentation model according to an embodiment of the present invention,
[0065]
[0036] [Fig. 4] Figure 4 represents an embodiment of the segmentation process according to an embodiment of the present invention.
[0066] Description of the implementation methods
[0067]
[0037] The present description relates to the field of medical images obtained by medical imaging devices. It concerns more particularly the training of a segmentation model and the use of the trained segmentation model applied to a plurality of images obtained by a three-dimensional medical imaging device.
[0068]
[0038] Medical imaging devices can provide two-dimensional images, such as X-ray, mammography, ultrasound devices, or three-dimensional images, such as computed tomography (CT scan), magnetic resonance imaging (MRI), tomography (PET scan), 3D ultrasound imaging.
[0069]
[0039] The training of the segmentation model can be applied to images that are not necessarily from the medical field, but it is relevant to train it on medical images when it is used to segment medical images, in order to increase the reliability of the model.
[0070]
[0040] Furthermore, even if the model as used in the present invention is used to segment a plurality of medical images representing a series of two-dimensional slices of a three-dimensional volume, the model can be trained from a set of training images that do not represent a series of two-dimensional slices of a three-dimensional volume.
[0071]
[0041] Medical imaging devices that produce three-dimensional images generate images comprising numerous slices, spaced a few millimeters apart, thus producing a volume composed of voxels. It is useful to obtain a comprehensive, or volumetric, view of the organ to be segmented, for example, to determine a pathology. This requires segmenting all the slices that make up the object, or at least a sufficient proportion, to reconstruct a three-dimensional, or volumetric, view of the organ. The present invention can accelerate the acquisition of this three-dimensional view by using the mask obtained for an image i to segment the image i+1, for at least a plurality of images among those captured.
[0072]
[0042] Figure 1 shows an example of a medical image, representing a plurality of organs, including a liver. A bounding box is drawn on this liver, encompassing both the liver and pixels not located within the liver. A bounding box has a rectangular (or square) shape.
[0073]
[0043] Figure 2 represents a segmented liver. It can be seen in particular that in this image, the segmentation of the liver is very approximate and does not include all the pixels of the liver.
[0074]
[0044] Figure 3 illustrates an embodiment of training a segmentation model as used in this disclosure. This model is based on a model known as "segment anything," also known by the acronym "SAM."
[0075]
[0045] The architecture of a "SAM" model is based on three main components, namely an image encoder 11, an instruction encoder 12 and a mask decoder 13.
[0076]
[0046] In the remainder of this disclosure, the term instruction encoder may also be read as indication encoder, prompt encoder, or simply prompt encoder. Similarly, when referring to an instruction, it may be understood as indication, prompt, or prompt.
[0077]
[0047] The image encoder 11 is preferably based on a so-called architecture of
[0078] The "transformer" is similar to the SAM model, but in some embodiments it can be based on a convolutional neural network (CNN) architecture that extracts features from the input images. In some embodiments, the architecture can be hybrid, composed of CNNs and transformers.
[0079]
[0048] The image encoder divides the image into patches, also called tiles, of intermediate size (typically but not exclusively 16x16 pixels) and associates a latent vector or representative vector with each patch. It produces as output a high-dimensional representation generated by the image encoder, which captures the global and local characteristics of the image.
[0080]
[0049] Transformers are particularly well-suited for processing data types that are not images, such as sequences or positions in a given space. Transformers operate by capturing spatial relationships and dependencies between different inputs (e.g., the positions of points or bounding boxes) through attention mechanisms. The role of the point encoder is to convert this spatial information into a representation that the model can use to generate segments.
[0081]
[0050] The instruction encoder 12, also called the prompt or indication encoder, may also preferably be based on transformers or similar mechanisms. In some embodiments, it may be based on CNNs. In some embodiments, the architecture may be hybrid, composed of CNNs and transformers.
[0082]
[0051] The segmentation model also includes a convolution module located before the mask decoder 13. The convolution module is used to reduce the dimensionality of the vectors obtained from the image encoder 11. It allows the high-dimensional representations to be transformed into a more compact form that is easier for the mask decoder to process. By reducing the dimensionality, the computational load is also reduced, allowing the model to process information more quickly while retaining the most important features.
[0083]
[0052] The convolution module refines the image features to extract important local details for segmentation. Convolutions are particularly effective at capturing spatial information, such as edges, shapes, and textures, which are essential for determining the precise boundaries of the objects to be segmented. This refinement process allows the mask decoder to have more relevant inputs for generating more accurate masks. The convolution module acts as a preprocessing step to ensure that the image features are in an optimal format for merging with the instruction (prompt) features.
[0053] The mask decoder 13 combines information from the image encoder and the prompt encoder. Its architecture is preferably a hybrid architecture based on a CNN network and transformers.In other embodiments, it can be a CNN or transformer architecture. It takes as input the high-dimensional representation generated by the image encoder and the representations extracted by the prompt encoder. The goal of the mask decoder is to merge these two types of information to produce accurate masks. This fusion is generally performed using a cross-attention mechanism.
[0084]
[0054] Once the image and prompt information has been merged by the attention mechanism, the mask decoder produces a segmentation map or binary mask. This segmentation map indicates which pixels of the image belong to the object or region specified by the user. The mask decoder predicts a patch-scale mask. After the image and prompt information have been merged, the decoder's features can be upsampled to generate a segmentation mask with the same resolution as the original image. A score is also calculated from a cost function by comparing the output mask to the target (original mask). This score, also called the loss function, is used to optimize the weights of the network by stochastic gradient descent.
[0085]
[0055] The final mask can be interpreted as a binary image where each pixel is either inside or outside the segmented object.
[0086]
[0056] The present disclosure advantageously proposes to train the model from a different prompt than those proposed by the SAM model.
[0087]
[0057] In the SAM model, three training modes are offered, namely:
[0088] - a "dot"-to-mask type training task, where the prompt decoder receives a dot as input,
[0089] - a training task of the "bounding box" type towards a mask,
[0090] - a training task of the type "bounding box" and / or several points and mask to mask.
[0091]
[0058] In this last scenario of the SAM model, the model is not trained to receive only a mask as input and then produce a new, refined mask, but rather it is trained to perform mask editing based on an existing mask and a bounding box and / or points entered by a user into the instruction decoder. Therefore, the model weights obtained during the training of the SAM model, even when it receives as input a mask to be edited and elements allowing for refining this mask, are not suitable for receiving a mask alone as input.
[0092]
[0059] To this end, the present disclosure proposes a method for training a segmentation model based on a neural network, said segmentation model receiving a plurality of input images and an instruction for an area to be segmented in said input images, said method comprising:
[0093] - an encoding of the features of the input images by a convolutional neural network, including a patch decomposition of the input image and a vector representation of said patches,
[0094] - An encoding of said at least one instruction to obtain a vector representation of said at least one instruction,
[0095] - A mask decoding representative of the segmented area from the vector representation of said patches and of said at least one vector representation of said at least one instruction, in which said at least one instruction is a mask obtained from at least one transformation of a mask of said area to be segmented.
[0096]
[0060] Thus, the training aims to refine an erroneous mask to obtain an exact or target mask, and this is achieved through transformations of the exact or target mask. The transformation must be a compromise between a transformation that, if it is too far removed, can resemble a bounding box instruction, and one that is too close, can make the training task too simple.
[0097]
[0061] Thus, the present disclosure uses, as a prompt, a mask or, more precisely, a transformation of a mask, and not a point or a bounding box. This transformation may be one or more transformations chosen from one or more of the following:
[0098] - a rotation,
[0099] - a translation,
[0100] - a dilation,
[0101] - erosion.
[0102]
[0062] This can advantageously improve the model's accuracy by obtaining a more precise mask and therefore more accurate segmentation.
[0063] To obtain the prompt masks used during training, a set of target masks is determined (or positioned) on objects of interest in medical images. These target masks may have been precisely determined, for example, by radiologists. An algorithmic transformation, 14, is applied to these target masks to introduce noise and make them erroneous. During training, the model's task is to correct this erroneous mask to recover the mask before the transformation.
[0103]
[0064] This transformation can advantageously simulate a difference between the masks of two consecutive slices of a set of images when the model is used as described with reference to Figure 3 when used in the context of this disclosure.
[0104]
[0065] Figure 4 represents a segmentation method based on the trained segmentation model as shown with reference to Figure 3.
[0105]
[0066] The segmentation method segments medical images captured by a 3D medical imaging device. The images represent a series of two-dimensional slices of a three-dimensional volume, comprising one or more parts of the human body. At least one of these parts is segmented by the segmentation method.
[0106]
[0067] The three-dimensional medical imaging device can capture images along different axes. Thus, image slices can be obtained along several axes. The method may include selecting an axis prior to segmenting the image slice obtained along that axis. It may also be envisaged to perform the segmentation along several (all or at least some) axes and compare the results to select the best segmentation result.
[0107]
[0068] In known image segmentation techniques for representing a series of two-dimensional slices of a three-dimensional volume, it is tedious to have to enter a prompt for each image of the slice to be segmented. One solution is to reduce the number of segmented images of the slice and reconstruct the volume based on these segmented images, but this considerably reduces the object's accuracy. Another technique is to segment all the images, but this is very time-consuming.
[0069] The present invention proposes using, as a prompt for an image, an instruction consisting of a selection of points obtained from the representative mask of the object of interest obtained for the previously segmented image of said plurality of images. The selection of points obtained from the representative mask of the object of interest obtained for the previously segmented image can be the mask obtained for the previously segmented image.
[0108]
[0070] This can advantageously increase the model's accuracy. Indeed, since the mask of image i and image i+1 are very close, the model performs very well. This can also eliminate the need to manually enter a prompt for each image in the volume.
[0109]
[0071] The model being trained to receive a mask as input as a prompt, is efficient when it receives the mask of the previous image as input as a prompt.
[0110]
[0072] To that end, the present disclosure relates to a method for segmenting a plurality of medical images captured by a medical imaging device, said plurality of medical images representing a series of two-dimensional slices of a three-dimensional volume, comprising
[0111] - a selection of at least one point on a first image selected from said plurality of images, said at least one point being positioned on an object of interest in said image,
[0112] - an application of a segmentation model trained according to one of the embodiments of the training method of this disclosure, said segmentation model receiving as input said selected image and an instruction consisting of said at least one point selected on said at least one first selected image, to obtain a mask representative of said segmented object of interest
[0113] - an iterative application of said segmentation model, on at least a subset of said plurality of images, such that said segmentation model receives as input for a current image other than said first image, an instruction consisting of a selection of points obtained from the representative mask of said object of interest obtained for the previously segmented image of said plurality of images.
[0114]
[0073] Preferably, the selection of points obtained from the representative mask of said object of interest is determined to be:
[0115] - the mask of the object of interest obtained for the previously segmented image, or - a box encompassing the mask of the object of interest obtained for the previously segmented image, or
[0116] - a point on the mask of the object of interest obtained for the previously segmented image,
[0117] - the barycenter of the mask of the object of interest obtained for the previously segmented image.
[0118]
[0074] The use of the mask of the object of interest obtained for the previously segmented image allows automation of the process for all or part of the images and precision and speed of segmentation of all the images.
[0119]
[0075] The use of a box encompassing the mask of the object of interest obtained for the previously segmented image can allow adjustment of the model during segmentation of all images, therefore the process is not automated on all images.
[0120]
[0076] Using the centroid of the mask of the object of interest obtained for the previously segmented image can advantageously allow the selection of a central point of the object to be selected, minimizing or reducing the risk that the model segments an object that is not the object of interest. More generally, when a point is selected, it is preferably not positioned on the edges of the object but inside the object and preferably in a central area.
[0121]
[0077] To improve the performance of the method, it may be useful to select the first image to be segmented. For this purpose, the first selected image may be the image containing an object of interest whose size exceeds a certain threshold. The first slice may be preferentially selected by a user. To this end, the method may receive a selection of the first image to be segmented or an indication of the first image to be segmented.
[0122]
[0078] Thus, when the first image selected is not one of the images located at one end of the slice, but is an image located between the images of the slice, the process can be repeated to proceed in a first direction, for example towards the image above the slice, and then in a second direction, towards the image below the slice. Of course, the first direction can be towards the image below the slice and the second direction towards the image above the slice.
[0123]
[0079] The selection of the first image can be performed by applying the trained model to several images of the slice and selecting, from among these images, the one with the largest segmented object size. The selected images can be distributed throughout the slice, or they can be selected randomly.
[0124]
[0080] Another method for selecting the first image can consist of segmenting the two images at the two ends of the slice and comparing the size of the segmented object in the two segmented images. The first image is the one with the largest segmented object. If, between the first and last images, the size of the segmented object is less than a predetermined threshold, a new image, taken between the two end images, can be selected. This new image can be the first image in which the size of the segmented object is greater than or equal to the predetermined threshold. A new image is selected as long as the size of the segmented object is less than a threshold. The threshold is dependent on (or determined based on) the size of the organ to be segmented.
[0125]
[0081] The method does not have a mask for segmenting the first selected image. Thus, selecting at least one point for the first image involves selecting a set of points forming a bounding box around the object of interest.
[0126]
[0082] The set of images in the image slice may include images that do not contain the object of interest. Indeed, the object of interest may be contained in fewer images if its volume is small, or in more images if its volume is large. A stopping condition for the process can therefore be determined to avoid segmenting images in the slice on which the object of interest does not appear, or based on its size within those images.
[0127]
[0083] Thus, the iterative application of said segmentation model is repeated for at least a subset of said plurality of images as long as one or a combination of the following conditions is satisfied:
[0128] - the size of the object of interest in the current image is greater than a size determined based on the average size of said object,
[0129] - the size of the object of interest in the current image is greater than a percentage of the size of said object of interest in said first image,
[0130] - the intersection between said representative masks of said object in said current image and in said previous image is less than a percentage of the size of said mask in the previous image.
[0084] It may be necessary to readjust the segmentation during iterations. Indeed, a deviation in the shape of the organ, and therefore in the segmentation, may be observed during the iterations. The method may include:
[0131] - for at least one of the iterations, other than the first iteration, called the adjustment iteration, a reception of an adjustment of said segmentation, said adjustment consisting of a selection of at least one point positioned on the segmented object of the previous image
[0132] - for the adjustment iteration, an application of said segmentation model, said segmentation model receiving as input said at least one point selected on the image corresponding to the current iteration, to obtain a mask representative of said segmented object of interest
[0133] - an iterative application of said segmentation model for iterations following the adjustment iteration, such that said segmentation model receives as input a selection of points obtained from the representative mask of said object of interest obtained for the previously segmented image of said plurality of images.
[0134]
[0085] An adjustment may be necessary, causing a deviation in the shape of the organ, depending on the size and / or shape of the organ, the texture, the color, a pathology (tumor for example), artifacts in the data recording, differences in data distribution (for example the age or ethnicity of the patient), the brand of the image capture device, the type of device (CT scan, MRI....), the device setting...).
[0135]
[0086] One of the applications of the method described above is its application to determining the volume of the segmented object. To this end, the method may include reconstructing said object of interest in three dimensions from the segmentation of said object on said two-dimensional image slices.
[0136]
[0087] Although the present invention has been described with reference to specific embodiments, it is evident that modifications and changes can be made to these examples without departing from the general scope of the invention as defined by the claims. In particular, individual features of the various embodiments illustrated / mentioned can be combined in additional embodiments. Therefore, the description and drawings should be considered in an illustrative rather than a restrictive sense.
[0088] It is also evident that all the features described with reference to a method are applicable, alone or in combination, to a device, and conversely, all the features described with reference to a device are applicable, alone or in combination, to a method.
Claims
Demands
1. A method for training a segmentation model based on a neural network, said segmentation model receiving a plurality of input images and an instruction for an area to be segmented in said input images, said method comprising: - an encoding (11) of the characteristics of the input images, comprising a patch decomposition of the input image and a vector representation of said patches, - an encoding (12) of said at least one instruction to obtain a vector representation of said at least one instruction, - a mask decoding (13) representative of the segmented area from the vector representation of said patches and of said at least one vector representation of said at least one instruction, in which said at least one instruction is a mask obtained from at least one transformation (14) of a target mask of said area to be segmented.
2. A method according to claim 1 wherein said transformation may be one or more transformations selected from one or more of the following: - a rotation, - a translation, - a dilation, - erosion.
3. A method for segmenting a plurality of medical images captured by a medical imaging device, said plurality of medical images representing a series of two-dimensional slices of a three-dimensional volume, said method comprising - a selection of at least one point on a first image selected from said plurality of images, said at least one point being positioned on an object of interest in said image, - an application of a segmentation model trained according to claim 1 or 2, said segmentation model receiving said image as input selected and an instruction consisting of said at least one point selected on said at least one first selected image, to obtain a mask representative of said segmented object of interest - an iterative application of said segmentation model, on at least a subset of said plurality of images, such that said segmentation model receives as input for a current image other than said first image, an instruction consisting of a selection of points obtained from the representative mask of said object of interest obtained for the previously segmented image of said plurality of images.
4. A method according to claim 3, wherein said point selection obtained from the representative mask of said object of interest is determined to be: - the mask of the object of interest obtained for the previously segmented image or, - a box encompassing the mask of the object of interest obtained for the previously segmented image or, - a point on the mask of the object of interest obtained for the previously segmented image, - the barycenter of the mask of the object of interest obtained for the previously segmented image.
5. A method according to any one of claims 3 or 4 wherein said first selected image is the image comprising a size of the object of interest greater than a threshold.
6. A method according to any one of claims 3 to 5 wherein said selection of at least one point for said first image comprises the selection of a set of points forming a bounding box around said object of interest.
7. A method according to any one of claims 3 to 6 wherein the iterative application of said segmentation model is repeated for at least a subset of said plurality of images as long as one or a combination of the following conditions is satisfied: - the size of the object of interest in the current image is greater than a size determined based on the average size of said object, - the size of the object of interest in the current image is greater than a percentage of the size of said object of interest in said first image, - the intersection between the said representative masks of said object in said current image and in said previous image is less than a percentage of the size of said mask of the previous image.
8. A method according to any one of claims 3 to 7 comprising: - for at least one of the iterations, other than the first iteration, called the adjustment iteration, a reception of an adjustment of said segmentation, said adjustment consisting of a selection of at least one point positioned on the segmented object of the previous image - for the adjustment iteration, an application of said segmentation model, said segmentation model receiving as input said at least one point selected on the image corresponding to the current iteration, to obtain a mask representative of said segmented object of interest - an iterative application of said segmentation model for iterations following the adjustment iteration, such that said segmentation model receives as input a selection of points obtained from the representative mask of said object of interest obtained for the previously segmented image of said plurality of images.
9. A method according to any one of claims 3 to 8 comprising: - the reconstruction of said object of interest in three dimensions from the segmentation of said object on said slices of two-dimensional images.
10. Computer program comprising instructions for carrying out the steps of the process according to any one of claims 1 to 2 or carrying out the steps of the process according to any one of claims 3 to 9 when said program is executed by a computer.
11. A computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the process according to any one of claims 1 to 2 or carrying out the steps of the process according to any one of claims 3 to 9.